CN102999939B - Coordinate acquiring device, system and method for real-time three-dimensional reconstruction, stereo interaction device - Google Patents

Coordinate acquiring device, system and method for real-time three-dimensional reconstruction, stereo interaction device Download PDF

Info

Publication number
CN102999939B
CN102999939B CN201210353512.7A CN201210353512A CN102999939B CN 102999939 B CN102999939 B CN 102999939B CN 201210353512 A CN201210353512 A CN 201210353512A CN 102999939 B CN102999939 B CN 102999939B
Authority
CN
China
Prior art keywords
image
point
camera
infrared
coordinates
Prior art date
Application number
CN201210353512.7A
Other languages
Chinese (zh)
Other versions
CN102999939A (en
Inventor
王伟
黄宇聪
魏益群
刘翩
Original Assignee
魏益群
王伟
黄宇聪
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 魏益群, 王伟, 黄宇聪 filed Critical 魏益群
Priority to CN201210353512.7A priority Critical patent/CN102999939B/en
Publication of CN102999939A publication Critical patent/CN102999939A/en
Application granted granted Critical
Publication of CN102999939B publication Critical patent/CN102999939B/en

Links

Classifications

    • GPHYSICS
    • G01MEASURING; TESTING
    • G01CMEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C11/00Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
    • G01C11/04Interpretation of pictures
    • G01C11/06Interpretation of pictures by comparison of two or more pictures of the same area
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K9/00Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
    • G06K9/00335Recognising movements or behaviour, e.g. recognition of gestures, dynamic facial expressions; Lip-reading
    • G06K9/00355Recognition of hand or arm movements, e.g. recognition of deaf sign language
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/50Depth or shape recovery
    • G06T7/55Depth or shape recovery from multiple images
    • G06T7/593Depth or shape recovery from multiple images from stereo images
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20Image signal generators
    • H04N13/204Image signal generators using stereoscopic image cameras
    • H04N13/239Image signal generators using stereoscopic image cameras using two 2D image sensors having a relative position equal to or related to the interocular distance
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20Image signal generators
    • H04N13/204Image signal generators using stereoscopic image cameras
    • H04N13/254Image signal generators using stereoscopic image cameras in combination with electromagnetic radiation sources for illuminating objects
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10004Still image; Photographic image
    • G06T2207/10012Stereo images
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N2013/0074Stereoscopic image analysis
    • H04N2013/0081Depth or disparity estimation from stereoscopic image signals

Abstract

本申请公开了一种基于计算机视觉的实时三维目标重建系统和方法及所使用的三维坐标信息获取装置、以及一种立体交互设备。 The present application discloses a three-dimensional object reconstruction based on real-time system and method for computer vision and three-dimensional coordinate information acquiring apparatus used, and a three-dimensional interaction device. 其中,坐标获取装置包括:图像采集模块,用于通过红外双目摄像机的第一摄像头和第二摄像头对同一目标物进行拍摄,分别得到第一图像和第二图像;区域选择模块,用于基于图像灰度特征在第一图像中确定出感兴趣区域;边缘点提取模块,用于对感兴趣区域进行边缘点提取;深度计算模块,用于在第二图像中,搜索与第一图像的边缘点对应的匹配点,根据边缘点的坐标及匹配点的坐标,得到匹配点的三维坐标。 Wherein the coordinate acquiring apparatus comprising: an image acquisition module for photographing the same object by a first camera and a second camera binocular infrared camera, respectively the first and second images; area selection module, based on wherein determining a first gray-scale image in an image region of interest; edge point extraction means for the edge region of interest point extraction; depth calculation means for edge in the second image, the first image search matching point corresponding to the point, according to the matching point coordinates and edge points, to obtain three-dimensional coordinates of the matching point. 本申请提供的方案适用于近距离三维目标重建。 The application provides solutions suitable for short-range three-dimensional object reconstruction.

Description

坐标获取装置、实时三维重建系统和方法、立体交互设备 Coordinate acquiring device, system and method for real-time three-dimensional reconstruction, stereo interaction device

技术领域 FIELD

[0001] 本申请涉及计算机视觉技术领域,尤其涉及一种基于计算机视觉的实时三维目标重建系统和方法及所使用的三维坐标信息获取装置、以及一种立体交互设备。 [0001] The present application relates to the field of computer vision technology, particularly to a three-dimensional object reconstruction based on real-time system and method for computer vision and three-dimensional coordinate information acquiring apparatus used, and a three-dimensional interaction device.

背景技术 Background technique

[0002] 随着人们对智能化人机交互领域的需求越来越高,如何使机器理解人成为一个亟待解决的问题。 [0002] As the demand for intelligent human-computer interaction field of higher and higher, people understand how to make the machine becomes a problem to be solved. 研究表明,如果能够借助各种传感设备实时提取人的行为的三维(3D)信息并设定某些规则使机器理解这些信息,将是解决这个问题的关键环节,其中3D目标重建是实现该关键环节的基础。 Research shows that if we can extract with a variety of sensing devices in real-time three-dimensional human behavior (3D) information and set some rules to make the machine understand this information will be the key to solving this problem, which is to achieve the 3D reconstruction goals the basis of the key. 3D目标重建是指在一个有限的3D视场中,提取目标在空间直角坐标系的X方向、Y方向以及Z方向的位置信息,据此实现3D重建。 3D reconstruction refers to a certain limited field of view in 3D, extracting target position information in the X-direction spatial rectangular coordinate system, Y and Z directions, whereby to achieve a 3D reconstruction.

发明内容 SUMMARY

[0003] 根据本申请的第一方面,提供一种用于实时三维目标重建的坐标获取装置,包括: 图像采集模块,用于通过红外双目摄像机的第一摄像头和第二摄像头对同一目标物进行拍摄,分别得到第一图像和第二图像;区域选择模块,用于基于图像灰度特征在第一图像中确定出感兴趣区域;边缘点提取模块,用于对感兴趣区域进行边缘点提取;深度计算模块,用于在第二图像中,搜索与第一图像的边缘点对应的匹配点,根据边缘点的坐标及匹配点的坐标,得到匹配点的三维坐标。 [0003] According to a first aspect of the present disclosure, there is provided a three-dimensional object coordinates of the real-time reconstruction means for acquiring, comprising: an image acquisition module, through a first camera and a second camera, an infrared camera binocular same object shooting, to obtain a first image and the second image; area selection module, based on the image gradation characteristic determined in the first image region of interest; edge point extraction means for the edge region of interest point extraction ; depth calculation module configured in the second image, the edge point search of the first image corresponding matching point, according to the matching point coordinates and edge points, to obtain three-dimensional coordinates of the matching point.

[0004] 根据本申请的第二方面,提供一种使用上述坐标获取装置的实时三维目标重建系统,其还包括目标重构模块,用于在第一校准图像中,根据边缘点进行插值得到插值点, 在第二校准图像中搜索与插值点对应的匹配插值点,根据插值点的坐标及匹配插值点的坐标,结合双目摄像机的焦距及两摄像头中心点的距离进行计算,得到匹配插值点的三维坐标,根据匹配插值点的三维坐标实现目标物的三维重建。 [0004] According to a second aspect of the application, to provide real-time three-dimensional object reconstruction system using the coordinate acquisition device, which further includes a target reconstruction module for the first calibration image, by interpolating an interpolation based on the edge points obtained point, the second calibration image search corresponding to the interpolation point matching interpolation point, based on the coordinates match the coordinates of interpolation points and interpolation points, with the focal length of the binocular camera from the camera and the center point of the two calculated matching interpolation points obtained three-dimensional coordinates, three-dimensional reconstruction of the object in the three-dimensional coordinate matching interpolation point.

[0005] 根据本申请的第三方面,提供一种使用上述坐标获取装置或三维目标重建系统的立体交互设备。 [0005] According to a third aspect of the present disclosure, there is provided a stereoscopic interactive device using the coordinate acquisition device or the three-dimensional object reconstruction system.

[0006] 根据本申请的第四方面,提供一种实时三维目标重建方法,包括:图像采集步骤, 通过红外双目摄像机的第一摄像头和第二摄像头对同一目标物进行拍摄,分别得到第一图像和第二图像,所述第一图像包括第一红外图像和第一红外增强图像,所述第二图像为第二红外增强图像;区域选择步骤,针对第一红外增强图像中的每个像素点以及与该像素点对应的第一红外图像中的每个像素点,计算二者的灰度值的比值或灰度值的差值,比值大于第一预设阈值或差值大于第二预设阈值的像素点构成感兴趣区域;边缘点提取步骤,对感兴趣区域进行边缘点提取;深度计算步骤,在第二图像中,搜索与第一图像的边缘点对应的匹配点,根据边缘点的坐标及匹配点的坐标,结合双目摄像机的焦距及两摄像头中心点的距离进行计算,得到匹配点的三维坐标;目 [0006] According to a fourth aspect of the present disclosure, there is provided a real-time three-dimensional object reconstruction method, comprising: an image acquisition step, the same object is photographed by a first camera and a second camera binocular infrared camera, respectively the first image and the second image, the first image comprising a first image and a first infrared enhanced infrared image, the second image is a second infrared image enhancement; region selection step, each pixel in the enhanced image for the first infrared points, and each pixel corresponding to the first pixel of the infrared image, the difference between both values ​​is calculated gradation or gray scale values ​​of the ratio, the ratio is greater than a first predetermined threshold value or the difference is greater than a second pre- pixels constituting the set threshold region of interest; edge point extraction step, a region of interest of the edge point extraction; depth calculation step, in the second image, the edge point search of the first image corresponding matching point, based on the edge points coordinates and the coordinates of the matching points, in conjunction with the binocular camera focal distance of the camera and the center point of two is calculated, to obtain three-dimensional coordinates of the matching points; mesh 重构步骤,在第一图像中,根据边缘点进行插值得到插值点,在第二图像中搜索与插值点对应的匹配插值点,根据插值点的坐标及匹配插值点的坐标,结合双目摄像机的焦距及两摄像头中心点的距离进行计算,得到匹配插值点的三维坐标,根据匹配插值点的三维坐标实现目标物的三维重建。 Reconstruction step, in the first image, based on the edge points obtained by interpolating interpolation points, in the second image corresponding to the interpolation point searching matching interpolation point, based on the coordinates match the coordinates of interpolation points and interpolation points, in conjunction with the binocular camera focal distance of the camera and the center point of two calculated three-dimensional coordinates to obtain interpolation point matching, three-dimensional reconstruction of the object in the three-dimensional coordinate matching interpolation point.

[0007] 本申请的有益效果是:通过从第一图像中提取边缘点,在第二图像中找出该边缘点的匹配点,结合双目摄像机的焦距及两摄像头中心点的距离进行计算,从而得到匹配点的空间三维坐标,这种方案适用于近距离三维目标重建。 [0007] The beneficial effect of the present application are: extract the edge points from a first image, find the matching point of the edge points in the second image, the binding binocular camera focal distance of the camera and the center point of two is calculated, whereby the three-dimensional spatial coordinates of the matching point, this embodiment is suitable for short-range three-dimensional object reconstruction.

[0008] -种实施例中,结合红外光源的辅助可得到具有稳定亮度的图像,使得后续计算的准确度增加,区域选取时只采用简单的灰度特征,加快了处理速度,仅以选取出的感兴趣区域的边缘点进行计算,达到了减少计算量且又提高速度的目的。 [0008] - the accuracy of the kind of embodiment, in conjunction with an auxiliary infrared light source having a stable brightness of the image obtained, so that an increase in subsequent calculations, only a simple gradation characteristic when the region selection, faster processing, only the selected edge point of the region of interest is calculated to achieve the purpose of reducing the amount of calculation and because of the speed increase. 进一步地,通过腐蚀处理消除区域边界处的不可靠点,从而达到提高精度的目的。 Further, by etching treatment to eliminate the unreliable point at the boundary region, so as to improve accuracy.

附图说明 BRIEF DESCRIPTION

[0009] 图1为本申请一种实施例的坐标获取装置的结构示意图; [0009] Figure 1 is an embodiment of the application of a coordinate acquiring a schematic structure of the apparatus;

[0010] 图2为本申请一种实施例的深度计算方法的流程示意图; [0010] FIG. 2 is a diagram of a process application depth calculation method of the embodiment;

[0011] 图3为本申请一种实施例的双目摄像机的结构示意图; [0011] FIG. 3 is a schematic structural diagram of a binocular camera application example of an embodiment;

[0012] 图4为本申请另一种实施例的坐标获取装置的结构示意图; [0012] FIG. 4 is a schematic structural diagram of another embodiment of the coordinate acquisition means to an embodiment of the present application;

[0013] 图5为本申请一种实施例的校准和区域选择处理的结构示意图; [0013] FIG. 5 and the schematic structural diagram of the calibration area selecting one embodiment of the present application;

[0014] 图6为本申请另一种实施例的坐标获取装置的结构示意图; [0014] FIG. 6 is a schematic structural diagram of another embodiment of the coordinate acquisition means to an embodiment of the present application;

[0015] 图7为本申请另一种实施例的坐标获取装置的结构示意图; [0015] FIG. 7 another embodiment of the present application coordinates schematic configuration acquiring means;

[0016] 图8为本申请一种实施例的坐标获取装置的一种应用示意图。 [0016] Figure 8 embodiment of a coordinate diagram of an apparatus embodiment is an application of the present application acquired.

具体实施方式 Detailed ways

[0017] 下面通过具体实施方式结合附图对本发明作进一步详细说明。 [0017] The following figures present invention will be further described in detail by specific embodiments in combination.

[0018] 实施例1 [0018] Example 1

[0019] 由于目标在空间直角坐标系的X方向和Y方向上的实时坐标可以通过任意一个摄像头拍摄到的数据得到,而Z方向的坐标信息无法直接得到,所以如何计算Z方向的实时坐标是实现三维目标重建的关键点之一。 [0019] Since the target real-time coordinate in the X and Y directions spatial rectangular coordinate system may be captured by any camera data obtained, the coordinate information in the Z direction can not be obtained directly, so the real-time coordinate how to calculate the Z direction is one key point to achieve three-dimensional object reconstruction.

[0020] 本申请的一种用于实时三维目标重建的坐标获取装置可参考图1,包括:图像采集模块101、区域选择模块103、边缘点提取模块105和深度计算模块107。 [0020] The present application is a real-time three-dimensional object coordinate acquisition means for reconstructing refer to Figure 1, comprising: image acquisition module 101, selection module 103 region, the edge point extraction module 105 and a depth calculating module 107.

[0021] 图像采集模块101用于通过双目摄像机的第一摄像头和第二摄像头对同一目标物进行拍摄,分别得到第一图像和第二图像。 [0021] The image acquisition module 101 for the same object is photographed by a first camera and a second camera binocular camera, respectively the first and second images. 双目摄像机可采用常规的双目摄像机。 Binocular cameras using conventional binocular cameras. 区域选择模块103用于基于图像灰度特征在第一图像中确定出感兴趣区域。 Area selection module 103 based on the image gradation characteristic determined in the first image region of interest. 边缘点提取模块105 用于对感兴趣区域进行边缘点提取。 Edge point extraction module 105 region of interest to extract edge points. 深度计算模块107用于在第二图像中搜索与第一图像的边缘点对应的匹配点,根据边缘点的坐标及匹配点的坐标,结合双目摄像机的焦距及两摄像头中心点的距离,根据图像间立体视差反比于测量深度进行计算,得到匹配点的三维坐标。 Depth calculation module 107 is used in the second image matching the search edge point corresponding to the first image, and the matching point according to the coordinates of the edge points from the combined focal length of the camera and binocular camera center point of the two, according to the stereo disparity between images is inversely proportional to the measurement depth is calculated to obtain three-dimensional coordinates of the matching point.

[0022] 这里深度计算的方法可采用基于区域的方法进行计算,这样可以在保证精度的情况下提高速度。 [0022] The method herein may be employed to calculate the depth calculation region-based method, which can ensure the accuracy of the speed increase case. 如图2所示,最上方的虚线框内为待拍摄的目标物,中间为双目摄像机(含摄像头1和摄像头2),最下方的左边图为摄像头1拍摄得到的图像L,右边图为摄像头2拍摄得到的图像R,其中T为摄像头1和摄像头2的中心点距离,f为摄像头焦距,山和d2分别为两个摄像头拍摄到的图像在X方向上的偏移距离。 As illustrated, the uppermost dotted frame of the object to be photographed, the intermediate binocular camera (including a camera 1 and camera 2), the left picture shows the bottom of a camera 2 photographed image L, the right picture shows 2 the camera image captured by R, where T is the distance from the center point of the camera 1 and camera 2, f is the focal length of the camera, and d2 are the mountain captured by two cameras to image shift distance in the X direction. 在图像L中以提取的边缘点为中心Pc,选取m*m(m为像素点的数目)大小的扫描区域,扫描图像R中以与Pd立置相同的像素点为中心的相同大小(即m*m)的区域,对图像L和图像R中的相应的扫描区域中像素灰度值差的绝对值求和,所求和的值最小的中心点位置就是图像L中边缘点在图像R中的匹配点的相对位置,由此可以得到d2的值,利用如下公式: In the image L to the extracted edge point as the center Pc, Xuanqu m * m (m is the number of pixels) of area to be scanned, the scanned image R to to Pd Li facing the same pixel as the center of the same size (i.e. m * m) region, summing the absolute values ​​of the respective scan region images L and R in the pixel gray value difference, the sum of the minimum values ​​is the center point position of the image edge points in the image L R the relative position of the matching points, whereby the value d2 is obtained by the following formula:

[0023] [0023]

Figure CN102999939BD00061

[0024] 可以得到该匹配点在Z方向上的坐标,至此,目标物在X方向、Y方向以及Z方向的坐标全部确定。 [0024] The coordinates of the matching point can be obtained in the Z-direction, thus, an object in the X direction, Y direction and Z direction coordinate of all determined.

Figure CN102999939BD00062

[0025] 基于该坐标获取装置可实现一种实时三维目标重建系统,即该系统包括该坐标获取装置以及目标重构模块,其中,目标重构用于在第一校准图像中,根据边缘点进行插值得到插值点,在第二校准图像中搜索与插值点对应的匹配插值点,根据插值点的坐标及匹配插值点的坐标,结合双目摄像机的焦距及两摄像头中心点的距离进行计算,得到匹配插值点的三维坐标,根据匹配插值点的三维坐标实现目标物的三维重建。 [0025] The device may implement a real-time three-dimensional reconstruction of the target system, i.e. the system includes the acquisition device and the target coordinate reconstruction module, wherein the first calibration target for reconstructed image, according to the coordinates of the edge points acquired based interpolation points interpolated in the second calibration image search corresponding to the interpolation point matching interpolation point, based on the coordinates and the coordinates of the matching interpolation point interpolation points, with the focal length of the binocular camera from the camera and the center point of two is calculated, to obtain match three-dimensional coordinates of interpolation points, three-dimensional reconstruction of the object in the three-dimensional coordinate matching interpolation point. 其中,搜索匹配插值点的方法可采用如前述的基于区域的方法进行计算,插值方式可采用线性插值,即针对感兴趣区域内的非边缘点Pa,可以通过在同一行上左右两个方向搜索最近的边缘点,记为aL和aR,通讨公式, Wherein the matching search method may be employed as the interpolation point is calculated based on the area, the interpolation linear interpolation methods can be used, i.e., for non-edge points in the region of interest Pa, by searching on the same line in the left and right directions the nearest edge point, denoted by aL and aR, through discussions formula,

[0026] [0026]

[0027] 兴1r·X、·y、·z刀、力u衣小τ仍、扮、即1」、外、同/艾刀问的值,根据该公式计算可得到该非边缘点的插值深度值。 [0027] Xing 1r · X, · y, · z knives, force τ u clothing still small, play, i.e. 1 ", the same / Ai knife Q value, the interpolation calculation of the non-edge points obtained based on the formula depth value.

[0028]由于在近距离拍摄中绝大多数情况下得到的内部区域满足平滑条件,因此这样的插值理论是比较精确的,这里平滑条件是指感兴趣区域内部总体是近似一个二维平面,没有复杂的褶皱;近距离是根据计算机理决定的,例如几十厘米等。 [0028] Since the close-ups in the interior region obtained in most cases provide smooth conditions, this theory is more accurate interpolation, smoothing condition here refers to the internal region of interest generally is approximately a two-dimensional plane, no complex folds; close processing computer is determined, for example, several tens of centimeters and the like.

[0029] 本申请还提供了一种立体交互设备,其包括本实施例的坐标获取装置或实时三维目标重建系统。 [0029] The present application also provides an interactive three-dimensional apparatus, which comprises a target acquisition device or real-time reconstruction of a three-dimensional coordinate system embodiment of the present embodiment.

[0030] 本实施例通过从第一图像中提取边缘点,在第二图像中找出该边缘点的匹配点, 结合双目摄像机的焦距及两摄像头中心点的距离进行计算,从而得到匹配点的空间三维坐标,这种方案适用于近距离三维目标重建。 [0030] In this embodiment, the image extracted from the first edge points, the edge point find a match point in the second image, the binding binocular camera focal distance of the camera and the center point of two is calculated to obtain a matching point the three-dimensional coordinate space, this program is suitable for short-range three-dimensional object reconstruction.

[0031] 基于本实施例的坐标获取装置,本申请还提供一种实时三维目标重建方法,包括如下步骤: [0031] Gets the present embodiment based on the coordinate device, the present application also provides a real-time three-dimensional object reconstruction method, comprising the steps of:

[0032]图像采集步骤,通过双目摄像机的第一摄像头和第二摄像头对同一目标物进行拍摄,分别得到第一图像和第二图像; [0032] The image acquisition step, the same object is photographed by a first camera and a second camera binocular camera, respectively the first and second images;

[0033] 区域选择步骤,基于图像灰度特征在第一图像中确定出感兴趣区域; [0033] The region selecting step, based on the image gradation characteristic determined in the first image region of interest;

[0034] 边缘点提取步骤,对感兴趣区域进行边缘点提取; [0034] The edge point extraction step, a region of interest of the edge point extraction;

[0035] 深度计算步骤,用于在第二图像中,搜索与第一图像的边缘点对应的匹配点,根据边缘点的坐标及匹配点的坐标,得到匹配点的三维坐标。 [0035] The depth calculation step of in the second image, the edge point search of the first image corresponding matching point, according to the matching point coordinates and edge points, to obtain three-dimensional coordinates of the matching point.

[0036]目标重构步骤,在第一校准图像中,根据边缘点进行插值得到插值点,在第二校准图像中搜索与插值点对应的匹配插值点,根据插值点的坐标及匹配插值点的坐标,结合双目摄像机的焦距及两摄像头中心点的距离进行计算,得到匹配插值点的三维坐标,根据匹配插值点的三维坐标实现目标物的三维重建。 [0036] The target reconstruction step, the first calibration image, interpolated to obtain interpolation points based on the edge points, match the search interpolation point in the interpolation point corresponding to the second calibration image, based on the coordinates and the matching interpolation point interpolation point the coordinates of the center point of the first distance, and the focal length of the camera in conjunction with the binocular imaging two calculated three-dimensional coordinates to obtain interpolation point matching, three-dimensional reconstruction of the object in the three-dimensional coordinate matching interpolation point.

[0037] 该方法的各步骤可采用本实施例的实时三维目标重建系统中涉及的对应部分实现,在此不作详述。 [0037] Real-time three-dimensional object reconstruction corresponding portions of the system may employ the steps of the method according to the present embodiment relates to the realization, not detailed herein.

[0038] 实施例2 [0038] Example 2

[0039] 本实施例的一种用于实时三维目标重建的坐标获取装置可参考图4,包括:图像采集模块401、校准模块402、区域选择模块403、边缘点提取模块405和深度计算模块407。 [0039] a real-time three-dimensional object coordinates for the reconstruction of the present embodiment may refer to FIG. 4 acquiring apparatus, comprising: image acquisition module 401, a calibration module 402, selection module 403 region, and the edge point extraction depth calculation module 405 module 407 .

[0040]图像采集模块401用于通过双目摄像机的第一摄像头和第二摄像头对同一目标物进行拍摄,分别得到第一图像和第二图像。 [0040] The image acquisition module 401 for the same object is photographed by a first camera and a second camera binocular camera, respectively the first and second images. 双目摄像机采用本申请提供的红外双目摄像机,如图3所示,该红外双目摄像机的结构包括两个普通摄像头(摄像头1和摄像头2)、红外光源、红外滤光片以及相应的控制电路。 Binocular binocular camera using infrared cameras provided herein, shown in Figure 3, the infrared camera binocular comprises two common cameras (camera 1 and camera 2), the infrared light source, the infrared filter and the corresponding control circuit. 图示中红外光源为红外LED,当然,也可以是其他类型的红外光源。 Infrared light source is an infrared illustrated in the LED, of course, may also be other types of infrared light. 红外光源设置在摄像头1和摄像头2的镜头外围,红外光源可以是均匀分布在该外围以达到最佳拍摄效果;红外滤光片位于镜头和摄像头的图像传感器之间;控制电路用于根据需要控制各摄像头的拍摄。 Infrared light source disposed on a camera lens and camera peripheral 2, the infrared light source can be evenly distributed in the peripheral in order to achieve the best results; infrared filter positioned between the image sensor and the lens of the camera; a control circuit necessary for controlling shooting each camera. 红外双目摄像机各元器件之间的连接关系及控制方式可采用本领域常用的技术实现,在此不作详述。 Binocular camera and an infrared connection relationship between the control components can be commonly used in the art to achieve the present technology, it is not described in detail herein.

[0041] 本实施例提供的红外双目摄像机相比普通双目摄像机的优势在于:普通双目摄像机在出现光线很亮或光线很暗时,图像的亮度极不稳定,成像结果会偏亮、偏暗或出现局部阴影,这对于后续主要依靠图像的灰度进行图像分析和运算即为不利,因为图像亮度发生变化,意味着后续分析和运算结果也可能发生变化,最终导致结果不准确。 [0041] IR binocular camera of the present embodiment is provided in that the common advantages compared binocular camera: normal binocular camera appears very bright light or dim light, the brightness of the image is very unstable, the imaging result will be biased on, appears dim or partial shade, this is the image analysis and computation unfavorable, because a change in image brightness for grayscale images rely mainly on the follow-up, the mean follow-up analysis and calculation results may change, leading to inaccurate results. 而采用本申请的双目摄像机在红外光源开启与关闭状态下生成的红外增强图像和红外图像,利用红外增强图像和红外图像的灰度差抓取感兴趣区域,使得当外界光线强度出现频繁变化时,减小抓取感兴趣区域的误差。 The use of the binocular camera of the present application is generated at the infrared light source on and off state of the enhanced image and the infrared image IR, by infrared and infrared images enhanced gradation difference gripping region of interest, so that when the ambient light intensity changes occur frequently , the gripping error is reduced region of interest.

[0042]图像采集模块401在红外光源处于关闭状态时,利用摄像头1拍摄图像得图像I1QFF,在红外光源处于开启状态时分别利用摄像头1和摄像头2拍摄得到图像I1(]N和I胃,图像I1(]FF为红外图像,图像I1(]N和12(]N为红外增强图像。 Respectively, using the time [0042] The image acquisition module 401 when the infrared light source is turned off, using the camera captured image obtained image I1QFF, the infrared light source is in the ON state camera 1 to obtain images I1 and camera 2 photographed (] N and I stomach, image I1 (] FF is the image, image I1 (] N and 12 (] N is an infrared image enhancement.

[0043] 校准模块402用于基于双目摄像机的标定参数,对图像I1QFF、I1QN和I2QN进行校准并输出到区域选择模块403。 [0043] The calibration module 402 for binocular camera calibration parameters based on the image I1QFF, I1QN and I2QN calibration and outputs to the area selection module 403. 具体校准过程可采用已有的校准方法实现,即根据摄像头定标后获得的单目内参数据(焦距、成像原点、畸变系数)和双目相对位置关系(旋转矩阵和平移向量),分别对摄像头1和摄像头2拍摄的图像进行消除畸变和行对准,使得拍摄得到的图像的成像原点坐标一致、两摄像头光轴平行、左右成像平面共面、对极线行对齐。 DETAILED calibration process can be implemented prior calibration method, monocular internal reference data (focal length, the origin of the imaging, a distortion coefficient) that is scaled according to the camera and binocular obtained relative positional relationship (the rotation matrix and translation vector), each of the camera head 1 and the image captured by the camera 2 and to eliminate distortion rows aligned such that coordinates of the image origin of imaging obtained by imaging the same, two parallel to the optical camera, the imaging plane about coplanar alignment of epipolar lines.

[0044]区域选择模块403用于基于图像灰度特征在第一图像中确定出感兴趣区域。 [0044] The selection module 403 based on area gradation image area of ​​interest is determined that the feature in the first image. 本实施例中,区域选择模块包括计算选择单元,用于针对校准后的图像I1(]FF中的每个像素点以及与该像素点对应的校准后的图像中的每个像素点,计算二者的灰度值的比值或灰度值的差值,比值大于第一预设阈值或差值大于第二预设阈值的像素点构成感兴趣区域。第一预设阈值和第二预设阈值可根据实际实验数据确定,为经验值。易理解,灰度值的比值或差值与设备(即三维重建系统或立体交互设备)所处的环境有关,在光线较亮的环境下,该比值可很小以防止无法采集到感兴趣区域,在光线较暗的环境下,该比值可适当提高以防止采集到因红外光源影响变化的背景区域。 Embodiment, the region selection module selecting comprises calculating means for calibrated image I1 for (] FF in each pixel and each pixel of the image after calibration point corresponding to the pixel in the present embodiment, two computing difference in gray value or gray value by the ratio, the ratio is greater than a first predetermined threshold or difference of pixels greater than a predetermined threshold value constituting the second region of interest. the first predetermined threshold and the second predetermined threshold value may be determined based on actual experimental data, an empirical value. easy to understand, the ratio or difference with the device (i.e., a perspective or three-dimensional reconstruction system interaction device) gradation values ​​related to the environment, in the light of a bright environment, the ratio may be very small in order to prevent the region of interest can not be collected, in low light environments, the ratio may be appropriate to prevent the increase affected by the infrared light source to capture varying background area.

[0045] 本实施例中校准模块和区域选择模块的处理工作流程如图5所示,包括: [0045] In the present embodiment, the calibration module and a selection module of the processing areas shown in Figure 5 flow, comprising:

[0046] 关闭时拍摄步骤,红外光源处于关闭状态,利用摄像头1拍摄图像,并经校准后得到图像Ref; [0046] Step shooting off, the infrared light source is turned off, using the camera captured image and the image obtained after calibrated Ref;

[0047] 开启时拍摄步骤,红外光源处于开启状态,利用摄像头1和2拍摄图像,并经校准后得到图像L和图像R; When photographing step [0047] on, the infrared light source is turned on, an image captured using the camera 2 and 1, and after calibration to obtain images L and R & lt;

[0048] 灰度计算步骤,将图像L和图像Ref逐点遍历,计算图像L和图像Ref中相对应的每个像素点的灰度值的比值或差值,把比值超过第一预设阈值或差值超过第二预设阈值的像素点提取出来,从而得到感兴趣区域。 [0048] calculation step gradation, the images L and Ref points traversed by a ratio or difference of gradation values ​​of each pixel is calculated in the images L and Ref corresponding to the ratio exceeds a first predetermined threshold value pixels or difference exceeds a second predetermined threshold value is extracted, thereby obtaining the region of interest.

[0049] 边缘点提取模块405用于对感兴趣区域进行边缘点提取。 [0049] The edge point extraction module 405 region of interest for extracting the edge point. 边缘点提取可采用图像处理中常用的点提取算法。 Edge point extraction processing can be commonly used in the image point extraction algorithm. 一种简单的边缘点提取方法可通过逐点判断得到,即判断感兴趣区域中的点是否是边缘点就以该点为中心点,看该中心点的邻域(例如3*3邻域,即周围8个点)中是否存在某个点不在感兴趣区域内,如果有,则该中心点为边缘点。 A simple method of extracting the edge point-by-point determination can be obtained, that is, whether the region of interest point is a point on the edge to the point as the center point, the center of the neighborhood to see (e.g. 3 * 3 neighborhood, i.e. around 8 points) whether there is not a point of interest within the area, if there is, then the center point as an edge point. 之所以提取边缘点,是因为通常这些点在立体交互设备中代表了操作点,这样就可以免于计算整个感兴趣区域的所有像素点,从而减少计算量。 The reason why the extracted edge point, since these points usually represents a perspective view of the operating point of interaction device, so that all the pixels can be calculated from the entire region of interest, thereby reducing the amount of calculation.

[0050] 深度计算模块407采用如实施例1的深度计算模块,在此不做重述。 [0050] As the depth calculation module 407 uses the depth calculation module according to Embodiment 1, which is not repeated here.

[0051] 本实施例的区域选择采用的算法仅基于图像的灰度值,可使得处理速度加快。 Region [0051] embodiment of the present embodiment selection algorithm based on the gray value image, so that only the processing may be faster. 此外,为实现实时区域选择,可以使红外光源以一定频率闪烁,即一个闪烁流程周期要小于人眼可分辨的最小延迟。 Furthermore, for the real-time zone selection can be made an infrared light source flashes at a frequency, i.e., a blinking cycle of the process is less than the minimum delay resolution of the human eye.

[0052] 类似地,基于本实施例的坐标获取装置,还可提供一种实时三维目标重建系统,其包括该坐标获取装置以及目标重构模块,其中,目标重构的过程可参考实施例1,在此不做重述。 [0052] Similarly, the present embodiment is based on the coordinate acquisition means may also provide a real-time three-dimensional object reconstruction system, the coordinate acquiring device comprising a reconstruction module and a target, wherein the target process may be reconstituted Reference Example 1 , which is not repeated here. 本申请还提供了一种立体交互设备,其包括本实施例的坐标获取装置或实时三维目标重建系统。 The present application also provides an interactive three-dimensional apparatus, which comprises a target acquisition device or real-time reconstruction of a three-dimensional coordinate system embodiment of the present embodiment.

[0053] 基于本实施例的坐标获取装置,本申请还提供了一种实时三维目标重建方法,包括: [0053] Gets the coordinates based on the present embodiment apparatus, the present application also provides a real-time three-dimensional object reconstruction method, comprising:

[0054] 图像采集步骤,通过红外双目摄像机的第一摄像头和第二摄像头对同一目标物进行拍摄,分别得到第一图像和第二图像,所述第一图像包括第一红外图像和第一红外增强图像,所述第二图像为第二红外增强图像; [0054] The image acquisition step, performed by a first camera and a second camera infrared binocular camera shooting the same object, respectively the first and second images, the first image comprising a first image and the first infrared IR enhanced image, the second image is a second infrared image enhancement;

[0055] 校准步骤,基于双目摄像机的标定参数,对第一红外图像、第一红外增强图像和第二红外增强图像进行校准并输出以用于后续步骤。 [0055] The calibration step, calibration parameters based on binocular camera, a first infrared image, the first image and the second infrared IR blaster enhanced image and outputs the calibrated for a subsequent step.

[0056] 区域选择步骤,针对第一红外增强图像中的每个像素点以及与该像素点对应的第一红外图像中的每个像素点,计算二者的灰度值的比值或灰度值的差值,比值大于第一预设阈值或差值大于第二预设阈值的像素点构成感兴趣区域; [0056] The region selecting step, each pixel enhanced image and each pixel corresponding to the pixel in the first image for a first infrared infrared, or calculating a ratio of the gradation value of the gradation values ​​of both the difference ratio is greater than a first predetermined threshold or difference of pixels greater than a predetermined threshold value constituting the second region of interest;

[0057] 边缘点提取步骤,对感兴趣区域进行边缘点提取; [0057] The edge point extraction step, a region of interest of the edge point extraction;

[0058] 深度计算步骤,在第二红外增强图像中,搜索与第一红外增强图像的边缘点对应的匹配点,根据边缘点的坐标及匹配点的坐标,结合双目摄像机的焦距及两摄像头中心点的距离进行计算,得到匹配点的三维坐标; [0058] Depth calculating step, in a second infrared image enhancement, edge enhancement searches the first infrared matching point corresponding to the point image, and the matching point according to the coordinates of the edge points, with the focal length of the camera and two binocular camera calculating distance between the center, to obtain three-dimensional coordinates of the matching points;

[0059] 目标重构步骤,在第一红外增强图像中,根据边缘点进行插值得到插值点,在第二红外增强图像中搜索与插值点对应的匹配插值点,根据插值点的坐标及匹配插值点的坐标,结合双目摄像机的焦距及两摄像头中心点的距离进行计算,得到匹配插值点的三维坐标,根据匹配插值点的三维坐标实现目标物的三维重建。 [0059] The target reconstruction step, the first infrared image enhancement, based on the edge point interpolation points interpolated in the second infrared image enhancement searches matching interpolation points corresponding to the interpolation point, and the coordinates of an interpolation point of the interpolated matching the coordinate point, from the combined focal length of the camera and binocular camera center point of two calculated three-dimensional coordinates to obtain interpolation point matching, three-dimensional reconstruction of the object in the three-dimensional coordinate matching interpolation point.

[0060] 该方法的各步骤可采用本实施例的坐标获取装置中涉及的对应部分实现,在此不作详述。 [0060] Each step of the process according to the present embodiment may employ a corresponding portion of the coordinate acquisition device, implemented according to, this will not be described in detail.

[0061] 实施例3 [0061] Example 3

[0062] 本实施例的一种用于实时三维目标重建的坐标获取装置可参考图6,包括:图像采集模块501、区域选择模块503、校准模块504、边缘点提取模块505和深度计算模块507。 [0062] a real-time three-dimensional object coordinates for the reconstruction of the present embodiment may refer to acquiring means 6, comprising: an image acquisition module 501, an area selection module 503, a calibration module 504, the edge point extraction module 505 and a depth calculating module 507 . 图像采集模块401采用如实施例2的图像采集模块,在此不做重述。 The image acquisition module 401 using the image acquisition module as in Example 2, which is not repeated here. 区域选择模块503类似实施例2的区域选择模块,不同之处在于,此时是直接基于拍摄得到的图像进行处理,该图像尚未经过校准,处理后再通过校准模块504进行校准,区域选择模块503、校准模块504、 边缘点提取模块505和深度计算模块507的实现可参考实施例2中对应的模块,在此不做重述。 Area selection module 503 region selection module analogous to Example 2, except that, at this time are processed directly obtained based on the image pickup, the image has not been calibrated, then the processing is calibrated by calibration area selection module 503 module 504 achieve calibration module 504, the edge point extraction module 505 and a depth calculating module 507 may reference Example 2 corresponding module embodiment, which is not repeated here.

[0063] 类似地,基于本实施例的坐标获取装置,还可提供一种实时三维目标重建系统,其包括该坐标获取装置以及目标重构模块,其中,目标重构的过程可参考实施例1,在此不做重述。 [0063] Similarly, the present embodiment is based on the coordinate acquisition means may also provide a real-time three-dimensional object reconstruction system, the coordinate acquiring device comprising a reconstruction module and a target, wherein the target process may be reconstituted Reference Example 1 , which is not repeated here. 本申请还提供了一种立体交互设备,其包括本实施例的坐标获取装置或实时三维目标重建系统。 The present application also provides an interactive three-dimensional apparatus, which comprises a target acquisition device or real-time reconstruction of a three-dimensional coordinate system embodiment of the present embodiment.

[0064] 基于本实施例的坐标获取装置,本申请还提供了一种实时三维目标重建方法,包括:图像采集步骤、区域选择步骤、校准步骤、边缘点提取步骤、深度计算步骤和目标重构步骤。 [0064] The coordinate acquisition device according to the present embodiment based on the embodiment, the present application also provides a real-time three-dimensional object reconstruction method, comprising: an image acquisition step, the region selecting step, the calibration step, the edge point extraction step, and the step of calculating the target depth reconstruction step.

[0065] 该方法的各步骤可采用本实施例的实时三维目标重建系统中涉及的对应部分实现,在此不作详述。 [0065] Real-time three-dimensional object reconstruction corresponding portions of the system may employ the steps of the method according to the present embodiment relates to the realization, not detailed herein.

[0066] 实施例4 [0066] Example 4

[0067] 本实施例的一种用于实时三维目标重建的坐标获取装置如图7所示,包括:图像采集模块601、校准模块602、区域选择模块603、无效点去除模块604、边缘点提取模块605 和深度计算模块607,其中,图像采集模块601、校准模块602、区域选择模块603、边缘点提取模块605和深度计算模块607可采用实施例3或实施例4的对应模块实现,也就是说,本实施例的校准模块和区域选择模块的位置顺序可以是如图7所示先校准后区域选择,也可以是如实施例3的先区域选择后校准方式。 [0067] The present coordinates for reconstructing a real-time three-dimensional object acquisition apparatus embodiment shown in Figure 7, comprising: image acquisition module 601, a calibration module 602, selection module 603 region, removing the null point module 604, an edge point extraction depth calculation module 605 and module 607, wherein the image acquisition module 601, a calibration module 602, selection module 603 region, the edge point extraction module 605 and a depth calculating module 607 employed in Example 3 or Example 4 to the corresponding module implemented embodiment, i.e. He said calibration module of the present embodiment and the region selection module position order may be selected after the first calibration area shown in Figure 7, may be selected as in Example 3 after the first calibration area embodiment. 无效点去除模块604用于在提取边缘点之前对感兴趣区域进行形态学处理。 Null point removing module 604 for performing the morphological process on the region of interest prior to extracting edge points. 本实施例中采用腐蚀操作,目的是消除感兴趣区域边界处的不可靠点。 Employed in the present embodiment, the etching operation, the purpose of eliminating the unreliable point at the boundary region of interest. 腐蚀操作的次数与物体的厚度及与双目摄像机的距离有关,通常距离越大厚度越小,则腐蚀次数越小。 The thickness of the etching operation and the number of times the distance of the object about the binocular camera, generally the smaller the greater the distance in the thickness, the smaller the number of corrosion.

[0068] 如图8所示为采用本实施例的一种实例的示意图,左上方的图为红外光源关闭时拍摄得到的图像Ref,左下方的图为红外光源开启状态拍摄的图像L,中间图像中的"手"为确定出的感兴趣区域,最右图为腐蚀结果图。 [0068] As shown in the schematic diagram of an example of embodiment of the present embodiment, an image photographed Ref infrared light source off when the upper left picture shows, the lower left picture shows the infrared light source L turned on 8 image capturing state, the intermediate image "hand" is determined as the region of interest, the most right is the result of corrosion FIG.

[0069] 本领域技术人员可以理解,上述实施方式中各种方法的全部或部分步骤可以通过程序来指令相关硬件完成,该程序可以存储于一计算机可读存储介质中,存储介质可以包括:只读存储器、随机存储器、磁盘或光盘等。 [0069] Those skilled in the art will appreciate that the above described embodiments all or part of the various steps of the method may be completed by a program instructing relevant hardware, the program may be stored in a computer-readable storage medium, the storage medium may include: only read memory, random access memory, a magnetic disk or optical disk.

[0070] 本申请针对近距离应用条件下,结合红外LED的辅助,精确提取有效区域,去除无效点,并计算深度信息,从而达到实现实时3D目标重建的目的,实时是指可以动态提取目标的3D信息,并且其延迟时间小于人眼识别动态目标的最小延迟时间;同时还可利用这些3D信息结合实际应用场景下设定的操作规则来实现3D体感操控。 [0070] The present application is directed under conditions close applications, auxiliary binding infrared LED, accurate extraction effective area, removes invalid point, and calculates the depth information, so as to achieve the purpose of real-time 3D reconstruction of the target, means may be real-time dynamic extraction target 3D information, and the delay time is less than the human eye recognition dynamic target minimum delay; 3D information but can also use the actual application scenario at the set operating rules to achieve 3D somatosensory control.

[0071] 以上内容是结合具体的实施方式对本发明所作的进一步详细说明,不能认定本发明的具体实施只局限于这些说明。 [0071] The above is described in further detail with reference to specific embodiments of the present invention made by the present invention should not be considered limited to these specific embodiments described. 对于本发明所属技术领域的普通技术人员来说,在不脱离本发明构思的前提下,还可以做出若干简单推演或替换。 Those of ordinary skill in the art for the present invention, without departing from the spirit of the present invention, can make various simple deduction or replacement.

Claims (9)

1. 一种用于实时三维目标重建的坐标获取装置,其特征在于,包括: 图像采集模块,用于通过红外双目摄像机的第一摄像头对目标物进行拍摄,得到第一图像,通过所述红外双目摄像机的第二摄像头对所述目标物进行拍摄,得到第二图像; 区域选择模块,用于基于图像灰度特征在第一图像中确定出感兴趣区域; 边缘点提取模块,用于对感兴趣区域进行边缘点提取; 深度计算模块,用于在第二图像中,搜索与第一图像的边缘点对应的匹配点,根据边缘点的坐标及匹配点的坐标,结合双目摄像机的焦距及两摄像头中心点的距离进行计算,得到匹配点的三维坐标; 其中所述图像采集模块包括具第一摄像头和第二摄像头的双目摄像机、红外光源、红外滤光片和控制电路;所述红外光源设置在所述第一摄像头和第二摄像头的镜头外围;所述红外滤光片位 A real-time coordinates of the three-dimensional object reconstruction for acquiring apparatus comprising: image capture means for photographing the target object by a first infrared camera binocular camera, obtain the first image, through the a second binocular camera infrared camera shooting the object to obtain a second image; area selection module, based on the image gradation characteristic determined in the first image region of interest; edge point extraction means for the edge region of interest point extraction; depth calculation module configured in the second image, the edge point search of the first image corresponding matching point, according to the matching point coordinates and edge points, in conjunction with the binocular camera two focal distance of the camera and the center point is calculated to obtain three-dimensional coordinates of the matching points; wherein said binocular image acquisition module comprises a video camera having a first camera and a second camera, an infrared light source, the infrared filter and a control circuit; the said infrared light source disposed on the periphery of the lens of the first camera and the second camera; the bit infrared filter 镜头和摄像头的图像传感器之间;所述控制电路用于控制第一摄像头在红外光源为关闭状态时拍摄以得到第一红外图像,控制第一摄像头和第二摄像头在红外光源为开启状态时拍摄以得到第一红外增强图像和第二红外增强图像,所述第一红外图像和所述第一增强图像为所述第一图像,所述第二红外增强图像为所述第二图像。 Between the image sensor and a camera lens; a control circuit for controlling the first camera is an infrared light source in the closed state imaging to obtain a first infrared image, the control of the first camera and the second camera capturing the infrared light source is in the ON state to obtain a first image and the second infrared IR blaster enhanced image, the infrared image and the first image is a first reinforcing the first image, the second image is a second infrared image enhancement.
2. 如权利要求1所述的坐标获取装置,其特征在于, 所述区域选择模块包括:计算选择单元,用于针对第一红外增强图像中的每个像素点以及与该像素点对应的第一红外图像中的每个像素点,计算二者的灰度值的比值或灰度值的差值,比值大于第一预设阈值或差值大于第二预设阈值的像素点构成感兴趣区域; 所述坐标获取装置还包括:校准模块,用于基于双目摄像机的标定参数,对所述图像采集模块得到的第二红外增强图像和由所述计算选择单元选出的感兴趣区域的第一红外增强图像进行校准并输出; 所述深度计算模块中的第二图像为校准后的第二红外增强图像。 2. The coordinates of the harvesting device of claim 1, wherein said area selection module comprising: selecting calculation means for enhancement for each pixel in the first infrared image and the corresponding pixel of each pixel in an infrared image, calculating the difference or ratio of the gradation value of the gradation values ​​of both, the ratio is greater than a first predetermined threshold or difference of pixels greater than a predetermined threshold value constituting the second region of interest ; the coordinate acquiring device further comprising: a calibration module for calibrating the parameter based on binocular camera, said second infrared image acquisition module and the enhanced image obtained by the first calculation means selecting the selected region of interest a calibrated infrared and outputting the enhanced image; a second image of the depth calculation module is calibrated a second infrared-enhanced image.
3. 如权利要求1所述的坐标获取装置,其特征在于, 所述坐标获取装置还包括:校准模块,用于基于双目摄像机的标定参数,对第一红外图像、第一红外增强图像和第二红外增强图像进行校准并输出; 所述深度计算模块中的第二图像为校准后的第二红外增强图像; 所述区域选择模块包括:计算选择单元,用于针对所述校准模块校准后的第一红外增强图像中的每个像素点以及与该像素点对应的校准后的第一红外图像中的每个像素点,计算二者的灰度值的比值或灰度值的差值,比值大于第一预设阈值或差值大于第二预设阈值的像素点构成感兴趣区域。 3. The coordinates of the acquisition device of claim 1, wherein the coordinate obtaining device further comprising: a calibration module for binocular camera calibration parameters based on the first infrared image, the first image and the IR blaster a second infrared calibration and outputs the enhanced image; a second image of the depth calculation module is calibrated a second infrared image enhancement; the region selection module comprising: calculating means selecting, for said calibration module for calibrating the a first infrared reinforcing each pixel image and each pixel of the first IR image after calibration corresponding to the pixel points, calculating the difference or ratio of the gradation value of the gradation values ​​of both, ratio is greater than a first predetermined threshold or difference of pixels greater than a predetermined threshold value constituting the second region of interest.
4. 如权利要求1-3任一项所述的坐标获取装置,其特征在于,还包括:无效点去除模块,用于在提取边缘点之前,对感兴趣区域进行腐蚀处理。 4. The coordinates of any one of claims 1-3 acquisition apparatus characterized by further comprising: a null point removal means for extracting edge points before, etching a region of interest.
5. 如权利要求1所述的坐标获取装置,其特征在于,所述深度计算模块包括: 搜索单元,用于在第一图像中以提取的每个边缘点为中心点选定预定大小的扫描区域,在第二图像中以与每个边缘点的中心点的坐标位置对应的像素点选取与预定大小相同的扫描区域,计算这两个扫描区域内各相应的像素点的灰度值之差的绝对值之和,与所求和的最小值对应的中心点即是第二图像中与第一图像的边缘点对应的匹配点; 坐标计算单元,根据边缘点的坐标及匹配点的坐标,结合双目摄像机的焦距及两摄像头中心点的距离,根据图像间立体视差反比于测量深度进行计算,得到匹配点的三维坐标。 5. The coordinates of the acquisition device of claim 1, wherein said depth computing module comprising: a search unit for scanning in the first image to each of the extracted edge point is the center point of the selected predetermined size region, the position coordinates of each edge point and the center point of the corresponding pixel in the second image of the same selected scanning area of ​​a predetermined size, calculates the difference between the grayscale values ​​corresponding to each pixel in the two scanning region sum of absolute values, and the minimum value of the sum corresponding to the center point, i.e., the second image is an edge point and the matching point corresponding to the first image; coordinate calculation unit, based on the coordinates and the coordinates of the matching point of the edge points, binding binocular camera focal length and the distance of the center point of the two cameras, a stereoscopic image according to the disparity between the measurements is inversely proportional to depth is calculated to obtain three-dimensional coordinates of the matching point.
6. -种实时三维目标重建系统,其特征在于,包括如权利要求1-5任一项的坐标获取装置,还包括:目标重构模块,用于在第一校准图像中,根据边缘点进行插值得到插值点, 在第二校准图像中搜索与插值点对应的匹配插值点,根据插值点的坐标及匹配插值点的坐标,结合双目摄像机的焦距及两摄像头中心点的距离进行计算,得到匹配插值点的三维坐标,根据匹配插值点的三维坐标实现目标物的三维重建。 6. - kind of real-time three-dimensional object reconstruction system, characterized in that the coordinates of any one of claims 1-5 comprising as claimed in claim acquisition apparatus further comprising: a first calibration image, based on the edge point target for reconstructing module, for interpolation points interpolated in the second calibration image search corresponding to the interpolation point matching interpolation point, based on the coordinates and the coordinates of the matching interpolation point interpolation points, with the focal length of the binocular camera from the camera and the center point of two is calculated, to obtain match three-dimensional coordinates of interpolation points, three-dimensional reconstruction of the object in the three-dimensional coordinate matching interpolation point.
7. -种立体交互设备,其特征在于,包括如权利要求1-5任一项所述的坐标获取装置或如权利要求6所述的实时三维目标重建系统。 7. - stereochemical interaction device, characterized by comprising a coordinate according to any one of claims 1-5 acquisition device or real-time three-dimensional object as claimed in claim 6, the reconstruction system.
8. -种实时三维目标重建方法,其特征在于,包括: 图像采集步骤,通过红外双目摄像机的第一摄像头和第二摄像头对同一目标物进行拍摄,分别得到第一图像和第二图像,所述第一图像包括第一红外图像和第一红外增强图像, 所述第二图像为第二红外增强图像; 区域选择步骤,针对第一红外增强图像中的每个像素点以及与该像素点对应的第一红外图像中的每个像素点,计算二者的灰度值的比值或灰度值的差值,比值大于第一预设阈值或差值大于第二预设阈值的像素点构成感兴趣区域; 边缘点提取步骤,对感兴趣区域进行边缘点提取; 深度计算步骤,在第二图像中,搜索与第一图像的边缘点对应的匹配点,根据边缘点的坐标及匹配点的坐标,结合双目摄像机的焦距及两摄像头中心点的距离进行计算,得到匹配点的三维坐标; 目标重构步骤,在第 8. - kind of real-time three-dimensional object reconstruction method, characterized by comprising: an image acquisition step, the same object is photographed by a first camera and a second camera binocular infrared camera, respectively the first and second images, the first image including a first image and a first infrared enhanced infrared image, the second image is a second infrared image enhancement; region selection step, for each pixel in the first infrared enhanced image and the pixel each pixel corresponding to a first infrared image, calculating a ratio of the difference between both the gray value or gray value of pixel ratio is greater than a first predetermined threshold or greater than a second preset threshold difference constituting region of interest; edge point extraction step, a region of interest of the edge point extraction; depth calculation step, in the second image, the edge point search image corresponding to the first match point, and based on the coordinates of the matching edge point coordinates, distances and the focal length of the camera in conjunction with two binocular camera center point is calculated to obtain three-dimensional coordinates of the matching points; target reconstruction step, the first 一图像中,根据边缘点进行插值得到插值点,在第二图像中搜索与插值点对应的匹配插值点,根据插值点的坐标及匹配插值点的坐标,结合双目摄像机的焦距及两摄像头中心点的距离进行计算,得到匹配插值点的三维坐标,根据匹配插值点的三维坐标实现目标物的三维重建。 In an image, based on the edge points obtained by interpolating interpolation points, in the second image corresponding to the interpolation point searching matching interpolation point, based on the coordinates match the coordinates of interpolation points and interpolation points, with the focal length of the camera and two binocular camera center distance of a point calculated to obtain three-dimensional coordinates of the interpolation point matching, three-dimensional reconstruction of the object in the three-dimensional coordinate matching interpolation point.
9. 如权利要求8所述的实时三维目标重建方法,其特征在于,还包括: 无效点去除步骤,在提取边缘点之前对感兴趣区域进行腐蚀处理; 校准步骤,执行于区域选择步骤之前或之后,基于双目摄像机的标定参数,对第一红外图像、第一红外增强图像和第二红外增强图像进行校准并输出。 9. Real-time three-dimensional object reconstruction method according to claim 8, characterized in that, further comprising: a null point removal step, etching the region of interest before the extraction of the edge points; calibration step performed prior to the step of selecting a region or Thereafter, the calibration parameters based on binocular camera, a first infrared image, the first image and the second infrared enhanced infrared calibration and outputs the enhanced image.
CN201210353512.7A 2012-09-21 2012-09-21 Coordinate acquiring device, system and method for real-time three-dimensional reconstruction, stereo interaction device CN102999939B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201210353512.7A CN102999939B (en) 2012-09-21 2012-09-21 Coordinate acquiring device, system and method for real-time three-dimensional reconstruction, stereo interaction device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201210353512.7A CN102999939B (en) 2012-09-21 2012-09-21 Coordinate acquiring device, system and method for real-time three-dimensional reconstruction, stereo interaction device
PCT/CN2013/083092 WO2014044126A1 (en) 2012-09-21 2013-09-09 Coordinate acquisition device, system and method for real-time 3d reconstruction, and stereoscopic interactive device

Publications (2)

Publication Number Publication Date
CN102999939A CN102999939A (en) 2013-03-27
CN102999939B true CN102999939B (en) 2016-02-17

Family

ID=47928467

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201210353512.7A CN102999939B (en) 2012-09-21 2012-09-21 Coordinate acquiring device, system and method for real-time three-dimensional reconstruction, stereo interaction device

Country Status (2)

Country Link
CN (1) CN102999939B (en)
WO (1) WO2014044126A1 (en)

Families Citing this family (27)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102999939B (en) * 2012-09-21 2016-02-17 魏益群 Coordinate acquiring device, system and method for real-time three-dimensional reconstruction, stereo interaction device
CN103236083B (en) * 2013-05-06 2016-06-22 北京建筑工程学院 Real-time three-dimensional measurement method based on stereoscopic video library
CN104424630A (en) * 2013-08-20 2015-03-18 华为技术有限公司 Three-dimension reconstruction method and device, and mobile terminal
CN104714840B (en) * 2013-12-13 2019-01-15 联想(北京)有限公司 A kind of information processing method and electronic equipment
CN105528599B (en) * 2014-09-30 2019-07-26 联想(北京)有限公司 Handle the method and terminal device of image
CN108924428A (en) * 2014-09-30 2018-11-30 华为技术有限公司 Auto-focus method and device, and electronic equipment
CN105528060B (en) * 2014-09-30 2018-11-09 联想(北京)有限公司 A terminal device and a control method
CN105491307B (en) * 2014-10-13 2019-06-25 联想(北京)有限公司 Depth sensing system
CN104709696B (en) * 2014-12-31 2018-03-16 深圳市中智科创机器人有限公司 Placing pallet stacker / cargo extraction method, apparatus and system
CN106293012A (en) * 2015-05-28 2017-01-04 深圳柔石科技有限公司 System and method for three-dimensional somatosensory two-way interactions
CN105082860A (en) * 2015-07-10 2015-11-25 青岛亿辰电子科技有限公司 Rapid manufacturing method for 3D mini-portrait
CN105277169B (en) * 2015-09-25 2017-12-22 安霸半导体技术(上海)有限公司 Binocular ranging method based on image segmentation
CN105430501B (en) * 2015-12-03 2019-06-04 青岛海信电器股份有限公司 A kind of volume adjusting method and system
CN105592367A (en) * 2015-12-23 2016-05-18 青岛海信电器股份有限公司 Image display parameter adjusting method and system
CN105872516A (en) * 2015-12-28 2016-08-17 乐视致新电子科技(天津)有限公司 Method and device for obtaining parallax parameters of three-dimensional film source
CN105704472A (en) * 2016-01-13 2016-06-22 青岛海信电器股份有限公司 Television control method capable of identifying child user and system thereof
CN105681861A (en) * 2016-03-04 2016-06-15 青岛海信电器股份有限公司 Adjusting method and system for display subtitle of terminal
CN105946718B (en) * 2016-06-08 2019-04-05 深圳芯智汇科技有限公司 The method of car-mounted terminal and its switching display reverse image
CN106170086B (en) * 2016-08-19 2019-03-15 深圳奥比中光科技有限公司 Method and device thereof, the system of drawing three-dimensional image
CN106384382A (en) * 2016-09-05 2017-02-08 山东省科学院海洋仪器仪表研究所 Three-dimensional reconstruction system and method based on binocular stereoscopic vision
CN106530389B (en) * 2016-09-23 2019-04-05 西安电子科技大学 Stereo reconstruction method based on medium-wave infrared facial image
CN107041729A (en) * 2016-12-30 2017-08-15 西安中科微光影像技术有限公司 Binocular near-infrared imaging system and blood vessel recognition method
CN106931906A (en) * 2017-03-03 2017-07-07 浙江理工大学 Object 3D size simple measurement method based on binocular stereo vision
CN106773509B (en) * 2017-03-28 2019-07-09 成都通甲优博科技有限责任公司 A kind of photometric stereo three-dimensional rebuilding method and beam splitting type photometric stereo camera
CN108269279A (en) * 2017-07-17 2018-07-10 杭州先临三维科技股份有限公司 Method and device for three-dimensional reconstruction based on monocular three-dimensional scanning system
CN107657245A (en) * 2017-10-16 2018-02-02 维沃移动通信有限公司 Face recognition method and terminal device
CN108480239A (en) * 2018-02-10 2018-09-04 浙江工业大学 Stereoscopic-vision-based rapid workpiece sorting method and device

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1484628A1 (en) * 2003-06-07 2004-12-08 Zeiss Optronik GmbH System and method for generating three-dimensional images
CN1946195A (en) * 2006-10-26 2007-04-11 上海交通大学 Scene depth restoring and three dimension re-setting method for stereo visual system
CN101924953A (en) * 2010-09-03 2010-12-22 南京农业大学 Simple matching method based on datum point
KR101012691B1 (en) * 2010-07-05 2011-02-09 주훈 3d stereo thermal imaging camera system
CN102074005A (en) * 2010-12-30 2011-05-25 杭州电子科技大学 Interest-region-oriented stereo matching method

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120098971A1 (en) * 2010-10-22 2012-04-26 Flir Systems, Inc. Infrared binocular system with dual diopter adjustment
CN102999939B (en) * 2012-09-21 2016-02-17 魏益群 Coordinate acquiring device, system and method for real-time three-dimensional reconstruction, stereo interaction device

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1484628A1 (en) * 2003-06-07 2004-12-08 Zeiss Optronik GmbH System and method for generating three-dimensional images
CN1946195A (en) * 2006-10-26 2007-04-11 上海交通大学 Scene depth restoring and three dimension re-setting method for stereo visual system
KR101012691B1 (en) * 2010-07-05 2011-02-09 주훈 3d stereo thermal imaging camera system
CN101924953A (en) * 2010-09-03 2010-12-22 南京农业大学 Simple matching method based on datum point
CN102074005A (en) * 2010-12-30 2011-05-25 杭州电子科技大学 Interest-region-oriented stereo matching method

Also Published As

Publication number Publication date
WO2014044126A1 (en) 2014-03-27
CN102999939A (en) 2013-03-27

Similar Documents

Publication Publication Date Title
US8787656B2 (en) Method and apparatus for feature-based stereo matching
CN104685513B (en) The feature-based high-resolution low-resolution image is captured using an array source estimation
US20130187919A1 (en) 3D Body Modeling, from a Single or Multiple 3D Cameras, in the Presence of Motion
JP5160643B2 (en) Three-dimensional object recognition system and method from 2D images
Zhuo et al. Defocus map estimation from a single image
US9214013B2 (en) Systems and methods for correcting user identified artifacts in light field images
US20130335535A1 (en) Digital 3d camera using periodic illumination
JP2015502058A (en) Multi-spectral imaging system
CN101432776B (en) System and method for three-dimensional object reconstruction from two-dimensional images
CN101276460A (en) Three-dimensional object imaging device
CN103207664B (en) An image processing method and apparatus
CN103221975B (en) Three-dimensional imaging system
JP4692371B2 (en) Image processing apparatus, image processing method, image processing program, and a recording medium recording an image processing program and a moving object detection system,
KR20140000195A (en) Autofocus for stereoscopic camera
CN102687515B (en) 3D image interpolation device,3d imaging device,and 3d image interpolation method
CN101996406A (en) No-reference structural sharpness image quality evaluation method
CN103868460B (en) Automatic parallax binocular stereo vision measurement optimization algorithm
CN102830793B (en) Gaze tracking method and apparatus
JP2012118698A (en) Image processing system
CN102472619A (en) Image capture device and image capture method
CN103578116A (en) Apparatus and method for tracking object
CN107025635A (en) Image saturation processing method and processing device based on depth of field, and electronic device
CN100373394C (en) Detection method of petoscope based on bionic oculus
US20150256813A1 (en) System and method for 3d reconstruction using multiple multi-channel cameras
KR20120058828A (en) System for extracting 3-dimensional coordinate and method thereof

Legal Events

Date Code Title Description
C06 Publication
C10 Entry into substantive examination
C14 Grant of patent or utility model
CF01