US8908913B2 - Voting-based pose estimation for 3D sensors - Google Patents

Voting-based pose estimation for 3D sensors Download PDF

Info

Publication number
US8908913B2
US8908913B2 US13/329,493 US201113329493A US8908913B2 US 8908913 B2 US8908913 B2 US 8908913B2 US 201113329493 A US201113329493 A US 201113329493A US 8908913 B2 US8908913 B2 US 8908913B2
Authority
US
United States
Prior art keywords
pair
boundary
points
pose
pair features
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US13/329,493
Other languages
English (en)
Other versions
US20130156262A1 (en
Inventor
Yuichi Taguchi
Oncel Tuzel
Srikumar Ramalingam
Changhyun Choi
Ming-Yu Liu
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Mitsubishi Electric Research Laboratories Inc
Original Assignee
Mitsubishi Electric Research Laboratories Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mitsubishi Electric Research Laboratories Inc filed Critical Mitsubishi Electric Research Laboratories Inc
Priority to US13/329,493 priority Critical patent/US8908913B2/en
Assigned to MITSUBISHI ELECTRIC RESEARCH LABORATORIES, INC. reassignment MITSUBISHI ELECTRIC RESEARCH LABORATORIES, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: CHOI, Changhyun, LIU, MING-YU, TAGUCHI, YUICHI, RAMALINGAM, SRIKUMAR, TUZEL, ONCEL
Priority to JP2014514266A priority patent/JP5726378B2/ja
Priority to DE112012005350.8T priority patent/DE112012005350B4/de
Priority to CN201280063083.8A priority patent/CN104040590B/zh
Priority to PCT/JP2012/081864 priority patent/WO2013094441A1/en
Publication of US20130156262A1 publication Critical patent/US20130156262A1/en
Application granted granted Critical
Publication of US8908913B2 publication Critical patent/US8908913B2/en
Active legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G06T7/75—Determining position or orientation of objects or cameras using feature-based methods involving models
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/521—Depth or shape recovery from laser ranging, e.g. using interferometry; from the projection of structured light
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30164—Workpiece; Machine component

Definitions

  • This invention relates generally to estimating poses of 3D objects, and more particularly to estimating poses from data acquired by 3D sensors.
  • the pose of an object has 6-degree-of-freedom (6-DoF), i.e., 3D translation and 3D rotation.
  • 6-DoF 6-degree-of-freedom
  • the problem of pose estimation refers to finding the pose of an object with respect to a reference coordinate system, usually the coordinate system of a sensor.
  • the pose can be acquired using measurements from sensors, e.g., 2D images, 3D point clouds, with 3D models.
  • Pose estimation plays a major role in many robotics applications such as bin picking, grasping, localization, autonomous navigation, and 3D reconstruction.
  • pose estimation was primarily done using 2D images because cameras are cost effective and allow fast image acquisition.
  • the main problem with 2D images is to match the 2D features with their corresponding 3D features in the model. This becomes challenging due to various illumination conditions and different viewpoints of the camera, causing changes in rotation and scale in the image space. Furthermore, some views of the object can theoretically lead to ambiguous poses.
  • Several invariant feature descriptors are known to determine the correspondences between an input image and a database of images, where the 2D keypoints are matched with the 3D coordinates.
  • 3D data obtained with 3D sensors have a lot less variants in contrast to 2D data.
  • the main challenge is to solve the correspondence problem in the presence of sensor noise, occlusions, and clutter.
  • the correspondence problem refers to finding a one-to-one matching between features in the data and features in the model.
  • the features are usually constructed to characterize the size and shape of the object.
  • 3D feature descriptors using the distributions of surface points and normals, and matching procedures are known. Those descriptors are generally invariant to rigid body transformation, but sensitive to noise and occlusion. Furthermore, those features require dense point clouds, which may not be available.
  • Pose estimation is feasible with various kinds of correspondences between sensor 3D data and the model: 3 point correspondences, 2 line correspondences, and 6 points to 3 or more planes. Typically those correspondences are used in a hypothesize-and-test framework such as RANdom SAmple Consensus (RANSAC) to determine the pose. Alternatively, the pose can be retrieved from the mode of the hypothesized pose distribution either using a Hough voting scheme, or clustering in the parameter space.
  • RANSAC RANdom SAmple Consensus
  • the pose can be retrieved from the mode of the hypothesized pose distribution either using a Hough voting scheme, or clustering in the parameter space.
  • Those approaches suffer from two problems when only 3D sensor data are available without images or other prior information. Points, lines, and planes are not very discriminative individually and are combinatorial to match, and it is difficult to achieve fast computation without doing any prior processing on the model.
  • a pair feature can be defined by a distance and relative orientations between two oriented points on the surface of an object.
  • An object is represented by a set of oriented point pair features, which is stored in a hash table for fast retrieval. Random two points are sampled from the sensor data and each such pair votes for a particular pose. The required pose corresponds to the one with a largest number of votes.
  • a simpler pair feature consisting of the depth difference between a pixel and an offset pixel is used for human pose estimation with a random forest ensemble classifier.
  • the embodiments of the invention provide a voting-based method for estimating a pose of an object from data acquired by a 3D sensor.
  • oriented surface points points on the surface of an object with normals
  • those surface points are not compact and discriminative enough for many industrial and real-world objects, which are planar.
  • edges play a key role in 2D registration, while depth discontinuities are crucial in 3D.
  • the embodiments provide a family of pose estimation methods that better exploit this boundary information.
  • oriented surface points we use two other geometric primitives, including oriented boundary points (points on the boundaries of an object with directions) and boundary line segments.
  • FIG. 1 is a schematic of a robotic assembly apparatus that can use the embodiments of the invention
  • FIG. 2A is a block diagram of a method for determining a pose of a 3D object using a voting-based method according to embodiments of the invention
  • FIG. 2B is a block diagram of details of the method for determining a pose of a 3D object using a voting-based method
  • FIGS. 3A-3D are schematics of pair features for voting-based pose estimation according to embodiments of the invention.
  • FIG. 4 is a schematic of a transform between a line pair feature obtained from the sensor data, and a line pair feature obtained from the model according to embodiments of the invention.
  • FIG. 1 shows a system 100 for estimating a pose of an object.
  • the system includes a 6-axis robotic arm 110 with a gripper 120 .
  • a 3D sensor 130 is arranged on the arm.
  • the gripper picks objects 140 up from a bin 150 depending on their pose 101 . It should be noted that the bin can contain different objects.
  • 3D sensor uses structured light generated by an infrared laser. Other sensors are also possible.
  • the sensor acquires 3D “point clouds” 160 as depth maps of 640 ⁇ 480 pixels.
  • the 3D sensor is calibrated with respect to the robot arm, thereby allowing grasping and picking of an object using the pose.
  • the 3D point clouds are processed by a method performed in a processor 170 .
  • the processor can include memory and input/output interfaces as known in the art.
  • the method determines the pose, which can be fed back to a controller 180 to direct the arm to pick the object 140 .
  • FIG. 2A shows a block diagram of the method 200 , which includes the following steps: 3D sensing 210 of the scene, voting-based pose estimation 220 , pose refinement 230 , and grasping 240 of the object according to its pose.
  • the system scans the bin of objects using the 3D sensor to detect an object and picks it up according to the pose determined by the method. Given a 3D CAD model of each different object in the bin, the voting-based method performs detection and pose estimation of the object using a scanned 3D point cloud 160 . This step provides multiple coarse poses. The system selects several best coarse poses and individually refines the poses using an iterative-closest point (ICP) procedure.
  • ICP iterative-closest point
  • the ICP minimizes a difference between two 3D point clouds.
  • the ICP is often used to reconstruct a complete 3D model from multiple partial scans acquired at different locations and to localize robots with respect to a 3D environmental model.
  • the procedure can be performed in real time by iteratively revising the transformation (translation and rotation) needed to minimize the distance between the corresponding points.
  • the input is the two 3D point clouds, an initial estimate of the transformation, and a termination criterion.
  • the output is the refined transformation.
  • the primary steps include finding corresponding points by a nearest neighbor criterion, estimating transformation parameters using a mean squared error as a cost function, transforming the points using the estimated parameters, and iterating.
  • the ICP procedure in our system renders the CAD model of the object using the current pose estimate and generates a 3D point cloud for the model by sampling the surface of the rendered model. Then, the ICP determines the closest 3D point in the scanned point cloud for each 3D point in the model and updates the pose estimate using the 3D point correspondences.
  • a registration error is measured as an average distance between the corresponding 3D points in the scanned point cloud and the model.
  • the registration error can be high when the ICP converges to an incorrect pose, or when a part of the object is missing due to occlusion from other objects. Therefore, if the registration error is high, our system does not use the estimated pose for grasping. In addition to the registration error, our system also checks whether the object at the pose is reachable by the robot arm to determine whether the object can be picked up.
  • the sensor is expressed in terms of a sensor (camera) coordinate system C.
  • the model of the object is expressed in terms of an object coordinate system O.
  • the object in the scene uses a world coordinate system W.
  • the sensor coordinate system S can be calibrated to the world coordinate system W because, in general, the sensor is at a known location and orientations in the real world.
  • the pose estimation method determines a transformation between the sensor coordinate system and the object coordinate system. Then, relationship between the sensor and world coordinate systems can be used to manipulate the object in the scene.
  • pair features are based on pairs of oriented points or lines, which we refer to as pair features.
  • a pair feature consisting of a pair of oriented surface points is denoted by S 2 S, because we construct the feature vector asymmetrically from one oriented point (reference) to another (referred).
  • S 2 S A pair feature consisting of a pair of oriented surface points
  • Point pair feature descriptors F S2S , F B2B , F S2B , and F B2S , are defined by the relative position f 1 and orientations f 2 ,f 3 , and f 4 of a pair of oriented points (m, n), where points indicate either surface points with surface normals, or boundary points with directions.
  • the line pair feature descriptor F L2L is defined by the minimum distance between two infinite lines f 1 , the acute angle between two line directions f 2 , and the maximum distance between the two line segments f 3 .
  • a surface pair feature S 2 S is defined using two points on the object surface and their normals. Given an oriented point from the scene and a corresponding primitive from the model, the 3D pose can be recovered up to a planar rotation by aligning point locations and their normals. To resolve the rotation ambiguity and recover the full 6-DoF pose, at least correspondences between two pairs of scene and model primitives are necessary.
  • the second and third components f 2 and f 3 are angles between the vector d and the surface normal vectors n r and n i , respectively.
  • the last component f 4 is the angle between the two normal vectors.
  • the S 2 S feature is shown in FIG. 3A . If the object spans a wide range of normals, then this feature provides a good description of the object.
  • the S 2 S feature fails to provide a good description for shapes that do not span a wide range of surface normals. Unfortunately, many industrial parts used during robotic assembly are planar and have a very small set of normal directions. Additionally, due to noise in the 3D data, it is difficult to estimate the normals accurately in high curvature regions on the surface, which further complicates the problem.
  • boundary points do not have well defined normals.
  • To extract the line segments on object boundaries we first determine the edges in the depth map using a Canny edge detector. The edges are stored in an edge map. Points from the edge map are randomly sampled and 3D lines are fit on local regions centered around these points using a RANSAC procedure. By iteratively locating and removing line segments with maximum inliers, we recover all the line segments. These line segments are further refined by applying least squares to the inliers. After line fitting, we uniformly sample boundary points on the 3D line segments.
  • n r and n i are directions of the 3D lines. Note that the directions are not uniquely determined. Therefore, we consider two possible directions n and ⁇ n when we use the B 2 B feature.
  • Object boundaries are highly informative. Compared to S 2 S, B 2 B provides more concise modeling because there are fewer boundary points compared to surface points. Additionally, the orientations from local line segments are more resilient to noise compared to surface normals.
  • Boundary pair features are associated with the depth edges of the object. Such a feature depending solely on boundary points may not be the best choice for an object with high surface curvatures. For example, any point on the surface of a spherical object can potentially become a depth edge based on the pose, whereas depth edges on a polyhedral object are more stable, and always appear on plane intersections.
  • the S 2 B feature descriptor F S2B ⁇ is defined by
  • the B 2 S feature is defined by selecting an oriented boundary point as the reference point and an oriented surface point as the referred point.
  • c r and c i are the closest points on each line with respect to each other.
  • the acute angle between two line segments with directions v 1 and v 2 is:
  • ⁇ a ⁇ ( v 1 , v 2 ) ⁇ ⁇ ⁇ ( v 1 , v 2 ) if ⁇ ⁇ ⁇ ⁇ ( v 1 , v 2 ) ⁇ ⁇ 2 ⁇ - ⁇ ( ⁇ v 1 , v 2 ) otherwise ⁇ ⁇
  • ⁇ ⁇ a ⁇ ( v 1 , v 2 ) ⁇ [ 0 ; ⁇ 2 ] .
  • the first component is the distance between the closest points c r and c i (i.e., the distance between the two infinite lines), and the second component is the acute angle between the two line segments.
  • the last component represents the maximum distance between the two line segments. This maximum distance can be the maximum of all possible distances between an end point in one line segment with an end point in another line segment. This maximum distance is useful to prune false matches between two line segments having a similar distance and angle. For example, any pair of coplanar orthogonal lines has the same distance and angle. However, as described above, this maximum distance is not reliable due to the breaking of the line segments. Thus, we use a larger quantization step for this component in pose estimation. We do not construct the L 2 L feature for pairs of parallel line segments, because such pairs lead to ambiguity in pose.
  • the line pair feature provides very efficient matching because the number of line segments is less than the number of surface points or boundary points. This feature is particularly efficient for polyhedral objects and objects having long boundary line segments.
  • surface points are obtained by scanning and subsampling the depth image, 3D lines are determined via our RANSAC procedure that estimates 3D line segments from the 3D scan, and the boundary points are then obtained by subsampling along the line segments.
  • geometric primitives oriented surface points for S 2 S, oriented boundary points for B 2 B, both oriented surface and boundary points for S 2 B and B 2 S, and 3D line segments for L 2 L.
  • geometric primitives can be determined from either 3D scanned data with known calibration between the sensor and the object, or synthetic depth data rendered from a known CAD model.
  • model pair features 201 S 2 S, B 2 B, S 2 B, B 2 S, and L 2 L we represent each particular object 140 using a set of model pair features 201 S 2 S, B 2 B, S 2 B, B 2 S, and L 2 L, as shown in FIG. 2A .
  • model pair features For efficient feature matching, we store model pair features in a hash table 205 . Initially, we discretize the feature pairs where the distances and the angles are quantized with step sizes of ⁇ d and ⁇ a , respectively. It is important to define the quantization levels appropriately. Using large step sizes reduces discriminative power of the descriptors, whereas using small step sizes make the method sensitive to noise. Following, discretized pair feature descriptors are used as the key for the hash function and the pair features are inserted into bins accordingly. With appropriate quantization steps, similar pair features are grouped together and matching and voting can be done in constant time.
  • Constructing object representation can be a preprocessing step that can be performed off-line.
  • FIG. 2A shows three different objects that could be randomly commingled in the bin for later picking.
  • a voting scheme which reduces the voting space to a 2D space using intermediate coordinate transformations.
  • a scene point pair (s r , s i ) is searched in the hash table , and then a corresponding model point pair (m r , m i ) is found.
  • reference points of the pairs are transformed to an intermediate coordinate system such that their positions correspond to the origin of the coordinate system and their normals are aligned with the x-axis.
  • the referred points, m i and s i are aligned by rotating the object model around the x-axis with an angle ⁇ .
  • a reference point s r in the scene is paired with the other points s i in the scene, and then the model pair features (M r , m i ), which are similar to the scene pair feature (s r , s i ) are obtained from the hash table based on their descriptors.
  • the rotation angle ⁇ is determined in the intermediate coordinate system and then votes are cast for the pair (m r ⁇ ).
  • elements greater than a predetermined threshold are selected, from which candidate poses (rigid transformations between the model coordinates and the scene coordinates) 206 are computed as in Eqn. (10). This process is repeated by selecting different reference points in the scene.
  • This voting scheme is applicable to S 2 S, B 2 B, S 2 B, and B 2 S pair features.
  • the L 2 L feature is defined by a pair of line segments.
  • the fundamental idea is the same: We align two pair features in an intermediate coordinate system, as shown in FIG. 4 .
  • the candidate poses 206 are obtained for each reference primitive (point or line) in the scene. Because the object is modeled by multiple feature pairs, it is expected to have multiple candidate poses over different reference primitives, points m r or lines l m r , supporting a consistent pose hypothesis. Thus, we aggregate similar poses from different scene primitives.
  • the proximity testing is done with fixed thresholds for translation and rotation. Distance computation and averaging for translation are performed in the 3D Euclidean space, while those for rotation are performed using quaternion representation of rotation matrices.
  • the clusters are sorted in decreasing order of the total number of votes, which determines confidence of the estimated poses.
  • the clusters with the largest numbers of votes correspond to the best poses 208 .
  • a set of pair features, using oriented surface points, oriented boundary points, and boundary line segments, is provided to model a variety of objects.
  • a bin-picking system implemented according to embodiments of the invention has a pickup success rate of higher than 98%, and pose estimation error less than 0.3 mm and 0.3° for translations and rotations.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Optics & Photonics (AREA)
  • Image Analysis (AREA)
  • Length Measuring Devices By Optical Means (AREA)
US13/329,493 2011-12-19 2011-12-19 Voting-based pose estimation for 3D sensors Active 2032-03-15 US8908913B2 (en)

Priority Applications (5)

Application Number Priority Date Filing Date Title
US13/329,493 US8908913B2 (en) 2011-12-19 2011-12-19 Voting-based pose estimation for 3D sensors
JP2014514266A JP5726378B2 (ja) 2011-12-19 2012-12-04 物体の姿勢を推定するための方法
DE112012005350.8T DE112012005350B4 (de) 2011-12-19 2012-12-04 Verfahren zum Schätzen der Stellung eines Objekts
CN201280063083.8A CN104040590B (zh) 2011-12-19 2012-12-04 用于估计物体的姿态的方法
PCT/JP2012/081864 WO2013094441A1 (en) 2011-12-19 2012-12-04 Method for estimating pose of object

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US13/329,493 US8908913B2 (en) 2011-12-19 2011-12-19 Voting-based pose estimation for 3D sensors

Publications (2)

Publication Number Publication Date
US20130156262A1 US20130156262A1 (en) 2013-06-20
US8908913B2 true US8908913B2 (en) 2014-12-09

Family

ID=47436153

Family Applications (1)

Application Number Title Priority Date Filing Date
US13/329,493 Active 2032-03-15 US8908913B2 (en) 2011-12-19 2011-12-19 Voting-based pose estimation for 3D sensors

Country Status (5)

Country Link
US (1) US8908913B2 (enrdf_load_stackoverflow)
JP (1) JP5726378B2 (enrdf_load_stackoverflow)
CN (1) CN104040590B (enrdf_load_stackoverflow)
DE (1) DE112012005350B4 (enrdf_load_stackoverflow)
WO (1) WO2013094441A1 (enrdf_load_stackoverflow)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150138322A1 (en) * 2013-11-19 2015-05-21 Canon Kabushiki Kaisha Image processing device and its control method, imaging apparatus, and storage medium
US20150290805A1 (en) * 2014-04-11 2015-10-15 Axium Inc. Vision-Assisted System and Method for Picking of Rubber Bales in a Bin
US20160253807A1 (en) * 2015-02-26 2016-09-01 Mitsubishi Electric Research Laboratories, Inc. Method and System for Determining 3D Object Poses and Landmark Points using Surface Patches
CN106091921A (zh) * 2015-04-28 2016-11-09 三菱电机株式会社 用于确定场景中的尺寸的方法
DE102017201169A1 (de) 2017-01-25 2018-07-26 Siemens Aktiengesellschaft Rechnergestütztes Bildverarbeitungsverfahren
WO2020047575A1 (en) * 2018-09-04 2020-03-12 Fastbrick Ip Pty Ltd Vision system for a robotic machine
US20200290496A1 (en) * 2017-12-05 2020-09-17 Guima Palfinger S.A.S. Truck-mountable detection system
US11420334B2 (en) 2020-04-14 2022-08-23 Hong Kong Applied Science And Technology Research Institute Co., Ltd. Candidate six dimensional pose hypothesis selection
US20230137715A1 (en) * 2021-10-28 2023-05-04 Ubtech North America Research And Development Center Corp Vision-guided picking and placing method, mobile robot and computer-readable storage medium

Families Citing this family (52)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130201339A1 (en) * 2012-02-08 2013-08-08 Honeywell International Inc. System and method of optimal video camera placement and configuration
US9098913B2 (en) * 2012-05-11 2015-08-04 Cornell University Prediction of successful grasps by end of arm tooling
ITPR20120039A1 (it) * 2012-06-20 2012-09-19 Gevis S R L Dispositivo e metodo di misura di un pezzo
US8913825B2 (en) * 2012-07-16 2014-12-16 Mitsubishi Electric Research Laboratories, Inc. Specular edge extraction using multi-flash imaging
JP6370038B2 (ja) * 2013-02-07 2018-08-08 キヤノン株式会社 位置姿勢計測装置及び方法
US9305364B2 (en) * 2013-02-19 2016-04-05 Caterpillar Inc. Motion estimation systems and methods
US9280827B2 (en) * 2013-07-03 2016-03-08 Mitsubishi Electric Research Laboratories, Inc. Method for determining object poses using weighted features
JP6245880B2 (ja) * 2013-07-31 2017-12-13 キヤノン株式会社 情報処理装置および情報処理手法、プログラム
US11412334B2 (en) * 2013-10-23 2022-08-09 Cochlear Limited Contralateral sound capture with respect to stimulation energy source
US9412040B2 (en) * 2013-12-04 2016-08-09 Mitsubishi Electric Research Laboratories, Inc. Method for extracting planes from 3D point cloud sensor data
US10043112B2 (en) * 2014-03-07 2018-08-07 Qualcomm Incorporated Photo management
ES2617307T3 (es) * 2014-04-14 2017-06-16 Softbank Robotics Europe Un procedimiento de localización de un robot en un plano de localización
WO2016040473A1 (en) * 2014-09-10 2016-03-17 Vangogh Imaging, Inc. Real-time dynamic three-dimensional adaptive object recognition and model reconstruction
US9747493B2 (en) 2014-09-23 2017-08-29 Keylemon Sa Face pose rectification method and apparatus
DE102015100983A1 (de) 2015-01-23 2016-07-28 Sick Ag Verfahren zur Lokalisierung von Greifpunkten von Objekten
CN104899918B (zh) * 2015-05-14 2018-02-27 深圳大学 一种无人机的三维环境建模方法及系统
WO2016195927A1 (en) 2015-06-02 2016-12-08 Life Technologies Corporation Systems and methods for calibrating a structured illumination imaging system and for capturing a structured illumination image
DE102015212932A1 (de) * 2015-07-10 2017-01-12 Kuka Roboter Gmbh Verfahren zum Steuern eines Roboters und/oder eines autonomen fahrerlosen Transportsystems
EP3185171B1 (en) * 2015-12-24 2020-03-04 Dassault Systèmes 3d object localization with descriptor
US11044390B2 (en) * 2016-02-10 2021-06-22 Karl Storz Imaging, Inc. Imaging system for identifying a boundary between active and inactive portions of a digital image
US9868212B1 (en) * 2016-02-18 2018-01-16 X Development Llc Methods and apparatus for determining the pose of an object based on point cloud data
CN107220580B (zh) * 2016-03-22 2022-08-09 敦泰电子有限公司 基于投票决策和最小二乘法的图像识别随机取样一致算法
US10371512B2 (en) * 2016-04-08 2019-08-06 Otis Elevator Company Method and system for multiple 3D sensor calibration
US10380767B2 (en) * 2016-08-01 2019-08-13 Cognex Corporation System and method for automatic selection of 3D alignment algorithms in a vision system
US10380762B2 (en) 2016-10-07 2019-08-13 Vangogh Imaging, Inc. Real-time remote collaboration and virtual presence using simultaneous localization and mapping to construct a 3D model and update a scene based on sparse data
US10116915B2 (en) * 2017-01-17 2018-10-30 Seiko Epson Corporation Cleaning of depth data by elimination of artifacts caused by shadows and parallax
US10839585B2 (en) 2018-01-05 2020-11-17 Vangogh Imaging, Inc. 4D hologram: real-time remote avatar creation and animation control
US10957072B2 (en) * 2018-02-21 2021-03-23 Cognex Corporation System and method for simultaneous consideration of edges and normals in image features by a vision system
US10529089B2 (en) * 2018-02-23 2020-01-07 GM Global Technology Operations LLC Crowd-sensed point cloud map
US11080540B2 (en) 2018-03-20 2021-08-03 Vangogh Imaging, Inc. 3D vision processing using an IP block
US10810783B2 (en) 2018-04-03 2020-10-20 Vangogh Imaging, Inc. Dynamic real-time texture alignment for 3D models
CN108537805B (zh) * 2018-04-16 2021-09-21 中北大学 一种基于特征几何收益的目标识别方法
US11170224B2 (en) 2018-05-25 2021-11-09 Vangogh Imaging, Inc. Keyframe-based object scanning and tracking
CN109785389A (zh) * 2019-01-18 2019-05-21 四川长虹电器股份有限公司 一种基于哈希描述与迭代最近点的三维物体检测方法
CN109903326B (zh) 2019-02-28 2022-02-22 北京百度网讯科技有限公司 用于确定工程机械设备的旋转角度的方法和装置
JP7115376B2 (ja) * 2019-03-18 2022-08-09 日本電信電話株式会社 回転状態推定装置、方法及びプログラム
US11232633B2 (en) 2019-05-06 2022-01-25 Vangogh Imaging, Inc. 3D object capture and object reconstruction using edge cloud computing resources
US11170552B2 (en) 2019-05-06 2021-11-09 Vangogh Imaging, Inc. Remote visualization of three-dimensional (3D) animation with synchronized voice in real-time
CN110348310A (zh) * 2019-06-12 2019-10-18 西安工程大学 一种霍夫投票3d彩色点云识别方法
JP7316134B2 (ja) * 2019-07-22 2023-07-27 ファナック株式会社 位置姿勢特定装置、位置姿勢特定方法及び位置姿勢特定プログラム
CN110992427B (zh) * 2019-12-19 2023-10-13 深圳市华汉伟业科技有限公司 一种形变物体的三维位姿估计方法及定位抓取系统
US11335063B2 (en) 2020-01-03 2022-05-17 Vangogh Imaging, Inc. Multiple maps for 3D object scanning and reconstruction
EP4081933A4 (en) * 2020-01-29 2024-03-20 Intrinsic Innovation LLC SYSTEMS AND METHODS FOR CHARACTERIZING OBJECT POSE DETECTION AND MEASUREMENT SYSTEMS
CN111453401B (zh) * 2020-03-25 2021-04-16 佛山缔乐视觉科技有限公司 工件的自动拾取方法和装置
DE102021109036A1 (de) 2021-04-12 2022-10-13 Robert Bosch Gesellschaft mit beschränkter Haftung Vorrichtung und verfahren zum lokalisieren von stellen von objekten aus kamerabildern der objekte
CN113486470B (zh) * 2021-07-15 2023-05-23 西北工业大学 一种考虑非理想表面接触状态的装配体位姿计算方法
CN113853632B (zh) * 2021-08-18 2024-08-06 香港应用科技研究院有限公司 用于3d扫描之间全局配准的方法和系统
CN114676528B (zh) * 2022-04-07 2024-02-23 西北工业大学 基于组合基元法的非理想模型装配偏差算法
CN115810086A (zh) * 2022-12-22 2023-03-17 香港中文大学(深圳)未来智联网络研究院 一种三维场景重建方法、装置、计算机设备及存储介质
GB2633375A (en) * 2023-09-08 2025-03-12 Continental Automotive Tech Gmbh A computer-implemented method for performing a 3D object detection using RANSAC and a device thereof
CN117237451B (zh) * 2023-09-15 2024-04-02 南京航空航天大学 一种基于轮廓重建和几何引导的工业零件6d位姿估计方法
CN118832591B (zh) * 2024-08-16 2025-04-29 贵州大学 一种基于重采样姿态解析的机器人多阶行为规划方法

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2005017820A1 (en) 2003-08-15 2005-02-24 Scape A/S Computer-vision system for classification and spatial localization of bounded 3d-objects
US20070070069A1 (en) * 2005-09-26 2007-03-29 Supun Samarasekera System and method for enhanced situation awareness and visualization of environments
US20080013836A1 (en) * 2006-06-19 2008-01-17 Akira Nakamura Information Processing Device, Information Processing Method, and Program
US7366645B2 (en) * 2002-05-06 2008-04-29 Jezekiel Ben-Arie Method of recognition of human motion, vector sequences and speech
US20120185095A1 (en) * 2010-05-20 2012-07-19 Irobot Corporation Mobile Human Interface Robot
US20130093852A1 (en) * 2011-10-12 2013-04-18 Board Of Trustees Of The University Of Arkansas Portable robotic device
US8467596B2 (en) * 2011-08-30 2013-06-18 Seiko Epson Corporation Method and apparatus for object pose estimation
US8558883B2 (en) * 2007-07-27 2013-10-15 Sportvision, Inc. Providing graphics in images depicting aerodynamic flows and forces

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
ATE452379T1 (de) * 2007-10-11 2010-01-15 Mvtec Software Gmbh System und verfahren zur 3d-objekterkennung
JP5088278B2 (ja) * 2008-09-17 2012-12-05 株式会社安川電機 物体検出方法と物体検出装置およびロボットシステム
CN101510308A (zh) * 2009-03-26 2009-08-19 山东理工大学 产品模型点云边界特征快速提取方法
EP2385483B1 (en) * 2010-05-07 2012-11-21 MVTec Software GmbH Recognition and pose determination of 3D objects in 3D scenes using geometric point pair descriptors and the generalized Hough Transform

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7366645B2 (en) * 2002-05-06 2008-04-29 Jezekiel Ben-Arie Method of recognition of human motion, vector sequences and speech
WO2005017820A1 (en) 2003-08-15 2005-02-24 Scape A/S Computer-vision system for classification and spatial localization of bounded 3d-objects
US20070127816A1 (en) * 2003-08-15 2007-06-07 Ivar Balslev Computer-vision system for classification and spatial localization of bounded 3d-objects
US7822264B2 (en) * 2003-08-15 2010-10-26 Scape A/S Computer-vision system for classification and spatial localization of bounded 3D-objects
US20070070069A1 (en) * 2005-09-26 2007-03-29 Supun Samarasekera System and method for enhanced situation awareness and visualization of environments
US20080013836A1 (en) * 2006-06-19 2008-01-17 Akira Nakamura Information Processing Device, Information Processing Method, and Program
US8558883B2 (en) * 2007-07-27 2013-10-15 Sportvision, Inc. Providing graphics in images depicting aerodynamic flows and forces
US20120185095A1 (en) * 2010-05-20 2012-07-19 Irobot Corporation Mobile Human Interface Robot
US8467596B2 (en) * 2011-08-30 2013-06-18 Seiko Epson Corporation Method and apparatus for object pose estimation
US20130093852A1 (en) * 2011-10-12 2013-04-18 Board Of Trustees Of The University Of Arkansas Portable robotic device

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Bertram Drost et al. "Model Globally Match Locally: Efficient and Robust 3D Object Recognition," 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 13-18, 2010, San Francisco, CA, USA. IEEE, Piscataway, NJ, US. Jun. 13, 2010. p. 998-1005.
Changhyun Choi et al. "Voting-based pose estimation for robotic assembly using a 3D sensor," Robotics and Automation (ICRA), 2012 IEEE International Conference on, IEEE, May 14, 2012, pp. 1724-1731.

Cited By (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150138322A1 (en) * 2013-11-19 2015-05-21 Canon Kabushiki Kaisha Image processing device and its control method, imaging apparatus, and storage medium
US10311595B2 (en) * 2013-11-19 2019-06-04 Canon Kabushiki Kaisha Image processing device and its control method, imaging apparatus, and storage medium
US20150290805A1 (en) * 2014-04-11 2015-10-15 Axium Inc. Vision-Assisted System and Method for Picking of Rubber Bales in a Bin
US10239210B2 (en) * 2014-04-11 2019-03-26 Symbotic Canada Ulc Vision-assisted system and method for picking of rubber bales in a bin
US20160253807A1 (en) * 2015-02-26 2016-09-01 Mitsubishi Electric Research Laboratories, Inc. Method and System for Determining 3D Object Poses and Landmark Points using Surface Patches
US10515259B2 (en) * 2015-02-26 2019-12-24 Mitsubishi Electric Research Laboratories, Inc. Method and system for determining 3D object poses and landmark points using surface patches
CN106091921B (zh) * 2015-04-28 2019-06-18 三菱电机株式会社 用于确定场景中的尺寸的方法
CN106091921A (zh) * 2015-04-28 2016-11-09 三菱电机株式会社 用于确定场景中的尺寸的方法
DE102017201169A1 (de) 2017-01-25 2018-07-26 Siemens Aktiengesellschaft Rechnergestütztes Bildverarbeitungsverfahren
WO2018137935A1 (de) 2017-01-25 2018-08-02 Siemens Aktiengesellschaft Rechnergestütztes bildverarbeitungsverfahren
US20200290496A1 (en) * 2017-12-05 2020-09-17 Guima Palfinger S.A.S. Truck-mountable detection system
US11685302B2 (en) * 2017-12-05 2023-06-27 Guima Palfinger S.A.S. Truck-mountable detection system
WO2020047575A1 (en) * 2018-09-04 2020-03-12 Fastbrick Ip Pty Ltd Vision system for a robotic machine
US20210192784A1 (en) * 2018-09-04 2021-06-24 Fastbrick Ip Pty Ltd. Vision system for a robotic machine
CN113508012A (zh) * 2018-09-04 2021-10-15 快砖知识产权私人有限公司 用于机器人机械的视觉系统
US12002240B2 (en) * 2018-09-04 2024-06-04 Fastbrick Ip Pty Ltd. Vision system for a robotic machine
US11420334B2 (en) 2020-04-14 2022-08-23 Hong Kong Applied Science And Technology Research Institute Co., Ltd. Candidate six dimensional pose hypothesis selection
US20230137715A1 (en) * 2021-10-28 2023-05-04 Ubtech North America Research And Development Center Corp Vision-guided picking and placing method, mobile robot and computer-readable storage medium
US12030191B2 (en) * 2021-10-28 2024-07-09 Ubkang (Qingdao) Technology Co., Ltd. Vision-guided picking and placing method, mobile robot and computer-readable storage medium

Also Published As

Publication number Publication date
DE112012005350T5 (de) 2014-09-11
US20130156262A1 (en) 2013-06-20
DE112012005350B4 (de) 2022-03-31
CN104040590B (zh) 2017-02-22
WO2013094441A1 (en) 2013-06-27
JP2014528057A (ja) 2014-10-23
CN104040590A (zh) 2014-09-10
JP5726378B2 (ja) 2015-05-27

Similar Documents

Publication Publication Date Title
US8908913B2 (en) Voting-based pose estimation for 3D sensors
Choi et al. Voting-based pose estimation for robotic assembly using a 3D sensor
JP6216508B2 (ja) ３ｄシーンにおける３ｄ物体の認識および姿勢決定のための方法
CN108369741B (zh) 用于配准数据的方法和系统
Oehler et al. Efficient multi-resolution plane segmentation of 3D point clouds
Thompson et al. Three-dimensional model matching from an unconstrained viewpoint
JP5455873B2 (ja) シーンにおける物体の姿勢を求めるための方法
US7822264B2 (en) Computer-vision system for classification and spatial localization of bounded 3D-objects
EP2081133B1 (en) System and method for deformable object recognition
Wu et al. A triangulation-based hierarchical image matching method for wide-baseline images
US20110273442A1 (en) Recognition and pose determination of 3d objects in 3d scenes
Läbe et al. Automatic relative orientation of images
Urban et al. Finding a good feature detector-descriptor combination for the 2D keypoint-based registration of TLS point clouds
Drost et al. Local hough transform for 3d primitive detection
Bergamasco et al. Cylinders extraction in non-oriented point clouds as a clustering problem
Weinmann et al. Geometric point quality assessment for the automated, markerless and robust registration of unordered TLS point clouds
CN107610174B (zh) 一种鲁棒的基于深度信息的平面检测方法及系统
Lane et al. Tutorial: Overview of stereo matching research
Liu et al. Pose estimation in heavy clutter using a multi-flash camera
Azad et al. Accurate shape-based 6-dof pose estimation of single-colored objects
Zhang et al. Method for matching general stereo planar curves
Lin et al. Image features-based mobile robot visual SLAM
Bodensteiner et al. Single frame based video geo-localisation using structure projection
Sarkar et al. Feature-augmented Trained Models for 6DOF Object Recognition and Camera Calibration.
Markiewicz et al. THE COMPARISON OF 2D AND 3D DETECTORS FOR TLS DATA REGISTRATION–PRELIMINARY RESULTS

Legal Events

Date Code Title Description
AS Assignment

Owner name: MITSUBISHI ELECTRIC RESEARCH LABORATORIES, INC., M

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:TAGUCHI, YUICHI;TUZEL, ONCEL;RAMALINGAM, SRIKUMAR;AND OTHERS;SIGNING DATES FROM 20120302 TO 20120314;REEL/FRAME:027897/0974

STCF Information on status: patent grant

Free format text: PATENTED CASE

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551)

Year of fee payment: 4

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 8