CN111226237A - 用于稳健抓取和瞄准物体的机器人系统和方法 - Google Patents

用于稳健抓取和瞄准物体的机器人系统和方法 Download PDF

Info

Publication number
CN111226237A
CN111226237A CN201880067484.8A CN201880067484A CN111226237A CN 111226237 A CN111226237 A CN 111226237A CN 201880067484 A CN201880067484 A CN 201880067484A CN 111226237 A CN111226237 A CN 111226237A
Authority
CN
China
Prior art keywords
computer
implemented method
object models
sensor
grabbing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201880067484.8A
Other languages
English (en)
Chinese (zh)
Inventor
肯尼思·伊盖尔·金山
杰弗里·布莱恩·马勒
马修·马特尔
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
University of California San Diego UCSD
Original Assignee
University of California San Diego UCSD
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University of California San Diego UCSD filed Critical University of California San Diego UCSD
Publication of CN111226237A publication Critical patent/CN111226237A/zh
Pending legal-status Critical Current

Links

Images

Classifications

    • BPERFORMING OPERATIONS; TRANSPORTING
    • B25HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25JMANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00Program-controlled manipulators
    • B25J9/16Program controls
    • B25J9/1628Program controls characterised by the control loop
    • B25J9/163Program controls characterised by the control loop learning, adaptive, model based, rule based expert control
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B25HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25JMANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00Program-controlled manipulators
    • B25J9/16Program controls
    • B25J9/1602Program controls characterised by the control system, structure, architecture
    • B25J9/161Hardware, e.g. neural networks, fuzzy logic, interfaces, processor
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B25HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25JMANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00Program-controlled manipulators
    • B25J9/16Program controls
    • B25J9/1694Program controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
    • B25J9/1697Vision controlled systems
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/60Type of objects
    • G06V20/64Three-dimensional [3D] objects
    • G06V20/647Three-dimensional [3D] objects by matching two-dimensional images to three-dimensional objects
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B25HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25JMANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00Program-controlled manipulators
    • B25J9/16Program controls
    • B25J9/1612Program controls characterised by the hand, wrist, grip control
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05BCONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00Program-control systems
    • G05B2219/30Nc systems
    • G05B2219/39Robotics, robotics to robotics hand
    • G05B2219/39082Collision, real time collision avoidance
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05BCONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00Program-control systems
    • G05B2219/30Nc systems
    • G05B2219/39Robotics, robotics to robotics hand
    • G05B2219/39505Control of gripping, grasping, contacting force, force distribution
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05BCONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00Program-control systems
    • G05B2219/30Nc systems
    • G05B2219/39Robotics, robotics to robotics hand
    • G05B2219/39531Several different sensors integrated into hand
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05BCONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00Program-control systems
    • G05B2219/30Nc systems
    • G05B2219/39Robotics, robotics to robotics hand
    • G05B2219/39542Plan grasp points, grip matrix and initial grasp force
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05BCONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00Program-control systems
    • G05B2219/30Nc systems
    • G05B2219/40Robotics, robotics mapping to robotics vision
    • G05B2219/40053Pick 3-D object from pile of objects
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05BCONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00Program-control systems
    • G05B2219/30Nc systems
    • G05B2219/40Robotics, robotics mapping to robotics vision
    • G05B2219/40155Purpose is grasping objects
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05BCONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00Program-control systems
    • G05B2219/30Nc systems
    • G05B2219/40Robotics, robotics mapping to robotics vision
    • G05B2219/40571Camera, vision combined with force sensor

Landscapes

  • Engineering & Computer Science (AREA)
  • Robotics (AREA)
  • Mechanical Engineering (AREA)
  • Physics & Mathematics (AREA)
  • Automation & Control Theory (AREA)
  • Orthopedic Medicine & Surgery (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Fuzzy Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Image Analysis (AREA)
  • Manipulator (AREA)
CN201880067484.8A 2017-09-01 2018-04-04 用于稳健抓取和瞄准物体的机器人系统和方法 Pending CN111226237A (zh)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US201762553589P 2017-09-01 2017-09-01
US62/553,589 2017-09-01
PCT/US2018/026122 WO2019045779A1 (en) 2017-09-01 2018-04-04 ROBOTIC SYSTEMS AND METHODS FOR ROBUST STITCHING AND TARGETING OF OBJECTS

Publications (1)

Publication Number Publication Date
CN111226237A true CN111226237A (zh) 2020-06-02

Family

ID=65525862

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201880067484.8A Pending CN111226237A (zh) 2017-09-01 2018-04-04 用于稳健抓取和瞄准物体的机器人系统和方法

Country Status (7)

Country Link
US (1) US11446816B2 (https=)
EP (1) EP3676766A4 (https=)
JP (1) JP2020532440A (https=)
CN (1) CN111226237A (https=)
AU (1) AU2018326171A1 (https=)
CA (1) CA3073516A1 (https=)
WO (1) WO2019045779A1 (https=)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111906781A (zh) * 2020-07-08 2020-11-10 西安交通大学 基于图神经网络的机器人自主工具构建方法、系统及相关设备
CN113787521A (zh) * 2021-09-24 2021-12-14 上海微电机研究所(中国电子科技集团公司第二十一研究所) 基于深度学习的机器人抓取方法、系统、介质和电子设备
CN114603599A (zh) * 2020-12-08 2022-06-10 山东新松工业软件研究院股份有限公司 一种机器人碰撞检测方法、装置、计算机设备及存储介质
CN114952836A (zh) * 2022-05-19 2022-08-30 中国科学院自动化研究所 多指手机器人抓取方法、装置与机器人系统
CN114986519A (zh) * 2022-07-21 2022-09-02 东南大学 一种基于深度强化学习的混杂场景机械臂精细抓取方法
CN116416444A (zh) * 2021-12-29 2023-07-11 广东美的白色家电技术创新中心有限公司 物体抓取点估计、模型训练及数据生成方法、装置及系统
CN118811238A (zh) * 2024-09-14 2024-10-22 启东泽厚智能科技有限公司 贴标机取料输送用夹爪自适应控制方法及装置
CN121535759A (zh) * 2026-01-19 2026-02-17 成都航天凯特机电科技有限公司 一种高扭矩密度机器人关节模组的控制方法和系统

Families Citing this family (36)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10899011B2 (en) * 2018-11-14 2021-01-26 Fetch Robotics, Inc. Method and system for selecting a preferred robotic grasp of an object-of-interest using pairwise ranking
US11235466B2 (en) * 2018-04-23 2022-02-01 University Of Southern California Setup planning and parameter selection for robotic finishing
JP7154815B2 (ja) * 2018-04-27 2022-10-18 キヤノン株式会社 情報処理装置、制御方法、ロボットシステム、コンピュータプログラム、及び記憶媒体
DE102018211044A1 (de) * 2018-07-04 2020-01-09 Kuka Deutschland Gmbh Verfahren und System zum Analysieren und/oder Konfigurieren einer industriellen Anlage
JP6833777B2 (ja) * 2018-09-10 2021-02-24 株式会社東芝 物体ハンドリング装置及びプログラム
US11325252B2 (en) * 2018-09-15 2022-05-10 X Development Llc Action prediction networks for robotic grasping
US11185978B2 (en) * 2019-01-08 2021-11-30 Honda Motor Co., Ltd. Depth perception modeling for grasping objects
US20200301510A1 (en) * 2019-03-19 2020-09-24 Nvidia Corporation Force estimation using deep learning
US11312581B2 (en) * 2019-04-16 2022-04-26 Abb Schweiz Ag Object grasp system and method
US11185980B2 (en) * 2019-04-16 2021-11-30 Abb Schweiz Ag Machine learning-based systems and methods for controlling robotic object picking and placement in training and run time environments
EP3921801B1 (en) * 2019-04-18 2023-06-07 Alma Mater Studiorum - Università di Bologna Creating training data variability in machine learning for object labelling from images
JP7358994B2 (ja) * 2020-01-08 2023-10-11 オムロン株式会社 ロボット制御装置、ロボット制御システム、ロボット制御方法及びロボット制御プログラム
US11701777B2 (en) * 2020-04-03 2023-07-18 Fanuc Corporation Adaptive grasp planning for bin picking
CN112115812B (zh) * 2020-08-31 2025-03-14 深圳市联合视觉创新科技有限公司 人体运动信号的标注方法、装置及计算设备
US11654564B2 (en) * 2020-09-10 2023-05-23 Fanuc Corporation Efficient data generation for grasp learning with general grippers
JP7492440B2 (ja) * 2020-11-10 2024-05-29 株式会社日立製作所 ロボット制御システム、ロボット制御方法及びプログラム
CN112720459B (zh) * 2020-12-02 2022-07-12 达闼机器人股份有限公司 目标物体抓取方法、装置、存储介质及电子设备
US12072419B2 (en) * 2020-12-17 2024-08-27 Gm Cruise Holdings Llc Procedurally generated three-dimensional environment for use in autonomous vehicle simulations
DE102021202759B4 (de) * 2021-03-22 2024-11-07 Robert Bosch Gesellschaft mit beschränkter Haftung Vorrichtung und Verfahren zum Trainieren eines maschinellen Lernmodells zum Steuern eines Roboters
WO2022251881A2 (en) * 2021-05-27 2022-12-01 Ambi Robotics, Inc. System and method for planning and adapting to object manipulation by a robotic system
US12017355B2 (en) * 2021-06-08 2024-06-25 Fanuc Corporation Grasp learning using modularized neural networks
EP4112238A1 (de) 2021-06-28 2023-01-04 Bayer Aktiengesellschaft Autonomes manipulieren einer biegeschlaffen primärverpackung in einer formstabilen sekundärverpackung mittels roboter
US20240386606A1 (en) * 2021-09-15 2024-11-21 Yamaha Hatsudoki Kabushiki Kaisha Image processing device, component gripping system, image processing method and component gripping method
DE102021210993A1 (de) 2021-09-30 2023-03-30 Robert Bosch Gesellschaft mit beschränkter Haftung Verfahren zum Erzeugen von Trainingsdaten für überwachtes Lernen zum Trainieren eines neuronalen Netzes
KR102756075B1 (ko) * 2021-11-23 2025-01-21 주식회사 노비텍 광환경 변화를 고려한 피킹 로봇용 비젼 분석 장치
CN114241131A (zh) * 2021-12-13 2022-03-25 深圳鹏行智能研究有限公司 机器人抓取数据集的构建方法及其构建装置和存储介质
JP2025528839A (ja) * 2022-08-17 2025-09-02 アンビ ロボティクス, インコーポレイテッド ロボットパッケージ取扱システムおよび方法
JP7745854B2 (ja) * 2023-03-28 2025-09-30 Pciソリューションズ株式会社 重量推定装置及び教師データ作成方法
CN116188883B (zh) * 2023-04-28 2023-08-29 中国科学技术大学 一种抓握位置分析方法及终端
CN116460848B (zh) * 2023-04-28 2025-11-21 西安交通大学 基于机械手指尖三维力信息的目标属性识别方法及系统
WO2025038086A1 (en) * 2023-08-16 2025-02-20 Siemens Corporation Closed-loop data generation for fine-tuning grasp neural networks
US12600031B2 (en) 2023-10-20 2026-04-14 Tacta Systems Inc. Tactile robotic training platform
KR102952086B1 (ko) * 2023-12-07 2026-04-14 한국전자기술연구원 운용환경에 따른 로봇 파지점 추출 알고리즘 학습 방법 및 시스템
US20260028224A1 (en) 2024-07-29 2026-01-29 Tacta Systems Inc. Embedded Digital Sensor Structure
US12602110B2 (en) 2024-08-09 2026-04-14 Tacta Systems Inc. Sensing gloves for performing tasks
CN121132591B (zh) * 2025-11-19 2026-02-24 成都航天凯特机电科技有限公司 一种基于pid的伺服驱动控制方法和系统

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150127150A1 (en) * 2013-11-01 2015-05-07 Brain Corporation Apparatus and methods for haptic training of robots
CN105598965A (zh) * 2015-11-26 2016-05-25 哈尔滨工业大学 基于立体视觉的机器人欠驱动手自主抓取方法
US9393693B1 (en) * 2014-07-10 2016-07-19 Google Inc. Methods and systems for determining and modeling admissible gripper forces for robotic devices
CN106737692A (zh) * 2017-02-10 2017-05-31 杭州迦智科技有限公司 一种基于深度投影的机械手爪抓取规划方法及控制装置

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3583450B2 (ja) * 1993-09-28 2004-11-04 川崎重工業株式会社 3次元物体の自動検査装置および方法
US8924021B2 (en) 2006-04-27 2014-12-30 Honda Motor Co., Ltd. Control of robots from human motion descriptors
FI20105732A0 (fi) 2010-06-24 2010-06-24 Zenrobotics Oy Menetelmä fyysisten kappaleiden valitsemiseksi robottijärjestelmässä
JP5623358B2 (ja) * 2011-09-06 2014-11-12 三菱電機株式会社 ワーク取り出し装置
US9248611B2 (en) * 2013-10-07 2016-02-02 David A. Divine 3-D printed packaging
US9321176B1 (en) * 2014-04-01 2016-04-26 University Of South Florida Systems and methods for planning a robot grasp based upon a demonstrated grasp
JP6632208B2 (ja) * 2015-03-24 2020-01-22 キヤノン株式会社 情報処理装置、情報処理方法、プログラム
JP6541397B2 (ja) * 2015-04-06 2019-07-10 キヤノン株式会社 情報処理装置、情報処理方法、およびプログラム
JP6522488B2 (ja) * 2015-07-31 2019-05-29 ファナック株式会社 ワークの取り出し動作を学習する機械学習装置、ロボットシステムおよび機械学習方法
DE102016015936B8 (de) * 2015-07-31 2024-10-24 Fanuc Corporation Vorrichtung für maschinelles Lernen, Robotersystem und maschinelles Lernsystem zum Lernen eines Werkstückaufnahmevorgangs
EP3414710B1 (en) * 2016-03-03 2022-11-02 Google LLC Deep machine learning methods and apparatus for robotic grasping
US10166676B1 (en) * 2016-06-08 2019-01-01 X Development Llc Kinesthetic teaching of grasp parameters for grasping of objects by a grasping end effector of a robot
US10967507B2 (en) * 2018-05-02 2021-04-06 X Development Llc Positioning a robot sensor for object classification

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150127150A1 (en) * 2013-11-01 2015-05-07 Brain Corporation Apparatus and methods for haptic training of robots
US9393693B1 (en) * 2014-07-10 2016-07-19 Google Inc. Methods and systems for determining and modeling admissible gripper forces for robotic devices
CN105598965A (zh) * 2015-11-26 2016-05-25 哈尔滨工业大学 基于立体视觉的机器人欠驱动手自主抓取方法
CN106737692A (zh) * 2017-02-10 2017-05-31 杭州迦智科技有限公司 一种基于深度投影的机械手爪抓取规划方法及控制装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
JEFFREY MAHLER 等: "Dex-Net 2.0:Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics" *

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111906781A (zh) * 2020-07-08 2020-11-10 西安交通大学 基于图神经网络的机器人自主工具构建方法、系统及相关设备
CN111906781B (zh) * 2020-07-08 2021-07-13 西安交通大学 基于图神经网络的机器人自主工具构建方法、系统及相关设备
CN114603599A (zh) * 2020-12-08 2022-06-10 山东新松工业软件研究院股份有限公司 一种机器人碰撞检测方法、装置、计算机设备及存储介质
CN113787521A (zh) * 2021-09-24 2021-12-14 上海微电机研究所(中国电子科技集团公司第二十一研究所) 基于深度学习的机器人抓取方法、系统、介质和电子设备
CN116416444A (zh) * 2021-12-29 2023-07-11 广东美的白色家电技术创新中心有限公司 物体抓取点估计、模型训练及数据生成方法、装置及系统
CN114952836A (zh) * 2022-05-19 2022-08-30 中国科学院自动化研究所 多指手机器人抓取方法、装置与机器人系统
CN114986519A (zh) * 2022-07-21 2022-09-02 东南大学 一种基于深度强化学习的混杂场景机械臂精细抓取方法
CN118811238A (zh) * 2024-09-14 2024-10-22 启东泽厚智能科技有限公司 贴标机取料输送用夹爪自适应控制方法及装置
CN121535759A (zh) * 2026-01-19 2026-02-17 成都航天凯特机电科技有限公司 一种高扭矩密度机器人关节模组的控制方法和系统
CN121535759B (zh) * 2026-01-19 2026-04-24 成都航天凯特机电科技有限公司 一种高扭矩密度机器人关节模组的控制方法和系统

Also Published As

Publication number Publication date
CA3073516A1 (en) 2019-03-07
WO2019045779A1 (en) 2019-03-07
EP3676766A4 (en) 2021-05-26
EP3676766A1 (en) 2020-07-08
US20200198130A1 (en) 2020-06-25
US11446816B2 (en) 2022-09-20
JP2020532440A (ja) 2020-11-12
AU2018326171A1 (en) 2020-04-23

Similar Documents

Publication Publication Date Title
CN111226237A (zh) 用于稳健抓取和瞄准物体的机器人系统和方法
Mahler et al. Learning deep policies for robot bin picking by simulating robust grasping sequences
Kumra et al. Antipodal robotic grasping using generative residual convolutional neural network
D’Avella et al. A study on picking objects in cluttered environments: Exploiting depth features for a custom low-cost universal jamming gripper
Mahler et al. Dex-net 3.0: Computing robust vacuum suction grasp targets in point clouds using a new analytic model and deep learning
JP6586532B2 (ja) ロボットの把持のための深層機械学習方法および装置
Saxena et al. Robotic grasping of novel objects
JP5458885B2 (ja) 物体検出方法と物体検出装置およびロボットシステム
Sorokin et al. People helping robots helping people: Crowdsourcing for grasping novel objects
JP2020532440A5 (https=)
Suzuki et al. Grasping of unknown objects on a planar surface using a single depth image
Hossain et al. Pick-place of dynamic objects by robot manipulator based on deep learning and easy user interface teaching systems
JP2022515238A (ja) 空の容器の検出
Dyrstad et al. Grasping virtual fish: A step towards robotic deep learning from demonstration in virtual reality
Eppner et al. Visual detection of opportunities to exploit contact in grasping using contextual multi-armed bandits
WO2023150238A1 (en) Object placement
US10933526B2 (en) Method and robotic system for manipulating instruments
Haidacher et al. Estimating finger contact location and object pose from contact measurements in 3d grasping
CN116018240A (zh) 信息处理装置、信息处理方法和计算机程序
Hasan et al. Model-free, vision-based object identification and contact force estimation with a hyper-adaptive robotic gripper
Chen et al. Benchmarking multi-object grasping
Kyprianou et al. Bin-picking in the industry 4.0 era
Sefat et al. Robotic grasping in agile production
Pinzón-Arenas et al. Path Planning for Robotic Training in Virtual Environments using Deep Learning
Bekiroglu et al. Learning grasp stability based on haptic data

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20200602

WD01 Invention patent application deemed withdrawn after publication