JP7023450B2 - カメラによって生成される画像情報に基づいて、動作計画を実行する方法および計算システム - Google Patents

カメラによって生成される画像情報に基づいて、動作計画を実行する方法および計算システム Download PDF

Info

Publication number
JP7023450B2
JP7023450B2 JP2020569197A JP2020569197A JP7023450B2 JP 7023450 B2 JP7023450 B2 JP 7023450B2 JP 2020569197 A JP2020569197 A JP 2020569197A JP 2020569197 A JP2020569197 A JP 2020569197A JP 7023450 B2 JP7023450 B2 JP 7023450B2
Authority
JP
Japan
Prior art keywords
camera
image information
estimation
object structure
end effector
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2020569197A
Other languages
English (en)
Japanese (ja)
Other versions
JP2022503406A (ja
Inventor
ヨウ，シュタオ
ラートクンターノン，プティチャイ
ニコラエフ デアンコウ，ロセン
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Mujin Inc
Original Assignee
Mujin Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mujin Inc filed Critical Mujin Inc
Publication of JP2022503406A publication Critical patent/JP2022503406A/ja
Priority to JP2022012853A priority Critical patent/JP7624663B2/ja
Application granted granted Critical
Publication of JP7023450B2 publication Critical patent/JP7023450B2/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Program-controlled manipulators
    • B25J9/16—Program controls
    • B25J9/1694—Program controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
    • B25J9/1697—Vision controlled systems
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J13/00—Controls for manipulators
    • B25J13/08—Controls for manipulators by means of sensing devices, e.g. viewing or touching devices
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J15/00—Gripping heads and other end effectors
    • B25J15/0052—Gripping heads and other end effectors multiple gripper units or multiple end effectors
    • B25J15/0061—Gripping heads and other end effectors multiple gripper units or multiple end effectors mounted on a modular gripping structure
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J19/00—Accessories fitted to manipulators, e.g. for monitoring, for viewing; Safety devices combined with or specially adapted for use in connection with manipulators
    • B25J19/02—Sensing devices
    • B25J19/021—Optical sensing devices
    • B25J19/023—Optical sensing devices including video camera means
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Program-controlled manipulators
    • B25J9/16—Program controls
    • B25J9/1602—Program controls characterised by the control system, structure, architecture
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Program-controlled manipulators
    • B25J9/16—Program controls
    • B25J9/1612—Program controls characterised by the hand, wrist, grip control
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Program-controlled manipulators
    • B25J9/16—Program controls
    • B25J9/1628—Program controls characterised by the control loop
    • B25J9/1653—Program controls characterised by the control loop parameters identification, estimation, stiffness, accuracy, error analysis
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Program-controlled manipulators
    • B25J9/16—Program controls
    • B25J9/1656—Program controls characterised by programming, planning systems for manipulators
    • B25J9/1664—Program controls characterised by programming, planning systems for manipulators characterised by motion, path, trajectory planning
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Program-controlled manipulators
    • B25J9/16—Program controls
    • B25J9/1656—Program controls characterised by programming, planning systems for manipulators
    • B25J9/1669—Program controls characterised by programming, planning systems for manipulators characterised by special application, e.g. multi-arm co-operation, assembly, grasping
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Program-controlled manipulators
    • B25J9/16—Program controls
    • B25J9/1656—Program controls characterised by programming, planning systems for manipulators
    • B25J9/1671—Program controls characterised by programming, planning systems for manipulators characterised by simulation, either to verify existing program or to create and verify new program, CAD/CAM oriented, graphic oriented programming systems
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
    • B65G—TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS
    • B65G59/00—De-stacking of articles
    • B65G59/02—De-stacking from the top of the stack
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B19/00—Program-control systems
    • G05B19/02—Program-control systems electric
    • G05B19/18—Numerical control [NC], i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of program data in numerical form
    • G05B19/4155—Numerical control [NC], i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of program data in numerical form characterised by program execution, i.e. part program or machine function execution, e.g. selection of a program
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G06T7/74—Determining position or orientation of objects or cameras using feature-based methods involving reference images or patches
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/50—Constructional details
    • H04N23/54—Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/695—Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/30—Nc systems
    • G05B2219/39—Robotics, robotics to robotics hand
    • G05B2219/39509—Gripping, grasping, links embrace, encircle, envelop object to grasp
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/30—Nc systems
    • G05B2219/39—Robotics, robotics to robotics hand
    • G05B2219/39527—Workpiece detector, sensor mounted in, near hand, gripper
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/30—Nc systems
    • G05B2219/39—Robotics, robotics to robotics hand
    • G05B2219/39542—Plan grasp points, grip matrix and initial grasp force
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/30—Nc systems
    • G05B2219/40—Robotics, robotics mapping to robotics vision
    • G05B2219/40014—Gripping workpiece to place it in another place
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/30—Nc systems
    • G05B2219/40—Robotics, robotics mapping to robotics vision
    • G05B2219/40269—Naturally compliant robot arm
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/30—Nc systems
    • G05B2219/40—Robotics, robotics mapping to robotics vision
    • G05B2219/40584—Camera, non-contact sensor mounted on wrist, indep from gripper
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20112—Image segmentation details
    • G06T2207/20164—Salient point detection; Corner detection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30244—Camera pose

Landscapes

  • Engineering & Computer Science (AREA)
  • Mechanical Engineering (AREA)
  • Robotics (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Signal Processing (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Data Mining & Analysis (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Automation & Control Theory (AREA)
  • Computing Systems (AREA)
  • Medical Informatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Software Systems (AREA)
  • Databases & Information Systems (AREA)
  • Manufacturing & Machinery (AREA)
  • Geometry (AREA)
  • Orthopedic Medicine & Surgery (AREA)
  • Manipulator (AREA)
  • Image Analysis (AREA)
  • Numerical Control (AREA)
JP2020569197A 2019-12-12 2020-10-29 カメラによって生成される画像情報に基づいて、動作計画を実行する方法および計算システム Active JP7023450B2 (ja)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2022012853A JP7624663B2 (ja) 2019-12-12 2022-01-31 カメラによって生成される画像情報に基づいて、動作計画を実行する方法および計算システム

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US201962946973P 2019-12-12 2019-12-12
US62/946,973 2019-12-12
PCT/US2020/058008 WO2021118702A1 (en) 2019-12-12 2020-10-29 Method and computing system for performing motion planning based on image information generated by a camera

Related Child Applications (1)

Application Number Title Priority Date Filing Date
JP2022012853A Division JP7624663B2 (ja) 2019-12-12 2022-01-31 カメラによって生成される画像情報に基づいて、動作計画を実行する方法および計算システム

Publications (2)

Publication Number Publication Date
JP2022503406A JP2022503406A (ja) 2022-01-12
JP7023450B2 true JP7023450B2 (ja) 2022-02-22

Family

ID=76316260

Family Applications (4)

Application Number Title Priority Date Filing Date
JP2020569197A Active JP7023450B2 (ja) 2019-12-12 2020-10-29 カメラによって生成される画像情報に基づいて、動作計画を実行する方法および計算システム
JP2021505688A Active JP7015031B2 (ja) 2019-12-12 2020-12-09 カメラによって生成される画像情報に基づいて、物体検出またはロボット相互作用計画作成を行う方法および計算システム
JP2022003304A Active JP7586436B2 (ja) 2019-12-12 2022-01-12 カメラによって生成される画像情報に基づいて、物体検出またはロボット相互作用計画作成を行う方法および計算システム
JP2022012853A Active JP7624663B2 (ja) 2019-12-12 2022-01-31 カメラによって生成される画像情報に基づいて、動作計画を実行する方法および計算システム

Family Applications After (3)

Application Number Title Priority Date Filing Date
JP2021505688A Active JP7015031B2 (ja) 2019-12-12 2020-12-09 カメラによって生成される画像情報に基づいて、物体検出またはロボット相互作用計画作成を行う方法および計算システム
JP2022003304A Active JP7586436B2 (ja) 2019-12-12 2022-01-12 カメラによって生成される画像情報に基づいて、物体検出またはロボット相互作用計画作成を行う方法および計算システム
JP2022012853A Active JP7624663B2 (ja) 2019-12-12 2022-01-31 カメラによって生成される画像情報に基づいて、動作計画を実行する方法および計算システム

Country Status (4)

Country Link
US (5) US11103998B2 (cg-RX-API-DMAC10.html)
JP (4) JP7023450B2 (cg-RX-API-DMAC10.html)
CN (2) CN113272106A (cg-RX-API-DMAC10.html)
WO (2) WO2021118702A1 (cg-RX-API-DMAC10.html)

Families Citing this family (45)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9102055B1 (en) * 2013-03-15 2015-08-11 Industrial Perception, Inc. Detection and reconstruction of an environment to facilitate robotic interaction with the environment
CN113894826B (zh) 2016-02-08 2025-03-11 伯克希尔格雷运营股份有限公司 用于提供使用运动规划的各种物体的处理的系统和方法
US10894676B2 (en) * 2017-07-17 2021-01-19 Symbolic Llc Apparatus and method for building a pallet load
CN107571273B (zh) * 2017-10-17 2020-06-16 清华大学 一种大口径光学元件精密洁净装配的自动化抓取装置与方法
US10614340B1 (en) * 2019-09-23 2020-04-07 Mujin, Inc. Method and computing system for object identification
EP4069215B1 (en) 2019-12-06 2026-07-01 Jlabs Beauty LLC Topical compositions containing rose oil and cannabidiol, and processes for their preparation and use
CN115552476A (zh) 2020-02-06 2022-12-30 伯克希尔格雷营业股份有限公司 用于利用可编程运动装置的铰接臂上的位置未知的基准物的照相机校准的系统和方法
JP6796901B1 (ja) * 2020-03-05 2020-12-09 株式会社Ｍｕｊｉｎ 容器検出および物体検出を行うための方法ならびに計算システム
CN113264302A (zh) * 2020-06-12 2021-08-17 深圳市海柔创新科技有限公司 仓储机器人的控制方法、装置、机器人和仓储系统
US12390931B2 (en) * 2020-07-15 2025-08-19 Duke University Autonomous robot packaging of arbitrary objects
US12626379B1 (en) * 2021-06-17 2026-05-12 Apple Inc. Modeling an environment based on limited data
JP7775582B2 (ja) * 2021-06-22 2025-11-26 コニカミノルタ株式会社 対象物認識装置およびプログラム
JP7417882B2 (ja) * 2021-08-09 2024-01-19 株式会社Mujin 計算システム、方法及び非一時的コンピュータ可読媒体
CN116071742A (zh) * 2021-08-09 2023-05-05 牧今科技 用于物体检测的系统和方法
US12538008B2 (en) * 2021-08-30 2026-01-27 Nec Corporation Determination device, identification method and recording medium
US11986955B2 (en) * 2021-09-09 2024-05-21 Intrinsic Innovation Llc In-hand pose refinement for pick and place automation
CN114030843B (zh) * 2021-10-27 2022-11-18 因格(苏州)智能技术有限公司 物品流转方法与系统
EP4444512A1 (en) * 2021-12-10 2024-10-16 Boston Dynamics, Inc. Systems and methods for object detection and pick order determination
TWI853201B (zh) * 2021-12-21 2024-08-21 財團法人工業技術研究院 調整機構及取放系統
EP4212288A1 (en) 2022-01-18 2023-07-19 Robert Bosch GmbH Device and method for controlling a robot
WO2023150238A1 (en) * 2022-02-02 2023-08-10 Intrinsic Innovation Llc Object placement
CN114157813B (zh) * 2022-02-07 2022-05-03 深圳市慧为智能科技股份有限公司 电子秤摄像头运动控制方法、装置、控制终端及存储介质
US12343884B2 (en) * 2022-09-21 2025-07-01 Accenture Global Solutions Limited Robotic assembly instruction generation from a video
EP4627532A4 (en) * 2022-12-04 2026-03-25 Equashield Medical Ltd PHARMACEUTICAL PREPARATION DEVICE USING IMAGE-ENHANCED PREPARATION PROCESS INGREDIENT POSITIONING
US20240371127A1 (en) * 2023-05-07 2024-11-07 Plus One Robotics, Inc. Machine vision systems and methods for robotic picking and other environments
JP7878158B2 (ja) * 2023-05-26 2026-06-23 トヨタ自動車株式会社 ピッキング装置
CN116713993A (zh) * 2023-06-14 2023-09-08 极限人工智能(北京)有限公司 一种六轴串联机器人d-h参数标定方法及系统
EP4488012A1 (en) 2023-07-02 2025-01-08 Mujin, Inc. Robotic system and method with object detection
US12521894B1 (en) * 2023-09-20 2026-01-13 Agility Robotics, Inc. Generating references for robot-carried objects and related technology
US20250104323A1 (en) * 2023-09-22 2025-03-27 Proprio, Inc. METHODS AND SYSTEMS FOR GENERATING THREE-DIMENSIONAL RENDERINGS OF A SCENE USING A MOBILE SENSOR ARRAY, SUCH AS NEURAL RADIANCE FIELD (NeRF) RENDERINGS
US12682478B1 (en) * 2023-10-05 2026-07-14 Brunswick Corporation Systems and methods for confirming dimensions of a marine vessel using a vision system on the marine vessel
CN117021121B (zh) * 2023-10-09 2024-06-14 浪潮(山东)计算机科技有限公司 一种基于机器视觉的内存条自动安装设备及方法
CN117067218B (zh) * 2023-10-13 2024-04-05 宁德时代新能源科技股份有限公司 电芯抓取系统及其控制方法、产线模块
KR102694987B1 (ko) * 2023-12-20 2024-08-16 주식회사 플라잎 인공지능을 이용하여 객체에 대한 로봇의 엔드 이펙터의 포즈를 추정하는 포즈 추정 장치 및 방법과 이를 포함하는 로봇 제어 시스템
US20250249577A1 (en) * 2024-02-07 2025-08-07 Logistics and Supply Chain MultiTech R&D Centre Limited A learning assisted robotic system, a learning assisted method and a gripper subassembly
US12636777B1 (en) 2024-05-15 2026-05-26 Agility Robotics, Inc. Object placing via a mobile robot including nudging and related technology
US20250370476A1 (en) * 2024-05-31 2025-12-04 Boston Dynamics, Inc. Methods and apparatus for object quality detection
CN118809616B (zh) * 2024-08-23 2024-12-17 海信集团控股股份有限公司 一种机器人的运动规划方法、装置、机器人及存储介质
US12673422B2 (en) * 2024-08-06 2026-07-07 GM Global Technology Operations LLC Systems and methods for manufacturing with an end effector having two or more independent tools
US20260115936A1 (en) * 2024-10-28 2026-04-30 Sony Group Corporation Enhanced end effector
WO2026095128A1 (ko) * 2024-10-31 2026-05-07 엘지전자 주식회사 인공 지능 장치, 그의 동작 방법 및 기록 매체
CN120107342A (zh) * 2025-01-27 2025-06-06 未来机器人（深圳）有限公司 一种对齐状态确认方法、控制系统及搬运设备
CN120107344A (zh) * 2025-01-27 2025-06-06 未来机器人（深圳）有限公司 一种对齐状态确认方法、控制系统及搬运设备
US12637339B1 (en) * 2025-04-28 2026-05-26 Visionnav Robotics Usa Inc. Material handling equipment, controller, and method of detecting handling object
CN121071706B (zh) * 2025-11-07 2026-06-26 海南大学三亚南繁研究院 一种丘陵山地天然橡胶园智慧巡检方法及装置

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010127721A (ja) 2008-11-26 2010-06-10 Toyota Motor Corp 対象物認識装置および対象物認識方法
JP2013255972A (ja) 2012-06-14 2013-12-26 Shinnichi Kogyo Co Ltd ワーク搬送装置およびその制御方法
JP2014034075A (ja) 2012-08-08 2014-02-24 Canon Inc ロボット装置
JP2016222377A (ja) 2015-05-28 2016-12-28 株式会社東芝 荷役装置及びその動作方法
JP2019063984A (ja) 2017-10-02 2019-04-25 キヤノン株式会社 情報処理装置、方法及びロボットシステム
JP2019151421A (ja) 2018-02-28 2019-09-12 株式会社エヌテック 物品移載装置及び荷取位置検出装置
JP2019185239A (ja) 2018-04-05 2019-10-24 オムロン株式会社 物体認識処理装置及び方法、並びに、物体ピッキング装置及び方法

Family Cites Families (32)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0671088U (ja) * 1993-03-18 1994-10-04 株式会社小松製作所 物流品収納ケースの搬送装置
JP3442140B2 (ja) 1994-04-20 2003-09-02 ファナック株式会社 ３次元視覚センサを用いた位置計測装置及び位置ずれ補正装置
JP3101674B2 (ja) * 1995-12-15 2000-10-23 岡山大学長 Ｃａｄ情報を用いた３次元認識手法及び装置
US10555775B2 (en) * 2005-05-16 2020-02-11 Intuitive Surgical Operations, Inc. Methods and system for performing 3-D tool tracking by fusion of sensor and/or camera derived data during minimally invasive robotic surgery
DE102007060653A1 (de) * 2007-12-15 2009-06-18 Abb Ag Positionsermittlung eines Objektes
JP4565023B2 (ja) * 2008-07-04 2010-10-20 ファナック株式会社 物品取り出し装置
US8923602B2 (en) * 2008-07-22 2014-12-30 Comau, Inc. Automated guidance and recognition system and method of the same
KR20110095700A (ko) * 2010-02-19 2011-08-25 현대중공업 주식회사 작업대상물 픽업을 위한 산업용 로봇 제어방법
US8912492B2 (en) * 2010-10-13 2014-12-16 Lasermax, Inc. Thermal marking systems and methods of control
US9102055B1 (en) * 2013-03-15 2015-08-11 Industrial Perception, Inc. Detection and reconstruction of an environment to facilitate robotic interaction with the environment
KR102016551B1 (ko) 2014-01-24 2019-09-02 한화디펜스 주식회사 위치 추정 장치 및 방법
US9272417B2 (en) * 2014-07-16 2016-03-01 Google Inc. Real-time determination of object metrics for trajectory planning
US9498887B1 (en) * 2014-07-24 2016-11-22 X Development Llc Two-faced linearly actuated gripper
US9327406B1 (en) * 2014-08-19 2016-05-03 Google Inc. Object segmentation based on detected object-specific visual cues
US9507995B2 (en) * 2014-08-29 2016-11-29 X Development Llc Combination of stereo and structured-light processing
US9694498B2 (en) * 2015-03-30 2017-07-04 X Development Llc Imager for detecting visual light and projected patterns
WO2016205074A1 (en) * 2015-06-15 2016-12-22 United States Postal Service Dimensional measuring system
JP6711591B2 (ja) * 2015-11-06 2020-06-17 キヤノン株式会社 ロボット制御装置およびロボット制御方法
EP3192616B1 (en) 2016-01-14 2024-09-25 Jungheinrich AG Robot to pick up and transport objects and method using such a robot
US10596700B2 (en) * 2016-09-16 2020-03-24 Carbon Robotics, Inc. System and calibration, registration, and training methods
WO2018146769A1 (ja) * 2017-02-09 2018-08-16 三菱電機株式会社 位置制御装置及び位置制御方法
US10269108B2 (en) 2017-09-01 2019-04-23 Midea Group Co., Ltd. Methods and systems for improved quality inspection of products using a robot
EP3701271B1 (en) 2017-10-27 2025-05-21 Beckman Coulter, Inc. Sample container recognition
GB201719058D0 (en) * 2017-11-17 2018-01-03 Ocado Innovation Ltd Control device and method for a robot system
CN110292558A (zh) 2018-03-21 2019-10-01 安徽深蓝医疗科技股份有限公司 一种冷敷冷疗凝胶及其制备方法
JP7157547B2 (ja) * 2018-04-20 2022-10-20 株式会社キーエンス 形状測定装置、形状測定方法及び形状測定プログラム
US20200094401A1 (en) * 2018-09-21 2020-03-26 Beijing Jingdong Shangke Information Technology Co., Ltd. System and method for automatic learning of product manipulation
JP6511681B1 (ja) * 2018-10-15 2019-05-15 株式会社Ｍｕｊｉｎ 形状情報生成装置、制御装置、積み降ろし装置、物流システム、プログラム、及び、制御方法
CN110405810B (zh) * 2018-10-30 2021-05-04 牧今科技 自动包裹配准系统、装置和方法
US10456915B1 (en) 2019-01-25 2019-10-29 Mujin, Inc. Robotic system with enhanced scanning mechanism
US10510155B1 (en) 2019-06-11 2019-12-17 Mujin, Inc. Method and processing system for updating a first image generated by a first camera based on a second image generated by a second camera
US10565737B1 (en) 2019-07-09 2020-02-18 Mujin, Inc. Method and system for performing automatic camera calibration for a scanning system

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010127721A (ja) 2008-11-26 2010-06-10 Toyota Motor Corp 対象物認識装置および対象物認識方法
JP2013255972A (ja) 2012-06-14 2013-12-26 Shinnichi Kogyo Co Ltd ワーク搬送装置およびその制御方法
JP2014034075A (ja) 2012-08-08 2014-02-24 Canon Inc ロボット装置
JP2016222377A (ja) 2015-05-28 2016-12-28 株式会社東芝 荷役装置及びその動作方法
JP2019063984A (ja) 2017-10-02 2019-04-25 キヤノン株式会社 情報処理装置、方法及びロボットシステム
JP2019151421A (ja) 2018-02-28 2019-09-12 株式会社エヌテック 物品移載装置及び荷取位置検出装置
JP2019185239A (ja) 2018-04-05 2019-10-24 オムロン株式会社 物体認識処理装置及び方法、並びに、物体ピッキング装置及び方法

Also Published As

Publication number Publication date
JP2022504009A (ja) 2022-01-13
JP2022503406A (ja) 2022-01-12
WO2021118702A1 (en) 2021-06-17
US20210178593A1 (en) 2021-06-17
JP2022044830A (ja) 2022-03-17
US11717971B2 (en) 2023-08-08
JP7624663B2 (ja) 2025-01-31
US11103998B2 (en) 2021-08-31
CN113365786A (zh) 2021-09-07
US20210178583A1 (en) 2021-06-17
JP7015031B2 (ja) 2022-02-02
US20240017417A1 (en) 2024-01-18
CN113272106A (zh) 2021-08-17
US12138815B2 (en) 2024-11-12
JP2022036243A (ja) 2022-03-04
JP7586436B2 (ja) 2024-11-19
WO2021119083A1 (en) 2021-06-17
US20240165820A1 (en) 2024-05-23
US11883966B2 (en) 2024-01-30
US20210347051A1 (en) 2021-11-11

Similar Documents

Publication Publication Date Title
JP7624663B2 (ja) カメラによって生成される画像情報に基づいて、動作計画を実行する方法および計算システム
JP7433609B2 (ja) 物体識別のための方法および計算システム
JP7398662B2 (ja) ロボット多面グリッパアセンブリ及びその操作方法
JP6805465B2 (ja) センサ誘導式ロボットを用いたボックスの位置特定、分離、およびピッキング
KR101902678B1 (ko) 궤적 계획을 위한 물체 메트릭들의 실시간 결정
US10870204B2 (en) Robotic system control method and controller
JP6945209B1 (ja) 物体検出のために安全ボリュームリストを生成するための方法および計算システム
JP7272568B2 (ja) ロボット運動計画およびリポジトリ検出を実行するための方法および計算システム
CN113043282B (zh) 用于物体检测或机器人交互规划的方法和系统
CN119546423A (zh) 用于机器人箱中取物的箱壁碰撞检测
EP4309858B1 (en) METHODS, SYSTEMS AND COMPUTER PRODUCTS FOR MANIPULATING ACCESSIBILITY CONSTRAINTS FOR HEIGHT THRESHOLD SCENARIOS IN ROBOTIC DEPALETTIZATION
CN113219900B (zh) 基于相机生成的图像信息执行运动规划的方法和计算系统
Pedrosa et al. A skill-based architecture for pick and place manipulation tasks
JP2024094664A (ja) 物体検出装置、ロボットシステム及び物体検出方法
WO2025154234A1 (ja) 物体認識装置、物体認識方法、プログラム
CN121661281A (zh) 用于简化描绘环境的点云的方法

Legal Events

Date Code Title Description
A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20210212

A871 Explanation of circumstances concerning accelerated examination

Free format text: JAPANESE INTERMEDIATE CODE: A871

Effective date: 20210212

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20211126

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20211222

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20220106

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20220131

R150 Certificate of patent or registration of utility model

Ref document number: 7023450

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150