JP6796901B1 - 容器検出および物体検出を行うための方法ならびに計算システム - Google Patents

容器検出および物体検出を行うための方法ならびに計算システム Download PDF

Info

Publication number
JP6796901B1
JP6796901B1 JP2020150960A JP2020150960A JP6796901B1 JP 6796901 B1 JP6796901 B1 JP 6796901B1 JP 2020150960 A JP2020150960 A JP 2020150960A JP 2020150960 A JP2020150960 A JP 2020150960A JP 6796901 B1 JP6796901 B1 JP 6796901B1
Authority
JP
Japan
Prior art keywords
container
spatial structure
orientation
information
camera
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2020150960A
Other languages
English (en)
Japanese (ja)
Other versions
JP2021139879A (ja
Inventor
ヨウ，シュタオ
和都 村瀬
和都 村瀬
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Mujin Inc
Original Assignee
Mujin Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US16/929,854 external-priority patent/US11130237B1/en
Application filed by Mujin Inc filed Critical Mujin Inc
Priority to JP2020186970A priority Critical patent/JP2021139882A/ja
Application granted granted Critical
Publication of JP6796901B1 publication Critical patent/JP6796901B1/ja
Publication of JP2021139879A publication Critical patent/JP2021139879A/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
    • G06Q10/047—Optimisation of routes or paths, e.g. travelling salesman problem
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/16—Programme controls
    • B25J9/1694—Programme controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
    • B25J9/1697—Vision controlled systems
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J13/00—Controls for manipulators
    • B25J13/08—Controls for manipulators by means of sensing devices, e.g. viewing or touching devices
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J19/00—Accessories fitted to manipulators, e.g. for monitoring, for viewing; Safety devices combined with or specially adapted for use in connection with manipulators
    • B25J19/02—Sensing devices
    • B25J19/021—Optical sensing devices
    • B25J19/023—Optical sensing devices including video camera means
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/16—Programme controls
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/16—Programme controls
    • B25J9/1656—Programme controls characterised by programming, planning systems for manipulators
    • B25J9/1664—Programme controls characterised by programming, planning systems for manipulators characterised by motion, path, trajectory planning
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/16—Programme controls
    • B25J9/1656—Programme controls characterised by programming, planning systems for manipulators
    • B25J9/1664—Programme controls characterised by programming, planning systems for manipulators characterised by motion, path, trajectory planning
    • B25J9/1666—Avoiding collision or forbidden zones
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/16—Programme controls
    • B25J9/1679—Programme controls characterised by the tasks executed
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
    • G01S17/894—3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
    • G01S7/4808—Evaluating distance, position or velocity data
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/08—Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
    • G06Q10/087—Inventory or stock management, e.g. order filling, procurement or balancing against orders
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/695—Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
    • G01B11/25—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/30—Nc systems
    • G05B2219/37—Measurements
    • G05B2219/37555—Camera detects orientation, position workpiece, points of workpiece
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/30—Nc systems
    • G05B2219/45—Nc applications
    • G05B2219/45063—Pick and place manipulator
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20021—Dividing image into blocks, subimages or windows
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30204—Marker
    • G06T2207/30208—Marker matrix
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30241—Trajectory

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Robotics (AREA)
  • Mechanical Engineering (AREA)
  • General Physics & Mathematics (AREA)
  • Business, Economics & Management (AREA)
  • Theoretical Computer Science (AREA)
  • Human Resources & Organizations (AREA)
  • Economics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Strategic Management (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Quality & Reliability (AREA)
  • General Business, Economics & Management (AREA)
  • Tourism & Hospitality (AREA)
  • Development Economics (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Electromagnetism (AREA)
  • Marketing (AREA)
  • Operations Research (AREA)
  • Signal Processing (AREA)
  • Human Computer Interaction (AREA)
  • Finance (AREA)
  • Accounting & Taxation (AREA)
  • Game Theory and Decision Science (AREA)
  • Manipulator (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Image Analysis (AREA)
JP2020150960A 2020-03-05 2020-09-09 容器検出および物体検出を行うための方法ならびに計算システム Active JP6796901B1 (ja)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2020186970A JP2021139882A (ja) 2020-03-05 2020-11-10 容器検出および物体検出を行うための方法ならびに計算システム

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US202062985336P 2020-03-05 2020-03-05
US62/985,336 2020-03-05
US16/929,854 2020-07-15
US16/929,854 US11130237B1 (en) 2020-03-05 2020-07-15 Method and computing system for performing container detection and object detection

Related Child Applications (1)

Application Number Title Priority Date Filing Date
JP2020186970A Division JP2021139882A (ja) 2020-03-05 2020-11-10 容器検出および物体検出を行うための方法ならびに計算システム

Publications (2)

Publication Number Publication Date
JP6796901B1 true JP6796901B1 (ja) 2020-12-09
JP2021139879A JP2021139879A (ja) 2021-09-16

Family

ID=73646911

Family Applications (2)

Application Number Title Priority Date Filing Date
JP2020150960A Active JP6796901B1 (ja) 2020-03-05 2020-09-09 容器検出および物体検出を行うための方法ならびに計算システム
JP2020186970A Pending JP2021139882A (ja) 2020-03-05 2020-11-10 容器検出および物体検出を行うための方法ならびに計算システム

Family Applications After (1)

Application Number Title Priority Date Filing Date
JP2020186970A Pending JP2021139882A (ja) 2020-03-05 2020-11-10 容器検出および物体検出を行うための方法ならびに計算システム

Country Status (4)

Country Link
US (1) US20240253237A1 (enExample)
JP (2) JP6796901B1 (enExample)
CN (1) CN113361740A (enExample)
DE (2) DE102020124221B4 (enExample)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023177476A3 (en) * 2022-03-18 2023-11-23 Hexagon Metrology, Inc. Recessed drawer opening apparatus

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12269164B2 (en) * 2021-05-04 2025-04-08 Mujin, Inc. Method and computing system for performing robot motion planning and repository detection
JP7757756B2 (ja) * 2021-12-09 2025-10-22 オムロン株式会社 画像処理装置、ロボット制御システムおよび制御方法

Family Cites Families (29)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS61206709A (ja) * 1985-03-07 1986-09-13 Motoda Electronics Co Ltd トレ−配送装置
JP4007204B2 (ja) * 2003-01-30 2007-11-14 松下電器産業株式会社 ロボットシステム
JP2006102881A (ja) * 2004-10-06 2006-04-20 Nagasaki Prefecture 把持ロボット装置
US7873549B1 (en) * 2006-03-27 2011-01-18 Amazon Technologies, Inc. Product dimension correction
JP5489000B2 (ja) * 2010-08-31 2014-05-14 株式会社安川電機 作業装置及び部品ピッキングシステム
JP5659787B2 (ja) * 2010-12-28 2015-01-28 トヨタ自動車株式会社 操作環境モデル構築システム、および操作環境モデル構築方法
JP5383836B2 (ja) 2012-02-03 2014-01-08 ファナック株式会社 検索ウィンドウを自動的に調整する機能を備えた画像処理装置
JP5469216B2 (ja) 2012-07-31 2014-04-16 ファナック株式会社 バラ積みされた物品をロボットで取出す装置
US9259844B2 (en) * 2014-02-12 2016-02-16 General Electric Company Vision-guided electromagnetic robotic system
CN112068526B (zh) * 2014-02-20 2024-05-17 M·奥利尼克 用于在机器人烹饪厨房中制备食物的方法和系统
US9824328B2 (en) * 2014-06-10 2017-11-21 Amazon Technologies, Inc. Arm-detecting overhead sensor for inventory system
US10518409B2 (en) * 2014-09-02 2019-12-31 Mark Oleynik Robotic manipulation methods and systems for executing a domain-specific application in an instrumented environment with electronic minimanipulation libraries
US9758301B2 (en) 2015-03-24 2017-09-12 Joseph Porat System and method for overhead warehousing
US9486921B1 (en) * 2015-03-26 2016-11-08 Google Inc. Methods and systems for distributing remote assistance to facilitate robotic object manipulation
JP6734728B2 (ja) * 2016-08-05 2020-08-05 株式会社日立製作所 ロボットシステム及びピッキング方法
JP2018161700A (ja) * 2017-03-24 2018-10-18 キヤノン株式会社 情報処理装置、システム、情報処理方法、製造方法
JP7045139B2 (ja) * 2017-06-05 2022-03-31 株式会社日立製作所 機械学習装置、機械学習方法、および機械学習プログラム
JP6923383B2 (ja) * 2017-07-27 2021-08-18 株式会社日立物流 ピッキングロボットおよびピッキングシステム
JP2019037349A (ja) * 2017-08-23 2019-03-14 ライフロボティクス株式会社 食器及びワーク
US10586418B2 (en) * 2017-09-29 2020-03-10 Omnicell, Inc. Apparatuses, systems, and methods for the automated dispensing of articles
JP7161857B2 (ja) * 2018-03-14 2022-10-27 オムロン株式会社 情報処理装置、情報処理方法、及びプログラム
US10740911B2 (en) * 2018-04-05 2020-08-11 Symbol Technologies, Llc Method, system and apparatus for correcting translucency artifacts in data representing a support structure
JP7057214B2 (ja) 2018-05-18 2022-04-19 トヨタ自動車株式会社 把持装置、タグが付された容器、対象物把持プログラムおよび対象物把持方法
US10369698B1 (en) 2019-03-07 2019-08-06 Mujin, Inc. Method and system for performing automatic camera calibration for robot control
JP2020168679A (ja) * 2019-04-03 2020-10-15 セイコーエプソン株式会社 ロボットシステム及び撮像方法
JP7204587B2 (ja) * 2019-06-17 2023-01-16 株式会社東芝 物体ハンドリング制御装置、物体ハンドリング装置、物体ハンドリング方法および物体ハンドリングプログラム
US12344488B2 (en) * 2019-10-16 2025-07-01 Symbotic Canada, Ulc Vision-assisted robotized depalletizer
WO2021118702A1 (en) * 2019-12-12 2021-06-17 Mujin, Inc. Method and computing system for performing motion planning based on image information generated by a camera
WO2023245208A1 (en) * 2022-06-17 2023-12-21 Wipro Limited Method and system for accurately counting items in a storehouse

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023177476A3 (en) * 2022-03-18 2023-11-23 Hexagon Metrology, Inc. Recessed drawer opening apparatus

Also Published As

Publication number Publication date
CN113361740A (zh) 2021-09-07
US20240253237A1 (en) 2024-08-01
JP2021139882A (ja) 2021-09-16
DE102020124221A1 (de) 2021-09-09
DE102021105152A1 (de) 2021-09-09
DE102020124221B4 (de) 2024-11-07
JP2021139879A (ja) 2021-09-16

Similar Documents

Publication Publication Date Title
JP7015031B2 (ja) カメラによって生成される画像情報に基づいて、物体検出またはロボット相互作用計画作成を行う方法および計算システム
JP6692107B1 (ja) 物体識別のための方法および計算システム
JP7671462B2 (ja) 高度化したスキャンメカニズムを有するロボットシステム
US11958202B2 (en) Method and computing system for performing container detection and object detection
KR102785170B1 (ko) 다중 카메라 이미지 처리
US20240253237A1 (en) Method and computing system for performing container detection and object detection
JP7495688B2 (ja) ロボットシステムの制御方法及び制御装置
JP6948033B1 (ja) グリップ領域検出を実施するための方法および計算システム
US20230052515A1 (en) System and method for robotic object placement
US20220375097A1 (en) Robotic system for object size detection
US20230286140A1 (en) Systems and methods for robotic system with object handling
US20240221350A1 (en) Method and computing system for generating a safety volume list for object detection
US12062182B2 (en) Method and computing system for performing or facilitating physical edge detection
CN113043282B (zh) 用于物体检测或机器人交互规划的方法和系统
CN112734783B (zh) 用于处理候选边缘的方法和计算系统
CN112288165B (zh) 用于执行容器检测和物体检测的方法和计算系统
CN113219900B (zh) 基于相机生成的图像信息执行运动规划的方法和计算系统
CN116061192A (zh) 具有物体处理的机器人系统的系统和方法

Legal Events

Date Code Title Description
A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20200924

A871 Explanation of circumstances concerning accelerated examination

Free format text: JAPANESE INTERMEDIATE CODE: A871

Effective date: 20200924

A975 Report on accelerated examination

Free format text: JAPANESE INTERMEDIATE CODE: A971005

Effective date: 20201012

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20201014

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20201110

R150 Certificate of patent or registration of utility model

Ref document number: 6796901

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150

S533 Written request for registration of change of name

Free format text: JAPANESE INTERMEDIATE CODE: R313533

R350 Written notification of registration of transfer

Free format text: JAPANESE INTERMEDIATE CODE: R350