CN113361740A - 用于执行容器检测和物体检测的方法和计算系统 - Google Patents

用于执行容器检测和物体检测的方法和计算系统 Download PDF

Info

Publication number
CN113361740A
CN113361740A CN202011045515.5A CN202011045515A CN113361740A CN 113361740 A CN113361740 A CN 113361740A CN 202011045515 A CN202011045515 A CN 202011045515A CN 113361740 A CN113361740 A CN 113361740A
Authority
CN
China
Prior art keywords
container
spatial structure
pose
computing system
information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011045515.5A
Other languages
English (en)
Chinese (zh)
Inventor
叶旭涛
村瀬和都
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Mu Jinkeji
Original Assignee
Mu Jinkeji
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US16/929,854 external-priority patent/US11130237B1/en
Application filed by Mu Jinkeji filed Critical Mu Jinkeji
Priority to CN202011186066.6A priority Critical patent/CN112288165B/zh
Publication of CN113361740A publication Critical patent/CN113361740A/zh
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
    • G06Q10/047—Optimisation of routes or paths, e.g. travelling salesman problem
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Program-controlled manipulators
    • B25J9/16—Program controls
    • B25J9/1694—Program controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
    • B25J9/1697—Vision controlled systems
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J13/00—Controls for manipulators
    • B25J13/08—Controls for manipulators by means of sensing devices, e.g. viewing or touching devices
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J19/00—Accessories fitted to manipulators, e.g. for monitoring, for viewing; Safety devices combined with or specially adapted for use in connection with manipulators
    • B25J19/02—Sensing devices
    • B25J19/021—Optical sensing devices
    • B25J19/023—Optical sensing devices including video camera means
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Program-controlled manipulators
    • B25J9/16—Program controls
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Program-controlled manipulators
    • B25J9/16—Program controls
    • B25J9/1656—Program controls characterised by programming, planning systems for manipulators
    • B25J9/1664—Program controls characterised by programming, planning systems for manipulators characterised by motion, path, trajectory planning
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Program-controlled manipulators
    • B25J9/16—Program controls
    • B25J9/1656—Program controls characterised by programming, planning systems for manipulators
    • B25J9/1664—Program controls characterised by programming, planning systems for manipulators characterised by motion, path, trajectory planning
    • B25J9/1666—Avoiding collision or forbidden zones
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Program-controlled manipulators
    • B25J9/16—Program controls
    • B25J9/1679—Program controls characterised by the tasks executed
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
    • G01S17/894—Three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver pixels, e.g. time-of-flight cameras or flash lidar
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
    • G01S7/4808—Evaluating distance, position or velocity data
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/08—Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
    • G06Q10/087—Inventory or stock management, e.g. order filling, procurement or balancing against orders
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/695—Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
    • G01B11/25—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/30—Nc systems
    • G05B2219/37—Measurements
    • G05B2219/37555—Camera detects orientation, position workpiece, points of workpiece
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/30—Nc systems
    • G05B2219/45—Nc applications
    • G05B2219/45063—Pick and place manipulator
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20021—Dividing image into blocks, subimages or windows
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30204—Marker
    • G06T2207/30208—Marker matrix
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30241—Trajectory

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Robotics (AREA)
  • Mechanical Engineering (AREA)
  • General Physics & Mathematics (AREA)
  • Business, Economics & Management (AREA)
  • Theoretical Computer Science (AREA)
  • Human Resources & Organizations (AREA)
  • Economics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Strategic Management (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Quality & Reliability (AREA)
  • General Business, Economics & Management (AREA)
  • Tourism & Hospitality (AREA)
  • Development Economics (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Electromagnetism (AREA)
  • Marketing (AREA)
  • Operations Research (AREA)
  • Signal Processing (AREA)
  • Human Computer Interaction (AREA)
  • Finance (AREA)
  • Accounting & Taxation (AREA)
  • Game Theory and Decision Science (AREA)
  • Manipulator (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Image Analysis (AREA)
CN202011045515.5A 2020-03-05 2020-09-29 用于执行容器检测和物体检测的方法和计算系统 Pending CN113361740A (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011186066.6A CN112288165B (zh) 2020-03-05 2020-09-29 用于执行容器检测和物体检测的方法和计算系统

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US202062985336P 2020-03-05 2020-03-05
US62/985,336 2020-03-05
US16/929,854 US11130237B1 (en) 2020-03-05 2020-07-15 Method and computing system for performing container detection and object detection
US16/929,854 2020-07-15

Related Child Applications (1)

Application Number Title Priority Date Filing Date
CN202011186066.6A Division CN112288165B (zh) 2020-03-05 2020-09-29 用于执行容器检测和物体检测的方法和计算系统

Publications (1)

Publication Number Publication Date
CN113361740A true CN113361740A (zh) 2021-09-07

Family

ID=73646911

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011045515.5A Pending CN113361740A (zh) 2020-03-05 2020-09-29 用于执行容器检测和物体检测的方法和计算系统

Country Status (4)

Country Link
US (1) US20240253237A1 (https=)
JP (2) JP6796901B1 (https=)
CN (1) CN113361740A (https=)
DE (2) DE102020124221B4 (https=)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12269164B2 (en) 2021-05-04 2025-04-08 Mujin, Inc. Method and computing system for performing robot motion planning and repository detection
JP7757756B2 (ja) * 2021-12-09 2025-10-22 オムロン株式会社 画像処理装置、ロボット制御システムおよび制御方法
US12600569B2 (en) * 2022-03-18 2026-04-14 Hexagon Metrology, Inc. Recessed drawer opening apparatus

Family Cites Families (29)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS61206709A (ja) * 1985-03-07 1986-09-13 Motoda Electronics Co Ltd トレ−配送装置
JP4007204B2 (ja) * 2003-01-30 2007-11-14 松下電器産業株式会社 ロボットシステム
JP2006102881A (ja) * 2004-10-06 2006-04-20 Nagasaki Prefecture 把持ロボット装置
US7873549B1 (en) * 2006-03-27 2011-01-18 Amazon Technologies, Inc. Product dimension correction
JP5489000B2 (ja) * 2010-08-31 2014-05-14 株式会社安川電機 作業装置及び部品ピッキングシステム
JP5659787B2 (ja) * 2010-12-28 2015-01-28 トヨタ自動車株式会社 操作環境モデル構築システム、および操作環境モデル構築方法
JP5383836B2 (ja) 2012-02-03 2014-01-08 ファナック株式会社 検索ウィンドウを自動的に調整する機能を備えた画像処理装置
JP5469216B2 (ja) * 2012-07-31 2014-04-16 ファナック株式会社 バラ積みされた物品をロボットで取出す装置
US9259844B2 (en) * 2014-02-12 2016-02-16 General Electric Company Vision-guided electromagnetic robotic system
CN112068526B (zh) * 2014-02-20 2024-05-17 M·奥利尼克 用于在机器人烹饪厨房中制备食物的方法和系统
US9824328B2 (en) * 2014-06-10 2017-11-21 Amazon Technologies, Inc. Arm-detecting overhead sensor for inventory system
US10518409B2 (en) * 2014-09-02 2019-12-31 Mark Oleynik Robotic manipulation methods and systems for executing a domain-specific application in an instrumented environment with electronic minimanipulation libraries
US9758301B2 (en) 2015-03-24 2017-09-12 Joseph Porat System and method for overhead warehousing
US9486921B1 (en) * 2015-03-26 2016-11-08 Google Inc. Methods and systems for distributing remote assistance to facilitate robotic object manipulation
JP6734728B2 (ja) * 2016-08-05 2020-08-05 株式会社日立製作所 ロボットシステム及びピッキング方法
JP2018161700A (ja) * 2017-03-24 2018-10-18 キヤノン株式会社 情報処理装置、システム、情報処理方法、製造方法
JP7045139B2 (ja) * 2017-06-05 2022-03-31 株式会社日立製作所 機械学習装置、機械学習方法、および機械学習プログラム
JP6923383B2 (ja) * 2017-07-27 2021-08-18 株式会社日立物流 ピッキングロボットおよびピッキングシステム
JP2019037349A (ja) * 2017-08-23 2019-03-14 ライフロボティクス株式会社 食器及びワーク
US10586418B2 (en) * 2017-09-29 2020-03-10 Omnicell, Inc. Apparatuses, systems, and methods for the automated dispensing of articles
JP7161857B2 (ja) * 2018-03-14 2022-10-27 オムロン株式会社 情報処理装置、情報処理方法、及びプログラム
US10740911B2 (en) * 2018-04-05 2020-08-11 Symbol Technologies, Llc Method, system and apparatus for correcting translucency artifacts in data representing a support structure
JP7057214B2 (ja) 2018-05-18 2022-04-19 トヨタ自動車株式会社 把持装置、タグが付された容器、対象物把持プログラムおよび対象物把持方法
US10369698B1 (en) 2019-03-07 2019-08-06 Mujin, Inc. Method and system for performing automatic camera calibration for robot control
JP2020168679A (ja) * 2019-04-03 2020-10-15 セイコーエプソン株式会社 ロボットシステム及び撮像方法
JP7204587B2 (ja) * 2019-06-17 2023-01-16 株式会社東芝 物体ハンドリング制御装置、物体ハンドリング装置、物体ハンドリング方法および物体ハンドリングプログラム
US12344488B2 (en) * 2019-10-16 2025-07-01 Symbotic Canada, Ulc Vision-assisted robotized depalletizer
WO2021118702A1 (en) * 2019-12-12 2021-06-17 Mujin, Inc. Method and computing system for performing motion planning based on image information generated by a camera
WO2023245208A1 (en) * 2022-06-17 2023-12-21 Wipro Limited Method and system for accurately counting items in a storehouse

Also Published As

Publication number Publication date
DE102020124221B4 (de) 2024-11-07
JP6796901B1 (ja) 2020-12-09
DE102020124221A1 (de) 2021-09-09
US20240253237A1 (en) 2024-08-01
DE102021105152A1 (de) 2021-09-09
JP2021139879A (ja) 2021-09-16
JP2021139882A (ja) 2021-09-16

Similar Documents

Publication Publication Date Title
US11958202B2 (en) Method and computing system for performing container detection and object detection
US20240253237A1 (en) Method and computing system for performing container detection and object detection
US11717971B2 (en) Method and computing system for performing motion planning based on image information generated by a camera
JP7433609B2 (ja) 物体識別のための方法および計算システム
CN113574563B (zh) 多相机图像处理
US20220375097A1 (en) Robotic system for object size detection
US20230052515A1 (en) System and method for robotic object placement
US20230286140A1 (en) Systems and methods for robotic system with object handling
US20240221350A1 (en) Method and computing system for generating a safety volume list for object detection
US12062182B2 (en) Method and computing system for performing or facilitating physical edge detection
CN113269801A (zh) 用于处理候选边缘的方法和计算系统
US20250005889A1 (en) Robotic system and method with object detection
CN112288165B (zh) 用于执行容器检测和物体检测的方法和计算系统
CN113219900B (zh) 基于相机生成的图像信息执行运动规划的方法和计算系统
JP2016148558A (ja) 情報処理装置、情報処理方法
US20250144807A1 (en) Designation device, robot system, designation method, and recording medium
CN116061192A (zh) 具有物体处理的机器人系统的系统和方法

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination