JP7551940B2 - 画像処理装置、部品把持システム、画像処理方法および部品把持方法 - Google Patents

画像処理装置、部品把持システム、画像処理方法および部品把持方法 Download PDF

Info

Publication number
JP7551940B2
JP7551940B2 JP2023548005A JP2023548005A JP7551940B2 JP 7551940 B2 JP7551940 B2 JP 7551940B2 JP 2023548005 A JP2023548005 A JP 2023548005A JP 2023548005 A JP2023548005 A JP 2023548005A JP 7551940 B2 JP7551940 B2 JP 7551940B2
Authority
JP
Japan
Prior art keywords
image
patch image
patch
grasping
unit
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2023548005A
Other languages
English (en)
Japanese (ja)
Other versions
JPWO2023042306A1 (https=
Inventor
惇史 山本
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Yamaha Motor Co Ltd
Original Assignee
Yamaha Motor Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Yamaha Motor Co Ltd filed Critical Yamaha Motor Co Ltd
Publication of JPWO2023042306A1 publication Critical patent/JPWO2023042306A1/ja
Application granted granted Critical
Publication of JP7551940B2 publication Critical patent/JP7551940B2/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G06T7/74—Determining position or orientation of objects or cameras using feature-based methods involving reference images or patches
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J19/00—Accessories fitted to manipulators, e.g. for monitoring, for viewing; Safety devices combined with or specially adapted for use in connection with manipulators
    • B25J19/02—Sensing devices
    • B25J19/04—Viewing devices
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J13/00—Controls for manipulators
    • B25J13/08—Controls for manipulators by means of sensing devices, e.g. viewing or touching devices
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J15/00—Gripping heads and other end effectors
    • B25J15/08—Gripping heads and other end effectors having finger members
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Program-controlled manipulators
    • B25J9/16—Program controls
    • B25J9/1694—Program controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
    • B25J9/1697—Vision controlled systems
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/7715—Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20076—Probabilistic image processing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20112—Image segmentation details
    • G06T2207/20132—Image cropping
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/06—Recognition of objects for industrial automation

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Robotics (AREA)
  • Mechanical Engineering (AREA)
  • Databases & Information Systems (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • Manipulator (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)
JP2023548005A 2021-09-15 2021-09-15 画像処理装置、部品把持システム、画像処理方法および部品把持方法 Active JP7551940B2 (ja)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2021/033962 WO2023042306A1 (ja) 2021-09-15 2021-09-15 画像処理装置、部品把持システム、画像処理方法および部品把持方法

Publications (2)

Publication Number Publication Date
JPWO2023042306A1 JPWO2023042306A1 (https=) 2023-03-23
JP7551940B2 true JP7551940B2 (ja) 2024-09-17

Family

ID=85602545

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2023548005A Active JP7551940B2 (ja) 2021-09-15 2021-09-15 画像処理装置、部品把持システム、画像処理方法および部品把持方法

Country Status (5)

Country Link
US (1) US20240386606A1 (https=)
JP (1) JP7551940B2 (https=)
CN (1) CN117999153A (https=)
DE (1) DE112021008230T5 (https=)
WO (1) WO2023042306A1 (https=)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2025192221A1 (ja) * 2024-03-12 2025-09-18 ソニーグループ株式会社 画像処理装置、画像処理方法、およびプログラム
CN119188772B (zh) * 2024-11-11 2025-12-02 天津博诺智创机器人技术有限公司 一种像素级抓取检测方法、装置和存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150235351A1 (en) 2012-09-18 2015-08-20 Iee International Electronics & Engineering S.A. Depth image enhancement method
JP2017030135A (ja) 2015-07-31 2017-02-09 ファナック株式会社 ワークの取り出し動作を学習する機械学習装置、ロボットシステムおよび機械学習方法
JP2017185578A (ja) 2016-04-05 2017-10-12 株式会社リコー 物体把持装置及び把持制御プログラム
JP2017185577A (ja) 2016-04-04 2017-10-12 ファナック株式会社 シミュレーション結果を利用して学習を行う機械学習装置，機械システム，製造システムおよび機械学習方法
JP2020015141A (ja) 2018-07-26 2020-01-30 Ntn株式会社 把持装置
JP2020532440A (ja) 2017-09-01 2020-11-12 カリフォルニア大学Ｔｈｅ Ｒｅｇｅｎｔｓ ｏｆ ｔｈｅ Ｕｎｉｖｅｒｓｉｔｙ ｏｆ Ｃａｌｉｆｏｒｎｉａ オブジェクトをロバストに把持し、ターゲティングするためのロボットシステムおよび方法

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11288472B2 (en) * 2011-08-30 2022-03-29 Digimarc Corporation Cart-based shopping arrangements employing probabilistic item identification
CN109843749A (zh) * 2016-08-04 2019-06-04 欧佩克斯公司 具有用于检测物品超出尺寸阈值的检测器的自动存储和取回系统
US10121262B2 (en) * 2016-12-20 2018-11-06 Canon Kabushiki Kaisha Method, system and apparatus for determining alignment data
US10373369B2 (en) * 2017-03-16 2019-08-06 Qualcomm Technologies, Inc. Three-dimensional pose estimation of symmetrical objects
JP6977337B2 (ja) * 2017-07-03 2021-12-08 富士通株式会社 部位認識方法、装置、プログラム、及び撮像制御システム
KR102476758B1 (ko) * 2018-05-09 2022-12-09 삼성전자주식회사 영상 정규화 장치 및 방법
US20230245319A1 (en) * 2020-05-21 2023-08-03 Sony Group Corporation Image processing apparatus, image processing method, learning device, learning method, and program
US11273552B2 (en) * 2020-07-14 2022-03-15 Vicarious Fpc, Inc. Method and system for object grasping
US20220084238A1 (en) * 2020-09-11 2022-03-17 Fanuc Corporation Multiple transparent objects 3d detection
US11568625B2 (en) * 2021-01-07 2023-01-31 Nvidia Corporation Targeted object detection in image processing applications
US12223710B2 (en) * 2021-06-29 2025-02-11 7-Eleven, Inc. Image cropping using depth information
US20240346798A1 (en) * 2021-09-15 2024-10-17 Yamaha Hatsudoki Kabushiki Kaisha Image processing device, component gripping system, image processing method and component gripping method
EP4401049B1 (en) * 2023-01-12 2026-03-04 Siemens Aktiengesellschaft Runtime assessment of suction grasp feasibility

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150235351A1 (en) 2012-09-18 2015-08-20 Iee International Electronics & Engineering S.A. Depth image enhancement method
JP2017030135A (ja) 2015-07-31 2017-02-09 ファナック株式会社 ワークの取り出し動作を学習する機械学習装置、ロボットシステムおよび機械学習方法
JP2017185577A (ja) 2016-04-04 2017-10-12 ファナック株式会社 シミュレーション結果を利用して学習を行う機械学習装置，機械システム，製造システムおよび機械学習方法
JP2017185578A (ja) 2016-04-05 2017-10-12 株式会社リコー 物体把持装置及び把持制御プログラム
JP2020532440A (ja) 2017-09-01 2020-11-12 カリフォルニア大学Ｔｈｅ Ｒｅｇｅｎｔｓ ｏｆ ｔｈｅ Ｕｎｉｖｅｒｓｉｔｙ ｏｆ Ｃａｌｉｆｏｒｎｉａ オブジェクトをロバストに把持し、ターゲティングするためのロボットシステムおよび方法
JP2020015141A (ja) 2018-07-26 2020-01-30 Ntn株式会社 把持装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
○松村 遼（大阪大）， 原田 研介（大阪大／産総研）， 堂前 幸康（産総研），バラ積みされた対象物のピッキングに対する物理シミュレータを用いた学習，ロボティクスメカトロニクス講演会２０１７講演会論文集 2017 JSME Conference on Robotics and Mechatronics ，一般社団法人 日本機械学会 The Japan Society of Mechanical Engineers

Also Published As

Publication number Publication date
DE112021008230T5 (de) 2024-09-19
US20240386606A1 (en) 2024-11-21
CN117999153A (zh) 2024-05-07
WO2023042306A1 (ja) 2023-03-23
JPWO2023042306A1 (https=) 2023-03-23

Similar Documents

Publication Publication Date Title
JP7787268B2 (ja) 画像処理装置、部品把持システム、画像処理方法および部品把持方法
JP7481427B2 (ja) 取り出しシステム及び方法
CN112297013B (zh) 一种基于数字孪生和深度神经网络的机器人智能抓取方法
Karaoguz et al. Object detection approach for robot grasp detection
CN109483573B (zh) 机器学习装置、机器人系统以及机器学习方法
CN111331607B (zh) 一种基于机械臂的自主抓取与码垛方法及系统
CN109986560B (zh) 一种面向多目标种类的机械臂自适应抓取方法
Berscheid et al. Improving data efficiency of self-supervised learning for robotic grasping
US20130006423A1 (en) Target object gripping apparatus, method for controlling the same and storage medium
JP6671694B1 (ja) 機械学習装置、機械学習システム、データ処理システム及び機械学習方法
JP5458885B2 (ja) 物体検出方法と物体検出装置およびロボットシステム
US12017355B2 (en) Grasp learning using modularized neural networks
JP7551940B2 (ja) 画像処理装置、部品把持システム、画像処理方法および部品把持方法
CN110216671A (zh) 一种基于计算机仿真的机械抓手训练方法及系统
CN108748149B (zh) 一种复杂环境下基于深度学习的无标定机械臂抓取方法
JP7051751B2 (ja) 学習装置、学習方法、学習モデル、検出装置及び把持システム
US20220391638A1 (en) Network modularization to learn high dimensional robot tasks
Zhi et al. 3dflowaction: Learning cross-embodiment manipulation from 3d flow world model
US20210241476A1 (en) Training a pose detection algorithm, and deriving an object pose using a trained pose detection algorithm
WO2022097353A1 (ja) データセット生成装置、方法、プログラム及びシステム
CN116984269A (zh) 一种基于图像识别的煤矸石抓取方法及系统
CN118081758A (zh) 一种杂乱场景下机器人自主拾取与放置位姿获取方法
WO2021085561A1 (ja) 学習データ生成方法
CN109816728B (zh) 基于生成查询网络的机械臂抓取点定位检测的方法
CN118876054B (zh) 一种电芯抓取方法、系统、装置、设备以及存储介质

Legal Events

Date Code Title Description
A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20231129

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20240521

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20240717

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20240903

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20240904

R150 Certificate of patent or registration of utility model

Ref document number: 7551940

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150