JP7581320B2 - ロボット装置からの視覚的出力を補強するシステム及び方法 - Google Patents

ロボット装置からの視覚的出力を補強するシステム及び方法 Download PDF

Info

Publication number
JP7581320B2
JP7581320B2 JP2022503979A JP2022503979A JP7581320B2 JP 7581320 B2 JP7581320 B2 JP 7581320B2 JP 2022503979 A JP2022503979 A JP 2022503979A JP 2022503979 A JP2022503979 A JP 2022503979A JP 7581320 B2 JP7581320 B2 JP 7581320B2
Authority
JP
Japan
Prior art keywords
drivable
robotic device
robot
drivable area
environment
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2022503979A
Other languages
English (en)
Japanese (ja)
Other versions
JP2022542241A (ja
JP2022542241A5 (enExample
Inventor
マー ジェレミー
Original Assignee
トヨタ リサーチ インスティテュート，インコーポレイティド
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by トヨタ リサーチ インスティテュート，インコーポレイティド filed Critical トヨタ リサーチ インスティテュート，インコーポレイティド
Publication of JP2022542241A publication Critical patent/JP2022542241A/ja
Publication of JP2022542241A5 publication Critical patent/JP2022542241A5/ja
Application granted granted Critical
Publication of JP7581320B2 publication Critical patent/JP7581320B2/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Program-controlled manipulators
    • B25J9/16—Program controls
    • B25J9/1602—Program controls characterised by the control system, structure, architecture
    • B25J9/1605—Simulation of manipulator lay-out, design, modelling of manipulator
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Program-controlled manipulators
    • B25J9/16—Program controls
    • B25J9/1656—Program controls characterised by programming, planning systems for manipulators
    • B25J9/1661—Program controls characterised by programming, planning systems for manipulators characterised by task planning, object-oriented languages
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Program-controlled manipulators
    • B25J9/16—Program controls
    • B25J9/1656—Program controls characterised by programming, planning systems for manipulators
    • B25J9/1664—Program controls characterised by programming, planning systems for manipulators characterised by motion, path, trajectory planning
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Program-controlled manipulators
    • B25J9/16—Program controls
    • B25J9/1656—Program controls characterised by programming, planning systems for manipulators
    • B25J9/1671—Program controls characterised by programming, planning systems for manipulators characterised by simulation, either to verify existing program or to create and verify new program, CAD/CAM oriented, graphic oriented programming systems
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Program-controlled manipulators
    • B25J9/16—Program controls
    • B25J9/1694—Program controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
    • B25J9/1697—Vision controlled systems
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/28—Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics
    • G06T19/20—Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G06T7/248—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving reference images or patches
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G06T7/74—Determining position or orientation of objects or cameras using feature-based methods involving reference images or patches
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Program-controlled manipulators
    • B25J9/16—Program controls
    • B25J9/1628—Program controls characterised by the control loop
    • B25J9/163—Program controls characterised by the control loop learning, adaptive, model based, rule based expert control
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Program-controlled manipulators
    • B25J9/16—Program controls
    • B25J9/1656—Program controls characterised by programming, planning systems for manipulators
    • B25J9/1664—Program controls characterised by programming, planning systems for manipulators characterised by motion, path, trajectory planning
    • B25J9/1666—Avoiding collision or forbidden zones
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/30—Nc systems
    • G05B2219/37—Measurements
    • G05B2219/37567—3-D vision, stereo vision, with two cameras
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/30—Nc systems
    • G05B2219/40—Robotics, robotics mapping to robotics vision
    • G05B2219/40136—Stereo audio and vision
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/30—Nc systems
    • G05B2219/40—Robotics, robotics mapping to robotics vision
    • G05B2219/40543—Identification and location, position of components, objects
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/30—Nc systems
    • G05B2219/40—Robotics, robotics mapping to robotics vision
    • G05B2219/40564—Recognize shape, contour of object, extract position and orientation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G06N20/10—Machine learning using kernel methods, e.g. support vector machines [SVM]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00—Indexing scheme for image data processing or generation, in general
    • G06T2200/04—Indexing scheme for image data processing or generation, in general involving 3D image data
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20092—Interactive image processing based on input by user
    • G06T2207/20104—Interactive definition of region of interest [ROI]

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Robotics (AREA)
  • Mechanical Engineering (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Mathematical Physics (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Architecture (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • Automation & Control Theory (AREA)
  • Manipulator (AREA)
  • Image Analysis (AREA)
  • Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
  • Processing Or Creating Images (AREA)
  • User Interface Of Digital Computer (AREA)
JP2022503979A 2019-07-23 2020-06-05 ロボット装置からの視覚的出力を補強するシステム及び方法 Active JP7581320B2 (ja)

Applications Claiming Priority (9)

Application Number Priority Date Filing Date Title
US201962877792P 2019-07-23 2019-07-23
US201962877791P 2019-07-23 2019-07-23
US201962877793P 2019-07-23 2019-07-23
US62/877,792 2019-07-23
US62/877,791 2019-07-23
US62/877,793 2019-07-23
US16/570,540 2019-09-13
US16/570,540 US11694432B2 (en) 2019-07-23 2019-09-13 System and method for augmenting a visual output from a robotic device
PCT/US2020/036428 WO2021015868A1 (en) 2019-07-23 2020-06-05 System and method for augmenting a visual output from a robotic device

Publications (3)

Publication Number Publication Date
JP2022542241A JP2022542241A (ja) 2022-09-30
JP2022542241A5 JP2022542241A5 (enExample) 2023-06-12
JP7581320B2 true JP7581320B2 (ja) 2024-11-12

Family

ID=74189690

Family Applications (4)

Application Number Title Priority Date Filing Date
JP2022503979A Active JP7581320B2 (ja) 2019-07-23 2020-06-05 ロボット装置からの視覚的出力を補強するシステム及び方法
JP2022503981A Active JP7686618B2 (ja) 2019-07-23 2020-06-05 ディープネットワークの訓練方法
JP2022503936A Active JP7693648B2 (ja) 2019-07-23 2020-06-10 視角エンベディングに基づいた自律タスク実行
JP2022503980A Active JP7693649B2 (ja) 2019-07-23 2020-07-22 移動操作システムの視覚的教示と繰り返し

Family Applications After (3)

Application Number Title Priority Date Filing Date
JP2022503981A Active JP7686618B2 (ja) 2019-07-23 2020-06-05 ディープネットワークの訓練方法
JP2022503936A Active JP7693648B2 (ja) 2019-07-23 2020-06-10 視角エンベディングに基づいた自律タスク実行
JP2022503980A Active JP7693649B2 (ja) 2019-07-23 2020-07-22 移動操作システムの視覚的教示と繰り返し

Country Status (5)

Country Link
US (6) US11580724B2 (enExample)
EP (4) EP4004670A4 (enExample)
JP (4) JP7581320B2 (enExample)
CN (4) CN114127805B (enExample)
WO (4) WO2021015869A1 (enExample)

Families Citing this family (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP4742722A3 (en) * 2018-12-28 2026-07-15 Apple Inc. Providing verified claims of user identity
US11580724B2 (en) 2019-07-23 2023-02-14 Toyota Research Institute, Inc. Virtual teach and repeat mobile manipulation system
US11685045B1 (en) * 2019-09-09 2023-06-27 X Development Llc Asynchronous robotic control using most recently selected robotic action data
US20210101280A1 (en) * 2019-10-02 2021-04-08 Baker Hughes Oilfield Operations, Llc Telemetry harvesting and analysis from extended reality streaming
WO2021147026A1 (en) * 2020-01-22 2021-07-29 Abb Schweiz Ag Method and electronic device, system and computer readable medium for time calibration
US11328170B2 (en) * 2020-02-19 2022-05-10 Toyota Research Institute, Inc. Unknown object identification for robotic device
US11679498B2 (en) * 2020-05-27 2023-06-20 Intrinsic Innovation Llc Robot execution system
US11813748B2 (en) * 2020-10-13 2023-11-14 Google Llc Simulating multiple robots in virtual environments
WO2022077113A1 (en) * 2020-10-14 2022-04-21 One Cup Productions Ltd. Animal visual identification, tracking, monitoring and assessment systems and methods thereof
CN116635193A (zh) 2020-12-21 2023-08-22 波士顿动力公司 监督式自主抓取
US12669817B2 (en) * 2020-12-23 2026-06-30 Qatar Foundation For Education, Science And Community Development Telepresence control schemes for hazardous environments
EP4288254A1 (en) * 2021-02-08 2023-12-13 Diligent Robotics, Inc. Systems, apparatuses, and methods for robotic learning and execution of skills including navigation and manipulation functions
IT202100003821A1 (it) * 2021-02-19 2022-08-19 Univ Pisa Procedimento di interazione con oggetti
DE102021202570A1 (de) * 2021-03-16 2022-09-22 Robert Bosch Gesellschaft mit beschränkter Haftung Vorrichtung und verfahren zum lokalisieren von stellen von objekten aus kamerabildern der objekte
US12065051B2 (en) * 2021-05-05 2024-08-20 ABB E-mobility B.V. Systems and methods for electric vehicle charging using machine learning
EP4370286B1 (en) 2021-07-15 2025-02-19 Schaeffler Technologies AG & Co. KG Teaching a robot system using hand gesture control and visual-inertial-odometry
US12002257B2 (en) * 2021-11-29 2024-06-04 Google Llc Video screening using a machine learning video screening model trained using self-supervised training
US12515332B2 (en) * 2022-02-09 2026-01-06 eBots Inc. System and method for sequencing assembly tasks
GB2616001A (en) * 2022-02-17 2023-08-30 Roke Manor Res Limited Remote visual inspection guidance
US20230286161A1 (en) * 2022-03-11 2023-09-14 Boston Dynamics, Inc. Systems and Methods for Robotic Manipulation Using Extended Reality
US12539609B2 (en) * 2023-08-29 2026-02-03 Nvidia Corporation Data generation of robotic devices performing tasks
CN116823838B (zh) * 2023-08-31 2023-11-14 武汉理工大学三亚科教创新园 高斯先验标签分配与特征解耦的海洋船舶检测方法与系统
WO2025184405A1 (en) * 2024-02-28 2025-09-04 Clutterbot, Inc. Clutter tidying robot for non-standard storage locations
DE102024112986A1 (de) * 2024-05-08 2025-11-13 Helmholtz-Zentrum Dresden - Rossendorf E. V. Verfahren und Vorrichtung zur Bildverarbeitung und Computerlesbares Speichermedium
WO2026015471A1 (en) * 2024-07-08 2026-01-15 Bria Artificial Intelligence Ltd Controllable and attributable generation of machine-generated content

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010152835A (ja) 2008-12-26 2010-07-08 Ihi Aerospace Co Ltd 無人移動体システム
JP2014085829A (ja) 2012-10-23 2014-05-12 Sharp Corp 機器制御装置、および自走式電子機器
US20160165795A1 (en) 2014-12-15 2016-06-16 Irobot Corporation Robot lawnmower mapping
JP2017033399A (ja) 2015-08-04 2017-02-09 トヨタ自動車株式会社 移動体監視システム
US20190015985A1 (en) 2017-07-12 2019-01-17 Lg Electronics Inc. Moving robot and controlling method
JP2019515407A (ja) 2016-05-11 2019-06-06 ブレーン コーポレーションＢｒａｉｎ Ｃｏｒｐｏｒａｔｉｏｎ ロボットが学習した経路を自律的に走行するように初期化するためのシステムおよび方法
JP2019106060A (ja) 2017-12-13 2019-06-27 東芝ライフスタイル株式会社 自律走行体
WO2019138640A1 (ja) 2018-01-10 2019-07-18 ソニー株式会社 情報処理装置、情報処理方法、および、プログラム
JP2019121364A (ja) 2018-01-05 2019-07-22 アイロボット・コーポレーション 状況認識のためのモバイル清掃ロボット人工知能

Family Cites Families (97)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2000287166A (ja) * 1999-01-29 2000-10-13 Sony Corp データ記述方法及びデータ処理装置
US7082497B2 (en) * 2001-12-28 2006-07-25 Hewlett-Packard Development Company, L.P. System and method for managing a moveable media library with library partitions
JP3834297B2 (ja) * 2003-05-12 2006-10-18 ファナック株式会社 画像処理装置
EP2302590A1 (en) * 2003-07-04 2011-03-30 Mitsubishi Electric Information Technology Centre Europe B.V. Method and apparatus for representing a group of images
ATE524783T1 (de) * 2004-03-27 2011-09-15 Harvey Koselka Autonomer persönlicher dienstroboter
JP2005308993A (ja) * 2004-04-20 2005-11-04 Matsushita Electric Ind Co Ltd 学習支援システム
DE102005009437A1 (de) * 2005-03-02 2006-09-07 Kuka Roboter Gmbh Verfahren und Vorrichtung zum Einblenden von AR-Objekten
JP2007026309A (ja) * 2005-07-20 2007-02-01 Fuji Xerox Co Ltd 画像形成装置及びその制御方法
US7836437B2 (en) * 2006-02-10 2010-11-16 Microsoft Corporation Semantic annotations for virtual objects
WO2007138987A1 (ja) * 2006-05-25 2007-12-06 Takehiro Ishizaki 作業ロボット
WO2008073962A2 (en) 2006-12-12 2008-06-19 Rutgers, The State University Of New Jersey System and method for detecting and tracking features in images
JP5037248B2 (ja) * 2007-07-17 2012-09-26 株式会社日立製作所 情報収集システムおよび情報収集ロボット
DE602008005318D1 (de) * 2007-08-22 2011-04-14 Honda Res Inst Europe Gmbh Schätzung der ordnungsgemässen bewegung von objekten mithilfe optischer fluss-, kinematik- und tiefeninformationen
FR2946160B1 (fr) 2009-05-26 2014-05-09 Aldebaran Robotics Systeme et procede pour editer et commander des comportements d'un robot mobile.
CA2928262C (en) * 2010-12-30 2018-06-26 Irobot Corporation Mobile robot system
JP2013022705A (ja) 2011-07-25 2013-02-04 Sony Corp ロボット装置及びロボット装置の制御方法、コンピューター・プログラム、並びにロボット・システム
US9621796B2 (en) * 2012-03-15 2017-04-11 Nokia Technologies Oy Method, apparatus and computer program for capturing images with multiple image capture and image modification
US8996167B2 (en) * 2012-06-21 2015-03-31 Rethink Robotics, Inc. User interfaces for robot training
US9186793B1 (en) * 2012-08-31 2015-11-17 Brain Corporation Apparatus and methods for controlling attention of a robot
US9940553B2 (en) * 2013-02-22 2018-04-10 Microsoft Technology Licensing, Llc Camera/object pose from predicted coordinates
US9102055B1 (en) * 2013-03-15 2015-08-11 Industrial Perception, Inc. Detection and reconstruction of an environment to facilitate robotic interaction with the environment
US9542626B2 (en) 2013-09-06 2017-01-10 Toyota Jidosha Kabushiki Kaisha Augmenting layer-based object detection with deep convolutional neural networks
US10888998B2 (en) * 2013-10-07 2021-01-12 Abb Schweiz Ag Method and device for verifying one or more safety volumes for a movable mechanical unit
US9259844B2 (en) * 2014-02-12 2016-02-16 General Electric Company Vision-guided electromagnetic robotic system
US9579799B2 (en) * 2014-04-30 2017-02-28 Coleman P. Parker Robotic control system using virtual reality input
KR101708659B1 (ko) * 2014-06-17 2017-02-22 주식회사 유진로봇 이동 로봇의 맵을 업데이트하기 위한 장치 및 그 방법
CN104299006B (zh) * 2014-07-23 2018-11-09 中国传媒大学 一种基于深度神经网络的车牌识别方法
US10518409B2 (en) 2014-09-02 2019-12-31 Mark Oleynik Robotic manipulation methods and systems for executing a domain-specific application in an instrumented environment with electronic minimanipulation libraries
US9969337B2 (en) * 2014-09-03 2018-05-15 Sharp Laboratories Of America, Inc. Methods and systems for mobile-agent navigation
US9579790B2 (en) * 2014-09-17 2017-02-28 Brain Corporation Apparatus and methods for removal of learned behaviors in robots
US9630318B2 (en) 2014-10-02 2017-04-25 Brain Corporation Feature detection apparatus and methods for training of robotic navigation
US9754188B2 (en) * 2014-10-23 2017-09-05 Microsoft Technology Licensing, Llc Tagging personal photos with deep networks
JP2016107379A (ja) * 2014-12-08 2016-06-20 ファナック株式会社 拡張現実対応ディスプレイを備えたロボットシステム
WO2016103065A1 (en) * 2014-12-23 2016-06-30 Husqvarna Ab Robotic vehicle with adjustable operating area
CN104732518B (zh) 2015-01-19 2017-09-01 北京工业大学 一种基于智能机器人地面特征的ptam改进方法
CN104808590B (zh) 2015-02-14 2017-01-25 浙江大学 一种基于关键帧策略的移动机器人视觉伺服控制方法
US9643314B2 (en) * 2015-03-04 2017-05-09 The Johns Hopkins University Robot control, training and collaboration in an immersive virtual reality environment
US9922271B2 (en) 2015-03-20 2018-03-20 Netra, Inc. Object detection and classification
US9868211B2 (en) * 2015-04-09 2018-01-16 Irobot Corporation Restricting movement of a mobile robot
US9646655B2 (en) 2015-08-31 2017-05-09 Fuji Xerox Co., Ltd. Systems and methods for interaction-based analysis and visualization of remote collaboration sessions
WO2017132636A1 (en) * 2016-01-29 2017-08-03 Pointivo, Inc. Systems and methods for extracting information about objects from scene information
JP7009389B2 (ja) * 2016-05-09 2022-01-25 グラバンゴ コーポレイション 環境内のコンピュータビジョン駆動型アプリケーションのためのシステムおよび方法
EP3402633B1 (en) * 2016-05-20 2020-05-13 Google LLC Machine learning methods and apparatus related to predicting motion(s) of object(s) in a robot's environment based on image(s) capturing the object(s) and based on parameter(s) for future robot movement in the environment
CN106023211B (zh) 2016-05-24 2019-02-26 深圳前海勇艺达机器人有限公司 基于深度学习的机器人图像定位方法及系统
US10105847B1 (en) * 2016-06-08 2018-10-23 X Development Llc Detecting and responding to geometric changes to robots
US10055667B2 (en) * 2016-08-03 2018-08-21 X Development Llc Generating a model for an object encountered by a robot
US9964955B2 (en) 2016-08-04 2018-05-08 Canvas Technology, Inc. System and methods of determining a geometric pose of a camera based on spatial and visual mapping
US10346710B2 (en) * 2016-09-29 2019-07-09 Datacolor Inc. Multi-agent training of a color identification neural network
JP2018055429A (ja) * 2016-09-29 2018-04-05 ファナック株式会社 物体認識装置および物体認識方法
CN108072370A (zh) * 2016-11-18 2018-05-25 中国科学院电子学研究所 基于全局地图的机器人导航方法及用该方法导航的机器人
US10162360B2 (en) * 2016-12-01 2018-12-25 GM Global Technology Operations LLC Vehicle environment imaging systems and methods
KR101850410B1 (ko) * 2016-12-26 2018-04-20 한국생산기술연구원 가상 현실 기반 로봇 교시를 위한 시뮬레이션 장치 및 방법
JP2018122376A (ja) * 2017-01-31 2018-08-09 セイコーエプソン株式会社 画像処理装置、ロボット制御装置、及びロボット
EP3585569B1 (en) * 2017-02-25 2023-06-07 Diligent Robotics, Inc. Systems, apparatus, and methods for robotic learning and execution of skills
US11709489B2 (en) * 2017-03-02 2023-07-25 RobArt GmbH Method for controlling an autonomous, mobile robot
CN107092926A (zh) * 2017-03-30 2017-08-25 哈尔滨工程大学 基于深度学习的服务机器人物体识别算法
US10742865B2 (en) * 2017-04-14 2020-08-11 International Business Machines Corporation Configuring cognitive robot vision
CN109213306B (zh) * 2017-06-30 2022-02-01 沈阳新松机器人自动化股份有限公司 一种机器人远程控制平台及其设计方法
CA3071332A1 (en) * 2017-07-25 2019-01-31 Mbl Limited Systems and methods for operations a robotic system and executing robotic interactions
US11494915B2 (en) * 2017-08-14 2022-11-08 Rakuten Group, Inc. Image processing system, image processing method, and program
CN107748569B (zh) * 2017-09-04 2021-02-19 中国兵器工业计算机应用技术研究所 用于无人机的运动控制方法、装置及无人机系统
CN107610579A (zh) * 2017-09-05 2018-01-19 芜湖瑞思机器人有限公司 基于vr系统控制的工业机器人示教系统及其示教方法
US10354139B1 (en) * 2017-09-07 2019-07-16 X Development Llc Generating and utilizing spatial affordances for an object in robotics applications
JP6822929B2 (ja) * 2017-09-19 2021-01-27 株式会社東芝 情報処理装置、画像認識方法および画像認識プログラム
WO2019060787A1 (en) * 2017-09-21 2019-03-28 Lexset.Ai Llc DETECTION OF AT LEAST ONE OBJECT IN AN IMAGE, OR A SEQUENCE OF IMAGES, AND DETERMINATION OF A CATEGORY AND AT LEAST ONE DESCRIPTOR FOR EACH OF THESE OBJECTS, GENERATION OF SYNTHETIC LEARNING DATA, AND FORMATION OF A NEURONAL NETWORK USING SYNTHETIC LEARNING DATA
US10607079B2 (en) * 2017-09-26 2020-03-31 Toyota Research Institute, Inc. Systems and methods for generating three dimensional skeleton representations
EP3701488B1 (en) 2017-10-24 2026-04-15 L'oreal System and method for image processing using deep neural networks
US11703334B2 (en) * 2017-10-31 2023-07-18 Hewlett-Packard Development Company. L.P. Mobile robots to generate reference maps for localization
US10346721B2 (en) * 2017-11-01 2019-07-09 Salesforce.Com, Inc. Training a neural network using augmented training datasets
JP6972962B2 (ja) * 2017-11-22 2021-11-24 コニカミノルタ株式会社 物体追跡装置、物体追跡方法、および、物体追跡プログラム
US11771283B2 (en) * 2017-12-06 2023-10-03 BISSELL , Inc. Method and system for manual control of autonomous floor cleaner
US10380751B1 (en) * 2017-12-12 2019-08-13 The Government Of The United States Of America, As Represented By The Secretary Of The Navy Robot vision in autonomous underwater vehicles using the color shift in underwater imaging
US10810427B1 (en) * 2017-12-15 2020-10-20 AI Incorporated Methods for an autonomous robotic device to identify locations captured in an image
US10981272B1 (en) * 2017-12-18 2021-04-20 X Development Llc Robot grasp learning
JP6641447B2 (ja) * 2017-12-26 2020-02-05 キヤノン株式会社 撮像装置及びその制御方法、プログラム、記憶媒体
US10913154B2 (en) * 2018-01-02 2021-02-09 General Electric Company Systems and method for robotic learning of industrial tasks based on human demonstration
US11097418B2 (en) * 2018-01-04 2021-08-24 X Development Llc Grasping of an object by a robot based on grasp strategy determined using machine learning model(s)
CN108333941A (zh) * 2018-02-13 2018-07-27 华南理工大学 一种基于混合增强智能的云机器人协作学习方法
US11340630B2 (en) * 2018-03-30 2022-05-24 Brain Corporation Systems and methods for robust robotic mapping
CN108619723A (zh) * 2018-04-18 2018-10-09 咪咕互动娱乐有限公司 一种应用操作的处理方法、装置以及存储介质
US10332261B1 (en) * 2018-04-26 2019-06-25 Capital One Services, Llc Generating synthetic images as training dataset for a machine learning network
CN108592920B (zh) * 2018-04-28 2022-04-12 上海智蕙林医疗科技有限公司 一种机器人的路径规划方法及系统
CN108681711A (zh) * 2018-05-17 2018-10-19 常州信息职业技术学院 一种面向移动机器人的自然路标提取方法
US11189078B2 (en) * 2018-06-20 2021-11-30 Google Llc Automated understanding of three dimensional (3D) scenes for augmented reality applications
US10636114B2 (en) * 2018-08-04 2020-04-28 Beijing Jingdong Shangke Information Technology Co., Ltd. System and method for scan-matching oriented visual slam
US11030458B2 (en) * 2018-09-14 2021-06-08 Microsoft Technology Licensing, Llc Generating synthetic digital assets for a virtual scene including a model of a real-world object
CN109543600A (zh) * 2018-11-21 2019-03-29 成都信息工程大学 一种实现可行驶区域检测方法及系统和应用
US11238309B2 (en) * 2018-12-26 2022-02-01 Intel Corporation Selecting keypoints in images using descriptor scores
CN109816686A (zh) * 2019-01-15 2019-05-28 山东大学 基于物体实例匹配的机器人语义slam方法、处理器及机器人
US10824923B1 (en) * 2019-01-23 2020-11-03 Facebook Technologies, Llc System and method for improving localization and object tracking
US10870204B2 (en) * 2019-01-25 2020-12-22 Mujin, Inc. Robotic system control method and controller
CN109886141B (zh) * 2019-01-28 2023-06-06 同济大学 一种基于不确定性优化的行人再辨识方法
JP7203474B2 (ja) * 2019-02-20 2023-01-13 富士フイルム株式会社 マンモグラフィ装置及びプログラム
CN109828588A (zh) * 2019-03-11 2019-05-31 浙江工业大学 一种基于多传感器融合的机器人室内路径规划方法
CN109974707B (zh) * 2019-03-19 2022-09-23 重庆邮电大学 一种基于改进点云匹配算法的室内移动机器人视觉导航方法
US11580724B2 (en) 2019-07-23 2023-02-14 Toyota Research Institute, Inc. Virtual teach and repeat mobile manipulation system
US11529737B2 (en) * 2020-01-30 2022-12-20 Raytheon Company System and method for using virtual/augmented reality for interaction with collaborative robots in manufacturing or industrial environment

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010152835A (ja) 2008-12-26 2010-07-08 Ihi Aerospace Co Ltd 無人移動体システム
JP2014085829A (ja) 2012-10-23 2014-05-12 Sharp Corp 機器制御装置、および自走式電子機器
US20160165795A1 (en) 2014-12-15 2016-06-16 Irobot Corporation Robot lawnmower mapping
JP2017033399A (ja) 2015-08-04 2017-02-09 トヨタ自動車株式会社 移動体監視システム
JP2019515407A (ja) 2016-05-11 2019-06-06 ブレーン コーポレーションＢｒａｉｎ Ｃｏｒｐｏｒａｔｉｏｎ ロボットが学習した経路を自律的に走行するように初期化するためのシステムおよび方法
US20190015985A1 (en) 2017-07-12 2019-01-17 Lg Electronics Inc. Moving robot and controlling method
JP2019106060A (ja) 2017-12-13 2019-06-27 東芝ライフスタイル株式会社 自律走行体
JP2019121364A (ja) 2018-01-05 2019-07-22 アイロボット・コーポレーション 状況認識のためのモバイル清掃ロボット人工知能
WO2019138640A1 (ja) 2018-01-10 2019-07-18 ソニー株式会社 情報処理装置、情報処理方法、および、プログラム

Also Published As

Publication number Publication date
WO2021015868A1 (en) 2021-01-28
JP2022542858A (ja) 2022-10-07
WO2021016394A1 (en) 2021-01-28
JP2022542241A (ja) 2022-09-30
EP4004670A1 (en) 2022-06-01
JP7693649B2 (ja) 2025-06-17
US11694432B2 (en) 2023-07-04
WO2021015883A1 (en) 2021-01-28
US12131529B2 (en) 2024-10-29
US20220165057A1 (en) 2022-05-26
CN114127805B (zh) 2026-02-10
CN114097004A (zh) 2022-02-25
EP4004828A1 (en) 2022-06-01
JP2022542239A (ja) 2022-09-30
US11580724B2 (en) 2023-02-14
JP2022544007A (ja) 2022-10-17
EP4003666A1 (en) 2022-06-01
EP4004670A4 (en) 2023-09-06
US20230154015A1 (en) 2023-05-18
US11288883B2 (en) 2022-03-29
CN114127805A (zh) 2022-03-01
EP4003668A1 (en) 2022-06-01
WO2021015869A1 (en) 2021-01-28
US20210027097A1 (en) 2021-01-28
EP4003666A4 (en) 2023-09-06
US20210027058A1 (en) 2021-01-28
US20210023703A1 (en) 2021-01-28
EP4004828A4 (en) 2023-10-18
US11113526B2 (en) 2021-09-07
JP7693648B2 (ja) 2025-06-17
US20210023707A1 (en) 2021-01-28
CN114097004B (zh) 2026-05-01
CN114127806A (zh) 2022-03-01
JP7686618B2 (ja) 2025-06-02
US11741701B2 (en) 2023-08-29
EP4003668A4 (en) 2023-09-06
CN114080583B (zh) 2023-12-08
CN114080583A (zh) 2022-02-22

Similar Documents

Publication Publication Date Title
JP7693648B2 (ja) 視角エンベディングに基づいた自律タスク実行
CN110202583B (zh) 一种基于深度学习的仿人机械手控制系统及其控制方法
Sayour et al. Autonomous robotic manipulation: real‐time, deep‐learning approach for grasping of unknown objects
US12521888B2 (en) Synergies between pick and place: task-aware grasp estimation
WO2022197386A1 (en) System(s) and method(s) of using imitation learning in training and refining robotic control policies
US20200338722A1 (en) Machine learning methods and apparatus for semantic robotic grasping
CN109108942A (zh) 基于视觉实时示教与自适应dmps的机械臂运动控制方法和系统
Xue et al. Gesture-and vision-based automatic grasping and flexible placement in teleoperation
Mania et al. Imagination-enabled robot perception
US10933526B2 (en) Method and robotic system for manipulating instruments
CN118809616B (zh) 一种机器人的运动规划方法、装置、机器人及存储介质
Zhang et al. Recent advances on vision-based robot learning by demonstration
WO2023100282A1 (ja) データ生成システム、モデル生成システム、推定システム、学習済みモデルの製造方法、ロボット制御システム、データ生成方法、およびデータ生成プログラム
Zeng et al. Object manipulation learning by imitation
CN120219671A (zh) 智能体任务规划的方法和相关装置
Xie Applications of Computer Vision and Machine Learning to Three Engineering Problems
CN121625153A (zh) 对象控制方法、装置、计算机设备、可读存储介质和程序产品
Lulu AI and Vision 3D-based Robot Arm for Object Grasping and Placing

Legal Events

Date Code Title Description
A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20230602

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20230602

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20240523

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20240528

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20240828

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20241001

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20241030

R150 Certificate of patent or registration of utility model

Ref document number: 7581320

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150

S111 Request for change of ownership or part of ownership

Free format text: JAPANESE INTERMEDIATE CODE: R313113

R350 Written notification of registration of transfer

Free format text: JAPANESE INTERMEDIATE CODE: R350