JP7224559B2 - 遠隔制御マニピュレータシステムおよび遠隔制御支援システム - Google Patents

遠隔制御マニピュレータシステムおよび遠隔制御支援システム Download PDF

Info

Publication number
JP7224559B2
JP7224559B2 JP2022568347A JP2022568347A JP7224559B2 JP 7224559 B2 JP7224559 B2 JP 7224559B2 JP 2022568347 A JP2022568347 A JP 2022568347A JP 2022568347 A JP2022568347 A JP 2022568347A JP 7224559 B2 JP7224559 B2 JP 7224559B2
Authority
JP
Japan
Prior art keywords
model
manipulator
camera
image
surrounding environment
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2022568347A
Other languages
English (en)
Japanese (ja)
Other versions
JPWO2022124398A1 (https=
JPWO2022124398A5 (https=
Inventor
翔太 楢崎
昇 川口
正樹 平野
宗晴 桑田
勝重 諏訪
俊行 安藤
仁深 尾野
彰裕 藤江
吉孝 神田
亨 佐藤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Mitsubishi Electric Corp
Original Assignee
Mitsubishi Electric Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mitsubishi Electric Corp filed Critical Mitsubishi Electric Corp
Publication of JPWO2022124398A1 publication Critical patent/JPWO2022124398A1/ja
Application granted granted Critical
Publication of JP7224559B2 publication Critical patent/JP7224559B2/ja
Publication of JPWO2022124398A5 publication Critical patent/JPWO2022124398A5/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Program-controlled manipulators
    • B25J9/16—Program controls
    • B25J9/1679—Program controls characterised by the tasks executed
    • B25J9/1689—Teleoperation
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J13/00—Controls for manipulators
    • B25J13/06—Control stands, e.g. consoles, switchboards
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J3/00—Manipulators of leader-follower type, i.e. both controlling unit and controlled unit perform corresponding spatial movements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—Three-dimensional [3D] image rendering
    • G06T15/10—Geometric effects
    • G06T15/20—Perspective computation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three-dimensional [3D] modelling for computer graphics
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/111—Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
    • H04N13/117—Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/204—Image signal generators using stereoscopic image cameras
    • H04N13/239—Image signal generators using stereoscopic image cameras using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular distance
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/332—Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/366—Image reproducers using viewer tracking
    • H04N13/38—Image reproducers using viewer tracking for tracking vertical translational head movements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/398—Synchronisation thereof; Control thereof
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/30—Nc systems
    • G05B2219/40—Robotics, robotics mapping to robotics vision
    • G05B2219/40195—Tele-operation, computer assisted manual operation
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/30—Nc systems
    • G05B2219/40—Robotics, robotics mapping to robotics vision
    • G05B2219/40264—Human like, type robot arm
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2210/00—Indexing scheme for image generation or computer graphics
    • G06T2210/61—Scene description
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N2013/0074—Stereoscopic image analysis
    • H04N2013/0081—Depth or disparity estimation from stereoscopic image signals

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Mechanical Engineering (AREA)
  • Robotics (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Manipulator (AREA)
JP2022568347A 2020-12-10 2021-12-10 遠隔制御マニピュレータシステムおよび遠隔制御支援システム Active JP7224559B2 (ja)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
JP2020205183 2020-12-10
JP2020205183 2020-12-10
PCT/JP2021/045527 WO2022124398A1 (ja) 2020-12-10 2021-12-10 遠隔制御マニピュレータシステムおよび遠隔制御支援システム

Publications (3)

Publication Number Publication Date
JPWO2022124398A1 JPWO2022124398A1 (https=) 2022-06-16
JP7224559B2 true JP7224559B2 (ja) 2023-02-17
JPWO2022124398A5 JPWO2022124398A5 (https=) 2023-02-17

Family

ID=81974596

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2022568347A Active JP7224559B2 (ja) 2020-12-10 2021-12-10 遠隔制御マニピュレータシステムおよび遠隔制御支援システム

Country Status (3)

Country Link
US (1) US11926064B2 (https=)
JP (1) JP7224559B2 (https=)
WO (1) WO2022124398A1 (https=)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023053368A1 (ja) * 2021-09-30 2023-04-06 ファナック株式会社 教示装置及びロボットシステム
US20250143820A1 (en) * 2022-02-17 2025-05-08 Momentis Surgical Ltd. Control system and method for robotic systems
WO2024013895A1 (ja) * 2022-07-13 2024-01-18 日本電信電話株式会社 遠隔制御システム、遠隔制御方法、および遠隔制御プログラム
US12356064B2 (en) * 2022-12-02 2025-07-08 National Kaohsiung University Of Science And Technology Measurement assistance system and method
US20250289141A1 (en) * 2023-12-22 2025-09-18 Sanctuary Cognitive Systems Corporation Systems, Computer Program Products, and Methods for Controlling Robots Through a Graphical User Interface

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5684531A (en) 1995-04-10 1997-11-04 The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration Ranging apparatus and method implementing stereo vision system
JP2000042960A (ja) 1998-07-29 2000-02-15 Gifu Prefecture マニピュレータの遠隔操作装置
JP2009006410A (ja) 2007-06-26 2009-01-15 Fuji Electric Systems Co Ltd 遠隔操作支援装置および遠隔操作支援プログラム
JP2009012106A (ja) 2007-07-03 2009-01-22 Fuji Electric Systems Co Ltd 遠隔操作支援装置および遠隔操作支援プログラム
JP2010094777A (ja) 2008-10-16 2010-04-30 Fuji Electric Systems Co Ltd 遠隔操作支援装置
JP2014056496A (ja) 2012-09-13 2014-03-27 Mitsubishi Heavy Ind Ltd 仮想カメラ画像生成装置、仮想カメラ画像生成システム、仮想カメラ画像生成方法、およびプログラム
US20180236666A1 (en) 2017-02-20 2018-08-23 Irobot Defense Holdings, Inc. Robotic gripper camera
WO2019059364A1 (ja) 2017-09-22 2019-03-28 三菱電機株式会社 遠隔制御マニピュレータシステムおよび制御装置
JP2020037145A (ja) 2018-09-03 2020-03-12 川崎重工業株式会社 ロボットシステム
JP2020040184A (ja) 2018-09-12 2020-03-19 キヤノン株式会社 ロボットシステム、ロボットシステムの制御方法、制御プログラム及び記録媒体
WO2021220915A1 (ja) 2020-04-27 2021-11-04 ファナック株式会社 産業機械の表示装置

Family Cites Families (31)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS6077589A (ja) * 1983-10-05 1985-05-02 Agency Of Ind Science & Technol 遠隔操作における作業環境表示装置
US5417210A (en) * 1992-05-27 1995-05-23 International Business Machines Corporation System and method for augmentation of endoscopic surgery
EP0641132B1 (en) * 1993-08-26 1999-04-14 Matsushita Electric Industrial Co., Ltd. Stereoscopic image pickup apparatus
US5510832A (en) * 1993-12-01 1996-04-23 Medi-Vision Technologies, Inc. Synthesized stereoscopic imaging system and method
CA2437286C (en) * 2002-08-13 2008-04-29 Garnette Roy Sutherland Microsurgical robot system
SE0203908D0 (sv) * 2002-12-30 2002-12-30 Abb Research Ltd An augmented reality system and method
US7236854B2 (en) * 2004-01-05 2007-06-26 Abb Research Ltd. Method and a system for programming an industrial robot
JP4375320B2 (ja) * 2005-10-27 2009-12-02 株式会社日立製作所 移動ロボット
JP4645678B2 (ja) * 2008-05-08 2011-03-09 ソニー株式会社 情報入出力装置及び情報入出力方法、並びにコンピュータ・プログラム
US9285586B2 (en) * 2011-05-13 2016-03-15 Sony Corporation Adjusting parallax barriers
WO2012157195A1 (ja) * 2011-05-19 2012-11-22 パナソニック株式会社 画像表示システムおよび３次元用メガネ
JP2012253690A (ja) * 2011-06-06 2012-12-20 Namco Bandai Games Inc プログラム、情報記憶媒体及び画像生成システム
US8880223B2 (en) * 2012-07-16 2014-11-04 Florida Institute for Human & Maching Cognition Anthro-centric multisensory interface for sensory augmentation of telesurgery
US10846942B1 (en) * 2013-08-29 2020-11-24 Ultrahaptics IP Two Limited Predictive information for free space gesture control and communication
US10078712B2 (en) * 2014-01-14 2018-09-18 Energid Technologies Corporation Digital proxy simulation of robotic hardware
US10203762B2 (en) * 2014-03-11 2019-02-12 Magic Leap, Inc. Methods and systems for creating virtual and augmented reality
JP6516234B2 (ja) * 2014-04-24 2019-05-22 Ｔｉａｎｍａ Ｊａｐａｎ株式会社 立体画像表示装置
US10768708B1 (en) * 2014-08-21 2020-09-08 Ultrahaptics IP Two Limited Systems and methods of interacting with a robotic tool using free-form gestures
JP6588196B2 (ja) * 2014-10-31 2019-10-09 株式会社ソニー・インタラクティブエンタテインメント 画像生成装置、画像生成方法および較正方法
WO2017091704A1 (en) * 2015-11-25 2017-06-01 Camplex, Inc. Surgical visualization systems and displays
US10471594B2 (en) * 2015-12-01 2019-11-12 Kindred Systems Inc. Systems, devices, and methods for the distribution and collection of multimodal data associated with robots
CN110192390A (zh) * 2016-11-24 2019-08-30 华盛顿大学 头戴式显示器的光场捕获和渲染
US10299880B2 (en) * 2017-04-24 2019-05-28 Truevision Systems, Inc. Stereoscopic visualization camera and platform
EP4067014B1 (en) * 2018-02-20 2024-10-16 Tecan Trading AG Virtual pipetting
JP7294776B2 (ja) * 2018-06-04 2023-06-20 オリンパス株式会社 内視鏡プロセッサ、表示設定方法、表示設定プログラムおよび内視鏡システム
WO2020048461A1 (zh) * 2018-09-03 2020-03-12 广东虚拟现实科技有限公司 三维立体显示方法、终端设备及存储介质
JP7119896B2 (ja) * 2018-10-24 2022-08-17 トヨタ自動車株式会社 コミュニケーションロボットおよびコミュニケーションロボットの制御プログラム
US11262887B2 (en) * 2019-09-13 2022-03-01 Toyota Research Institute, Inc. Methods and systems for assigning force vectors to robotic tasks
US20210327121A1 (en) * 2020-04-15 2021-10-21 Sirisilp Kongsilp Display based mixed-reality device
JP2023103499A (ja) * 2020-06-15 2023-07-27 ソニーグループ株式会社 医用画像処理システム、手術画像制御装置、及び、手術画像制御方法
JP7676122B2 (ja) * 2020-09-10 2025-05-14 キヤノン株式会社 画像処理装置、画像処理方法、記憶媒体およびプログラム

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5684531A (en) 1995-04-10 1997-11-04 The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration Ranging apparatus and method implementing stereo vision system
JP2000042960A (ja) 1998-07-29 2000-02-15 Gifu Prefecture マニピュレータの遠隔操作装置
JP2009006410A (ja) 2007-06-26 2009-01-15 Fuji Electric Systems Co Ltd 遠隔操作支援装置および遠隔操作支援プログラム
JP2009012106A (ja) 2007-07-03 2009-01-22 Fuji Electric Systems Co Ltd 遠隔操作支援装置および遠隔操作支援プログラム
JP2010094777A (ja) 2008-10-16 2010-04-30 Fuji Electric Systems Co Ltd 遠隔操作支援装置
JP2014056496A (ja) 2012-09-13 2014-03-27 Mitsubishi Heavy Ind Ltd 仮想カメラ画像生成装置、仮想カメラ画像生成システム、仮想カメラ画像生成方法、およびプログラム
US20180236666A1 (en) 2017-02-20 2018-08-23 Irobot Defense Holdings, Inc. Robotic gripper camera
WO2019059364A1 (ja) 2017-09-22 2019-03-28 三菱電機株式会社 遠隔制御マニピュレータシステムおよび制御装置
JP2020037145A (ja) 2018-09-03 2020-03-12 川崎重工業株式会社 ロボットシステム
JP2020040184A (ja) 2018-09-12 2020-03-19 キヤノン株式会社 ロボットシステム、ロボットシステムの制御方法、制御プログラム及び記録媒体
WO2021220915A1 (ja) 2020-04-27 2021-11-04 ファナック株式会社 産業機械の表示装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
後久卓哉 他，次世代マニピュレータによる廃棄物分離選別システムの開発 第３報 最終成果報告，第２９回日本ロボット学会学術講演会予稿集ＤＶＤ－ＲＯＭ ２０１１年 Ｔｈｅ ２９ｔｈ Ａｎｎｕａｌ Ｃｏｎｆｅｒｅｎｃｅ ｏｆ ｔｈｅ Ｒｏｂｏｔｉｃｓ Ｓｏｃｉｅｔｙ ｏｆ Ｊａｐａｎ ，日本，一般社団法人日本ロボット学会，2011年09月07日

Also Published As

Publication number Publication date
JPWO2022124398A1 (https=) 2022-06-16
US11926064B2 (en) 2024-03-12
WO2022124398A1 (ja) 2022-06-16
US20240033930A1 (en) 2024-02-01

Similar Documents

Publication Publication Date Title
JP7224559B2 (ja) 遠隔制御マニピュレータシステムおよび遠隔制御支援システム
KR102379245B1 (ko) 웨어러블 디바이스 기반의 이동 로봇 제어 시스템 및 제어 방법
EP3342561B1 (en) Remote control robot system
US20260042220A1 (en) Autonomous and teleoperated sensor pointing on a mobile robot
US20200055195A1 (en) Systems and Methods for Remotely Controlling a Robotic Device
Fritsche et al. First-person tele-operation of a humanoid robot
JP4844453B2 (ja) ロボットの教示装置及び教示方法
KR101988937B1 (ko) 카메라와 로봇 핸드의 캘리브레이션 방법 및 장치
KR102001214B1 (ko) 가상 현실 기반 양팔로봇 교시 장치 및 방법
JP2012011498A (ja) ロボットアーム操作システムおよびその操作方法
WO2018043525A1 (ja) ロボットシステム、ロボットシステム制御装置、およびロボットシステム制御方法
JP7454046B2 (ja) ロボット教示装置及び作業教示方法
JP2013071231A (ja) ロボットアームの教示装置、ロボット装置、教示方法、ロボットアームの制御装置、ロボットアームの制御プログラム、並びに、集積電子回路
CN112828916A (zh) 冗余机械臂遥操作组合交互装置及冗余机械臂遥操作系统
US20220193906A1 (en) User Interface for Supervised Autonomous Grasping
CN110834330A (zh) 柔性机械臂遥操作人机交互终端及方法
JP5198078B2 (ja) 計測装置および計測方法
JP2022163836A (ja) ロボット画像の表示方法、コンピュータープログラム、及び、ロボット画像の表示システム
CN115362049A (zh) 用于修正机器人的示教位置的装置、示教装置、机器人系统、示教位置修正方法以及计算机程序
JP7731104B2 (ja) 情報処理装置、ロボットの軌道生成方法、ロボットの制御方法、プログラム、移動体及びロボットシステム
CN120839787A (zh) 基于多模态动捕融合的工业人形机器人遥操作协同系统和方法
JP4910552B2 (ja) 動作データ作成装置及びその方法
Bai et al. Kinect-based hand tracking for first-person-perspective robotic arm teleoperation
WO2021192271A1 (ja) ロボット制御システム、ロボット制御方法、およびロボット制御プログラム
JP6343930B2 (ja) ロボットシステム、ロボット制御装置、及びロボット制御方法

Legal Events

Date Code Title Description
A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20221207

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20221207

A871 Explanation of circumstances concerning accelerated examination

Free format text: JAPANESE INTERMEDIATE CODE: A871

Effective date: 20221207

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20230110

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20230207

R150 Certificate of patent or registration of utility model

Ref document number: 7224559

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250