WO2015060393A1 - 遠隔行動誘導システム及びその処理方法 - Google Patents
遠隔行動誘導システム及びその処理方法 Download PDFInfo
- Publication number
- WO2015060393A1 WO2015060393A1 PCT/JP2014/078236 JP2014078236W WO2015060393A1 WO 2015060393 A1 WO2015060393 A1 WO 2015060393A1 JP 2014078236 W JP2014078236 W JP 2014078236W WO 2015060393 A1 WO2015060393 A1 WO 2015060393A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- camera
- time
- worker
- instructor
- video
- Prior art date
Links
- 230000009471 action Effects 0.000 title claims abstract description 32
- 238000003672 processing method Methods 0.000 title claims abstract description 21
- 238000012937 correction Methods 0.000 claims abstract description 15
- 239000000284 extract Substances 0.000 claims abstract 3
- 210000001508 eye Anatomy 0.000 claims description 18
- 230000006854 communication Effects 0.000 claims description 9
- 238000004891 communication Methods 0.000 claims description 9
- 238000006243 chemical reaction Methods 0.000 claims description 7
- 230000007175 bidirectional communication Effects 0.000 claims description 2
- 238000000034 method Methods 0.000 abstract description 8
- 230000003190 augmentative effect Effects 0.000 abstract description 4
- 230000004044 response Effects 0.000 abstract description 3
- 230000005540 biological transmission Effects 0.000 description 18
- 210000000707 wrist Anatomy 0.000 description 6
- 230000008859 change Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 239000011159 matrix material Substances 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 230000009466 transformation Effects 0.000 description 4
- 238000001514 detection method Methods 0.000 description 3
- 210000003128 head Anatomy 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 244000045947 parasite Species 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 210000005252 bulbus oculi Anatomy 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
- H04N7/147—Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1679—Programme controls characterised by the tasks executed
- B25J9/1689—Teleoperation
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
- G02B27/0172—Head mounted characterised by optical features
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B19/00—Programme-control systems
- G05B19/02—Programme-control systems electric
- G05B19/418—Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control [DNC], flexible manufacturing systems [FMS], integrated manufacturing systems [IMS] or computer integrated manufacturing [CIM]
- G05B19/41805—Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control [DNC], flexible manufacturing systems [FMS], integrated manufacturing systems [IMS] or computer integrated manufacturing [CIM] characterised by assembly
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
- H04N7/142—Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/15—Conference systems
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/0138—Head-up displays characterised by optical features comprising image capture systems, e.g. camera
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/014—Head-up displays characterised by optical features comprising information/image processing systems
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B2219/00—Program-control systems
- G05B2219/30—Nc systems
- G05B2219/31—From computer integrated manufacturing till monitoring
- G05B2219/31046—Aid for assembly, show display on screen next workpiece, task, position to be assembled, executed
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B2219/00—Program-control systems
- G05B2219/30—Nc systems
- G05B2219/32—Operator till task planning
- G05B2219/32014—Augmented reality assists operator in maintenance, repair, programming, assembly, use of head mounted display with 2-D 3-D display and voice feedback, voice and gesture command
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Definitions
- the present invention relates to a remote action guidance system in which an instructor at a remote location gives a work instruction to the worker while viewing the work image of the worker, and a processing method thereof, and more particularly, guidance to the worker by augmented reality technology.
- the present invention relates to a remote action guidance system and a processing method thereof.
- the work video and audio of a worker who performs a predetermined work at the work site are sent to an instructor at a remote location via a two-way communication network (line), and an instructor skilled in such work sends the work video.
- Remote action guidance systems that give work instructions to workers while watching are known.
- an operator uses a head-mounted display (HMD: Head Mount Display) to indicate an indicator of an operator's work site, and an object of an actual optical image of a work object through the monitor screen.
- HMD Head Mount Display
- a remote action guidance system that can be displayed in a superimposed manner on a region and can easily and accurately transmit instructions from an instructor.
- the HMD is mounted on the wearer's head and face, and images such as an organic EL display and a liquid crystal display device are projected directly onto the wearer's eyeball by an eyepiece optical system, as if the images were projected in the air. It is a device that makes it possible to observe a virtual image as if it were.
- an operator wears an HMD equipped with a camera unit, and transmits an image of a work object captured by the camera unit to a remote instructor. While the instructor observes this image on the monitor of the remote instruction device, the remote instruction device provides information on the image and voice to the worker via the HMD and gives a work instruction. According to the index given on the virtual image of the HMD, the operator can reach out to the instructed part while easily confirming the instructed part to improve workability.
- Non-Patent Document 1 discloses, as a PH (parasite human) system, for example, a remote action guidance system in which a skilled worker supports the work of an unskilled worker.
- An unskilled worker at the site and a skilled worker at a remote place wear a wearable camera, which is an HMD equipped with a camera unit, and a microphone.
- Skilled workers who are sent information from the site, operate in the virtual space through the monitor. From the image of the camera worn by the skilled worker, the image of the hand and arm of the skilled worker is extracted, or a CG image of the hand and arm is generated, and this is generated by the camera worn by the worker at the site. It is displayed overlaid on the video, and the operation of the field worker is guided by augmented reality (AR) technology.
- AR augmented reality
- Patent Document 2 describes a master-slave robot in which a robot operates in accordance with the movement of an operator at a remote place.
- a robot is also similar to the remote action guidance system described above.
- on-site workers workers
- the action is determined according to the operation information of the operator. That is, the determination timing of the action of the worker (robot) on site is different.
- the work image of the worker is sent to the instructor who is in a remote place, and the above-mentioned index or the work image of the instructor is superimposed on the work image, and again, When it is sent to the worker, a considerable amount of time is required depending on the amount of data and the state of the communication network (line). That is, due to a time delay, the returned work video may be significantly different from the current situation of the worker. In such a case, the workability of the worker is rejected.
- the present invention has been made in view of the situation as described above, and an object of the present invention is to provide an augmented reality (AR) to an operator who is in a remote place while viewing a work image of the worker. ) It is an object of the present invention to provide a remote behavior guidance system and a method for processing the same, which can suppress the obstruction of workability due to time delay when performing guidance by technology.
- AR augmented reality
- a remote behavior guidance system is a remote behavior guidance system in which an instructor in a remote location gives a work instruction to the worker while viewing a work image of the worker, the first camera provided to the worker, A first monitor, an operator computer for controlling the first camera and the first monitor, a second camera and a second monitor given to the instructor, and the second camera and the second monitor.
- An instruction side computer connected to the worker side computer via a two-way communication network, wherein the instruction side computer corresponds to an image from the first camera at time t1 at time t2.
- the work instruction video is extracted from the video of the second camera, and (1) the instructor-side computer adds the video of the first camera at time t1 to the video at time t2.
- the work instruction video is superimposed on the second monitor and displayed on the second monitor.
- the worker computer displays the work instruction video at time t2 on the video from the first camera at time t3.
- is given and superimposed, and displayed on the first monitor.
- the data transmission capacity is reduced and the data transmission time delay is reduced compared to the case where the instruction video is superimposed on the work video and transmitted.
- the hindrance to workability due to time delay can be suppressed, the worker can work accurately while receiving instructions from the instructor efficiently, and the situation at the time t3 corresponds to the situation at the scene at the time t3. it can.
- the time difference correction may be a homography conversion based on a comparison of videos from the first camera at time t1 and time t3. According to this invention, it is possible to correct the time difference more accurately, that is, it is possible to suppress the impediment to workability due to time delay, and the operator can perform the correct operation while receiving instructions from the instructor more efficiently. is there.
- the first camera may be a monocular device that covers one eye of the worker
- the second camera may be a binocular device that covers both eyes of the instructor.
- the first camera and the second camera may be binocular devices that cover both eyes of the operator and the instructor, respectively.
- the image visually recognized by the worker at the time of the operation can be almost the image at that time (time t3), and the time delay can be reduced, so that the work can be performed only with the binocular device and the sense of reality is improved. It can be done.
- the remote action guidance processing method is a remote action guidance processing method in which an instructor in a remote location gives a work instruction to the worker while viewing the work image of the worker.
- An instructor-side computer that controls a second monitor and is connected to the operator-side computer via a two-way communication network, wherein the instructor-side computer corresponds to an image from the first camera at time t1.
- a work instruction video is extracted from the video of the second camera, and (1) the instructor side computer performs the first instruction at time t1.
- the work instruction video at time t2 is superimposed on the video from the camera and displayed on the second monitor, and (2) the worker computer performs time on the video from the first camera at time t3.
- the data transmission capacity is reduced and the data transmission time delay is reduced as compared with the case where the instruction video is superimposed on the work video and transmitted.
- the time difference correction may be a homography conversion based on a comparison of images from the first camera at time t1 and time t3. According to this invention, it is possible to correct the time difference more accurately, that is, it is possible to suppress the impediment to workability due to time delay, and the operator can perform the correct operation while receiving instructions from the instructor more efficiently. is there.
- the first camera may be a monocular device that covers one eye of the operator
- the second camera may be a binocular device that covers both eyes of the instructor.
- the first camera and the second camera may be binocular devices that cover both eyes of the operator and the instructor, respectively.
- the image visually recognized by the worker at the time of the operation can be almost the image at that time (time t3), and the time delay can be reduced, so that the work can be performed only with the binocular device and the sense of reality is improved. It can be done.
- FIG. 1 is a block diagram illustrating a system according to the present invention. It is a figure which shows transmission time delay. It is a flowchart which shows the processing method by this invention. It is a block diagram which shows the processing method by this invention. It is a figure which shows the processing method by this invention. It is a figure which shows the processing method by this invention. It is a figure which shows the processing method by this invention.
- computers (PCs) 11 and 21 capable of at least video processing are installed on the worker P1 side and the instructor P2 side, respectively.
- the communication line 2 can partially include a wireless LAN, a dedicated line, or the like.
- a camera 14, a monitor 15, and an audio device (microphone and speaker) 16 are connected to the PC 11 on the worker side.
- a camera 24, a monitor 25, and an audio device (microphone and speaker) 26 are connected to the PC 21 on the instructor side. As will be described later, these are preferably incorporated as wearable devices from the viewpoint of workability.
- work status data including at least a work video (image) photographed by the camera 14 on the worker side is transmitted from the PC 11 via the line 2 to the instructor side. It transmits to PC21 (S101).
- the instructor side PC 21 displays the work image on the monitor 25.
- the instructor P2 who can grasp the work status inputs an instruction video (image) from the camera 24 and, if necessary, voice from the voice device 26 regarding the contents to be worked based on the experience. To do.
- the instruction data is transmitted from the instruction side PC 21 to the worker side PC 11 via the line 2 (S102).
- the worker side PC 11 reproduces the instruction data on the monitor 15 and the sound data by the sound device 16 with respect to the instruction data. Thereby, the worker P1 can work according to the instruction of the instructor P2.
- the instruction video of the content to be worked by the instructor P2 can be presented to the operator P1 by being superimposed on the work video showing the work environment photographed by the camera 14. According to this, it is possible for the worker P1 to grasp the contents to be visually worked with a sense of reality.
- the worker P1 can work by wearing a one-eye HMD (head display) including the monitor 15. That is, it is possible to visually recognize the actual work situation with one eye and the instruction work content shown on the HMD with the other eye. According to this, even if the work situation changes rapidly, since the actual work situation is visually recognized, a quick response can be made. Even in the case of a one-eye HMD, it is desired to further reduce the time delay ⁇ in order to eliminate the discomfort in the visual recognition of the worker P1.
- the one-eye HMD can remove the uncomfortable visual feeling, and the binocular HMD can be used to improve the sense of reality. obtain.
- the image is displayed as it is on the monitor 25 of the instructor P2 (S25).
- the instructor P2 operates from the operation tools A1, A2,... While looking at the binocular HMD monitor 25 on which the video C1 is displayed, that is, from the same viewpoint as the worker P1 at the work site.
- the control device A2 to be operated is selected, and the hand is extended and the operation is performed.
- the wrist image 6 of the instructor P2 who is about to perform the operation is photographed on the camera 24 of the instructor P2 as the video C2 (see FIG. 5B) together with the background 5 of the place. (S21).
- the video C2 is subjected to image processing in the PC 21. That is, the background 5 is erased from the video C2, and the video C2 'obtained by extracting the wrist image 6 of the instructor P2 is obtained (S22, see FIG. 5C).
- the image C2 is converted from the RGB space to the HSV space, and each value of hue (H: Hue), saturation (S: Saturation), and lightness (V: Value (Lightness)) is specified within a specific range. It can be extracted by filtering to the area inside.
- the video C2 'data is compressed and transmitted to the worker P1 side (S23).
- the video C2 ' is superimposed on the video C1 (S24) and is displayed on the monitor 25 of the binocular HMD of the instructor P2 (S25).
- the instructor P2 can perform an instruction operation with a sense of realism as if it were in the place of the worker P1.
- a homography transformation performed based on the movement of the feature points 31, 32, 33, 34... (only four points are shown here) in the video (b) at b (where t a ⁇ t b ) can be used.
- the feature points 31, 32, 33, 34... are, for example, FAST, SIFT, SURF, ORB, etc. existing on a stationary object.
- a homography matrix (planar projection transformation matrix) f is estimated based on the association of four or more feature points. Then, the specific image 8 in the video (a) is plane-projected into the specific image 8 ′ by the homography matrix f.
- the superimposed image is displayed on the monitor 15 (S15), and the worker P1 can work according to the movement of the wrist image 6 of the instructor P2 (S16).
- the remote behavior guidance system for extracting the CG image of the wrist image 6 transmitted from the instructor P2 and performing the planar projective transformation based on the estimated homography matrix and presenting it to the worker P1 and the processing method thereof are described. did.
- the work instruction video is extracted and transmitted to the worker P1
- the data transmission capacity is reduced and the time delay of data transmission is reduced compared to the case where the instruction video is superimposed on the work video and transmitted.
- , the image visually recognized by the worker P1 can be made almost the image at the time t t3. That is, it is possible to suppress the hindrance to workability due to time delay, and the worker can perform an accurate work while receiving instructions from the instructor P2 efficiently.
- voice apparatus (microphone and speaker) 16 (26) can also be used for PC11 (21), respectively.
- a part of the line 2 may include a wireless line.
- video transmission has been mainly described, audio transmission can also be performed in accordance with video.
- the required line capacity is very small compared to video, so that a transmission method can be appropriately considered in accordance with video by a known method.
- the CG image of the hand of the instructor P2 matched to the latest site situation is converted and displayed while the latest site image is displayed to the worker P1 at the site, so that the worker P1 Can give work instructions suitable for
- the conversion correction display corrects the instruction information in accordance with the on-site image that has changed between the time when the worker P1 transmits the on-site image to the instructor P2 and the time when the instruction information from the instructor P2 is received. Can be done. That is, since it is a change in the field image captured by the camera 14 of the worker P1, this corresponds to the movement of the camera 14, and typically a conversion technique such as homography conversion can be used.
- the moving object may be made translucent to clarify and be superimposed on the on-site image and displayed as described above.
- a method for detecting a moving object a method of constantly performing detection processing of the moving object on the on-site image of the camera 14 and recording it together with a detection time, or a comparison between the on-site images of the camera 14 at different times is compared. There is a method of performing detection processing of the moving object from a part.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Theoretical Computer Science (AREA)
- Mechanical Engineering (AREA)
- Quality & Reliability (AREA)
- Automation & Control Theory (AREA)
- Manufacturing & Machinery (AREA)
- Robotics (AREA)
- Computer Graphics (AREA)
- Computer Hardware Design (AREA)
- Software Systems (AREA)
- Optics & Photonics (AREA)
- Human Computer Interaction (AREA)
- Closed-Circuit Television Systems (AREA)
- User Interface Of Digital Computer (AREA)
- Studio Devices (AREA)
Abstract
Description
しかしながら、上記した遠隔行動誘導システムでは、指示者(熟練作業員)の動作情報を受けて現場の作業者(作業員)がかかる動作情報に基づいた自身の行動をその場で判断するのに対し、マスタースレーブロボットでは、その行動は操作者の動作情報に対応して決定される。すなわち、現場の作業者(ロボット)の行動の決定タイミングが異なるのである。
2 通信回線
11、21 PC
14、24 カメラ
15、25 モニタ
Claims (8)
- 作業者の作業映像を見ながら遠隔地に居る指示者が該作業者に作業指示を与える遠隔行動誘導システムであって、
前記作業者に与えられる第1カメラ及び第1モニタと、
前記第1カメラ及び前記第1モニタを制御する作業者側コンピュータと、
前記指示者に与えられる第2カメラ及び第2モニタと、
前記第2カメラ及び前記第2モニタを制御し前記作業者側コンピュータに双方向通信ネットワークで接続された指示者側コンピュータと、を含み、
前記指示者側コンピュータは、時刻t1の前記第1カメラからの映像に対応して、時刻t2において、前記第2カメラの映像から作業指示映像を抽出するとともに、
(1)前記指示者側コンピュータは、時刻t1の前記第1カメラからの映像に時刻t2の前記作業指示映像を重畳させて前記第2モニタに表示し、且つ、
(2)前記作業者側コンピュータは、時刻t3の前記第1カメラからの映像に対して時刻t2の前記作業指示映像をτ=|t3-t1|の関数である時間差補正を与えて重畳させて前記第1モニタに表示することを特徴とする遠隔行動誘導システム。 - 前記時間差補正は、時刻t1及び時刻t3の前記第1カメラからの映像の比較に基づくホモグラフィ変換であることを特徴とする請求項1記載の遠隔行動誘導システム。
- 前記第1カメラは前記作業者の片眼を覆う単眼デバイス、前記第2カメラは前記指示者の両眼を覆う両眼デバイスであることを特徴とする請求項2記載の遠隔行動誘導システム。
- 前記第1カメラ及び第2カメラは、それぞれ前記作業者及び前記指示者の両眼を覆う両眼デバイスであることを特徴とする請求項2記載の遠隔行動誘導システム。
- 作業者の作業映像を見ながら遠隔地に居る指示者が該作業者に作業指示を与える遠隔行動誘導の処理方法であって、
前記作業者に与えられる第1カメラ及び第1モニタと、
前記第1カメラ及び前記第1モニタを制御する作業者側コンピュータと、
前記指示者に与えられる第2カメラ及び第2モニタと、
前記第2カメラ及び前記第2モニタを制御し前記作業者側コンピュータに双方向通信ネットワークで接続された指示者側コンピュータと、からなるシステムにおいて、
前記指示者側コンピュータは、時刻t1の前記第1カメラからの映像に対応して、時刻t2において、前記第2カメラの映像から作業指示映像を抽出するとともに、
(1)前記指示者側コンピュータは、時刻t1の前記第1カメラからの映像に時刻t2の前記作業指示映像を重畳させて前記第2モニタに表示し、且つ、
(2)前記作業者側コンピュータは、時刻t3の前記第1カメラからの映像に対して時刻t2の前記作業指示映像をτ=|t3-t1|の関数である時間差補正を与えて重畳させて前記第1モニタに表示することを特徴とする遠隔行動誘導処理方法。 - 前記時間差補正は、時刻t1及び時刻t3の前記第1カメラからの映像の比較に基づくホモグラフィ変換であることを特徴とする請求項5記載の遠隔行動誘導処理方法。
- 前記第1カメラは前記作業者の片眼を覆う単眼デバイス、前記第2カメラは前記指示者の両眼を覆う両眼デバイスであることを特徴とする請求項6記載の遠隔行動誘導処理方法。
- 前記第1カメラ及び第2カメラは、それぞれ前記作業者及び前記指示者の両眼を覆う両眼デバイスであることを特徴とする請求項6記載の遠隔行動誘導処理方法。
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2015543910A JP6179927B2 (ja) | 2013-10-25 | 2014-10-23 | 遠隔行動誘導システム及びその処理方法 |
US15/031,672 US9538131B2 (en) | 2013-10-25 | 2014-10-23 | Remote behavior navigation system and processing method thereof |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2013-222493 | 2013-10-25 | ||
JP2013222493 | 2013-10-25 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2015060393A1 true WO2015060393A1 (ja) | 2015-04-30 |
Family
ID=52992981
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2014/078236 WO2015060393A1 (ja) | 2013-10-25 | 2014-10-23 | 遠隔行動誘導システム及びその処理方法 |
Country Status (3)
Country | Link |
---|---|
US (1) | US9538131B2 (ja) |
JP (1) | JP6179927B2 (ja) |
WO (1) | WO2015060393A1 (ja) |
Cited By (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106339094A (zh) * | 2016-09-05 | 2017-01-18 | 山东万腾电子科技有限公司 | 基于增强现实技术的交互式远程专家协作检修系统及方法 |
WO2017013986A1 (ja) * | 2015-07-17 | 2017-01-26 | シャープ株式会社 | 情報処理装置、端末、および、遠隔通信システム |
WO2017187807A1 (ja) * | 2016-04-28 | 2017-11-02 | ソニー株式会社 | 情報処理端末装置 |
CN108170273A (zh) * | 2017-12-28 | 2018-06-15 | 南京华讯方舟通信设备有限公司 | 一种基于hololens眼镜的专家远程指导系统及实现方法 |
CN112689114A (zh) * | 2021-03-11 | 2021-04-20 | 太平金融科技服务(上海)有限公司 | 对车辆的目标位置进行确定的方法、装置、设备和介质 |
WO2023281667A1 (ja) * | 2021-07-07 | 2023-01-12 | 日本電信電話株式会社 | メディア加工装置、メディア加工方法及びメディア加工プログラム |
WO2023281666A1 (ja) * | 2021-07-07 | 2023-01-12 | 日本電信電話株式会社 | メディア加工装置、メディア加工方法及びメディア加工プログラム |
KR20230146980A (ko) | 2022-04-13 | 2023-10-20 | 가부시키가이샤 히타치세이사쿠쇼 | 작업 지원 시스템 및 방법 |
WO2024057398A1 (ja) * | 2022-09-13 | 2024-03-21 | 日本電信電話株式会社 | 提示映像調整装置、提示映像調整方法及び提示映像調整プログラム |
Families Citing this family (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10454943B2 (en) | 2015-08-17 | 2019-10-22 | The Toronto-Dominion Bank | Augmented and virtual reality based process oversight |
US10339931B2 (en) | 2017-10-04 | 2019-07-02 | The Toronto-Dominion Bank | Persona-based conversational interface personalization using social network preferences |
US10460748B2 (en) | 2017-10-04 | 2019-10-29 | The Toronto-Dominion Bank | Conversational interface determining lexical personality score for response generation with synonym replacement |
EP3788510A4 (en) * | 2018-04-30 | 2021-12-08 | Hewlett-Packard Development Company, L.P. | VISUAL OVERLAYS BASED ON OPERATOR CHARACTERISTICS |
KR102305405B1 (ko) * | 2019-03-12 | 2021-09-29 | 주식회사 피앤씨솔루션 | 머리 착용형 디스플레이 장치 및 증강현실을 이용한 원격 업무지원 시스템 |
JP7339167B2 (ja) * | 2020-01-17 | 2023-09-05 | 株式会社日立製作所 | 作業支援システム及び作業支援方法 |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH11298872A (ja) * | 1998-04-07 | 1999-10-29 | Toshiba Corp | 映像通信システム |
JP2003037840A (ja) * | 2001-07-24 | 2003-02-07 | Nippon Telegr & Teleph Corp <Ntt> | レーザポインタの方向制御方法およびその装置 |
JP2003209832A (ja) * | 2002-01-11 | 2003-07-25 | National Institute Of Advanced Industrial & Technology | 遠隔環境ガイド装置 |
JP2006279648A (ja) * | 2005-03-30 | 2006-10-12 | National Institute Of Information & Communication Technology | ゆらぎ補正による画像表示方法及びその方法を用いた模型自動車等用リモートコントロールシステム |
JP2010541065A (ja) * | 2007-09-26 | 2010-12-24 | 本田技研工業株式会社 | 3次元飲料容器位置決定装置 |
Family Cites Families (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH08257948A (ja) | 1995-03-20 | 1996-10-08 | Yaskawa Electric Corp | ロボットの遠隔操作装置 |
JP4952204B2 (ja) | 2006-11-13 | 2012-06-13 | コニカミノルタホールディングス株式会社 | 遠隔作業支援システム、及びその表示方法 |
US10013531B2 (en) * | 2012-12-20 | 2018-07-03 | Accenture Global Services Limited | Context based augmented reality |
US9380261B2 (en) * | 2014-02-25 | 2016-06-28 | Cisco Technology, Inc. | Multi-camera access for remote video access |
US9641811B2 (en) * | 2014-06-16 | 2017-05-02 | Baker Hughes Incorporated | System and method for providing real-time maintenance, trouble-shooting, and process assurance for the oilfield |
US10142596B2 (en) * | 2015-02-27 | 2018-11-27 | The United States Of America, As Represented By The Secretary Of The Navy | Method and apparatus of secured interactive remote maintenance assist |
-
2014
- 2014-10-23 WO PCT/JP2014/078236 patent/WO2015060393A1/ja active Application Filing
- 2014-10-23 JP JP2015543910A patent/JP6179927B2/ja not_active Expired - Fee Related
- 2014-10-23 US US15/031,672 patent/US9538131B2/en active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH11298872A (ja) * | 1998-04-07 | 1999-10-29 | Toshiba Corp | 映像通信システム |
JP2003037840A (ja) * | 2001-07-24 | 2003-02-07 | Nippon Telegr & Teleph Corp <Ntt> | レーザポインタの方向制御方法およびその装置 |
JP2003209832A (ja) * | 2002-01-11 | 2003-07-25 | National Institute Of Advanced Industrial & Technology | 遠隔環境ガイド装置 |
JP2006279648A (ja) * | 2005-03-30 | 2006-10-12 | National Institute Of Information & Communication Technology | ゆらぎ補正による画像表示方法及びその方法を用いた模型自動車等用リモートコントロールシステム |
JP2010541065A (ja) * | 2007-09-26 | 2010-12-24 | 本田技研工業株式会社 | 3次元飲料容器位置決定装置 |
Cited By (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPWO2017013986A1 (ja) * | 2015-07-17 | 2018-06-14 | シャープ株式会社 | 情報処理装置、端末、遠隔通信システム、および、情報処理プログラム |
WO2017013986A1 (ja) * | 2015-07-17 | 2017-01-26 | シャープ株式会社 | 情報処理装置、端末、および、遠隔通信システム |
JPWO2017187807A1 (ja) * | 2016-04-28 | 2019-03-07 | ソニー株式会社 | 情報処理端末装置 |
WO2017187807A1 (ja) * | 2016-04-28 | 2017-11-02 | ソニー株式会社 | 情報処理端末装置 |
US10679581B2 (en) | 2016-04-28 | 2020-06-09 | Sony Corporation | Information processing terminal apparatus |
CN106339094B (zh) * | 2016-09-05 | 2019-02-26 | 山东万腾电子科技有限公司 | 基于增强现实技术的交互式远程专家协作检修系统及方法 |
CN106339094A (zh) * | 2016-09-05 | 2017-01-18 | 山东万腾电子科技有限公司 | 基于增强现实技术的交互式远程专家协作检修系统及方法 |
CN108170273A (zh) * | 2017-12-28 | 2018-06-15 | 南京华讯方舟通信设备有限公司 | 一种基于hololens眼镜的专家远程指导系统及实现方法 |
CN112689114A (zh) * | 2021-03-11 | 2021-04-20 | 太平金融科技服务(上海)有限公司 | 对车辆的目标位置进行确定的方法、装置、设备和介质 |
WO2023281667A1 (ja) * | 2021-07-07 | 2023-01-12 | 日本電信電話株式会社 | メディア加工装置、メディア加工方法及びメディア加工プログラム |
WO2023281666A1 (ja) * | 2021-07-07 | 2023-01-12 | 日本電信電話株式会社 | メディア加工装置、メディア加工方法及びメディア加工プログラム |
KR20230146980A (ko) | 2022-04-13 | 2023-10-20 | 가부시키가이샤 히타치세이사쿠쇼 | 작업 지원 시스템 및 방법 |
WO2024057398A1 (ja) * | 2022-09-13 | 2024-03-21 | 日本電信電話株式会社 | 提示映像調整装置、提示映像調整方法及び提示映像調整プログラム |
Also Published As
Publication number | Publication date |
---|---|
JP6179927B2 (ja) | 2017-08-16 |
US9538131B2 (en) | 2017-01-03 |
JPWO2015060393A1 (ja) | 2017-03-09 |
US20160277710A1 (en) | 2016-09-22 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP6179927B2 (ja) | 遠隔行動誘導システム及びその処理方法 | |
US11231897B2 (en) | Display system, display device, information display method, and program | |
TWI610097B (zh) | 電子系統、可攜式顯示裝置及導引裝置 | |
KR102233223B1 (ko) | 화상 표시 장치 및 화상 표시 방법, 화상 출력 장치 및 화상 출력 방법과, 화상 표시 시스템 | |
US10133532B2 (en) | Display system, display device, information display method, and program | |
JP2016224810A5 (ja) | ||
WO2016013272A1 (ja) | 情報処理装置及び情報処理方法、並びに画像表示システム | |
JP2013016020A (ja) | 作業支援システム、装置、方法及びプログラム | |
JP6590050B2 (ja) | 表示装置、表示システム、表示装置の制御方法、及び、プログラム | |
JP6419118B2 (ja) | 画像表示システム | |
JP7078568B2 (ja) | 表示装置、表示制御方法、及び表示システム | |
KR101090081B1 (ko) | 증강현실 제공 시스템 및 그 제공 방법 | |
JP5161544B2 (ja) | 画像表示システム | |
WO2021230363A1 (ja) | 遠隔制御システム、遠隔作業装置、映像処理装置およびプログラム | |
JP6733401B2 (ja) | 表示システム、表示装置、情報表示方法、及び、プログラム | |
JP2006340017A (ja) | 立体映像表示装置及び立体映像表示方法 | |
WO2020071144A1 (ja) | 情報処理装置、情報処理方法、及びプログラム | |
JP6725999B2 (ja) | 情報処理装置、情報処理装置の制御方法およびプログラム | |
WO2024057783A1 (ja) | 360度画像視点位置同定部を備える情報処理装置 | |
KR102420166B1 (ko) | 맞춤형 네트워크 전송 정보 퀄리티 조정 방법 및 시스템 | |
JP2017181605A (ja) | 情報処理装置、画像表示装置、画像表示システム、および情報処理方法 | |
WO2021106136A1 (ja) | 表示端末装置 | |
JP2023172084A (ja) | 情報処理装置、頭部装着型表示装置、情報処理方法、およびプログラム | |
JP2019029868A (ja) | 画像処理装置と、その処理方法、プログラム | |
JP2018045373A (ja) | 情報処理プログラム、情報処理端末及び情報処理方法 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 14855424 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2015543910 Country of ref document: JP Kind code of ref document: A |
|
WWE | Wipo information: entry into national phase |
Ref document number: 15031672 Country of ref document: US |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 14855424 Country of ref document: EP Kind code of ref document: A1 |