JP2006191243A - Remote operation support system, field-side device, indication-side device, and program and method thereof - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately transmit operation indication contents of an indicator to an operator. <P>SOLUTION: A field-side device 1 transmits photographed image data of an object to be operated which is inputted through a photographed image input section 1 to an indication-side device 1 and stores the data in an image memory 141 provided on a video processing section 14. An indication information reception section 13 receives indication information including an operated object range and a moving vector sent from the indication-side device 2. The video processing section 14 converts the received operated object range and moving vector to a coordinate system of a projector 4. An operated object range image is extracted from the converted operated object range and moving vector and the photographed image data stored in the image memory 141 to generate a projection image P1 on which moving operation is reflected. A projection image output section 15 projects the generated projection image P1 over the operated object by the projector 4. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to, for example, a remote operation support system, a site-side device, an instruction-side device, a program, and a method for supporting a worker at a work site from a remote location.

  Conventionally, when it is difficult for only an on-site worker to perform the work, the work is performed while receiving instructions from a telephone from a skilled worker or supervisor (hereinafter referred to as an instructor) in a remote place. However, in the work instruction by telephone, the instructor needs to explain the contents of the instruction in detail, and the operator needs to listen carefully to the instruction contents. For this reason, there is a possibility that it takes time and effort to transmit the contents of the instruction, and may cause a work error due to erroneous recognition.

  Therefore, in recent years, voice image communication such as a videophone has been effectively used to reduce the burden of remote work support using only voice. For example, a remote work support communication device has been proposed in which a work target position from an instructor is clearly indicated in real space using a laser pointer so that the worker can easily grasp the work target (for example, , See Patent Document 1 or Patent Document 2.)

JP 2000-125042 A

Japanese Patent Laid-Open No. 2001-45451

  However, in the above conventional remote work support communication device, the position of the work target can be easily grasped in the real space, but since the operation method for the work target is transmitted by voice, the worker is made aware of the details of the operation contents. Is difficult. For example, if the instruction content is “move this operation lever to the right”, the operator can recognize the corresponding operation lever, but clearly indicate how much the corresponding operation lever is moved to the right by voice. It is difficult.

  The present invention has been made paying attention to the above circumstances. The purpose of the present invention is to provide a remote work support system and an on-site device that can accurately convey the work instruction content of the instructor to the worker. An object of the present invention is to provide an instruction side apparatus and a program and method thereof.

  In order to achieve the above object, a remote work support system, a field device, an instruction device, and a program and method thereof according to the present invention include a worker, a work object, and an imaging device and a projection device. 1 is a remote work support system comprising an on-site device installed in one space and an instruction-side device installed in a second space where a work instructor exists and capable of transmitting information to and from the on-site device. Then, the first image information obtained by photographing the work object with the photographing device is acquired, and the input of information representing the work instruction content by the work instructor is received. Then, based on the acquired first image information and the information representing the accepted work instruction content, second image information representing the state of the work object reflecting the work instruction content is created. Then, the created second image information is associated with the position where the work object is present and projected onto the first space by the projection device.

With this configuration, it is possible to accurately convey the work instruction content of the instructor to the operator. For this reason, the worker can grasp not only the work target but also the work instruction in detail.
The present invention is also characterized by having the following various configurations.
In the first configuration, the pointing device is installed in the first space where the operator and the work object are present and the photographing device and the projection device are arranged, and is installed in the second space where the work instructor exists. Is a field-side device capable of transmitting information to and from the camera, the first object is acquired by causing the photographing device to photograph the work object, and represents the work instruction content input by the work instructor. Information is received from the pointing device. Then, based on the obtained first image information and the information representing the received work instruction content, second image information representing the state of the work object reflecting the work instruction content is created. Means to do. The created second image information is associated with the position where the work object is present and projected onto the first space by the projection device.

Thereby, since the work instruction content of the instructor is clearly displayed in association with the work object, the worker can accurately grasp the work instruction content.
According to a second configuration, the means for creating the second image information includes means for obtaining information representing a displacement locus of the work instruction operation based on the information representing the work instruction content, and the acquired second image information is obtained. 1 represents the displacement trajectory of the work object including the image of the operation end point and the afterimage up to the operation end point based on the image information of 1 and the information representing the displacement trajectory of the obtained work instruction operation. Create image information. With this configuration, not only the state at the time of the operation completion in the work instruction content but also the operation process of the work instruction operation is displayed, so that the worker can grasp the work instruction content in more detail.

  The third configuration is installed in the second space where the work instructor exists, and in the first space where the worker and the work object exist, and the first imaging device and the first projection device are arranged. An instruction-side device capable of transmitting information to and from an installed site-side device, wherein the first image information obtained by photographing the work object by the first imaging device is transmitted from the site-side device. Means for receiving, and means for receiving input of information representing the contents of work instructions by the work instructor. Then, based on the received first image information and the information representing the accepted work instruction content, second image information representing the state of the work object reflecting the work instruction content is created. . The created second image information is transmitted to the on-site device so as to be projected onto the first space by the first projection device in association with the position where the work object is present. Even if configured in this way, it is possible to achieve the same effect as the first configuration, and is particularly effective when the instruction-side device has higher processing performance than the field-side device.

  According to a fourth configuration, the means for creating the second image information includes means for obtaining information representing a displacement locus of the work instruction operation based on the information representing the work instruction content, and the acquired second image information is obtained. 1 represents the displacement trajectory of the work object including the image of the operation end point and the afterimage up to the operation end point based on the image information of 1 and the information representing the displacement trajectory of the obtained work instruction operation. Create image information. Even with this configuration, it is possible to achieve the same effect as the second configuration, and not only the state at the end of the operation in the work instruction content but also the operation process of the work instruction operation is displayed. The person can grasp the work instruction content in more detail.

  In the fifth configuration, when the second imaging device and the second projection device are arranged in the second space, the means for receiving input of information representing the work instruction content is the acquired first The first image is projected onto the second space by the second projection device, and the work instruction content indicated by the work instructor in the second space is displayed together with the projected first image in the second imaging. Photographing is performed by the apparatus to obtain third image information. Then, the work instruction content is recognized from the obtained third image information, and information representing the work instruction content is generated.

In this way, the work instructor can instruct the work content as if he / she operated the work object at the work site. Thus, the work instructor can instruct the work content more accurately and in detail.
In the sixth configuration, the pointing device is installed in the first space where the worker and the work object are present and the photographing device and the projection device are arranged, and is installed in the second space where the work instructor exists. The first image information obtained by photographing the work object with the photographing device is transmitted to the instruction-side device. Also, a second image representing the state of the work object reflecting the work instruction content, created based on the first image information and information representing the work instruction content input by the work instructor. Information is received from the pointing device. Then, the received second image information is associated with the position where the work object is present and is projected onto the first space by the projection device.

  Even with this configuration, the work instruction content of the instructor is clearly displayed in association with the work object, so that the worker can accurately grasp the work instruction content.

  In short, according to the present invention, it is possible to provide a remote work support system, an on-site device, an instruction device, and a program and method thereof that can accurately convey the work instruction contents of the instructor to the worker. .

(First embodiment)
FIG. 1 is a schematic configuration diagram showing a remote work support system according to the first embodiment of the present invention.
This remote work support system is capable of connecting via a transmission line 9 between a site-side device 1 arranged at a work site and an instruction-side device 2 arranged at a support center where an instructor is present. The transmission path 9 may be either a wired transmission path or a wireless transmission path, and may be configured, for example, via a LAN (Local Area Network) or a WAN (Wide Area Network) including the Internet. A camera 3 and a projector 4 are connected to the on-site device 1, respectively. Further, a display device 8 such as a liquid crystal display (LCD) and an input device 7 such as a keyboard and a mouse are connected to the instruction side device 2.

  Further, the camera 3 and the projector 4 arranged at the work site are fixed so that the shooting range of the camera 3 and the video projection range of the projector 4 coincide as shown in FIG. In this state, the camera 3 captures a range including the operation levers L1 to L3 as a work area, and at the same time, the projector 4 projects an image so as to coincide with the work area. That is, when an image captured by the camera 3 is output to the projector 4 as it is, the image captured by the operation levers L1 to L3 is projected onto the operation levers L1 to L3 that are subjects.

FIG. 2 is a block diagram showing the functional configuration of the site side apparatus 1 and the instruction side apparatus 2 shown in FIG.
The site-side apparatus 1 includes a captured image input unit 11, a video / audio transmission / reception unit 12, an instruction information reception unit 13, a video processing unit 14, and a projection image output unit 15. In addition, a speaker 51, a microphone 61, a camera 3, and a projector 4 are connected to the site side apparatus 1.

The captured image input unit 11 performs reception processing of captured image data sent from the camera 4 via a communication cable or the like. The video / audio transmission / reception unit 12 transmits / receives a transmission signal including a video signal and an audio signal to / from the instruction-side device 2 via the transmission path 9.
The worker's voice collected by the microphone 61 is converted into an electrical signal and then input to the video / audio transmission / reception unit 12 as an audio signal. The camera 3 captures the work area and inputs a video signal including the captured image data to the captured image input unit 11. The captured image input unit 11 outputs the input video signal to the video / audio transmission / reception unit 12 and the video processing unit 14, respectively. The video / audio transmitting / receiving unit 12 synthesizes the audio signal and the video signal to generate a transmission signal, and transmits the transmission signal to the instruction-side device 2 via the transmission path 9. In addition, an audio signal is extracted from the video / audio transmission / reception unit 12 from the transmission signal that has arrived from the instruction-side device 2 via the transmission line 9, and is output loudly by the speaker 51.

  The instruction information receiving unit 13 performs a process for receiving instruction information sent from the instruction-side device 2 via the transmission path 9. The instruction information will be described later. In addition, the video processing unit 14 creates projection image data representing work content based on the instruction information received by the instruction information receiving unit 13. FIG. 3 shows a detailed functional configuration of the video processing unit 14. The video processing unit 14 includes an image memory 141, a coordinate conversion unit 142, and an operation target extraction unit 143. The instruction information received by the instruction information receiving unit 13 is converted into the coordinate system of the projector 4 by the coordinate conversion unit 142. Further, the image memory 141 is configured by, for example, a RAM or the like, and an image photographed by the camera 3 is taken into the image memory 141 at regular intervals via the photographed image input unit 11. The operation target extraction unit 143 extracts the operation target image from the captured image captured in the image memory 141 based on the instruction information converted by the coordinate conversion unit 142.

  On the other hand, the instruction-side device 2 used by the instructor includes a video / audio transmission / reception unit 21, an image output unit 22, an instruction information reception unit 23, and an instruction information transmission unit 24. In addition, a speaker 52, a microphone 62, an input device 7, and a display device 8 are connected to the instruction side device 2.

  The video / audio transmission / reception unit 21 transmits / receives a transmission signal including a video signal and an audio signal to / from the site-side apparatus 1 via the transmission path 9. The transmission signal transmitted from the site-side apparatus 1 is received by the video / audio transmission / reception unit 21 via the transmission path 9 and then converted into a video signal and an audio signal. The converted video data is displayed on the display device 8 via the image output unit 22, and the audio signal is loudly output through the speaker 52. The voice of the instructor is converted into an electric signal by the microphone 62 and input to the video / audio transmitting / receiving unit 21 as an audio signal. In the video / audio transmission / reception unit 21, a transmission signal is generated from the input audio signal, and the generated audio signal is transmitted to the site-side apparatus 1 through the transmission path 9.

  The instruction information receiving unit 23 has a function of receiving work contents from the instructor input from the input device 7 and creating instruction information from the work contents. Detailed description regarding the instruction information will be described later. The instruction information transmission unit 24 transmits the created instruction information to the field device 1 via the transmission path 9.

Here, with reference to FIG. 4 thru | or 6, operation which inputs the operation | work content by the instruction | indication side apparatus 2 is demonstrated. FIG. 4 is an example of a screen displayed on the display device 8 of the instruction side device 2 and it is assumed that the operation levers L1 to L3 photographed by the camera 3 are displayed.
In this state, first, the instructor designates the range of the operation target with the input device 7. FIG. 5 shows a state where the range A1 of the operation target is designated on the display screen of the display device 8 of FIG. When the input device 7 is composed of a mouse or the like, the range A1 of the operation target is designated by operating the mouse. In addition, when the display device 8 and the input device 7 are configured as a touch panel, the range A1 of the operation target is specified by tracing the screen with a finger.

  The operation object range A1 designated by the input device 7 is sampled by the instruction information receiving unit 23 at a constant time period or at a constant movement distance interval of a mouse or the like, and is represented in the monitor coordinate system of the display device 8. Are converted into a plurality of coordinate values. The operation target range represented by the plurality of coordinate values is temporarily stored in the instruction information receiving unit 23 and then transmitted to the field device 1 via the transmission path 9.

  Further, as shown in FIG. 6, the instructor designates the movement start position S and the movement end position E of the operation target using the input device 7. A movement vector V1 is calculated from the difference between the movement start position S and the movement end position E, and the calculated movement vector V1 is temporarily stored in the instruction information receiving unit 23 and then transmitted via the transmission path 9 to the field side apparatus. 1 is transmitted. FIG. 15 shows an example of the transmitted instruction information, which is composed of the operation target range information A1 and the movement vector V1.

Next, the operation of the field side apparatus 1 will be described according to the transmitted instruction information. FIG. 14 is a flowchart showing a processing procedure for projecting an image representing a work instruction in the site-side apparatus 1 and its processing contents.
In step S <b> 14 a, the instruction information receiving unit 13 of the site-side apparatus 1 performs reception waiting processing for instruction information transmitted from the instruction-side apparatus 2. When the transmitted instruction information is received by the instruction information receiving unit 13, the process proceeds to step S <b> 14 b, and the instruction information shown in FIG. 15 is converted into the coordinate system of the projector 4 by the coordinate conversion unit 142. Is done. Then, the operation object extraction unit 143 extracts the image A1 of the operation object from the converted coordinate value of the operation object range and the image captured in the image memory 141.

  Subsequently, in step S14c, the operation target object image A1 extracted in step S14b is moved according to the previously converted movement vector V1. An image created by the operation object extraction unit 143 in this way is shown as a projection image P1 in FIG. This is an image obtained by moving the extracted operation object image A1 based on the movement vector V1. The portion other than the operation target object image A1 in the projection image P1 is configured in black so as to be a transparent background without being filled. The projection image P1 thus created is output to the projector 4 via the projection image output unit 15 in step S14d, and is projected toward the work area by the projector 4.

FIG. 8 shows a state in which the projection image P1 is projected superimposed on the work area. In FIG. 8, the projected operation object image A1 represents an operation instruction to the operation lever L2, and the operator operates the operation lever L2 according to the operation object image A1.
As described above, in the above-described embodiment, the site-side apparatus 1 transmits the captured image data of the operation target input by the captured image input unit 11 to the instruction-side apparatus 1 and the image memory provided in the video processing unit 14. 141 is stored. The instruction information receiving unit 13 receives instruction information including the operation target range and the movement vector transmitted from the instruction-side device 2. The video processing unit 14 converts the operation target range and movement vector received by the coordinate conversion unit 142 into the coordinate system of the projector 4. Then, an operation target range image is extracted from the converted operation target range and movement vector and the captured image data stored in the image memory 141, and a projection image P1 reflecting the movement operation is created. The projection image output unit 15 projects the created projection image P1 onto the operation target by the projector 4.

  Therefore, according to the first embodiment, as shown in FIG. 8, the image A <b> 1 with the work content from the instructor is projected onto the operation lever L <b> 2 to be worked on, so that the worker can The operation lever L2 may be operated according to the projected image A1. Therefore, when the instructor wants to give an instruction “move this operation lever to the right”, it is possible to clearly instruct which lever is to be moved to the right visually. For this reason, it is possible to accurately convey the work instruction content of the instructor to the worker, thereby allowing the operator to perform an operation according to the content of the instruction.

(Second Embodiment)
FIG. 9 is a block diagram showing functional configurations of the field side apparatus and the instruction side apparatus according to the second embodiment of the present invention. This figure shows another configuration example of the instruction-side device 2 in the first embodiment. In the figure, the same parts as those in FIG. .
The instruction side device 2A newly includes a photographed image input unit 251 and an image recognition unit 252 in the instruction information receiving unit 25. Further, the photographed image input unit 251 and the camera 32 are connected, and the image output unit 22 and the projector 42 are connected.

  FIG. 10 is a diagram showing the configuration of the camera 32 and the projector 42 connected to the instruction side device 2A. The projector 42 is arranged so that a photographed image of the camera 3 sent from the field side device 1 is projected on the hand of an instructor such as a desk. Further, the camera 32 and the projector 42 are fixed so that the shooting range of the camera 32 and the projection range of the projector 42 coincide with each other, similarly to the camera 3 and the projector 4 connected to the field side apparatus 1.

  FIG. 11 shows how the work instruction is input by the instructor. The instructor designates the range of the operation target by tracing the work target projected on the desk as shown in FIG. 11, and traces the position of the movement start position and the movement end position with the finger. These finger movements are photographed by the camera 32 connected to the instruction-side device 2, and the photographed images are input to the image recognition unit 252 via the photographed image input unit 251.

  The image recognition unit 252 detects the position of the fingertip from the captured image input by the captured image input unit 251, and calculates a coordinate value indicating the range of the imaging target and a movement vector indicating the movement state. The algorithm for fingertip position detection processing can be realized by utilizing a module such as commercially available image processing software such as skin color recognition technology. Then, instruction information including the calculated coordinate value indicating the range of the operation target object and a movement vector is generated, and the generated instruction information is transmitted to the site-side apparatus 1 via the instruction information transmitting unit 24. The transmitted instruction information is processed in the field side apparatus 1 in the same manner as in the first embodiment described above, and the operation content is projected onto the work area by the projector 4.

  In the second embodiment, a camera 32 and a projector 42 are connected to the instruction side device 2A instead of the input device 7 and the display device 8, and the instruction information receiving unit 25 includes a captured image input unit 251 and an image recognition unit. 252. Then, the video of the work area sent from the on-site device 1 is output from the image output unit 22 to the projector 42, and the projector 42 projects the video of the work area on a desk or the like. The operator designates the work range and the movement state by tracing the image of the work area projected on the desk or the like with a finger. These finger movements are photographed by the camera 32 connected to the instruction-side device 2, and the photographed video is input to the image recognition unit 252 via the photographed image input unit 251. The image recognition unit 252 performs image processing, calculates a work range and a movement vector, and transmits the calculated work range and movement vector to the site-side apparatus 1 via the instruction information transmission unit 24.

  With such a configuration, the instructor can instruct the operation content as if operating the operation target on the site. Further, the projection size projected by the projector 42 connected to the instruction-side device 2A is configured to match the projection size of the projector 4. In this way, since the instructor can give a work instruction based on the actual size projected image, the work content can be instructed more accurately and in detail.

  In the above embodiment, the range of the operation target and the movement position are specified with the fingertip. However, a marker may be attached to the fingertip and the marker position may be recognized as an image. In addition to the fingertip, any movement displacement may be detected, and a pen, an indicator bar, or the like may be used.

(Third embodiment)
In the third embodiment of the present invention, the instruction information transmitted from the instruction-side device 2 is accumulated for a predetermined time, and the displacement process from the start to the end of the operation is displayed. Since the configuration of the remote operation support system is the same as that of the first embodiment, the operation for projecting an image representing instruction information will be described below.

  FIG. 16 shows an example of a configuration in which instruction information that has arrived from the instruction-side device 2 is accumulated, and is configured by range information A3 of the operation target and a plurality of movement vectors V21 to 2n. The received range information A3 of the operation object and the plurality of movement vectors V21 to 2n are converted into the coordinate system of the projector 4 by the coordinate conversion unit 142, respectively. Then, based on the converted coordinate value of the range of the operation target object, an image A3 indicating the range of the operation target object is extracted from the captured image stored in the image memory 141.

  Then, an image obtained by moving the image A3 indicating the range of the extracted operation object based on the plurality of movement vectors V21 to 2n is created, and the projected images showing the displacement process by superimposing the movement images, respectively. Create The projection image P3 created in this way is shown in FIG. As in the first embodiment, the portion other than the image A3 showing the range of the plurality of operation objects is configured in black so as to be a transparent background without being filled. The projection image P3 thus created is output to the projector 4 via the projection image output unit 15, and is projected toward the work area by the projector 4.

  FIG. 13 shows a state in which a projection image P3 representing the displacement process of the operation is projected superimposed on the work area. In FIG. 13, the projected operation object image A3 represents a displacement process of the operation instruction to the operation lever L2, and the operator operates the operation lever L2 according to the operation object image A3.

  As described above, in the third embodiment, the projection image P3 representing the displacement process from the start to the end of the operation of the operation lever L2 that is the operation target is created, and the created projection image P3 is operated by the projector 4. It is superimposed and projected toward the lever L2. As a result, it is possible to visually recognize the operation process of the work instruction to the worker, and thus it is possible to convey the work instruction in more detail.

  In the third embodiment, the instruction information received from the instruction-side device 2 is stored in the field-side device 1, but the instruction-side device 2 stores and stores coordinate values representing the operation process. You may comprise so that the coordinate value showing the performed operation process may be transmitted to the field side apparatus 1. FIG.

(Other embodiments)
In each of the above-described embodiments, the case where the work object is moved has been described as an example, but it is also possible to represent an operation for instructing rotation or deformation of the work object. In this case, the instruction information includes a change amount such as a rotation amount and a deformed size instead of the movement vector. Moreover, it is also possible to instruct the work object in combination with the above operations, such as an operation for moving and rotating the work object in series.

Moreover, in each said embodiment, although comprised so that the image processing part 14 might be provided in the field side apparatus 1, you may comprise so that the video processing part 14 may be provided in the instruction | indication side apparatus 2. FIG. In this case, projection image data representing the work content is created in the instruction side device 2, and the created projection image data is transmitted to the field side device 1 via the transmission path 9. The on-site device 1 receives the transmitted projection image data, and outputs it to the projector 4 by the projection image output unit 15 for projection. This configuration is effective when, for example, the transmission capacity of the transmission line 9 is large and the instruction-side device 2 has higher processing performance than the field-side device 1. Furthermore, the configuration of each system, the processing procedure for creating projection image data, and the processing content can be variously modified without departing from the spirit of the present invention.
The field device 1 and the instruction device 2 of the present invention can be realized by a computer and a program, and the program can be stored in a recording medium or provided via a network.

  In short, the present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. Moreover, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the above embodiments. For example, some components may be deleted from all the components shown in each embodiment. Furthermore, you may combine the component covering different embodiment suitably.

1 is a schematic configuration diagram showing a remote work support system according to a first embodiment of the present invention. The block diagram which shows the function structure of the field side apparatus and instruction | indication side apparatus which are shown in FIG. The block diagram which shows the detailed structure of the video processing part shown in FIG. The figure which shows an example of the screen displayed on the display apparatus shown in FIG. The figure which shows a mode that the operation range was instruct | indicated in the display apparatus shown in FIG. The figure which shows a mode that movement operation was instruct | indicated in the display apparatus shown in FIG. The figure which shows an example of the projection image produced in the video processing part shown in FIG. The figure which shows a mode that the projection image shown in FIG. 7 was projected with the projector. The block diagram which shows the function structure of the field side apparatus and instruction | indication side apparatus concerning 2nd Embodiment of this invention. The figure which shows arrangement | positioning of the camera and projector connected to the instruction | indication side apparatus shown in FIG. The figure which shows a mode that a work instruction | indication is input in the projection image by the projector shown in FIG. The figure which shows an example of the projection image produced in 3rd Embodiment. The figure which shows a mode that the projection image was superimposed and projected on the work object. The flowchart which projects the image showing the work instruction input in the field side apparatus. The figure which shows an example of the instruction information in 1st Embodiment. The figure which shows an example of the instruction information in 3rd Embodiment.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Site side device, 2, 2A ... Instruction side device, 3, 32 ... Camera, 4, 42 ... Projector, 51, 52 ... Speaker, 61, 62 ... Microphone, 7 ... Input device, 8 ... Output device, 9 ... Transmission path, L1, L2, L3 ... control lever, 11 ... photographed image input unit, 12 ... video / audio transmitting / receiving unit, 13 ... instruction information receiving unit, 14 ... video processing unit, 15 ... projected image output unit, 21 ... video / audio Transmission / reception unit, 22 ... Image output unit, 23 ... Instruction information reception unit, 24, 25 ... Instruction information transmission unit, 141 ... Image memory, 142 ... Coordinate conversion unit, 143 ... Operation object extraction unit, A1, A2, A3 ... Operation target range, V1, V21 to 2n ... movement vector, P1, P3 ... projection image, 251 ... captured image input unit, 252 ... image recognition unit

Claims (10)

An on-site device installed in a first space where an operator and a work object exist and an imaging device and a projection device are arranged; and an on-site device installed in a second space where a work instructor exists A remote operation support system comprising an instruction side device capable of transmitting information between
Means for obtaining first image information obtained by photographing the work object with the photographing device;
Means for accepting input of information representing work instruction content by the work instructor;
Means for creating second image information representing the state of the work object reflecting the work instruction content based on the acquired first image information and information representing the accepted work instruction content; ,
Means for projecting the created second image information into the first space by the projection device in association with the position where the work object is present, either the on-site device or the work instruction side device. A remote operation support system characterized by comprising a crab. Information is transmitted to and from an instruction side device installed in a second space where a worker and a work object are present and in which a photographing device and a projection device are arranged and in which a work instruction person exists. Which can be used on site
Means for causing the photographing device to photograph the work object and obtaining first image information;
Means for receiving, from the instruction side device, information representing work instruction content input by the work instructor;
Based on the obtained first image information and the information representing the received work instruction content, means for creating second image information representing the state of the work object reflecting the work instruction content When,
A site-side apparatus comprising: means for projecting the created second image information to the first space by the projection apparatus in association with the position where the work object exists. The means for creating the second image information includes:
Means for obtaining information representing a displacement trajectory of the work instruction operation based on the information representing the work instruction content;
Based on the acquired first image information and information representing the obtained displacement trajectory of the work instruction operation, the work object includes an operation end point image and an afterimage up to the operation end point. The field side apparatus according to claim 2, further comprising means for creating image information representing a displacement trajectory. A field-side device installed in a second space where a work instructor is present, and in which a worker and a work object are present and in which a first imaging device and a first projection device are disposed An instruction side device capable of transmitting information to and from
Means for receiving first image information obtained by photographing the work object with the first photographing device from the field side device;
Means for accepting input of information representing work instruction content by the work instructor;
Means for creating second image information representing the state of the work object reflecting the work instruction content based on the received first image information and the information representing the accepted work instruction content; ,
Means for transmitting the created second image information to the site-side apparatus so that the second image information is projected to the first space by the first projection apparatus in association with the position where the work object exists. A pointing device comprising: a pointing device. The means for creating the second image information includes:
Means for obtaining information representing a displacement trajectory of the work instruction operation based on the information representing the work instruction content;
Based on the acquired first image information and information representing the obtained displacement trajectory of the work instruction operation, the work object includes an operation end point image and an afterimage up to the operation end point. 5. The pointing device according to claim 4, further comprising means for creating image information representing a displacement trajectory. When the second imaging device and the second projection device are arranged in the second space,
Means for receiving input of information representing the work instruction content,
Means for projecting the acquired first image onto the second space by the second projection device;
Means for obtaining third image information by photographing the work instruction content indicated by the work instructor in the second space together with the projected first image by the second photographing device;
5. The instruction side device according to claim 4, further comprising means for recognizing the work instruction content from the obtained third image information and generating information representing the work instruction content. Information is transmitted to and from an instruction side device installed in a second space where a worker and a work object are present and in which a photographing device and a projection device are arranged and in which a work instruction person exists. Which can be used on site
Means for transmitting first image information obtained by photographing the work object by the photographing device to the pointing device;
Second image information that is created based on the first image information and information representing the work instruction content input by the work instructor and that represents the state of the work object reflecting the work instruction content. Means for receiving from the instructing device;
Means for projecting the received second image information to the first space by the projection device in association with the position where the work object is present. By performing information transmission between the field side device installed in the first space where the worker and the work object are present and the instruction side device installed in the second space where the work instructor is present, In the remote work support method in which the work instructor supports the work of the worker on the work object,
Photographing the work object and obtaining first image information;
Accepting input of information representing work instruction content by the work instructor;
Creating second image information representing the state of the work object reflecting the work instruction content based on the acquired first image information and the information representing the accepted work instruction content; ,
And a step of projecting the created second image information onto the first space in association with an existing position of the work object. Information is transmitted to and from an instruction side device installed in a second space where a worker and a work object are present and in which a photographing device and a projection device are arranged and in which a work instruction person exists. A program used in a field side device provided with a computer that supports the work of the worker by performing
Processing for capturing the work object by the imaging device and obtaining first image information;
A process of receiving information representing work instruction contents input by the work instructor from the instruction side device;
A process of creating second image information representing the state of the work object reflecting the work instruction content based on the acquired first image information and the information representing the received work instruction content. When,
A program for causing the computer to execute a process of causing the projection device to project the created second image information in association with an existing position of the work object onto the first space. . It is installed in the second space where the work instructor exists, and information is transmitted between the site side device installed in the first space where the photographing device and the projection device are arranged while the worker and the work object exist. A program used in an instruction-side device including a computer that supports the work of the worker by performing
Processing for receiving, from the field side device, first image information obtained by photographing the work object by the photographing device;
Processing for accepting input of information representing work instruction content by the work instructor;
A process of creating second image information representing the state of the work object reflecting the work instruction content based on the received first image information and the information representing the accepted work instruction content; ,
A process of transmitting the created second image information to the on-site device so as to be projected onto the first space by the projection device in association with the position where the work object exists. A program characterized by being executed.

Images