JP2008124795A - Remote work support system and displaying method of the same - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a remote work support system which allows instructions to be transmitted easily and correctly without placing a burden on a person who gives instructions and an operator associated with presentation of instructions about a work and reception of the instructions, and to provide a displaying method of the same. <P>SOLUTION: The remote work support system comprises: head mount video displaying equipment including an electronic camera for photographing an object at work and a display unit capable of making the external world see-through; and remote instruction equipment which includes a monitor for displaying the video of the object at work photographed by the electronic camera and provides information for giving instructions on a region of the object the operator should work on to the head mount video displaying equipment. The head mount video displaying equipment includes an index display processing portion for displaying on the display unit an index for giving instructions on the region of the object the operator should work on based on the information provided by the remote instruction equipment. The index display processing portion displays the index in an overlapped manner on the corresponding region of an optical image of the object at work which the operator can observe on the screen of the display unit. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a remote work support system and a display method thereof, and more particularly, to a remote work support system including a head-mounted video display device and a display method thereof.

  Conventionally, an image (hereinafter also referred to as an image) that is detachably attached to the observer's head or face and obtained from an image display element such as a small CRT or a liquid crystal display element is displayed on the observer's eyeball by an eyepiece optical system. A head-mounted image display device, that is, an HMD (Head Mount Display) is known in which a virtual image can be observed as if the image is enlarged and projected in the air.

  The HMD is used as a display device for viewing content videos such as DVDs and videos, and for remote operation of industrial equipment or medical equipment, and has a wide variety of uses.

  For remote operation, an operator wears an HMD equipped with an electronic camera, and an instructor observes an image of a work object photographed by the electronic camera on a monitor such as a host computer provided at a remote location. However, a remote work support system is known in which appropriate information is provided to a worker by voice or video and work instructions are given.

  Here, there is a case where an instructor instructs the operator how to operate a panel (work object) on which a large number of switches having the same shape are arranged using such a remote work support system. In order to convey the operation of pressing the switch, conventionally, the instructor observes the image taken with the electronic camera on the monitor, for example, “the fourth switch from the top and the third switch from the left ...”. As you say, explain to the operator verbally the location of the switch to be operated and how to use the switch. The worker receives the instruction from the instructor by voice and executes the work.

  However, in such a conventional remote operation support system, the instructor specifies the details of the work instruction in a specific and detailed manner, such as “the fourth switch from the top and the third switch from the left ...”. There was a problem that it was very burdensome. In addition, the operator needs to listen carefully to the contents of instructions from the instructor, and like the instructor, there is a problem that it is very burdensome.

  Therefore, various remote work support systems have been studied in order to deal with such problems. For example, an operator works while observing an image of a work object photographed by an electronic camera provided in the HMD and displayed on the video display unit of the HMD, and the instructor is provided with the same video in a remote place. 2. Description of the Related Art A remote work support system is known in which a host computer is operated to give a work instruction by video to an operator while observing with a monitor of the host computer.

Specifically, in the video of the work target displayed on the monitor, when the switch is indicated by a mouse icon that the operator wants the operator to operate, the video display unit of the HMD worn by the worker is displayed. A mouse icon is displayed at the position of the switch in the image of the displayed work object (see Patent Document 1).
JP 2002-132487 A

  As described above, the remote work support system disclosed in Patent Document 1 includes a switch to be operated by the worker on the video of the work target displayed on the video display unit of the HMD worn by the worker. By overlaying and displaying a mouse icon indicating a place, the operator can easily visually recognize a switch to be operated.

  However, since the work object as an image is different in size and position from the optical image of the actual work object, even if the switch to be operated with the mouse icon can be confirmed on the image, the actual work object It is considered difficult to reach the switch by hand.

  The present invention has been made in view of the above problems, and can remotely transmit an instruction easily and accurately without imposing a burden on the instructor or the operator to present and receive a work instruction. An object is to provide a work support system and a display method thereof.

  The above object is achieved by the invention described in any one of 1 to 5 below.

1. An electronic camera that shoots a work object to be worked by an operator;
A head-mounted video display device including a display unit that displays video and allows an operator to see through the outside world;
A remote indication device that includes a monitor for displaying an image of the work object photographed by the electronic camera, and provides the head-mounted image display device with information for instructing a work site by an operator; Have
An operator wearing the head-mounted image display device is attached to the head by the remote instruction device based on the video of the work object photographed by the electronic camera displayed on the monitor by the instructor. A remote work support system for giving work instructions via a video display device,
The head-mounted image display device has an index display processing unit that displays an index indicating a part of work by an operator on the display unit based on the information provided from the remote instruction device,
The index display processing unit displays the index superimposed on a target portion in an optical image of the work target that can be observed by an operator through the screen of the display unit.

2. The remote instruction device, a field frame display processing unit that displays a field frame representing a see-through area of the display unit on the image of the work object photographed by the electronic camera and displayed on the monitor;
In the video of the work object showing the field frame displayed on the monitor by the field frame display processing unit, the coordinates of the position of the part where the operator should work are based on the position of the field frame. A coordinate detection unit to detect,
2. The remote instruction device according to claim 1, wherein the remote instruction device provides the head-mounted video display device with the coordinates of the position of the part to be worked by the worker detected by the coordinate detection unit as the information. Remote work support system.

3. The remote instruction device has an operation unit for designating a part to be worked by an operator by an operation of an instructor on an image of the work object displayed on the monitor,
3. The remote operation support system according to 2, wherein the coordinate detection unit detects the coordinates of the position of the part to be worked by the operator designated by the operation unit.

  4). The remote operation support system according to 3, wherein the operation unit is a light pen or a mouse.

5. A field frame representing a see-through area of the display unit is provided outside the image taken by an electronic camera provided in a head-mounted image display device provided with a display unit through which the user can see-through the outside world. Display method on a monitor,
A first step of matching the first index displayed at one corner of the screen of the display unit with an optical image of a second index for adjustment that a user can observe through the screen of the display unit;
When the first index and the second index coincide with each other in the first step, the position of the second index captured by the electronic camera and displayed on the monitor is displayed on the screen of the display unit. A second step of determining a corner of the field frame corresponding to the corner;
Repeating the second step to determine the position of the other corner of the field frame;
A fourth step of generating a field frame based on the positions of a plurality of corners of the field frame determined in the second step and the third step, and displaying the frame superimposed on the video imaged by the electronic camera; The display method characterized by having.

  According to the present invention, an indicator for indicating a part to be worked by an operator is displayed so as to be superimposed on a target part in an actual optical image of a work target that the worker can observe through the screen of the display unit. That is, the index is not displayed so as to be superimposed on the target part in the image of the work object as in the conventional case, but is displayed so as to be superimposed on the target part in the actual optical image. Thus, the instruction can be easily and accurately transmitted without imposing a burden on the instructor or the operator for presenting and receiving the work instruction. In addition, the operator can easily extend manually to the instructed portion, and workability can be improved.

  Embodiments of a remote work support system according to the present invention will be described below with reference to the drawings. In addition, although this invention is demonstrated based on embodiment of illustration, this invention is not limited to this embodiment.

  First, the configuration of the remote work support system 1 will be described with reference to FIG. FIG. 1 is a schematic diagram showing an outline of the overall configuration of a remote operation support system 1 according to the present invention.

  As shown in FIG. 1, the remote work support system 1 includes an HMD 2, a remote instruction device 4, and the like. The remote instruction device 4 includes a host computer 5, a video / audio recording server 6, a content server 7, and the like.

  The host computer 5 is connected to the video / audio recording server 6 and the content server 7 via the Ethernet EN, and the host computer 5 and the HMD 2 are connected to the Internet IN, mobile phone, PHS, a public network PN such as a wireless LAN, etc. Are tied together.

  In the remote work support system 1 having such a configuration, the operator B wears the HMD 2 equipped with the camera unit 28 described later, and the instructor A provides an image of the work object captured by the camera unit 28 at a remote place. While observing on the monitor 53 (to be described later) of the remote instruction device 4, the remote instruction device 4 provides the worker B with appropriate information by video and voice via the HMD 2 to give a work instruction.

  Next, the configuration of the HMD 2 corresponding to the head-mounted image display device according to the present invention will be described with reference to FIG. FIG. 2 is an overall configuration diagram of the HMD 2 according to the present invention.

  As shown in FIG. 2, the HMD 2 includes temples 21R and 21L, a bridge 22, nose pads 23R and 23L, transparent substrates 24R and 24L, a display unit 25 (LCD display unit 26, an eyepiece optical system 27), a camera unit 28, and an earphone. 29R, 29L, a microphone 30, a control unit 31, and the like.

  The HMD 2 is an index based on an image taken by the camera unit 28, an image transmitted from the remote instruction device 4, and information for instructing a part of work by the worker B provided from the remote instruction device 4. Is displayed on the LCD display unit 26 provided in the display unit 25, and an image obtained from the LCD display unit 26 is directly projected onto the eyeball of the wearer (hereinafter also referred to as worker B) by the eyepiece optical system 27. This makes it possible to observe a virtual image as if the image is enlarged and projected in the air.

  The temples 21R and 21L are long members made of a flexible elastic material or the like. The temples 21R and 21L are hooked on the wearer's ears or the temporal region and are held on the head of the wearer by the wearer. This is for adjusting the mounting position. The temples 21R and 21L are configured to be rotatable in the directions of arrows P and Q in the rotating portions 21Ra and 21La. When the HMD 2 is not used, the temples 21R and 21L are rotated in the directions of the arrows P and Q. It can be made compact by moving it along the transparent substrates 24R and 24L.

  The bridge 22 includes nose pads 23R and 23L that hold the HMD 2 on the face of the wearer, and is a short rod-like member that is stretched over a predetermined position on the transparent substrates 24R and 24L facing each other. The transparent substrate 24L is held in a relative positional relationship with a fixed gap.

  The transparent substrates 24R and 24L are substantially flat transparent bodies that form a U-shaped space 24Rs at a position corresponding to one eyeball. An eyepiece optical system 27 is fitted in a U-shaped space 24Rs formed by being surrounded by the transparent substrate 24R.

  The display unit 25 includes an LCD display unit 26, an eyepiece optical system 27, and the like. The remote instruction device 4 is connected via an image captured by the camera unit 28 or a network I / F 36 described later provided in the control unit 31. And an index based on the information transmitted from the remote instruction device 4 and information for instructing the part of the work by the worker B provided from the remote instruction device 4.

  The camera unit 28 corresponds to an electronic camera according to the present invention, and includes a zoom lens 281 (not shown), a CCD (Charge Coupled Device) 282, an image processing unit 284, and the like, and captures an external scene around the wearer. The subject optical image formed by the zoom lens 281 is photoelectrically converted by the CCD 282 to generate an image signal (video signal), and the image signal is subjected to predetermined image processing by the image processing unit 284 or the like. Video).

  Earphones 29R and 29L are inserted into the ears of the wearer and provide sound to the wearer.

  The microphone 30 converts the voice of the wearer into an audio signal.

  The control unit 31 includes a microcomputer and includes a power switch 351, an operation switch 352, and the like, and a display operation performed in the HMD 2 in response to signals from these various operation switches, a control signal from the remote instruction device 4, and the like. And image signal processing operations are comprehensively controlled.

  Here, the configuration of the display unit 25 will be described with reference to FIG. FIG. 3 is a side sectional view from the left side surface of the display unit 25 in the HMD 2 according to the present invention, and mainly shows the internal configuration of the display unit 25.

  As shown in FIG. 3, the display unit 25 includes a housing 261, an LED (Light Emitting Diode) 262, a collimator lens 263, an LCD (Liquid Crystal Display) 264 LCD display unit 26, a prism 271, and HOE (Holographic Optical). Element) 272 and the eyepiece optical system 27.

  In the state where the LED 262, the collimator lens 263, and the LCD 264 are built in the housing 261 of the LCD display unit 26, the housing 261 is diagonally upward and forward at the upper end portion of the prism 271 of the eyepiece optical system 27 (FIG. 3). Then, it is attached in such a manner as to protrude obliquely upward to the right.

  The LED 262 is a point light source composed of a white light emitting diode (LED) including a predetermined wavelength color.

  The collimator lens 263 converts the light from the LED 262 into substantially parallel light and projects it onto the LCD 264.

  The LCD 264 is provided from the video instruction signal generated by the camera unit 28, the video signal transmitted from the remote instruction device 4 via the network I / F 36 described later provided in the control unit 31, and the remote instruction device 4. An image (image) is generated based on the information for instructing the part of work performed by the worker B, and is a transmissive liquid crystal display panel, for example.

  The prism 271 is a substantially plate-shaped transparent member made of glass, transparent resin, or the like, and causes light from the LCD 264 to be reflected a plurality of times inside. The upper end portion of the prism 271 is formed in a wedge shape so that the front side (opposite side of the eyepiece surface) protrudes to be thicker upward so that most of the light from the LCD 264 can be collected inside. The thick part 271b thus formed is formed.

  An inclined surface 271a is formed at the lower end of the prism 271. The prism 271 is bonded (for example, bonded) to the inclined surface 24Ra formed on the transparent substrate 24R via the HOE 272. The front and back surfaces of the prism 271 are flush with the front and back surfaces of the transparent substrate 24R. Thereby, the prism 271 is integrated with the transparent substrate 24R in a single plate shape.

  The HOE 272 is a volume phase type holographic optical element having a positive power, which is constituted by a so-called free-form surface that is optically asymmetric with respect to the axis, and is supported at a lower end portion of the prism 271 with a predetermined inclination angle. The HOE 272 provides a hologram image to the eyeball E using the light interference phenomenon when irradiated with the light guided by the prism 271.

  In the display unit 25 having the above-described configuration, the light emitted from the LED 262 illuminates the LCD 264 through the collimator lens 263, and the image light generated by the LCD 264 by this illumination is a plurality of times in the prism 271. After reflection, the light is diffracted by the HOE 272 and guided to the wearer's eyeball E as a virtual image.

  Further, the prism 271 guides light incident from the front to the eyeball E of the wearer. As a result, the wearer can see through the outside world (a subject in front), and the remote instruction device 4 via an image captured and generated by the camera unit 28 or a network I / F 36 described later provided in the control unit 31. And an index or the like based on information transmitted from the remote control device 4 and information for instructing a part of work by the worker B provided from the remote instruction device 4 is superimposed on the outside (front subject) for visual recognition.

  Next, returning to FIG. 1, the configuration of the remote instruction device 4 will be described. The remote instruction device 4 includes a host computer 5 (hereinafter referred to as a host PC 5), an audio / video recording server 6, a content server 7, and the like.

  As shown in FIG. 1, the host PC 5 is, for example, a personal computer including a computer main body 51, a monitor 53, a mouse 55, a keyboard 57, and the like.

  The host PC 5 displays on the monitor 53 an image of the work object taken by the camera unit 28 of the HMD 2 attached to the worker B. The instructor A operates the host PC 5 with the mouse 55 and the keyboard 57 corresponding to the operation unit in the present invention, a light pen (not shown), etc. while observing the image of the work object displayed on the monitor 53, so that the HMD 2 Information necessary for work recorded in the content server 7, information for instructing a part of work by the worker B, and the like are provided, and the work instruction is given to the worker B.

  The audio / video recording server 6 stores work records such as video captured by the camera unit 28 of the HMD 2 and audio exchanged between the worker B and the instructor A.

  The content server 7 stores work support information such as video and audio necessary for the worker B to perform work.

  Next, the electric circuit configuration of the remote support system 1 will be described with reference to FIG. FIG. 4 is a block diagram of the electric circuit of the remote support system 1 according to the present invention. In FIG. 4, the same members as those shown in FIGS. 1 to 3 are given the same numbers.

  The main circuit block of the HMD 2 includes a display unit 25, a camera unit 28, a control unit 31, and the like.

  The display unit 25 includes an LCD display unit 26 and an eyepiece optical system 27. Since the operations performed in each part have been described above, the description thereof will be omitted.

  The camera unit 28 includes a zoom lens 281, a CCD (Charge Coupled Device) 282, an AFE (Analog Front End) 283, an image processing unit 284, and the like.

  The CCD 282 is a color area imaging sensor in which R (red) light, G (green) light, and B (blue) light transmission filters are arranged in a checkered pattern in pixel units (pixel units), and is connected by a zoom lens 281. The imaged subject light image is photoelectrically converted into image signals (signals consisting of a signal sequence of pixel signals received in units of pixels) of each color component of R (red) light, G (green) light, and B (blue) light. To convert.

  The AFE 283 controls the drive of the CCD 282 based on the reference clock transmitted from the control unit 31, performs a known analog signal processing on the video signal (image signal) read from the CCD 282, and converts it into a digital video signal To do.

  In this manner, the video signal read by the CCD 282 is subjected to predetermined processing by the AFE 283 and converted into a digital video signal. The digitized video signal is captured by the image processing unit 284 and subjected to predetermined processing. Hereinafter, the process for the digital video signal performed by the image processing unit 284 will be described.

  First, the digital video signal captured by the image processing unit 284 is written into an image memory 33 in the control unit 31 described later in synchronization with the readout of the video signal output from the CCD 282. That is, a digital video signal used for processing performed by the image processing unit 284 is once recorded in the image memory 33 and taken out from the image memory 33 and used for processing in each part in the image processing unit 284.

  The image processing unit 284 includes, for example, a black level correction unit, a pixel interpolation unit, a white balance control unit, a gamma correction unit, a matrix calculation unit, a shading correction unit, an image compression unit, and the like (not shown). The digital video signal is subjected to known image signal processing. Then, the digital video signal that has undergone predetermined processing at these parts is stored in the image memory 33 again.

  The control unit 31 includes a control unit 32, an image memory 33, a VRAM (Video Random Access Memory) 34, a switch group 35, a network I / F 36, and the like.

  The control unit 32 reads a ROM (Read Only Memory) that stores each control program (not shown), a RAM (Random Access Memory) that temporarily stores data such as arithmetic processing and control processing, and a control program from the ROM. Display operations and image signals performed by the HMD 2 in response to signals from the power switch 351 and various operation switches 352, control signals from the remote instruction device 4, and the like. It controls the overall processing operations.

  The index display processing unit 321 indicates the part of the work by the worker B on the display unit 25 based on the coordinates of the position of the part to be worked by the worker B detected by the coordinate detection unit 512 described later in the host PC 5. The indicator to be instructed is displayed so as to be superimposed on the target part in the optical image of the work target that can be observed through the screen of the display unit 25 by the worker B.

  The image memory 33 is a temporary memory used as a work area for performing various processing on the digital video signal by the control unit 32 in the control unit 31.

  The VRAM 34 has a video signal recording capacity corresponding to the number of pixels of the LCD 264 in the LCD display unit 26, and is a video signal buffer memory that constitutes an image reproduced and displayed on the LCD 264.

  The network I / F 36 connects the HMD 2 to the public line network PN, communicates with the server device 4, and inputs and outputs video, audio, and various control signals.

  Next, the main electric circuit block of the host PC 5 includes a computer main body 51, a monitor 53, a mouse 55, a keyboard 57, and the like.

  The computer main body 51 includes a field frame display processing unit 511, a coordinate detection unit 512, and the like.

  The field frame display processing unit 511 superimposes and displays a field frame representing the see-through area of the display unit 25 in the HMD 2 on the image of the work target photographed by the electronic camera 28 in the HMD 2 and displayed on the monitor 53. A method of displaying the field frame that is displayed so as to overlap the image of the work object displayed on the monitor 53 will be described later.

  The coordinate detection unit 512 is a position of a part on which the operator should work on the basis of the position of the field frame in the image of the work target displayed on the monitor 53 by the field frame display processing unit 511. Detect the coordinates of. Specifically, the instructor A operates the mouse 55 and the keyboard 57 to select the position of the part on which the worker should work on the monitor 53 with an icon, so that the field frame of the selected position is displayed. Calculate coordinates based on position.

  In the work support system 1 having such a configuration, the present invention makes it easy for the operator B to give instructions from the instructor A when the instructor A at a remote location instructs the worker B to work on a part to be worked. In order to be able to confirm, the index indicating the part to be worked by the worker B is controlled so as to be superimposed on the target part in the actual optical image of the work target that the worker B can observe through the screen of the display unit 25. To do.

  Here, the flow of the index display operation performed in the work support system 1 will be described with reference to FIGS. 5 and 6. FIG. 5 is a flowchart showing the flow of the index display operation performed in the remote operation support system 1 according to the present invention. FIG. 6 is a schematic diagram showing the flow of the index display operation in the remote operation support system according to the present invention.

  First, on the screen of the display unit 25 of the HMD 2 worn by the worker B, as shown in FIG. 6A, a see-through image S (optical image) of the work object X is displayed on the LCD display region (see-through field of view) D. (Step S1).

  On the other hand, on the monitor 53 of the remote instruction device 4 provided at a remote place, as shown in FIG. 6B, an image KI of the work object X photographed by the camera unit 28 of the HMD 2 is displayed. In the video KI, a field frame F representing the LCD display area (see-through field) D of the display unit 25 of the HMD 2 is displayed superimposed by the field frame display processing unit 511 (step S2). A method of displaying the field frame F displayed on the monitor 53 so as to overlap the image KI of the work target X will be described later.

  Next, the instructor A operates, for example, the mouse 55 on the monitor 53 on which the field frame F and the image KI of the work object X are displayed, and determines the position where the worker B should work, for example, the position of the switch SW. The selection is made with the mouse icon (step S3). When the switch SW is selected, the coordinate detection unit 512 calculates the coordinates T0 (T0x, T0y) of the position of the switch SW with reference to the coordinates of the positions of the four corners P, Q, R, and S of the field frame F. (Step S4), the calculated coordinates T0 (T0x, T0y) are sent to the HMD2.

Here, a method of calculating the coordinates T0 (T0x, T0y) will be described. The coordinates of the positions of the four corners P, Q, R, S and the switch SW of the field frame F with respect to the origin O (0, 0) of the monitor 53 are P (Px, Py), Q (Qx, Assuming that Qy), R (Rx, Ry), S (Sx, Sy), and T (Tx, Ty), the switch SW based on the coordinates of the positions of the four corners P, Q, R, and S of the field frame F The coordinates T0 (T0x, T0y) of the position are obtained by the following (Expression 1) and (Expression 2).
T0x = (Tx−Px) / (Qx−Px) (Formula 1)
T0y = (Ty−Py) / (Sy−Py) (Formula 2)
Next, the index display processing unit 321 of the HMD2 uses the coordinates T0 (T0x, T0y) of the position of the switch SW acquired from the coordinate detection unit 512 as the four corners of the LCD display area (see-through field of view) D of the display unit 25 of the HMD2. Conversion is made to coordinates t (tx, ty) with reference to the coordinates of the positions of p, q, r, and s (step S5).

Here, a conversion method to the coordinate t (tx, ty) will be described. If the coordinates of the four corners of the LCD display area (see-through visual field) D of the display unit 25 are p (0, 0), q (qx, 0), r (rx, ry), s (0, sy), the coordinate t (Tx, ty) is obtained by the following (formula 3) and (formula 4).
tx = T0x · qx (Formula 3)
ty = T0y · sy (Formula 4)
Next, as shown in FIG. 6C, the index display processing unit 321 is a part of the work by the worker B at the position of the coordinate t (tx, ty) of the LCD display area (see-through visual field) D of the display unit 25. A pointer PI, for example, a pointer PI is superimposed on the see-through image S (optical image) of the work object X that the worker B is observing through the screen of the display unit 25 (step S6).

  In this way, in the see-through image S (optical image) of the work object X that the worker B can observe through the screen of the display unit 25 of the HMD 2, an index (pointer PI) that indicates the part to be worked by the worker B is displayed. It can be displayed superimposed on the target site.

  Here, a display method of the visual field frame F displayed on the monitor 53 so as to overlap the image KI of the work target X will be described with reference to FIGS. FIG. 7 is a flowchart showing a flow of a method for displaying the field frame F displayed on the monitor 53 according to the present invention. FIG. 8 is a schematic diagram showing a flow of a method for displaying the field frame F displayed on the monitor 53 according to the present invention.

  First, as shown in FIG. 8 (a1), a marker M corresponding to the first index in the present invention is displayed at one corner s of the LCD display area (see-through visual field) D of the display unit 25 of the HMD2 (step S1). S1). Further, as shown in FIG. 8 (b1), the provisional visual field frame W is displayed on the monitor 53 of the host PC 5 (step S2).

  Next, as shown in FIG. 8 (a2), an adjustment chart C for adjustment corresponding to the second index in the present invention is installed at an arbitrary position in the outside world (for example, on the work object X) (step) S3). Then, as shown in FIG. 8 (b2), the image CI obtained by photographing the installed adjustment chart C with the camera unit 28 is displayed on the monitor 53 (step S4).

  Next, the worker B wearing the HMD 2 is displayed in one corner s of the LCD display area (see-through field of view) D of the display unit 25 by changing the direction of the head as shown in FIG. 8 (a3). The marker M and the optical image of the adjustment chart C that can be observed through the screen of the display unit 25 overlap each other so as to be visually recognized (step S5; first step). At this time, as shown in FIG. 8B3, the video CI of the adjustment chart C displayed on the monitor 53 and 1 of the provisional visual field frame W corresponding to one corner s of the LCD display area (see-through visual field) D are displayed. The position of the corner S may be shifted due to a difference in the position of the pupil for each worker.

  Therefore, as shown in FIG. 8 (b4), the instructor A moves the position of one corner S of the temporary visual field frame W displayed on the monitor 53 by operating the mouse 55 and the keyboard 57 to adjust. It is made to correspond with the image | video CI of the chart C (step S6). Then, the visual field frame display processing unit 511 determines the position of one corner S of the provisional visual field frame W at this time as a position corresponding to one corner s of the LCD display area (see-through visual field) D (step S7: second). Process). Similarly, the positions of the other three corners P, Q, R of the provisional field frame W respectively corresponding to the other three corners p, q, r of the LCD display area (see-through field of view) D are determined (step S8; (3rd process). Then, the field frame display processing unit 511 displays the field frame W having four corners of the determined coordinates P, Q, R, and S of the field frame (step S9; fourth step). If strict accuracy is not required for the position of the field frame F, the field frame F may be displayed based on only one corner of the field frame F or two diagonal positions.

  In this way, the visual field frame F corresponding to the LCD display area (see-through visual field) D of the display unit 25 of the HMD 2 can be displayed on the video KI of the work target X displayed on the monitor 53.

  As described above, in the remote work support system 1 according to the embodiment of the present invention, the operator B observes the index (pointer PI) indicating the part to be worked through the screen of the display unit 25 of the HMD 2. It was made to display on the target part in the see-through image S (optical image) of the work target X that can be displayed. That is, the index is not displayed so as to be superimposed on the target part in the image of the work object as in the conventional case, but is displayed so as to be superimposed on the target part in the actual optical image. Thus, the instruction can be easily and accurately transmitted without imposing a burden on the instructor or the operator for presenting and receiving the work instruction. In addition, the operator can easily extend manually to the instructed portion, and workability can be improved.

  The present invention has been described above with reference to the embodiments. However, the present invention should not be construed as being limited to the above-described embodiments, and can be changed or improved as appropriate. For example, in the above-described embodiment, the object in the optical image of the work object that can be observed through the screen of the display unit 25 by operating the mouse 55, the keyboard 57, the light pen, or the like to indicate the work site. Although it is displayed so as to be superimposed on the part, a symbol or character such as an arrow or “push here” is input by hand on the tablet PC, and the input symbol or character is superimposed on the target part in the optical image of the work object. You may make it display. Thereby, a more specific and specific instruction can be transmitted.

It is a schematic diagram which shows the whole structure of the remote work assistance system which concerns on this invention. 1 is an overall configuration diagram of an HMD according to the present invention. It is a sectional side view of the display unit in HMD concerning the present invention. It is an electric circuit block block diagram of the remote operation assistance system which concerns on this invention. It is a flowchart which shows the flow of the parameter | index display operation | movement in the remote operation assistance system which concerns on this invention. It is a schematic diagram which shows the flow of the index | index display operation | movement in the remote operation assistance system which concerns on this invention. It is a flowchart which shows the flow of the display method of the visual field frame displayed on the monitor which concerns on this invention. It is a schematic diagram which shows the flow of the display method of the visual field frame displayed on the monitor which concerns on this invention.

Explanation of symbols

1 Remote work support system 2 Head-mounted video display device (HMD)
21R, 21L Temple 22 Bridge 23R, 23L Nose pad 24R, 24L Transparent substrate 25 Display unit 26 LCD display unit 261 Case 262 LED
263 Collimator lens 264 LCD
27 Eyepiece optical system 271 Prism 272 HOE
28 Camera Unit 281 Zoom Lens 282 CCD
283 AFE
284 Image processing unit 29R, 29L Earphone 30 Microphone 31 Control unit 32 Control unit 321 Index display processing unit 33 Image memory 34 VRAM
35 Switch group 351 Power switch 352 Operation switch 36 Network I / F
4 Remote indication device 5 Host computer (host PC)
51 Computer Main Body 511 Field Frame Display Processing Unit 512 Coordinate Detection Unit 53 Monitor 55 Mouse 57 Keyboard 6 Video / Audio Recording Server 7 Content Server A Instructor B Worker C Adjustment Chart EN Ethernet E Eye IN Internet PN Public Network X Work Object

Claims (5)

An electronic camera that shoots a work object to be worked by an operator;
A head-mounted video display device including a display unit that displays video and allows an operator to see through the outside world;
A remote indication device that includes a monitor for displaying an image of the work object photographed by the electronic camera, and provides the head-mounted image display device with information for instructing a work site by an operator; Have
An operator wearing the head-mounted image display device is attached to the head by the remote instruction device based on the video of the work object photographed by the electronic camera displayed on the monitor by the instructor. A remote work support system for giving work instructions via a video display device,
The head-mounted image display device has an index display processing unit that displays an index indicating a part of work by an operator on the display unit based on the information provided from the remote instruction device,
The index display processing unit displays the index superimposed on a target portion in an optical image of the work target that can be observed by an operator through the screen of the display unit. The remote instruction device, a field frame display processing unit that displays a field frame representing a see-through area of the display unit on the image of the work object photographed by the electronic camera and displayed on the monitor;
In the video of the work object showing the field frame displayed on the monitor by the field frame display processing unit, the coordinates of the position of the part where the operator should work are based on the position of the field frame. A coordinate detection unit to detect,
The remote instruction device provides the head-mounted video display device with the coordinates of the position of the part to be worked by the worker detected by the coordinate detection unit as the information. Remote work support system. The remote instruction device has an operation unit for designating a part to be worked by an operator by an operation of an instructor on an image of the work object displayed on the monitor,
The remote work support system according to claim 2, wherein the coordinate detection unit detects coordinates of a position of a part to be worked by an operator designated by the operation unit. The remote operation support system according to claim 3, wherein the operation unit is a light pen or a mouse. A field frame representing a see-through area of the display unit is provided outside the image taken by an electronic camera provided in a head-mounted image display device provided with a display unit through which the user can see-through the outside world. Display method on a monitor,
A first step of matching the first index displayed at one corner of the screen of the display unit with an optical image of a second index for adjustment that a user can observe through the screen of the display unit;
When the first index and the second index coincide with each other in the first step, the position of the second index captured by the electronic camera and displayed on the monitor is displayed on the screen of the display unit. A second step of determining a corner of the field frame corresponding to the corner;
Repeating the second step to determine the position of the other corner of the field frame;
A fourth step of generating a field frame based on the positions of a plurality of corners of the field frame determined in the second step and the third step, and displaying the frame superimposed on the video imaged by the electronic camera; The display method characterized by having.

Images