JP2014197802A - Work support system and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a work support system and a program capable of obtaining a high resolution picture generated at an imaging time of an imaged picture taken before the present time.SOLUTION: An HMD 1 generates low resolution picture data from imaged picture data, and adds a time stamp X of the imaging time to preserve in a transmission buffer 52B (P1). The HMD 1 generates high resolution picture data from imaged picture data at each thinning time, and adds a time stamp X to preserve in a storage buffer 52C (P2). The HMD 1 successively transmits the low resolution picture data to a PC 80, so that the PC 80 successively display low resolution pictures (P3). The PC 80 transmits a time stamp Y added to each low resolution image when an instructor performs a capture operation, together with a capture request signal (P4). The HMD 1 acquires high resolution data having an added time stamp X close to the time stamp Y (P5), to transmit to the PC 80 (P6).

Description

  The present invention relates to a work support system and a program capable of transmitting a captured image taken by a work terminal to an instruction terminal via a network.

  There is known a system that can capture an image with a camera, transmit the captured image to a terminal device via a network, and display the image on a display unit of the terminal device (see, for example, Patent Document 1). Since time is required for data transfer over the network, the camera of Patent Document 1 transmits a captured image captured at a predetermined resolution with a relatively low resolution to the terminal device. When the user of the terminal device performs a zoom operation on the captured image, the camera captures a high-resolution image having a resolution higher than the resolution of the captured image and transmits the captured image to the terminal device. The terminal device can switch to the enlarged display of the high resolution image when receiving the high resolution image while enlarging the captured image in accordance with the zoom operation.

JP 2007-189503 A

  However, in Patent Document 1, when a user operates a captured image, a high-resolution image is captured, so that the captured time of the captured image and the high-resolution image are different. In addition, since there is a delay in data transfer or the like on the network, there is already a delay from when the camera captures an image until the user views the captured image. In addition, there is a delay required for the high-resolution image request signal to reach the camera and a delay required for the high-resolution image to be sent via the network as the user operates the captured image. There is a possibility that a large deviation occurs between the photographing time of the original photographed image and the photographing time of the high resolution image. Since a high-resolution image having a shooting time different from that of the captured image displayed on the display unit when the user performs an operation is displayed, the user may feel uncomfortable with the image.

  The present invention has been made to solve the above-described problems, and provides a work support system and a program capable of obtaining a high-resolution image generated at the photographing time of a photographed image photographed before the current time. The purpose is to do.

  According to the first aspect of the present invention, a work terminal that can be used by an operator, can receive an instruction for the work, and can be presented to the worker, and the work is used by the instructor to send the work instruction to the worker. An operation support system in which an instruction terminal that can be transmitted to a terminal is connected via a network, wherein the work terminal includes a photographing device capable of photographing a landscape around the worker, and photographing performed by the photographing device. A storage device that stores various data including at least captured image data that is image data and low-resolution image data in which the resolution of the captured image data is lowered, and the storage device stores the captured image data in a first predetermined format. A storage control unit that performs control to store time or a predetermined number, a first acquisition unit that acquires time information indicating a time at which the imaging device captured the captured image, and the first acquisition unit Association means for associating the time information with the captured image data and the low-resolution image data; first transmission means for transmitting the low-resolution image data associated with the time information to the instruction terminal; When request information including the time information, which is information for requesting captured image data, is received from the instruction terminal, the request time is the same as or the closest time to the request time that is the time based on the time information included in the request information. Second acquisition means for acquiring the captured image data associated with the time information shown from the storage device, and second transmission means for transmitting the captured image data acquired by the second acquisition means to the instruction terminal The instruction terminal includes a first receiving unit that receives a plurality of the low-resolution image data from the work terminal, and the low-level received by the first receiving unit. Third acquisition means for acquiring the time information associated with the low resolution image data desired by the instructor among the image image data, and the low resolution image data corresponding to the low resolution image data desired by the instructor. Third requesting means for transmitting the request information, which is information for requesting captured image data and including the time information acquired by the third acquisition means, to the work terminal, and transmitted from the work terminal based on the request information And a second receiving means for receiving the captured image data.

  The work terminal can acquire captured image data associated with time information that is the same as or close to the time information of the low-resolution image viewed by the instructor at the instruction terminal. Therefore, the instructor can obtain desired photographed image data, and it is difficult to feel a sense of incongruity between the original low-resolution image and the captured photographed image. In addition, even if the delay caused by transmission / reception of low-resolution image data and captured image data via the network changes depending on the network state, the work terminal supports the low-resolution image that the instructor is browsing. Since the captured image data is acquired retrospectively from the storage device based on the time information and transmitted to the instruction terminal, it is not affected by the delay. Therefore, the instructor can obtain a desired data photographed image, and it is difficult to feel a sense of incongruity between the original low resolution image and the captured photographed image.

It is a perspective view which shows the external appearance of HMD1. It is a block diagram which shows the electrical structure of HMD1 and PC80. It is a figure which shows the instruction | indication screen 100 displayed on the display 92 of PC80. It is a flowchart of the work support program for HMD installed in HMD1. It is a flowchart of the work support program for PC installed in PC80. It is a sequence diagram for demonstrating the flow of the process which HMD1 and PC80 perform. It is a figure which shows the T3 determination table 150. FIG.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The drawings to be referred to are used to explain technical features that can be adopted by the present invention. The configuration of the illustrated apparatus is not intended to be limited only to the form, but merely an illustrative example.

  As shown in FIG. 1, an image display device (hereinafter referred to as “head mounted display” or “HMD”) 1 according to the present invention is a projection device (hereinafter referred to as “head display” or “HD”) 10. A control device (hereinafter referred to as “control box” or “CB”) 50 is provided. An operator uses the HMD 1 with the HD 10 attached to the head and the CB 50 attached to a waist belt or an arm. The HMD 1 is connected to a personal computer (hereinafter referred to as “PC”, see FIG. 2) 80 that is used by the instructor and transmits instruction information for the worker to the HMD 1 via wireless communication or wired communication. Constitutes a work support system capable of remotely supporting workers. In the following description, the upper, lower, right diagonally downward, left diagonally upward, right diagonally upward and left diagonally downward of FIG. 1 are the upper, lower, forward, backward, right and left sides of the HMD 1, respectively.

  The HD 10 is used by being attached to spectacles 5 as a dedicated wearing tool. The HD 10 may be attached to other wearing tools such as glasses, helmets, and headphones that are used daily by workers. The HD 10 irradiates the operator's eyes with image light. The HD 10 is detachably connected to the CB 50 via the harness 7. The CB 50 controls the HD 10.

  The configuration of the HD 10 will be described. The HD 10 includes a housing 2. The housing 2 includes a half mirror 3 on the right end side (left side in FIG. 1) when viewed from the worker side. The half mirror 3 is disposed in front of the operator's eyes (for example, the left eye) when the operator wears the HD 10 on the head. The HD 10 includes a liquid crystal display (hereinafter referred to as “LCD”) 11 (see FIG. 2) and an eyepiece optical system (not shown) inside the housing 2. The LCD 11 displays an image based on an image signal transmitted from the CB 50 via the harness 7. The LCD 11 may be a retinal scanning display unit that displays an image by two-dimensionally scanning laser light having an intensity corresponding to an image signal, an organic EL (Organic Electro-Luminescence) display, or the like.

  The eyepiece optical system condenses image light indicating an image displayed on the LCD 11 in order to guide the light to the operator's eyes. At least a part (for example, half) of the image light emitted from the eyepiece optical system is reflected by the half mirror 3 provided on the left side of the housing 2, and is reflected on one (for example, the left) eyeball (not shown) of the operator. Incident. Since the half mirror 3 transmits at least part of the light from the real image in the outside world, the operator can see the image superimposed on the real image (outside landscape) in his field of view.

  The glasses 5 are configured to hold the HD 10 on the operator's head. The eyeglasses 5 include a support portion 4 at the upper right end (upper left end as viewed from the operator) of the rim portion that supports the left-eye lens in the frame 6. The support unit 4 holds the housing 10 of the HD 10 and is attached to the glasses 5. The support part 4 can adjust the holding position of the housing 2 in the vertical direction and the horizontal direction. The operator can place the half mirror 3 at a position that matches the position of the eyeball.

  The eyeglasses 5 include a camera 15, a microphone (hereinafter referred to as “microphone”) 16, and an earphone 17 in a temple portion to be hung on the right ear in the frame 6. The camera 15 captures an external landscape in front of the worker's field of view. The shooting direction and angle of view of the camera 15 are set in advance according to the field of view of the operator. More specifically, the shooting direction and angle of view of the camera 15 are based on the display of image light from the LCD 11 (see FIG. 2) in which the shooting range of the external scenery by the camera 15 can be superimposed on a real image that passes through the half mirror 3. It is set to almost match the range. The camera 15 may be configured so that it can be attached to the shoulder or chest, or removed and photographed. The microphone 16 collects the voice uttered by the worker. Earphone 17 outputs the voice of the instructor received from PC 80 (see FIG. 2). The captured image captured by the camera 15 is transmitted to the PC 80, and the instructor talks with the worker using the microphone 16 and the earphone 17, and instructs the work while referring to the captured image. Since the shape of the glasses 5 itself is similar to that of normal glasses, detailed description thereof is omitted.

  The configuration of the CB 50 will be described. The CB 50 has a box-shaped housing. The CB 50 includes an operation unit 62 including a power switch 63 that incorporates a power lamp 64. An operator can operate the power switch 63 to turn the power of the HMD 1 on or off. The operator can perform various settings in the HD 10 and various operations during use via the operation unit 62.

  The CB 50 performs known wireless communication or wired communication, is connected to the PC 80 (see FIG. 2) via the network 9, and transmits / receives various data including sound data and photographed image data to / from the PC 80. The CB 50 may include a USB interface and be connected to the PC 80 using a USB cable. Note that the PC 80 described later is a personal computer, but may be other devices such as a smartphone or a tablet-type mobile terminal.

  The electrical configuration of the HMD 1 will be described with reference to FIG. The HD 10 includes an LCD 11, an LCD driver 12, an image signal receiving unit 13, and a connection controller 14. The connection controller 14 is connected to the connection controller 55 of the CB 50 via the harness 7 and performs wired communication between the HD 10 and the CB 50. The image signal receiving unit 13 is connected to the connection controller 14 and receives an image signal transmitted from the image signal transmitting unit 60 of the CB 50. The LCD driver 12 causes the LCD 11 to display an image based on the image signal received by the image signal receiving unit 13.

  The connection controller 14 is connected to the camera 15, the microphone 16, and the earphone 17. The microphone 16 includes an A / D converter (not shown), digitally converts the input sound, and transmits the converted sound to the CB 50 via the harness 7. The earphone 17 incorporates a D / A converter (not shown), and analog-converts and outputs sound data received from the CB 50 via the harness 7. The camera 15 incorporates an encoder (not shown) that encodes the YUV signal output from the CMOS type light receiving cell, and transmits it to the CB 50 via the harness 7. The camera 15 of this embodiment captures images at a predetermined frame rate C [fps] (for example, 10 fps), and outputs captured image data that is captured image data to the CB 50 frame by frame.

  The electrical configuration of the CB 50 will be described. The CB 50 includes a SoC (System on a chip) 67, an operation unit 62, a wireless communication unit 65, and a wired communication unit 66. The SoC 67 is an integrated circuit in which various functions necessary for controlling the HMD 1 are integrated. The SoC 67 includes a CPU 51, a RAM 52, a program ROM 53, a flash ROM 54, a connection controller 55, a resolution processing unit 56, an image compression unit 57, an image processing unit 58, a video RAM 59, an image signal transmission unit 60 and an interface 61. The CPU 51 controls the entire HMD 1. The CPU 51 is connected to the RAM 52, the program ROM 53, the flash ROM 54, the connection controller 55, the resolution processing unit 56, the image compression unit 57, the image processing unit 58, the video RAM 59, the image signal transmission unit 60, and the interface 61 via the system bus.

  The RAM 52 temporarily stores various flags, various data, and the like. In the execution of an HMD work support program described later, the CPU 51 secures storage areas of the frame buffer 52A, the transmission buffer 52B, and the storage buffer 52C in the RAM 52. The frame buffer 52A is a storage area used as a so-called VRAM, receives captured image data captured by the camera 15 from the HD 10, and stores captured image data for one to several frames. The transmission buffer 52B is a storage area for storing low-resolution image data (described later) to be transmitted to the PC 80 based on a known first-in-first-out (FIFO) process. The storage buffer 52C is a storage area for storing high-resolution image data (described later) to be transmitted to the PC 80 in response to a request based on the FIFO processing.

  The program ROM 53 includes an HMD work support program described later, and stores various programs executed by the CPU 51, flags used by the various programs, initial values of data, and the like. Various programs including the HMD work support program are stored in the program ROM 53 when the HMD 1 is shipped. The CPU 51 can also execute a program stored in the flash ROM 54. The flash ROM 54 stores various programs installed after the shipment of the CB 50, flags used by the various programs, data setting values, and the like. Various programs including a work support program for HMD may be stored in a storage medium (not shown), read by a reading device (not shown) and installed in the flash ROM 54, or a server that can be connected via the network 9 ( It may be installed in the flash ROM 54 by downloading from (not shown). The connection controller 55 is connected to the connection controller 14 of the HD 10 via the harness 7 and performs wired communication.

  The resolution processing unit 56 performs processing for changing the resolution of the captured image data stored in the frame buffer 52A to a predetermined resolution and generating high-resolution captured image data and low-resolution captured image data. The high-resolution captured image data is image data with the maximum resolution among the captured image data output from the camera 15. The low-resolution captured image data is image data obtained by changing the captured image data to a resolution lower than that of the high-resolution image data. As an example, if the resolution of high-resolution captured image data is represented as the number of pixels, for example, 3 million pixels, the number of pixels of low-resolution captured image data is a relatively heavy load on transmission to the PC 80 via the network 9. It is preferable that the resolution is low (for example, about 300,000 pixels). The resolution of the high-resolution captured image data and the low-resolution captured image data can be arbitrarily set according to the resolution of the captured image data.

  The image compression unit 57 performs processing for compressing the high-resolution captured image data and the low-resolution captured image data generated by the resolution processing unit 56 into, for example, a JPEG image. The CPU 51 stores the high resolution image data generated by the image compression unit 57 by compressing the high resolution captured image data in the storage buffer 52C, and compresses the low resolution captured image data to generate the low resolution image. Data is stored in the transmission buffer 52B.

  The image processing unit 58 performs processing for forming an image to be displayed on the LCD 11 of the HD 10. The video RAM 59 forms a virtual screen that virtually displays the image formed by the image processing unit 58 in the storage area. The image signal transmission unit 60 generates an image signal to be displayed on the LCD 11 based on the virtual screen formed by the video RAM 59 and transmits the image signal to the image signal reception unit 13 of the HD 10 via the harness 7. The interface 61 is connected to an operation unit 62 including a power switch 63 and a power lamp 64, and performs input / output of an input signal of a user operation, a lamp lighting signal, and the like. The interface 61 is connected to the wireless communication unit 65 and the wired communication unit 66, and inputs / outputs various data transmitted / received to / from the PC 80 connected to the network 9. The wireless communication unit 65 and the wired communication unit 66 are wirelessly or wired connected to an access point (not shown) of the network 9 and communicate with the PC 80 connected to the network 9. Note that the wireless communication unit 65 and the wired communication unit 66 may communicate with the PC 80 directly or wirelessly without using the network 9.

  The PC 80 is a known personal computer used by the instructor. The PC 80 includes a CPU 81 that controls the entire PC 80. CPU 81 is electrically connected to ROM 82, RAM 83, and input / output bus (hereinafter referred to as “I / O bus”) 85 via a data bus. The ROM 82 is a read-only storage device that stores programs such as BIOS executed by the CPU 81. The RAM 83 is a readable / writable storage device that temporarily stores data. In the execution of a PC work support program described later, the CPU 81 secures a storage area of a reception buffer (not shown) in the RAM 83. The reception buffer is a storage area for storing low-resolution image data (described later) received from the HMD 1 based on a known FIFO process.

  Connected to the I / O bus 85 are a hard disk drive (hereinafter referred to as “HDD”) 84, a communication unit 86, an audio control unit 87, a display control unit 88, and an input device 93 such as a mouse and a keyboard. The HDD 84 is a storage device in which an OS, a program, and the like are installed. A PC work support program to be described later is installed in the HDD 84. The communication unit 86 connects to an access point (not shown) of the network 9 by wired communication or wireless communication, and connects the PC 80 to the network 9. The voice control unit 87 is connected to a headset 89 including a microphone 90 and an earphone 91, and controls voice input / output of an instructor who uses the PC 80. The display control unit 88 performs a drawing process for displaying an image or the like on the display 92. The input device 93 is used by the instructor to input operations of the PC 80.

  The work support program will be described with reference to FIGS. The work support program includes an HMD work support program installed on the HMD 1 and a PC work support program installed on the PC 80. As described above, the HMD work support program is a program executed by the CPU 51 of the HMD 1. Although details will be described later, the CPU 51 that executes the work support program for HMD generally generates low-resolution image data from the captured image data captured by the camera 15 and associates it with the time stamp indicating the capturing time, and the PC 80. Process to send to. When the CPU 51 receives a capture request signal and a time stamp from the PC 80, the CPU 51 performs processing for transmitting high-resolution image data having the same time stamp to the PC 80.

  The PC work support program is a program executed by the PC 80 CPU 81 as described above. Although details will be described later, the CPU 81 that executes the work support program for PC generally performs a process of displaying a low-resolution image on the display 92 based on the low-resolution image data received from the HMD 1. When the instructor performs a capture operation, the CPU 81 acquires a time stamp associated with the low resolution image being displayed, and transmits the capture request signal and the time stamp to the HMD 1. The instructor enlarges and displays the high-resolution image based on the high-resolution image data received from the HMD 1 by the PC 80 in response to the capture request signal and checks the details, or draws an instruction for the operator. be able to.

  An instruction screen 100 on which an instructor operates in the execution of the PC work support program will be described. The instruction screen 100 shown in FIG. 3 is a screen that is displayed on the display 92 of the PC 80, and the instructor gives instructions to the worker while viewing the captured image taken by the worker. In the upper right corner of the instruction screen 100, an end button 101 for ending the execution of the application (PC work support program) is arranged. On the lower left side of the instruction screen 100, a display area 110 for displaying a photographed image photographed by the operator with the camera 15 is arranged. The display area 110 displays the low resolution image 111 based on the low resolution image data transmitted to the PC 80 via the network 9 by converting the display image data to the low resolution by the CPU 51 of the HMD 1 and compressing the display image data. The low resolution image data is transmitted at a frame rate of C [fps] (for example, 10 fps), and the CPU 81 displays the low resolution image 111 as a video in the display area 110.

  On the upper left side of the instruction screen 100, a display area 120 for displaying an image to be transmitted to the HMD 1 is arranged. When the instructor performs a capture operation, the display area 120 displays a high-resolution image 121 corresponding to one frame of the low-resolution image 111 displayed in the display area 110 at the timing as a capture image. . On the upper right side of the instruction screen 100, a tool button 130 and a capture button 131 used when the instructor edits the high-resolution image 121 displayed in the display area 120 are arranged. The tool button 130 is operated when the instructor draws or erases a free line, a character, or the like on the high resolution image 121 as an instruction to the worker. The capture button 131 is a button that is clicked when performing a capture operation for capturing an image of one frame of the low resolution image 111 desired by the instructor. The capture button 131 is also operated when canceling the capture state. The tool button 130 becomes operable when the capture button 131 is clicked and is in the capture state. The display area 120 displays a camera image on the instructor side, document data such as a document that the instructor wants to display, an image obtained by enlarging the low-resolution image 111, and the like except during the capture operation.

  When the instructor performs a capture operation, a capture icon 122 indicating that a capture state is displayed is displayed in the upper left corner of the high resolution image 121 in the display area 120. While the capture icon 122 is displayed, the instructor can use various tools (for example, the pencil tool 123, the text box 124, and the like) that can be used in response to the operation of the tool button 130. The instructor can draw characters, line drawings, and the like using a variety of tools on a layer (not shown) displayed superimposed on the captured image (high-resolution image 121). Image data of an image displayed in the display area 120, that is, an image obtained by synthesizing the captured image and the image of the layer superimposed on the captured image is transmitted to the HMD1. The CPU 51 of the HMD 1 generates an image signal based on the image data received from the PC 80, transmits it to the HD 10, and displays an image similar to the image in the display area 120 on the LCD 11.

  Next, details of processing performed by the CPU 51 of the HMD 1 and the CPU 81 of the PC 80 in accordance with the execution of the work support program will be described. First, with reference to FIG. 4, processing performed by the CPU 51 of the HMD 1 by executing an HMD work support program will be described. When the operator operates the power switch 63 provided on the CB 50 of the HMD 1, the CPU 51 performs a predetermined operation at the time of startup according to execution of a main program (not shown). The CPU 51 operates the wireless communication unit 65 and the wired communication unit 66 and connects to the network 9 via a communication unit that can be connected to the network 9. That is, the CPU 51 connects to the network 9 via the wired communication unit 66 if a network cable (not shown) that connects to the network 9 is connected to the wired communication unit 66. When the CPU 51 finds an access point (not shown) to which the wireless communication unit 65 can be connected, the CPU 51 connects to the network 9 via the wireless communication unit 65.

  The CPU 51 starts execution of the HMD work support program and performs an initial setting process (not shown) that is performed at the time of activation. The CPU 51 initializes the flags and data stored in the RAM 52, and writes the flags and data initial values stored in the program ROM 53 and the flag and data set values stored in the flash ROM 54 to the RAM 52. The CPU 51 secures storage areas for the frame buffer 52A, the transmission buffer 52B, and the storage buffer 52C in the RAM 52, respectively. The CPU 51 is connected to the PC 80 via the network 9. The CPU 51 starts a process of acquiring captured image data frame by frame from the camera 15 and storing it in the frame buffer 52A. One to several frames of captured image data are temporarily stored in the frame buffer 52A. The CPU 51 starts processing for acquiring sound data from the microphone 16 and transmitting it to the PC 80 via the network 9. The CPU 51 starts processing for receiving sound data from the PC 80 via the network 9 and outputting it from the earphone 17. Further, the CPU 51 transmits an image signal generated based on the image data received from the PC 80 to the HD 10 and starts a process of displaying an image based on the image signal on the LCD 11.

  When the initial setting process is completed, the CPU 51 converts the captured image data stored in the frame buffer 52A (S1) into high resolution image data in the image processing unit 58, and JPEG compression in the image compression unit 57. The CPU 51 transmits a test signal and high resolution image data to the PC 80 (S3). When transmitting the test signal, the CPU 51 starts a timer for measuring the delay time T1 [seconds]. The delay time T1 is a time required for data transmission / reception between the HMD 1 and the PC 80 via the network 9. As will be described later, the CPU 81 of the PC 80 transmits a capture request signal when receiving a test signal and high-resolution image data. The CPU 51 of the HMD 1 waits for the capture request signal to be received (S5: NO), and when received (S5: YES), the time taken from receiving the test signal to receiving the capture request signal is set as the delay time T1. Ask.

  The CPU 51 determines the storage time T2 [seconds] based on the delay time T1 (S7). The storage time T2 is a time for holding the high resolution image data stored in the storage buffer 52C. The storage time T2 is the time taken for the low-resolution image data to be transmitted from the HMD 1 to the PC 80 in order for the instructor to browse the low-resolution image on the basis of the time when the camera 15 takes the image, and the low-resolution image It is obtained in consideration of the time taken for the capture request signal for requesting high resolution image data corresponding to the data to arrive from the PC 80 (ie, the delay time T1). Specifically, the storage time T2 is obtained by adding the margin α to the delay time T1. In the present embodiment, the margin α is set to “5” as an initial value, for example. That is, a time five times the delay time T1 is set as the storage time T2 for holding the high resolution image data in the storage buffer 52C on the basis of the image capturing time.

  The CPU 51 determines a thinning time T3 [second] based on the storage time T2 (S9). In order for the CPU 51 that has received the capture request signal to transmit the corresponding high-resolution image data to the PC 80, the storage buffer 52C has a high-resolution image corresponding to a delay time (that is, T2 time) in consideration of the margin α. Retain data. The storage capacity A [bit] of the storage buffer 52C is set in advance. When the amount of data corresponding to one frame of the high resolution image data is B [bit], the number of frames of the high resolution image that can be held in the storage buffer 52C is given by (A / B) [frame]. When the camera 15 captures a captured image at a frame rate C [fps], the number of frames of a high-resolution image for T2 hours is C × T2 [frames]. However, when the storage capacity A of the storage buffer 52C is small, the data amount corresponding to one frame of the high resolution image data is large, the frame rate C is large, etc. (A / B) <(C × T2) In this case, the high resolution image data for T2 hours cannot be held in the storage buffer 52C. In such a case, the CPU 51 secures high-resolution image data for T2 hours by thinning out the high-resolution image data based on the thinning-out time T3 and storing it in the storage buffer 52C. The thinning time T3 may be set as necessary based on the delay time considering the margin α and the time that can be stored in the storage buffer 52C.

  Specifically, the thinning-out time T3 is given as an integer multiple of (1 / C) equal to or greater than T2 / (A / B). Each time the thinning time T3 elapses, the CPU 51 saves the high resolution image data generated at that time in the storage buffer 52C. However, in the case of T3 <(1 / C), T3 is less than one frame time, which means that it is not particularly necessary to thin out. In this case, the CPU 51 stores all the high resolution image data generated at the frame rate C in the storage buffer 52C. When obtaining the thinning time T3, the CPU 51 starts a timer for measuring the thinning time T3. At the time of the first execution, the CPU 51 starts counting the timer from T3 seconds so that the timer of the thinning time T3 times out in the process of S21 described later.

  The CPU 51 stores one frame of captured image data newly captured by the camera 15 in the frame buffer 52A (S11). The CPU 51 acquires the current time as a time stamp (for convenience, “time stamp X”) from a time timer (not shown) that operates in accordance with the execution of the main program (not shown) (S13). The CPU 51 lowers the resolution of the captured image data stored in the frame buffer 52A in the resolution processing unit 56, and generates a low-resolution image data by performing a process of compressing it into a JPEG image in the image compression unit 57 (S15). . The CPU 51 associates the time stamp X acquired in the process of S13 with the low resolution image data (S17), and stores it in the transmission buffer 52B. For example, the time stamp X is added to the area for storing the properties of the low resolution image data. The CPU 51 transmits the low resolution image data stored in the transmission buffer 52B to the PC 80 via the network 9 (S19). In addition, in order to transmit the low resolution image data to the PC 80 according to the frame rate C, the transmission buffer 52B stores the low resolution image data for a plurality of frames. The CPU 51 transmits the low resolution image data with the oldest time indicated by the time stamp X to the PC 80 in order.

  The CPU 51 refers to a timer for measuring the thinning time T3. When the thinning time T3 has elapsed (S21: YES), the CPU 51 resets the timer for measuring the thinning time T3 to zero, starts a new time measurement, and proceeds to S23. Proceed. The CPU 51 compares the current time acquired in S13 with the time stamp X associated with the high resolution image data stored in the storage buffer 52C. The CPU 51 determines whether or not the high-resolution image data associated with the time stamp X of the time before the storage time T2 [seconds] from the current time is stored in the storage buffer 52C (S23). If the corresponding high-resolution image data is stored in the storage buffer 52C (S23: YES), the CPU 51 deletes the corresponding high-resolution image data and advances the process to S27 (S25). If the corresponding high resolution image data is not stored in the storage buffer 52C (S23: NO), the CPU 51 proceeds the process to S27 as it is.

  In the resolution processing unit 56, the CPU 51 generates photographic image data having the same high resolution as the resolution of the photographic image data stored in the frame buffer 52A. In the image compression unit 57, the CPU 51 performs processing for compressing the high-resolution captured image data into a JPEG image to generate high-resolution image data. The CPU 51 associates the generated high-resolution image data with the time stamp X acquired in the process of S13 (S27) and stores it in the storage buffer 52C (S29). As described above, the time stamp X is added to the area for storing the properties of the high resolution image data. As described above, by deleting the high-resolution image data exceeding the storage time T2 from the storage buffer 52C in the processing of S23 and S25, and saving the new high-resolution image data in the processing of S29, the CPU 51 In the storage buffer 52C, a storage process for holding high-resolution image data for a predetermined time (storage time T2) can be performed based on a so-called FIFO process.

  The CPU 51 advances the process to S31, and if the capture request signal has not been received from the PC 80 (S31: NO), advances the process to S39. If the operator does not operate the power switch 63 (S39: NO), the CPU 51 returns the process to S11. The CPU 51 performs processing for generating low-resolution image data from the captured image data of the next frame, associating with the time stamp X, and transmitting it to the PC 80 (S11 to S19). In S21, if the thinning time T3 has not elapsed (S21: NO), the CPU 51 advances the process to S31 and does not perform the process of saving the high resolution image data in the storage buffer 52C. When T3 <1 / C and the thinning-out time T3 is less than one frame time, the time interval for storing one frame of captured image data captured by the camera 15 in the processing of S11 is stored in the frame buffer 52A. Since the thinning time T3 has elapsed, the CPU 51 always advances the process to S23 in S21.

  Similarly to the above, if the capture request signal has not been received (S31: NO) and the power switch 63 has not been operated (S39: NO), the CPU 51 returns the process to S11 and captures the captured image data of the next frame in the frame buffer 52A. Save to. Similarly, the CPU 51 generates low-resolution image data from the photographed image data (S15), associates the time stamp X with it (S17), and transmits it to the PC 80 (S19). Only when the thinning time T3 has passed (S21: YES), the CPU 51 generates high-resolution image data, associates the time stamp X with it (S27), and stores it in the storage buffer 52C (S29).

  In S31, when a capture request signal is received from the PC 80 (S31: YES), the CPU 51 obtains a time stamp (referred to as “time stamp Y” for convenience) received together with the capture request signal. The time stamp Y is the time stamp X stored in the property of the low-resolution image 111 displayed in the display area 110 at the timing when the instructor performs the capture operation in the execution of the PC work support program described later. . The CPU 51 compares the time indicated by the time stamp Y acquired together with the capture request signal (hereinafter referred to as “request time”) with the time stamp X associated with the high resolution image data stored in the storage buffer 52C. The CPU 51 acquires high-resolution image data whose time indicated by the time stamp X is closest to the request time indicated by the time stamp Y (S33). In other words, the CPU 51 acquires the high-resolution image data having the smallest time difference between the time indicated by the time stamp X and the request time indicated by the time stamp Y among the high-resolution image data stored in the storage buffer 52C. .

  The CPU 51 transmits the acquired high resolution image data to the PC 80 via the network 9 (S35). The CPU 51 waits for reception of a capture stop signal from the PC 80 (S37: NO). When the CPU 51 receives the capture stop signal (S37: YES), if the power switch 63 is not operated (S39: NO), the process returns to S11. As described above, the CPU 51 resumes the process of repeatedly transmitting the low-resolution image data based on the captured image data and thinning and storing the high-resolution image data for each frame. When the operator operates the power switch 63 (S39: YES), the CPU 51 ends the execution of the HMD work support program.

  Next, a process performed by the CPU 81 of the PC 80 by executing the PC work support program will be described with reference to FIG. When the instructor operates the PC 80 to start the PC work support program, the CPU 81 executes the PC work support program installed in the HDD 84. The CPU 81 initializes flags and data stored in the RAM 83 in an initial setting process (not shown) performed when starting the PC work support program, operates the communication unit 86, and connects to the network 9. The CPU 81 starts processing for acquiring sound data from the microphone 90 of the headset 89 and transmitting it to the HMD 1 via the network 9. The CPU 81 starts processing for receiving sound data from the HMD 1 via the network 9 and outputting it from the earphone 91 of the headset 89. Further, the CPU 81 starts a process of generating image data based on an image displayed in the display area 120 of the instruction screen 100 (see FIG. 3) and transmitting it to the HMD 1. The CPU 81 secures a storage area for a reception buffer (not shown) in the RAM 83.

  When the initial setting process is completed, the CPU 81 waits for reception of a test signal and high resolution image data from the HMD 1 (S51: NO). When the CPU 81 receives the test signal and the high resolution image data (S51: YES), the CPU 81 transmits a capture request signal to the HMD 1 (S53). As described above, the CPU 51 of the HMD 1 determines the delay time T1 based on the capture request signal transmitted by the CPU 81 of the PC 80 in the process of S53.

  The CPU 81 receives the low-resolution image data transmitted by the CPU 51 of the HMD 1 in the process of S19 (S55) and stores it in the reception buffer of the RAM 83. The CPU 81 reads the low resolution image data stored in the reception buffer, and displays the low resolution image 111 based on the low resolution image data in the display area 110 (see FIG. 3) of the instruction screen 100 (S57). As described above, when the instructor performs a capture operation of clicking the capture button 131, the CPU 81 displays a capture image (a high-resolution image 121 corresponding to the low-resolution image 111) in the display area 120. If the capture operation has not been performed (S59: NO), the CPU 81 advances the process to S73. If the instructor has not performed an operation of clicking the end button 101 (S73: NO), the process returns to S55.

  The CPU 81 receives the low-resolution image data of the next frame, stores it in the reception buffer, and waits for the capture operation to be performed while performing the process of displaying it in the display area 110. In the display area 110, the low-resolution image 111 is displayed as a video that is replaced according to the frame rate C. If the capture operation is performed by the instructor in S59 (S59: YES), the CPU 81 uses the time stamp X stored in the property of the low resolution image 111 displayed in the display area 110 at that time as the time stamp. Obtained as Y (S61).

  The CPU 81 transmits a capture request signal and a time stamp Y to the HMD 1 (S63). As described above, when the CPU 51 of the HMD 1 receives the capture request signal, the CPU 51 of the HMD 1 acquires high-resolution image data whose time indicated by the time stamp X is closest to the request time indicated by the time stamp Y from the storage buffer 52C. , To the PC 80. The CPU 81 of the PC 80 receives the high resolution image data from the HMD 1 (S65) and stores it in the reception buffer. The CPU 81 reads the high-resolution image data stored in the reception buffer, and displays the high-resolution image 121 based on the high-resolution image data as a capture image in the display area 120 (see FIG. 3) of the instruction screen 100 (see FIG. 3). S67). As described above, the image data of the image displayed in the display area 120 is transmitted to the HMD 1 and displayed on the LCD 11. Note that a high-resolution image acquisition message is displayed until a high-resolution image is acquired and displayed in the display area 120 after the capture operation is performed. Alternatively, the low resolution 111 may be displayed in the display area 120 so that it can be edited.

  As shown in FIG. 3, the instructor can edit the captured image by operating the tool button 130 in the capture state. For example, when the pencil button 132 of the tool button 130 is clicked, the cursor changes to the pencil tool 123, and the instructor can draw a line drawing 125 or the like on the layer superimposed on the captured image. When the text button 133 of the tool button 130 is clicked, the text box 124 is displayed on the layer, and the instructor can write characters or the like in the text box 124. On the LCD 11 of the HMD 1, a line drawing 125 and a text box 124 drawn by the instructor are displayed on the captured image, and a work instruction is transmitted from the instructor to the operator.

  As shown in FIG. 5, the CPU 81 stands by while the instructor is editing the captured image (S69: NO). When the instructor finishes editing the captured image and clicks the capture button 131 (S69: YES), the CPU 81 cancels the capture state and transmits a capture stop signal to the HMD 1 (S71). As described above, when receiving the capture stop signal, the CPU 51 of the HMD 1 restarts the process of repeatedly transmitting the low resolution image data based on the captured image data and thinning out the high resolution image data for each frame. . The CPU 81 of the PC 80 advances the process to S73. If no application termination operation has been performed, the process returns to S55 (S73: NO), and the process of displaying the low-resolution image as a video in the display area 110 is resumed. When the instructor performs an operation of clicking the end button 101 in S73 (S73: YES), the CPU 81 ends the execution of the PC work support program.

  In the execution of the work support program described above, the flow of data transmitted and received between the HMD 1 and the PC 80 will be described with reference to the sequence diagram of FIG. For convenience of explanation, each process described in the sequence diagram is an excerpt of each process of the work support program (see FIGS. 4 and 5). Therefore, the same step numbers as those of the steps of the flowchart of the electronic conference program are given for explanation. In FIG. 6, a delimiter for explanation is indicated by a circled character “P” to which numbers are added. In FIG. 6, an arrow extending downward from the HMD 1 and the PC 80 in the drawing indicates a time axis.

  (P1) The CPU 51 of the HMD 1 stores the low-resolution image data to which the time stamp X is added in the transmission buffer 52B in the process of S17, and sequentially transmits it to the PC 80 in the process of S19. For example, when the frame rate is set to 10 fps, the CPU 51 transmits low-resolution image data to which a new time stamp X is added every 100 ms.

  (P2) In the process of S29, the CPU 51 stores the high resolution image data to which the time stamp X is added in the storage buffer 52C at every thinning time T3 (for example, 1 second). That is, the high-resolution image data to which the time stamp X indicating the same shooting time as the low-resolution image data transmitted every 100 ms is added is stored in the storage buffer 52C every second.

  (P3) The CPU 81 of the PC 80 displays the low-resolution image data received in the process of S55 as an image on the display area 110 of the instruction screen 100 sequentially in the process of S57. The low resolution image is displayed on the PC 80 with a delay from the photographing time indicated by the time stamp X due to a delay associated with transmission via the network 9.

  (P4) With the operator's capture operation, the CPU 81 transmits a capture request signal to the HMD 1 in the process of S63. The CPU 81 further sets the time indicated by the time stamp X of the low resolution image displayed in the display area 110 when the capture operation is performed (here, “15: 10: 15.200”) as the time stamp Y. And transmitted together with the capture request signal.

  (P5) In the process of S31, the CPU 51 of the HMD 1 that has received the capture request signal compares the time stamp Y with the time stamp X of the high-resolution image data stored in the storage buffer 52C in the process of S33. In the storage buffer 52C, at least high-resolution image data whose time stamp X indicates “15: 10: 15,000” and high-resolution image data whose time stamp X indicates “15: 10: 16,000”. And are stored.

  (P6) In the processing of S35, the CPU 51 sets the time stamp X indicating the time (“15: 10: 15,000”) closest to the request time (“15: 10: 15.200”) indicated by the time stamp Y. The associated high resolution image data is transmitted to the PC 80.

  (P7) When the CPU 81 of the PC 80 receives the high resolution image data from the HMD 1 in the process of S65, the CPU 81 displays the high resolution image in the display area 120 of the instruction screen 100 in the process of S67.

  (P8) When the instructor finishes editing the high-resolution image, the CPU 81 transmits a capture stop signal to the HMD 1 in the process of S71.

  (P9) In the process of S37, the CPU 51 of the HMD1 that has received the capture stop signal saves the low-resolution image data in the transmission buffer 52B in the process of S17, and stores it in the storage buffer 52C in the process of S29. The process of saving the high resolution image data is resumed.

  The present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the gist of the present invention. In the work support program for HMD, the CPU 51 performs the processing of S1 to S9 at the start of program execution, and determines the storage time T2 and the thinning time T3 according to the communication state of the network 9. For example, the CPU 51 may periodically perform the processing of S1 to S9 so that the storage time T2 and the thinning time T3 can be determined according to the communication state of the network 9 that changes every moment. . Specifically, for example, a timer for starting time measurement is provided in the initial setting process (not shown) of the work support program for HMD shown in FIG. 4, and the CPU 51 performs the process after S39: NO every time a predetermined time elapses. What is necessary is just to return to S1 instead of S11. In this case, the PC work support program shown in FIG. 5 is similarly provided with a timer, and the CPU 81 may return the process to S51 instead of S55 after S73: NO each time a predetermined time elapses. .

  In the processing of S17 and S19, the CPU 51 adds a time stamp X to all the low resolution image data and transmits it to the PC 80. For example, the CPU 51 may add the time stamp X only to the low-resolution image data corresponding to the high-resolution image data stored in the storage buffer 52C and transmit the data to the PC 80. For example, a process similar to S21 for determining whether T3 seconds have elapsed is added between the process of S15 and the process of S17, and when it has elapsed, the CPU 51 advances the process to S17 and has not elapsed. In that case, the process may proceed to S19. In this case, the CPU 81 of the PC 80 acquires the time stamp X every time it receives the low-resolution image data to which the time stamp X is added, and overwrites and saves it in the RAM 83 as the time stamp Y. This process may be added between the process of S55 and the process of S57, for example. If there is a capture operation by the instructor, the CPU 81 may acquire the latest time stamp Y overwritten and saved in the RAM 83 in the process of S61 and transmit it together with the capture request signal in the process of S63. The CPU 51 of the HMD 1 may transmit high-resolution image data associated with the time stamp X that matches the time stamp Y to the PC 80.

  The CPU 51 generates high-resolution image data at every thinning-out time T3. However, the CPU 51 generates low resolution between the timing when the high-resolution image data is generated and the time when new high-resolution image data is generated after T3 seconds. The image image data may be associated with a time stamp X associated with the previous high resolution image data. Specifically, for example, the CPU 51 performs a process of overwriting and holding the time stamp X acquired in S13 in the RAM 52 every T3 seconds, and updates the time stamp X every T3 seconds. The CPU 51 may associate the latest time stamp X overwritten and saved in the RAM 52 with the low resolution image data in the process of S17, and associate with the high resolution image data in the process of S27. In this way, all the low resolution image data can have a time stamp X indicating the same time as any of the high resolution image data generated by thinning.

  In this embodiment, the resolution of the captured image data is the same as that of the high-resolution image data, but the resolution of the high-resolution image data is slightly lower than the resolution of the captured image data (for example, 3 million pixels) by the resolution processing unit 56. The resolution may be changed (for example, 2 million pixels).

  Further, in this embodiment, the thinning time T3 is determined based on the delay time T1 that occurs in data transmission / reception via the network 9, but may be set according to the connection mode to the network 9. For example, at the start of execution of the work support program for HMD, the CPU 51 determines whether the connection form with the network 9 is a wireless connection performed via the wireless communication unit 65 or a wired connection performed via the wired communication unit 66. Check. Further, the CPU 51 acquires resolution information indicating the resolution of the high resolution image data generated by the resolution processing unit 56. The CPU 51 refers to the T3 determination table 150 shown in FIG. 7 (assumed to be stored in the program ROM 53 together with the HMD work support program), and based on the connection form with the network 9 and the resolution information, the thinning time is determined. T3 is determined. The T3 determination table 150 is a table in which the thinning time T3 is set longer as the resolution of the high-resolution image data is larger and when the wireless connection is used than the wired connection. If the thinning time T3 is determined with reference to the T3 determination table 150, the time required for measuring the delay time T1 can be shortened, and the load on the CPU 51 in the calculation of the storage time T2 and the thinning time T3 can be reduced. .

  In the present embodiment, the high resolution image data is held in the storage buffer 52C by performing the FIFO processing with the passage of the storage time T2 as a determination condition. However, the frame of the high resolution image that can be held in the storage buffer 52C is used. The high resolution image data may be held in the storage buffer 52 </ b> C by performing a FIFO process with the number (A / B) being exceeded. Note that the number of high-resolution image frames (A / B) that can be stored in the storage buffer 52C corresponds to the “predetermined number”.

  In the modified example, the CPU refers to the T3 determination table 150 in S9 and determines the thinning time T3 according to the connection form of the network 9 and the resolution information of the high resolution image data. Not limited to this, the CPU 51 refers to a table that determines T3 according to the above resolution information and the compression rate when the image compression unit 57 generates low-resolution image data in S15, and sets the thinning time T3. You may decide. As the compression rate increases, the CPU 51 takes longer to compress the captured image data in the image processing unit 58. Therefore, the larger the compression ratio of the captured image data, the longer the delay time that occurs in the stage before the low resolution image data is transmitted. That is, it takes a longer time for the low resolution image data to be transmitted from the HMD 1 and received by the PC 80 and displayed on the instruction screen 100 with respect to the time taken by the camera 15. Therefore, it is preferable to set the thinning time T3 according to the difference in the compression rate of the low resolution image data. As an example, when the compression rate is a two-stage setting of a high compression rate (for example, less than 80%) and a low compression rate (for example, 80% or more), a high compression rate is set for “wireless connection” in the T3 determination table 150 of FIG. Correspondingly, it is preferable to use a table in which the low compression rate is equivalent to “wired connection”. The compression rate may be arbitrarily set by the operator or the instructor.

  In addition, in the T3 determination table 150 of FIG. 7, not only the thinning time T3 but also the storage time T2 may be determined as a target to be determined based on the resolution information and the connection form of the network 9. Based on the determination result of this table, the CPU 51 may perform a FIFO process based on the storage time T2 in the processes of S23 and S25 of the present embodiment.

  As described above, in the work support system according to the present invention, the HMD 1 stores high-resolution image data associated with a time stamp that is the same as or close to the time stamp of the low-resolution image viewed by the instructor on the PC 80. Can be obtained as a captured image. Therefore, the instructor can obtain desired high resolution image data, and it is difficult for the instructor to feel discomfort between the original low resolution image and the captured high resolution image. In addition, even if the delay caused by transmission / reception of low-resolution image data or high-resolution image data via the network changes depending on the network status, the HMD 1 supports the low-resolution image that the instructor is browsing. Since the high-resolution image data to be acquired is acquired retroactively based on the time stamp and transmitted to the PC 80, it is not affected by the delay. Therefore, the instructor can obtain desired high resolution image data, and it is difficult for the instructor to feel discomfort between the original low resolution image and the captured high resolution image.

  The HMD 1 can store the high resolution image data in the storage buffer 52C for a long time by thinning and storing the high resolution image data in the storage buffer 52C. The PC 80 can obtain high-resolution image data associated with a time stamp indicating a photographing time close to the time when the low-resolution image viewed by the instructor is photographed. Therefore, the instructor is less likely to feel discomfort between the original low resolution image and the captured high resolution image.

  The HMD 1 can hold the high resolution image data for the storage time T2 or the number of frames (A / B) that can be held in the storage buffer 52C, and can delete the excess data so that the storage capacity of the storage buffer 52C is full. Therefore, it is not necessary to hold high-resolution image data, and it is not necessary to unnecessarily increase the storage capacity that the system requires to secure. If the HMD 1 holds the high-resolution image data for the storage time T2 or the number of holdable frames (A / B) without thinning out to the storage buffer 52C, the PC 80 browses the PC 80. High-resolution image data associated with a time stamp indicating the same time as the time when the low-resolution image was taken can be obtained. Therefore, the instructor can obtain desired high resolution image data, and does not feel discomfort between the original low resolution image and the captured high resolution image.

  Depending on whether the connection form via the network 9 is a wireless connection or a wired connection, the amount of delay in data transmission / reception differs. In addition, a delay that occurs in data transmission / reception (in this case, a delay that occurs before the low-resolution image data is transmitted) due to the difference in the compression rate at which the captured image data is compressed into low-resolution image data. The size of is different. Therefore, the storage time T2 or the number of frames (A / B) that can be held is set in consideration of at least one of the connection form via the network 9 and the size of the compression rate, and the storage time T2 is stored in the storage buffer 52C. By holding the high resolution image data corresponding to the number of frames or the number of holdable frames (A / B), the HMD 1 can reliably hold the high resolution image data desired by the instructor in the storage buffer 52C. .

  The HMD 1 sets the storage time T2 or the number of holdable frames (A / B) according to the storage capacity of the storage buffer 52C, and the storage buffer 52C has the storage time T2 or the number of holdable frames (A / B). By holding the high-resolution image data, it is possible to reliably hold the high-resolution image data desired by the instructor in the storage buffer 52C. Therefore, the instructor can obtain desired high resolution image data, and does not feel discomfort between the original low resolution image and the captured high resolution image. Even if the storage capacity of the storage buffer 52C is insufficient and the high-resolution image data corresponding to the storage time T2 or the number of frames (A / B) that can be held cannot be held, the HMD 1 thins out the high-resolution image data. Thus, the high resolution image data whose photographing time is close to the high resolution image data desired by the instructor can be held in the storage buffer 52C. Therefore, the instructor can obtain a high-resolution image whose photographing time is close to the desired high-resolution image data, and hardly feels a sense of incongruity between the original low-resolution image and the captured high-resolution image.

  The HMD 1 detects a delay time that occurs in data transmission / reception via the network, sets the storage time T2 or the number of holdable frames (A / B) according to the delay time T1, and stores the storage buffer 52C in the storage time T2 minutes. Alternatively, if the high resolution image data for the number of frames (A / B) that can be held is held, the high resolution image data desired by the instructor can be reliably held in the storage buffer 52C.

  In the present embodiment, communication between the HD 10 and the CB 50 via the harness 7 (transfer of image data displayed on the LCD 11, transfer of captured image data of the camera 15, transfer of audio data of the microphone 16 and the earphone 17, etc.) is performed by digital data communication. However, the data may be A / D / D / A converted in the CB 50 and analog data communication.

  In the present invention, the HMD 1 corresponds to a “work terminal”. The PC 80 corresponds to an “instruction terminal”. The camera 15 corresponds to a “photographing device”. The RAM 52 corresponds to a “storage device”. The storage time T2 corresponds to the “first predetermined time”. In S23, S25, and S29, the CPU 51 that stores the high-resolution image data for the storage time T2 in the storage buffer 52C based on the FIFO processing corresponds to the “storage control unit”. The time stamp corresponds to “time information”. The CPU 51 that acquires the time stamp X in S13 corresponds to “first acquisition means”. In S <b> 17, the CPU 51 that associates the time stamp X with the low-resolution image data corresponds to “association means”. The CPU 51 that transmits the low-resolution image data to the PC 80 in S19 corresponds to the “first transmission unit”. The capture request signal corresponds to “request information”. The CPU 51 that acquires the high-resolution image data associated with the time stamp X closest to the time stamp Y in S33 corresponds to the “second acquisition unit”. The CPU 51 that transmits the high-resolution image data to the PC 80 in S35 corresponds to a “second transmission unit”.

  The CPU 81 that receives the low-resolution image data from the HMD 1 in S55 corresponds to the “first receiving unit”. In S61, the CPU 81 that acquires the time stamp X from the low-resolution image data as the time stamp Y corresponds to the “third acquisition unit”. In S63, the CPU 81 that transmits the capture request signal and the time stamp Y to the HMD 1 corresponds to a “third transmission unit”. The CPU 81 that receives the high-resolution image data from the HMD 1 in S65 corresponds to the “second receiving unit”. The thinning time T3 corresponds to the “second predetermined time”. The CPU 51 that determines the thinning time T3 in S9 corresponds to “first determination means”. In S23, the CPU 51 that determines whether or not there is high-resolution image data indicating the time indicated by the time stamp X that is a time before the storage time T2 corresponds to “determination means”. In S25, the CPU 51 that deletes the high-resolution image data associated with the time stamp X indicating the time before the storage time T2 corresponds to the “deleting unit”. The CPU 51 that obtains the delay time T1 required from the transmission of the test signal in S3 to the reception of the capture request signal in S5 corresponds to “detection means”.

1 Head mounted display (HMD)
9 Network 15 Camera 51 CPU
52 RAM
52C Storage buffer 65 Wireless communication unit 66 Wired communication unit 80 Personal computer (PC)
81 CPU
83 RAM
86 Communication unit 92 Display A Storage capacity B Data amount equivalent to one frame of high resolution image data C Frame rate T1 Delay time T2 Storage time T3 Thinning time X, Y Time stamp

Claims (8)

A work terminal that can be used by an operator, can receive an instruction for the work, and can be presented to the worker, and an instruction terminal that can be used by the instructor and can transmit the work instruction to the worker to the work terminal. A work support system connected via
The work terminal is
A photographing device capable of photographing the scenery around the worker;
A storage device that stores various types of data including at least captured image data that is captured image data captured by the imaging device and low-resolution image data in which the resolution of the captured image data is reduced;
Storage control means for performing control for storing the photographed image data in the storage device for a first predetermined time or a predetermined number;
First acquisition means for acquiring time information indicating a time at which the imaging device has captured the captured image;
Association means for associating the time information acquired by the first acquisition means with the captured image data and the low resolution image data;
First transmission means for transmitting the low-resolution image data associated with the time information to the instruction terminal;
When request information including the time information, which is information requesting the captured image data, is received from the instruction terminal, the same time as the request time that is the time based on the time information included in the request information or the closest time Second acquisition means for acquiring the captured image data associated with the time information indicating from the storage device;
Second transmission means for transmitting the captured image data acquired by the second acquisition means to the instruction terminal;
With
The instruction terminal is
First receiving means for receiving a plurality of the low resolution image data from the work terminal;
Third acquisition means for acquiring the time information associated with the low resolution image data desired by the instructor among the low resolution image data received by the first reception means;
The request information including the time information acquired by the third acquisition unit, which is information requesting the captured image data corresponding to the low resolution image data desired by the instructor, is transmitted to the work terminal. Three transmission means;
Second receiving means for receiving the captured image data transmitted by the work terminal based on the request information;
A work support system characterized by comprising: The storage control unit of the work terminal determines a second predetermined time which is a time interval for thinning out a part of the captured image data of all the captured images captured by the imaging device and storing the thinned image data in the storage device. Including first determining means to
The work support system according to claim 1, wherein the storage control unit stores the captured image data captured at the second predetermined time determined by the first determination unit in the storage device.   When the second acquisition unit of the work terminal receives the request information from the instruction terminal, the second time acquisition unit associates the time information indicating the time with the smallest time difference between the time based on the time information and the request time. The work support system according to claim 2, wherein the captured image data is acquired from the storage device. The storage control means of the work terminal is
Among the captured image data stored in the storage device, whether there is the captured image data in which the elapsed time from the time based on the time information to the present has passed the first predetermined time or more, or in the storage device Determining means for determining whether or not the number of the captured image data to be stored is equal to or greater than the predetermined number;
When the determination means determines that there is the captured image data whose elapsed time has passed the first predetermined time or that the predetermined number or more of the captured image data is stored, from the storage device, Deleting means for deleting in order from the captured image data having the oldest time based on the time information;
The work support system according to claim 1, comprising:   The storage control means of the work terminal is based on at least one of a connection form with the instruction terminal via a network and a compression rate for compressing the captured image data into the low resolution image data. The work support according to any one of claims 1 to 4, wherein the first predetermined time or the predetermined number is determined, and control is performed to store the captured image data in the storage device. system. The storage control means of the work terminal includes at least one of a connection form with the instruction terminal via a network, a compression rate for compressing the captured image data into the low resolution image data, and the storage. Based on the storage capacity of the device,
When the captured image data for the first predetermined time or the predetermined number can be stored in the storage device, the length of the first predetermined time or the predetermined number is determined and stored in the storage device. Control to store the shot image data,
When the first predetermined time or the predetermined number of the captured image data cannot be stored in the storage device, the first selection means selects every second predetermined time determined by the first determination means. The work support system according to claim 2, wherein control is performed to store the captured image data in the storage device. The work terminal further includes detection means for detecting a delay time in data transmission / reception performed with the instruction terminal at the start of connection with the instruction terminal or periodically.
5. The work support according to claim 1, wherein the storage control unit determines the length of the first predetermined time or the predetermined number based on the delay time. 6. system. A work terminal used as a work auxiliary device used by a worker and configured to be connectable via a network to an instruction terminal used by an instructor, and capable of photographing a landscape around the worker And a storage device for storing various data including at least captured image data that is captured image data captured by the imaging device and low resolution image data in which the resolution of the captured image data is reduced, and the instruction A program for receiving a work instruction for the worker from a terminal and functioning as a work terminal that can be presented to the worker,
On the computer,
A storage control step of performing control for storing the photographed image data in the storage device for a first predetermined time or a predetermined number;
A first acquisition step of acquiring time information indicating a time at which the imaging device has captured the captured image;
Associating the time information acquired in the first acquisition step with the captured image data and the low-resolution image data;
A first transmission step of transmitting the low-resolution image data associated with the time information to the instruction terminal;
When request information including the time information, which is information requesting the captured image data, is received from the instruction terminal, the same time as the request time that is the time based on the time information included in the request information or the closest time A second acquisition step of acquiring, from the storage device, the captured image data associated with the time information indicating
A second transmission step of transmitting the captured image data acquired in the second acquisition step to the instruction terminal;
A program that executes

Images