JP2014045438A - Remote operation support system, remote operation support method, and remote operation support program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a remote operation support system, a remote operation support method, and a remote operation support program, capable of transmitting an easily perceptible video picture while preventing a sharer of the video picture from suffering from visually induced motion sickness.SOLUTION: A remote operation support system 1 includes: an imaging apparatus 2; and a remote work support apparatus 6. The imaging apparatus 2 includes: a photographing part 3 that photographs a subject; an acceleration detection part 4 that detects acceleration of the imaging apparatus 2; a video picture display control part 5 that, when the acceleration of the imaging apparatus 2 detected by the acceleration detection part 4 is equal to or higher than a predetermined value over a predetermined period, starts a process of reducing a frame rate of the video picture of the photographed subject; and a video transmission part 6 that transmits the video picture with the reduced frame rate to a remote operation support apparatus 7. The remote operation support apparatus 7 includes: a display 8 that displays the video picture transmitted from the imaging apparatus 2.

Description

  The present invention relates to a remote work support system, a remote work support method, and a remote work support program that support a remote worker using an imaging device.

  In order to provide remote work support for a remote worker who performs work in a remote place to instruct and manage the work by the work instructor, video communication can be performed between terminals used by both parties. There is a remote work support system.

  When the remote work support system is used, environment information, which is multimedia information such as voice and video, can be shared in real time between the remote worker and the work instructor. In the remote work support system, for example, a head-mounted device equipped with a camera is used as a terminal on the remote worker side.

  The reason why the head-mounted camera is used in the remote work support system is that the work instructor can obtain an experience image of the remote worker. When the work instructor shares the experience video of the remote worker, the remote worker can receive an appropriate instruction from the work instructor. In addition, the head-mounted camera does not bother the remote operator in order to capture the experience video. Therefore, the remote worker can use his / her hands freely.

  However, the shooting direction of the head-mounted camera moves with the movement of the head. For this reason, when the head moves or shakes at high speed, the image taken by the camera also moves or shakes at high speed. At this time, video sickness may occur in the work instructor who views the video taken by the camera.

  In recent years, commercialization of wearable cameras with small size and high image quality has been promoted. However, wearable cameras tend to produce large image shaking compared to general handy cameras. For this reason, when a wearable camera is used as a head-mounted camera, there is a high possibility that motion sickness will occur in a work instructor who views an image captured by the camera, as compared with the case where a handy camera is used.

  In such a situation, a remote work support system using an imaging device that can avoid image shaking caused by high-speed movement or shaking of the head is desired.

  In a head-mounted camera, a remote work support system that makes it easy to recognize and view the subject of the video shot by this camera even when the head moves or shakes at high speed is proposed (For example, refer to Patent Document 1).

JP 2011-101300 A

  However, the imaging device used in the remote work support system described in Patent Document 1 immediately decreases the frame rate value when the detected angular velocity of the head exceeds a specified value. For this reason, depending on how the image is shaken, the frame rate value decreases with a slight change in the image, and it becomes difficult for the work instructor who sees the captured image to recognize the image.

  Therefore, the present invention provides a remote work support system, a remote work support method, and a remote work support program in which an easy-to-recognize video is transmitted without causing video sickness to a video sharer.

  A remote work support system according to the present invention includes a photographing device mounted on a user, and a remote work support device connected to the photographing device via a communication network, the photographing device photographing a subject, An acceleration detection unit that detects the acceleration of the imaging device, and a process for reducing the frame rate of the image of the captured subject when the acceleration of the imaging device detected by the acceleration detection unit is equal to or greater than a predetermined value over a predetermined period A display unit for displaying the video transmitted from the imaging device, the video display control unit for starting the video, and a video transmission unit for transmitting the video with a reduced frame rate to the remote work support device. It is characterized by including.

  A remote work support method according to the present invention is a remote work support method that is connected to a remote work support device via a communication network and is executed by a photographing device attached to a user, wherein a photographing unit included in the photographing device includes Detecting the acceleration of the photographing device when photographing the subject, and reducing the frame rate of the photographed subject image when the detected acceleration of the photographing device is equal to or greater than a predetermined value over a predetermined period Is started, and a video with a reduced frame rate is transmitted to the remote work support device.

  A remote work support program according to the present invention is a remote work support program in a computer that is connected to a remote work support device via a communication network and is mounted on a photographing device worn by a user. When the acceleration of the imaging device detected by the imaging processing, the acceleration detection processing of detecting the acceleration of the imaging device, or the acceleration detection processing is equal to or higher than a predetermined value over a predetermined period, the frame rate of the image of the captured subject is set. A video display control process for starting a process for reducing, and a video transmission process for transmitting a video with a reduced frame rate to a remote work support apparatus are executed.

  ADVANTAGE OF THE INVENTION According to this invention, the image | video which it is easy to recognize is provided, preventing image sickness from occurring to the sharer of the image.

It is a system configuration figure showing an embodiment of the present invention. 2 is a block diagram illustrating a configuration example of a glasses-type camera unit 100. FIG. It is a flowchart which shows the processing flow until the vibration determination part 122 performs the determination which starts the fall of a frame rate. It is a flowchart which shows the processing flow until the vibration determination part 122 performs the determination which stops the fall of a frame rate. It is a block diagram which shows the outline | summary of the remote operation assistance system by this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a system configuration diagram showing a configuration example of an embodiment of a remote operation support system according to the present invention. The remote work support system shown in FIG. 1 includes a glasses-type camera unit 100, a remote data communication server 200, and an operator personal computer (hereinafter referred to as operator PC) 300.

  The remote data communication server 200 can communicate with a plurality of operator PCs. FIG. 1 shows an example in which the remote data communication server 200 transmits the video data and the ID of the glasses-type camera unit to the operator PC 300 that is one of the plurality of operator PCs via the communication network 500. It is shown. The ID is used by the operator PC to determine which glasses-type camera unit has captured the video data. 1 shows one operator PC 300, the number of operator PCs 300 may be two or more.

  A glasses-type camera unit 100 shown in FIG. 1 is a glasses-type camera that a remote worker A wears on the head and photographs a subject in a predetermined direction from the remote worker A. In addition, as long as it can be mounted on the head of the remote worker A, a camera other than the glasses type may be used. The glasses-type camera unit 100 includes a vibration acceleration sensor 110, an information processing unit 120, a photographing unit 130, and a distribution unit 140.

  The glasses-type camera unit 100 is connected to the remote data communication server 200 via the communication network 400 so that video data can be transmitted. Note that the communication network 400 is a communication network such as a wireless communication network. Therefore, the glasses-type camera unit 100 is in a state where it can communicate with the remote data communication server 200.

  The vibration acceleration sensor 110 detects the vibration acceleration of the glasses-type camera unit 100.

  The information processing unit 120 inputs vibration acceleration data detected by the vibration acceleration sensor 110. Then, the information processing unit 120 determines whether to reduce the frame rate of the video based on the input vibration acceleration data. After the determination, the information processing unit 120 outputs the determination result to the distribution unit 140.

  The photographing unit 130 photographs a subject such as a situation around the remote worker A. Then, the imaging unit 130 outputs the captured video data to the distribution unit 140.

  The distribution unit 140 distributes the video data photographed by the photographing unit 130. When distributing the video data, the distribution unit 140 transmits a frame rate video based on the determination result input from the information processing unit 120 to the remote data communication server 200.

  A remote data communication server 200 shown in FIG. 1 is a server provided at a location away from the glasses-type camera unit 100. Remote data communication server 200 includes a server reception unit 210, a server processing unit 220, and a server distribution unit 230.

  The remote data communication server 200 is communicable with a plurality of glasses-type camera units. FIG. 1 shows that the remote data communication server 200 is receiving video data via the communication network 400 from the glasses-type camera unit 100 which is one of a plurality of glasses-type camera units. 1 shows one eyeglass-type camera unit 100, the number of eyeglass-type camera units 100 may be two or more.

  The number of eyeglass-type camera units that can communicate with the remote data communication server 200 may not match the number of operator PCs. This is because the remote data communication server 200 can appropriately select a video data distribution destination.

  The server reception unit 210 receives video data transmitted from the distribution units of the plurality of glasses-type camera units via the communication network 400.

  The server processing unit 220 distributes to which operator PC the received video data is transmitted.

  The server delivery unit 230 transmits the video data distributed by the server processing unit 220 to the operator PC corresponding to the ID of the glasses-type camera unit that captured the video data via the communication network 500.

  An operator PC 300 shown in FIG. 1 is an apparatus used by the work instructor B when a remote work using the glasses-type camera unit 100 is performed.

  Operator PC 300 includes a PC receiving unit 310 and a PC display unit 320. The operator PC 300 can communicate with the remote data communication server 200. Specifically, operator PC 300 receives video data from remote data communication server 200 via communication network 500.

  The PC receiving unit 310 receives video data transmitted from the server distribution unit 230 of the remote data communication server 200 via the communication network 500.

  The PC display unit 320 displays a subject included in the video data received by the PC receiving unit 310. The work instructor B gives an instruction to the remote worker A while looking at the displayed subject.

  Next, a detailed configuration within the glasses-type camera unit 100 will be described with reference to FIG. FIG. 2 is a block diagram illustrating a configuration example of the glasses-type camera unit 100. The information processing unit 120 illustrated in FIG. 2 includes a sensor information collection unit 121 and a vibration determination unit 122.

  The sensor information collection unit 121 collects vibration acceleration data detected by the vibration acceleration sensor 110. The vibration acceleration data is held in the sensor information collection unit 121 until it is transmitted to the vibration determination unit 122.

  The vibration determination unit 122 determines whether to reduce the frame rate based on the vibration acceleration data acquired from the sensor information collection unit 121. Here, the movement of the glasses-type camera unit 100 when the acceleration of the glasses-type camera unit 100 is a predetermined value or more for a certain period of time is defined as “vibration”. The movement of the glasses-type camera unit 100 when the acceleration of the glasses-type camera unit 100 is 0 to less than a predetermined value for a certain time is defined as “fine movement”. The predetermined value is, for example, 10 cm / s @ 2. The predetermined value is not limited to this value.

  As a result of analyzing vibration acceleration data detected in a certain time zone, when the vibration acceleration that was initially less than a predetermined value subsequently changes to a predetermined value or more and the state continues for a certain period of time, a vibration determination unit 122 determines that the movement of the glasses-type camera unit 100 has changed from “fine movement” to “vibration”. At this time, the vibration determination unit 122 transmits a signal indicating that the frame rate reduction process is started to the distribution unit 140.

  Similarly, as a result of analyzing vibration acceleration data detected in a certain time zone, when the vibration acceleration that was initially a predetermined value or more changes to a value less than a predetermined value and the state continues for a certain period of time, The vibration determination unit 122 determines that the movement of the glasses-type camera unit 100 has changed from “vibration” to “fine movement”. At this time, the vibration determination unit 122 transmits a signal to the distribution unit 140 to stop the frame rate reduction process.

  The result determined by the vibration determination unit 122 is transmitted to the distribution unit 140. The distribution unit 140 reflects the determination result received from the vibration determination unit 122 in the frame rate of the video data received from the imaging unit 130. At this time, the distribution unit 140 performs reflection so that the time zone in which the vibration acceleration data that is the basis of the determination result is detected matches the time zone in which the video data that reflects the determination result is captured. Thereafter, the distribution unit 140 transmits the video data to the remote data communication server 200 at a frame rate that reflects the determination result.

  Hereinafter, the operation of the vibration determination unit 122 until it is determined to start the frame rate reduction process in a state where the movement of the glasses-type camera unit 100 is determined to be “fine movement” will be described with reference to FIG. 3. To do. FIG. 3 is a flowchart illustrating a processing flow until the vibration determination unit 122 determines to start the frame rate reduction process.

  First, the vibration determination unit 122 acquires vibration acceleration data from the sensor information collection unit 121 (step S101).

  The vibration determination unit 122 determines whether the detected vibration acceleration is 10 cm / s2 or more (step S102). When the detected vibration acceleration is a value less than 10 cm / s 2 (No in step S102), the vibration determination unit 122 “finely moves” the movement of the glasses-type camera unit 100 in the time zone in which the vibration acceleration is detected. (Step S105). In this case, the vibration determination unit 122 does not transmit a signal indicating that the frame rate reduction process is started to the distribution unit 140.

  Then, the vibration determination unit 122 restarts a timer (hereinafter referred to as a continuation timer) that measures the time during which the “vibration” state of the glasses-type camera unit 100 continues (step S106). Thereafter, the vibration determination unit 122 obtains vibration acceleration data from the sensor information collection unit 121 again (step S101), and continues the determination process after step S102.

  On the other hand, when the detected vibration acceleration has a value of 10 cm / s 2 or more (Yes in step S102), the vibration determination unit 122 measures the time during which the state in which the value of 10 cm / s 2 or more is detected continues. It is determined whether the measurement time of the continuation timer has reached a predetermined timeout value (step S103). When the predetermined time-out value has not been reached (No in step S103), the vibration determination unit 122 acquires vibration acceleration data from the sensor information collection unit 121 again (step S101), and continues the determination processing after step S102. . This timeout value is a predetermined value.

  On the other hand, when the predetermined time-out value is reached (Yes in step S103), the vibration determination unit 122 determines that the movement of the glasses-type camera unit 100 during the time period in which the vibration acceleration is detected is “vibration” (step S104). ). Then, vibration determination unit 122 transmits a signal to start distribution rate reduction processing to distribution unit 140.

  Next, the operation of the vibration determination unit 122 until it is determined to stop the frame rate reduction process in a state where the movement of the glasses-type camera unit 100 is determined to be “vibration” will be described with reference to FIG. To do. FIG. 4 is a flowchart illustrating a process flow until the vibration determination unit 122 determines to stop the frame rate reduction process.

  First, the vibration determination unit 122 acquires vibration acceleration data from the sensor information collection unit 121 (step S201).

  The vibration determination unit 122 determines whether the detected vibration acceleration is less than 10 cm / s 2 (step S202). When the detected vibration acceleration is a value of 10 cm / s2 or more (No in step S202), the vibration determination unit 122 determines the movement of the glasses-type camera unit 100 in the time zone in which the vibration acceleration is detected as “vibration”. Determination is made (step S205). In this case, the vibration determination unit 122 does not transmit a signal to stop the frame rate reduction process to the distribution unit 140.

  Then, the vibration determination unit 122 restarts a continuation timer that measures the time during which the “fine movement” state of the glasses-type camera unit 100 is continued (step S206). After that, the vibration determination unit 122 acquires vibration acceleration data from the sensor information collection unit 121 again (step S201), and continues the determination process after step S202.

  On the other hand, when the detected vibration acceleration is a value less than 10 cm / s 2 (Yes in step S202), vibration determination unit 122 measures the time during which the state where a value less than 10 cm / s 2 is detected continues. It is determined whether the measurement time of the continuation timer has reached a predetermined timeout value (step S203). When the predetermined time-out value has not been reached (No in step S203), the vibration determination unit 122 acquires vibration acceleration data from the sensor information collection unit 121 again (step S201), and continues the determination processing after step S202. . This timeout value is also a predetermined value.

  On the other hand, when the predetermined time-out value has been reached (Yes in step S203), the vibration determination unit 122 determines that the movement of the glasses-type camera unit 100 during the time period in which the vibration acceleration is detected is “fine movement” (step) S204). Then, vibration determination unit 122 transmits a signal to stop the frame rate reduction process to distribution unit 140.

  Next, a specific example when the remote worker A and the work instructor B use the remote work support system will be described.

  It is assumed that the remote worker A wearing the glasses-type camera unit 100 starts up the glasses-type camera unit 100 in a stationary state. The activated glasses-type camera unit 100 photographs a subject in a predetermined direction of the remote worker A. The captured video is displayed on operator PC 300 via remote data communication server 200. Starting from this point in time, the work instructor B shares the subject photographed by the remote worker A and the glasses-type camera unit 100.

  Thereafter, for example, it is assumed that the remote worker A in a stationary state has moved from the current location to the lower floor on the stairs according to the work instruction. At this time, the vibration determination unit 122 performs processing according to the flowchart illustrated in FIG.

  When the remote worker A descends the steps of the stairs, the glasses-type camera unit 100 in a stationary state also vibrates up and down together with the remote worker A. Vibration acceleration data detected by the vibration acceleration sensor 110 is transmitted to the vibration determination unit 122 via the sensor information collection unit 121. As a result of the determination by the vibration determination unit 122, it is assumed that the transmitted vibration acceleration data is determined to be a value of 10 cm / s 2 or more.

  After that, when the remote worker A continues to move down the steps of the stairs for a while and continues to detect vibration acceleration of 10 cm / s 2 or more, the duration of the detected state reaches a specified timeout value. At this time, the movement of the glasses-type camera unit 100 when the remote worker A descends the step of the stairs is determined as “vibration”.

  As a result, when the remote worker A descends the steps of the stairs, a process for reducing the frame rate of the video imaged by the glasses-type camera unit 100 is performed. As a result of the processing for reducing the frame rate, this video is displayed on the operator PC 300 by frame advance.

  When the continuation timer reaches a specified timeout value, the vibration determination unit 122 performs processing according to the flowchart illustrated in FIG. Therefore, when the remote worker A continues to move down the steps of the stairs and a vibration acceleration of 10 cm / s 2 or more is detected, the movement of the glasses-type camera unit 100 is determined as “vibration”. When it is determined as “vibration”, the continuation timer that measures the time during which the “fine movement” state of the glasses-type camera unit 100 is continued is restarted.

  At this time, since the signal for stopping the frame rate reduction process is not transmitted to the distribution unit 140, the frame rate of the video captured by the glasses-type camera unit 100 remains low.

  Next, for example, it is assumed that the remote worker A stops the operation of going down the steps of the stairs. At this time, the vibration determination unit 122 performs processing according to the flowchart illustrated in FIG.

  When the remote worker A stops moving down the steps of the stairs, the glasses-type camera unit 100, together with the remote worker A, eliminates vertical vibration. Vibration acceleration data detected by the vibration acceleration sensor 110 is transmitted to the vibration determination unit 122 via the sensor information collection unit 121. As a result of the determination by the vibration determination unit 122, it is assumed that the transmitted vibration acceleration data is determined to be a value less than 10 cm / s @ 2.

  If the remote worker A continues to stop on the stairs for a while after that and the state where vibration acceleration of less than 10 cm / s2 is detected continues, the duration of the detected state reaches a specified timeout value. At this time, the movement of the glasses-type camera unit 100 when the remote worker A continues to stop on the stairs is determined as “fine movement”.

  As a result, the process of lowering the frame rate of the video captured by the glasses-type camera unit 100 after the remote worker A stops the operation of descending the steps of the stairs is stopped. As a result of the processing for lowering the frame rate being stopped, this video is displayed on the operator PC 300 at the normal frame rate.

  When the continuation timer reaches a specified timeout value, the vibration determination unit 122 performs processing according to the flowchart illustrated in FIG. Therefore, when the remote worker A is still moving and continues to move slightly or move the neck, and when vibration acceleration of less than 10 cm / s2 is detected, the movement of the glasses-type camera unit 100 is as follows. , “Fine movement” is determined. By determining “fine movement”, the continuation timer that measures the time during which the vibration state of the glasses-type camera unit 100 is continued is restarted.

  At this time, since the signal for starting the frame rate reduction process is not transmitted to the distribution unit 140, the frame rate of the video captured by the glasses-type camera unit 100 remains at a normal value.

  When using the remote work support system of this embodiment, the work instructor can avoid motion sickness even when the head of the remote worker vibrates at high speed. Even if the remote worker's head alternately vibrates and stops, the frame rate value does not change immediately, so that the work instructor does not have difficulty in recognizing the video. .

  When using the remote operation support system of the present embodiment, a subject image captured by one glasses-type camera is distributed to a plurality of operator PCs, or a subject image captured by a plurality of glasses-type cameras is provided by one unit. Can be distributed to the operator PC.

  Next, the outline of the present invention will be described. FIG. 5 is a block diagram showing an outline of a remote work support system according to the present invention. A remote work support system 1 according to the present invention includes a photographing apparatus 2 (for example, a glasses-type camera unit 100) and a remote work support apparatus 7 (for example, an operator PC 300).

  The photographing device 2 is detected by a photographing unit 3 (for example, a photographing unit 130) that photographs a subject, an acceleration detecting unit 4 (for example, a vibration acceleration sensor 110) that detects acceleration of the photographing device 2, and an acceleration detecting unit 4. A video display control unit 5 (for example, an information processing unit 120) that starts a process of reducing the frame rate of the video of the captured subject when the acceleration of the imaging device 2 is equal to or greater than a predetermined value over a predetermined period of time; The video transmission unit 6 (for example, the distribution unit 140) that transmits the video with the reduced frame rate to the remote work support device 7 is included.

  The remote work support device 7 includes a display unit 8 (for example, a PC display unit 320) that displays an image transmitted from the imaging device 2.

  With such a configuration, the work instructor who shares the video experienced by the remote worker can be a subject of the video shot by the imaging device even when the remote worker's head vibrates at a high speed for a certain period of time. Can be easily recognized without causing motion sickness.

  Moreover, the imaging device 2 and the remote work support device 7 may be plural.

  Further, the imaging device 2 is identified between the imaging device 2 and the remote work support device 7, and the video is transmitted to the remote work support device 7 corresponding to the identified imaging device 2. Relay device 10 (for example, remote data communication server 200) may be communicably connected via a communication network.

  In addition, the video display control unit 5 detects that the acceleration of the imaging device 2 detected by the acceleration detection unit 4 is less than a predetermined value in a predetermined period while the process of reducing the video frame rate is performed. At this time, the process of reducing the frame rate of the video may be stopped.

A Remote worker B Work instructor 1 Remote work support system 2 Imaging device 3 Imaging unit 4 Acceleration detection unit 5 Video display control unit 6 Video transmission unit 7 Remote work support device 8 Display unit 10 Relay device 100 Glasses type camera unit 110 Vibration Acceleration sensor 120 Information processing unit 121 Sensor information collection unit 122 Vibration determination unit 130 Imaging unit 140 Distribution unit 200 Remote data communication server 210 Server reception unit 220 Server processing unit 230 Server distribution unit 300 Operator PC
310 PC Receiver 320 PC Display 400 Communication Network 500 Communication Network

Claims (6)

A photographing device attached to the user;
A remote operation support device connected to the imaging device via a communication network,
The imaging device
A shooting section for shooting the subject;
An acceleration detection unit for detecting the acceleration of the imaging device;
A video display control unit that starts a process of reducing a frame rate of a video of a photographed subject when the acceleration of the photographing device detected by the acceleration detection unit is equal to or greater than a predetermined value over a predetermined period;
A video transmission unit that transmits the video with a reduced frame rate to the remote operation support device,
The remote work support device is
A remote operation support system comprising a display unit for displaying an image transmitted from the photographing apparatus. The remote work support system according to claim 1, comprising a plurality of photographing devices and a plurality of remote work support devices. A relay device is connected between the imaging device and the remote operation support device so as to be communicable via a communication network,
The relay device is
The remote work support system according to claim 2, wherein the image capturing apparatus is identified from which image capture apparatus is transmitted, and the image is transmitted to the remote work support apparatus corresponding to the identified image capture apparatus. The video display control unit is configured to detect the video image when the acceleration of the imaging device detected by the acceleration detection unit is less than a predetermined value over a predetermined period during the process of reducing the frame rate of the video. The remote work support system according to any one of claims 1 to 3, wherein processing for lowering a frame rate is stopped. A remote work support method connected to a remote work support device via a communication network and executed by a photographing device attached to a user,
When a photographing unit included in the photographing apparatus photographs a subject,
Detecting the acceleration of the imaging device;
When the detected acceleration of the photographing device is equal to or greater than a predetermined value over a predetermined period, a process of reducing the frame rate of the photographed subject image is started.
A remote work support method, wherein the video with a reduced frame rate is transmitted to the remote work support device. A remote work support program in a computer connected to a remote work support device via a communication network and mounted on a photographing device worn by a user,
In the computer,
Shooting process to shoot the subject,
An acceleration detection process for detecting the acceleration of the imaging device;
A video display control process for starting a process of reducing a frame rate of a video of a photographed subject when the acceleration of the photographing apparatus detected by the acceleration detection process is equal to or greater than a predetermined value over a predetermined period;
A remote work support program for executing video transmission processing for transmitting the video with a reduced frame rate to the remote work support device.

Images