JP6134301B2 - Video transmission system for remote control of construction machinery - Google Patents

Description

  The present invention relates to a video transmission system for remote control of a construction machine such as a hydraulic excavator.

  Conventional construction machines such as hydraulic excavators are generally operated by operators on board the construction machine. Recently, cameras were installed on the construction machine and its work site, and were photographed by the camera. A system has been developed for remotely maneuvering from a remote cockpit via a wireless communication line while watching the video.

  In such a remote control system, the quality of the displayed video is affected by the performance of the camera or communication device or the state of the communication line, and a video delay may occur. Although the video delay itself can be tolerated to some extent by remote control if you get used to it, if you cannot accurately grasp the degree, work efficiency will be significantly reduced.

  Therefore, for example, Patent Document 1 below proposes a method of calculating a video delay time by simultaneously displaying a reference video and a delayed video using two monitors and analyzing the displayed two videos. . Further, in Patent Document 2 below, a control signal is transmitted from the cockpit to a remote location, a lamp is turned on based on the control signal at the remote location, and an image of the situation is transmitted to the cockpit. There has been proposed a method for detecting and measuring the delay time of an image by measuring the time lag between the transmission of a control signal and the timing at which the lamp image is lit in the image transmitted from a remote place to the cockpit. .

Japanese Patent No. 4251979 JP 2012-1427789 A

  By the way, in order to measure the video delay time, it is necessary to compare the reference video and the delayed video at the same timing. For this purpose, it is necessary to match the time axes of the reference video and the delayed video. However, in the above-described known example, it is necessary to install a monitor for displaying a reference image, a monitor for displaying a delayed image, and a camera for photographing two monitors at the same place in order to adjust the time axis. It is impossible to compare the reference image before transmission and the delayed image displayed in the cockpit transmitted through the communication line at the same timing. Therefore, it is difficult to measure the delay time of a video transmitted from a remote place, and the video delay time cannot be presented to the operator.

  In addition to the video signal delay between the remote location and the cockpit, there is a communication delay, so it is necessary to synchronize the absolute time between the remote location and the cockpit. Measurement cannot be performed. This synchronization requires an expensive and sophisticated system such as a method of using a GPS time signal and a method of installing a time synchronization server, and is difficult to introduce at all remote work sites and mobile construction machines.

  Therefore, the present invention has been devised to solve these problems, and its purpose is a novel construction capable of accurately measuring the transmission delay time of a video signal and notifying a remote operator accurately. A video transmission system for remote control of a machine is provided.

  In order to solve the above-mentioned problems, the first invention is a remote control system in which a remotely-controllable construction machine at a work site and its surroundings are photographed by a camera at the work site and the video signal is transmitted to a remote cockpit. A video transmission system, wherein the construction machine or its work site and the cockpit each include a measurement signal transmitter / receiver for transmitting / receiving a signal for measuring a communication delay time, and between the signal transmitter / receiver Video transmission for remote control of construction machinery, comprising a video delay time measuring device for measuring a transmission delay time of a video signal transmitted from the work site to the cockpit on the basis of a delay time of the measurement signal of System.

  According to such a configuration, the transmission delay time of the video signal transmitted from the work site to the cockpit is measured with reference to the delay time of the measurement signal between the signal transceivers arranged in the work site and the cockpit. Therefore, it is possible to accurately measure the transmission delay time of the video signal and accurately notify the remote pilot in the cockpit.

  According to a second aspect of the present invention, the first aspect includes a cockpit camera for photographing the cockpit, wherein each of the measurement signal transceivers includes a transmission / reception display unit for displaying a signal transmission / reception state, The work site camera images the work site including the transmission / reception display unit of the measurement signal transmitter / receiver, and the cockpit camera includes the transmission / reception display unit of the measurement signal transmitter / receiver disposed in the cockpit An image of a work site composed of a video signal transmitted from the work site to the cockpit is photographed so as to be captured on one screen, and the video delay time measuring device is configured to delay a signal between the measurement signal transceivers. Remote control of a construction machine, characterized in that a transmission delay time of a video signal transmitted from the work site is measured based on time and a display state of each transmission / reception display unit photographed by the cockpit camera It is a video transmission system.

  According to such a configuration, the video delay time measuring device recognizes the display state of each transmission / reception display unit imaged by the cockpit camera, so that the signal delay time between the measurement signal transceivers and the control signal can be controlled. The transmission delay time of the video signal transmitted from the work site can be quickly and accurately measured based on the display state of each transmission / reception display unit photographed by the room camera.

  According to a third aspect of the present invention, in the first or second aspect of the invention, the video delay time measuring apparatus is provided with a remote control monitor for displaying a video of a work site from a video signal transmitted from the work site in the cockpit. Is a video transmission system for remote control of a construction machine, wherein the measured transmission delay time of the video signal transmitted from the work site is displayed on the remote control monitor. According to such a configuration, the remote operator in the cockpit can easily and accurately grasp the transmission delay time of the image together with the image of the work site reflected on the monitor for remote control.

  In a fourth aspect based on the third aspect, the video delay time measuring device divides the measured transmission delay time of the video signal transmitted from the work site into a plurality of levels according to the magnitude, Is displayed on the remote control monitor. The video transmission system for remote control of a construction machine. According to such a configuration, the remote operator in the cockpit can easily grasp the delay time only by looking at the level of the work site image displayed on the remote control monitor.

  According to a fifth invention, in any one of the first to fourth inventions, the work site camera and the measurement signal transmitter / receiver disposed on the work site are integrated, and an image of the work site is displayed. A video transmission system for remote control of a construction machine, in which a monitor and a signal transmitter / receiver disposed in the cockpit are integrated. By integrating in this way, the installation space can be reduced and the number of parts can be reduced, which can contribute to cost reduction.

  According to the present invention, the transmission delay time of the video signal transmitted from the work site to the cockpit is measured with reference to the transmission delay time of the measurement signal between the signal transceivers respectively disposed at the work site and the cockpit. Therefore, it is possible to accurately measure the transmission delay time of the video signal and accurately notify the remote pilot in the cockpit. As a result, the remote pilot in the cockpit can efficiently perform remote control.

1 is an overall configuration diagram showing an embodiment of a video transmission system 100 for remote control of a construction machine according to the present invention. (A) is a front view showing a measurement signal transceiver 11 arranged in the work site 1, and (B) is a front view showing a measurement signal transceiver 21 arranged in the cockpit 2. It is explanatory drawing which shows an example of the measuring method of the video delay time by the video transmission system 100 for remote control of the construction machine which concerns on this invention. It is explanatory drawing which shows the other example of the measuring method of the video delay time by the video transmission system 100 for remote control of the construction machine which concerns on this invention.

  Next, embodiments of the present invention will be described with reference to the accompanying drawings. FIG. 1 shows an embodiment of a video transmission system 100 for remote control of a construction machine according to the present invention. In the figure, reference numeral 1 is a work site, and 2 is a cockpit remote from the work site. Here, the distance is not particularly limited in terms of distance, for example, a distance of several tens to several hundreds of kilometers or more, of course, above and below the ground, or between one country and another country or between the earth and the moon, It is a concept that includes interplanets.

  The work site 1 has a measurement signal transmitter / receiver (hereinafter referred to as “work site communication device”) 11 for the work site and an operation for photographing the work site so that the work site communication device 11 enters the same screen. A field camera 12 and a remotely operated mobile construction machine 13 are arranged. On the other hand, in the cockpit 2, a measurement signal transmitter / receiver for the cockpit (hereinafter referred to as a “cockpit communication device”) 21, a cockpit camera 22 for photographing the inside of the cockpit 2, and a remote pilot are provided. A remote control monitor 23 to be visually observed, a video delay confirmation monitor 24, a video delay time measuring device 25, and a remote control controller 26 operated by the remote operator 5 are arranged.

  The work site 1 and the cockpit 2 are connected to each other via a communication line 4 such as a satellite communication line or a mobile phone network so that bidirectional communication is possible. The work site communication device 11 and the cockpit communication device 21 are connected to each other. The delay time measurement signal is transmitted / received via the communication line 4. Similarly, a video signal photographed by the work site camera 12 installed in the work site 1 is also connected to the remote control monitor 23 and the video delay confirmation monitor 24 installed in the cockpit 2 through the communication line 4. Sent to and displayed. Then, as shown in the figure, on the screens of the remote control monitor 23 and the video delay confirmation monitor 24, the video of the on-site communication device 11 arranged there is simultaneously projected together with the video of the work site. .

  A remote control signal is also sent to the mobile construction machine 13 in the work site 1 via the communication line 4. Accordingly, the operator 5 in the cockpit 2 controls the remote control controller 26 while looking at the remote control monitor 23 and transmits the control signal to the work site 1 through the communication line 4 so as to move the mobile construction machine 13. Is to be remotely controlled. That is, at least three types of signals such as the remote control signal, the video signal, and the delay time measurement signal are transmitted between the work site 1 and the cockpit 2 via the communication line 4. Yes.

  As shown in FIGS. 2 (A) and 2 (B), transmission / reception display portions 11A and 21A are provided on the front surfaces of the on-site communication device 11 and the cockpit communication device 21 so as to be clearly distinguishable. The transmission / reception display units 11A and 21A are turned on simultaneously with the transmission display units (triangles in the figure) 11a and 21a that are turned on at the same time as transmitting a signal for measuring a communication delay time (hereinafter referred to as a measurement signal). Receiving display sections (circular in the figure) 11b and 21b.

  For example, when a measurement signal is transmitted from the on-site communication device 11 to the cockpit communication device 21, at the same time, the transmission display portion 11a of the transmission / reception display portion 11A of the on-site communication device 11 is turned on. The reception display unit 21b of the transmission / reception display unit 21A of the cockpit communication device 21 that has received the light is turned on. On the contrary, when a measurement signal is transmitted from the cockpit communication device 21 to the on-site communication device 11, the transmission display portion 21a of the transmission / reception display portion 21A of the cockpit communication device 21 is turned on at the same time. The reception display unit 11b of the transmission / reception display unit 11A of the on-site communication device 11 that has received the signal is turned on. This transmission / reception may be distinguished not only by the shape of the display unit but also by color.

  Returning to FIG. 1, in the cockpit camera 22, the video 31 displayed on the video delay confirmation monitor 24 arranged in the cockpit 2 and the cockpit communication device 21 arranged in the vicinity thereof enter the same screen. The captured video 3 is input to the video delay time measuring device 25 that is also disposed in the cockpit 2. On the other hand, the video delay time measuring device 25 is a transmission / reception display unit 11A (for the on-site communication device 11 and the cockpit communication device 21 included in the video 31 of the video delay confirmation monitor 24 input as described later. 11a, 11b) and 21A (21a, 21b) are recognized to measure the transmission delay time of the video signal and the transmission delay time of the obtained video signal is displayed on the remote control monitor 23 to display the remote pilot. 5 is informed. Specifically, the video delay time measuring device 25 is composed of a dedicated or general-purpose computer in which image processing software for performing a series of operations is incorporated.

  Next, the transmission delay time of this video signal will be described with reference to FIGS. In FIG. 3, time t0 shows a state before the start of measurement. Time t1 indicates the time when the cockpit communication device 21 subsequently transmits a measurement signal to the work site communication device 11 through the communication line 4. At the time t0, the transmission / reception display units 11A (11a, 11b) and 21A (21a, 21b) of the on-site communication device 11 and the cockpit communication device 21 are all turned off, but at the time t1, the control is performed. When the room communication device 21 transmits the measurement signal, the transmission display portion 21a is turned on, and this is shown in the captured image 3 captured by the cockpit camera 22.

  Time t2 indicates the time when the on-site communication device 11 subsequently receives this measurement signal. At time t2, the reception display unit 11b of the on-site communication device 11 that has received this measurement signal is lit, but since the video signal has not yet reached the cockpit 2 side, the video on the video delay confirmation monitor 24 is displayed. 31 and the reception display unit 11b of the on-site communication device 11 included in the captured video 3 are both turned off.

  Time t3 indicates the time when the video signal of the work site 1 at time t2 arrives at the cockpit 2 side. At time t3, the reception display unit 11b of the on-site communication device 11 included in the video 31 of the video delay confirmation monitor 24 and the captured video 3 is lit.

  Time t4 indicates the time at which the on-site communication device 11 at the work site 1 transmits a measurement signal at time t3. At time t4, the transmission display unit 11a of the on-site communication device 11 is lit, but since the measurement signal and the video signal have not reached the cockpit 2 side, the transmission / reception display unit 21A of the cockpit communication device 21 is provided. The transmission / reception display units 11A (11a, 11b) and 21A (21a, 21b) of the on-site communication device 11 and the cockpit communication device 21 shown in the video 31 of the video delay confirmation monitor 24 and the captured video 3 are: Both are turned off.

  Time t5 indicates the time when the cockpit communication device 21 in the cockpit 2 subsequently receives the measurement signal. At time t5, the reception display unit 21b of the cockpit communication device 21 is turned on by receiving the measurement signal. However, since the video signal arrives later than the measurement signal, it is displayed on the video delay confirmation monitor 24. The transmission display unit 11a of the on-site communication device 11 shown in the projected image 31 remains off.

  Time t6 indicates the time when the video signal arrives at the cockpit 2 from the work site 1 thereafter. At time t6, the transmission display unit 11a of the on-site communication device 11 shown in the video 31 of the video delay confirmation monitor 24 and the captured video 3 is in a lit state.

(Measurement of transmission delay time of measurement signal)
Next, a method for measuring the transmission delay time of the measurement signal between the on-site communication device 11 and the cockpit communication device 21 in the transmission system in which the transmission delay phenomenon as described above occurs will be described. First, when the on-site communication device 11 and the cockpit communication device 21 receive the delay time measurement signal from the other, they respectively transmit the delay time measurement signal to the other after a predetermined time T2. T2 is a wait time, and the set value is grasped as a known value.

  Thereafter, when the on-site communication device 11 or the cockpit communication device 21 transmits a measurement signal to the other through the communication line 4, this signal is received by the other after the elapse of time T1. The other communication device that has received the measurement signal transmits the measurement signal to the first communication device that has transmitted after the wait time T2. The measurement signal transmitted from the other communication device is received by the communication device that first transmitted after the time T1 has elapsed.

The communication device that has transmitted the first time measures the time until the measurement signal transmitted from the other communication device is received after the wait time T2 elapses after the measurement signal is transmitted to the other communication device. If this total time is T, T is a time measured by the communication device, and T2 is a known set value, so the delay time T1 of the communication signal can be obtained by the following equation.
T1 = (T−T2) / 2

  Then, the communication delay time T1 obtained in this way is input to the video delay time measuring device 25. The communication signal delay time T1 obtained in this way may be the communication delay time measured by either the on-site communication device 11 or the cockpit communication device 21, or both. You may employ | adopt the average value with the communication delay time measured by (1).

(Measurement of transmission delay time of video signal)
Next, a method for measuring the transmission delay time of the video signal between the work site 1 and the cockpit 2 in the transmission system in which the transmission delay phenomenon of the measurement signal as described above occurs will be described. The video delay time measuring device 25 arranged in the cockpit 2 includes transmission / reception display units 11a and 11b of the on-site communication device 11 included in the video 31 of the video delay confirmation monitor 24 among the input captured video 3 and The lighting and extinguishing states of the transmission / reception display units 21a and 21b of the cockpit communication device 21 are recognized, and the time when the transmission display unit 21a of the cockpit communication device 21 is turned on is stored as time t1.

  Thereafter, the video delay time measuring device 25 continues to monitor the video 31 of the video delay confirmation monitor 24, and if the video 31 recognizes that the reception display unit 11b of the on-site communication device 11 is lit, The time is stored as time t3, and the elapsed time from time t1 to time t3 is measured. Let the measured elapsed time be t '.

  In this case, the actual video delay time is the time from when the reception display unit 11b of the work site communication device 11 is actually turned on until it is displayed on the video 31 of the remote control monitor 24 in the cockpit 2, That is, since this is the time taken from time t2 to time t3, this time is set to t (<t ′).

The time that can be grasped in the cockpit 2 is t ′, which is the elapsed time from the time t1 to the time t3, and the delay time T1 of the known communication signal, that is, the time taken from the time t1 to the time t2. The video delay time t is obtained as follows.
t = t′−T1

  Thereafter, the video delay time measuring device 25 displays the video delay time t thus determined on the remote control monitor 23 to notify the operator 5 of the video delay time. The operator 5 can perform efficient remote control by operating the controller 25 in consideration of the video delay time displayed on the remote control monitor 23.

  Here, the video delay time t may be displayed as a numerical value, but the video delay time t is divided into a plurality of levels according to the length, and the level is displayed on the remote control monitor 23. May be displayed. Further, the level display may be a character such as a number or an alphabet, for example, or a part of the screen may be colored and displayed by the color coding. For example, when the delay time is less than 1 second, the letter “A” is displayed or part of the screen is colored “green”, and when the delay time is less than 1 second to less than 3 seconds, the letter “B” is displayed or displayed. May be displayed in such a manner that a part of is colored “yellow”, and when the delay time is 3 seconds or more, the letter “C” or a part of the screen is colored “red”.

  Still another method may be adopted as a method of measuring the video delay time t. For example, the video delay time measuring device 25 includes the transmission / reception display units 11 a and 11 b of the work site communication device 11 and the transmission / reception display units 21 a and 21 b of the cockpit communication device 21 included in the video 31 of the remote control monitor 24. The time when the reception display unit 21b of the cockpit communication device 21 is lit is stored as time t5.

  Thereafter, the video delay time measuring device 25 stores the time when the transmission display unit 11a of the work site communication device 11 included in the video 31 of the remote control monitor 24 is lit as time t6, and from time t5 to time t6. Measure elapsed time. The measured time is assumed to be t ″. In this case, the actual video delay time is displayed on the video 31 of the remote control monitor 24 in the cockpit 2 after the transmission display section 11a of the work site communication device 11 is actually turned on. The time until the transmission display unit 11a of the included work site communication device 11 is turned on, that is, the time from the time t4 to the time t6 is t.

The time that can be grasped in the cockpit 2 is the above-described time t ″ and the known communication delay time T1, that is, the time taken from the time t4 to the time t5, and the actual video delay time t is obtained as follows.
t = t ″ + T1

  Then, the video delay time T obtained in this way may be displayed on the remote control monitor 23 in numerical or level-divided states as described above. Further, the work site camera 12 and the work site communication device 11 may be integrated, or the video delay confirmation monitor 24 arranged in the cockpit 2 and the cockpit communication device 21 may be integrated. By integrating in this way, the installation space can be reduced and the number of parts can be reduced, which can contribute to cost reduction.

DESCRIPTION OF SYMBOLS 100 ... Video transmission system for remote control of construction machinery 1 ... Work site 2 ... Cockpit 3 ... Shooting video 4 ... Communication line 5 ... Pilot 11 ... Transmitter / receiver for measurement signal (communication device for work site)
DESCRIPTION OF SYMBOLS 11A ... Transmission / reception display part 11a ... Transmission display part 11b ... Reception display part 12 ... Camera for work site 13 ... Mobile construction machine 21 ... Transmitter / receiver for measurement signals (communication device for cockpit)
21A ... Transmission / reception display unit 21a ... Transmission display unit 21b ... Reception display unit 22 ... Control room camera 23 ... Remote control monitor 24 ... Video delay confirmation monitor 25 ... Video delay time measuring device 26 ... Remote control controller 31 ... Video Image of monitor 24 for delay confirmation

Claims (5)

A remote control video transmission system for photographing a remote controlable construction machine at a work site and its surroundings with a camera for the work site and transmitting the video signal to a remote cockpit,
A transmitter / receiver for measurement signals provided in the construction machine or its work site and the cockpit, respectively, for transmitting and receiving signals for measuring communication delay time;
A video delay time measuring device for measuring a transmission delay time of a video signal transmitted from the work site to the cockpit on the basis of a delay time of the measurement signal between the measurement signal transceivers;
And a camera cockpit to shoot the cockpit,
Each of the measurement signal transceivers has a transmission / reception display unit for displaying a signal transmission / reception state,
The work site camera photographs the work site including a transmission / reception display unit of the measurement signal transceiver,
The cockpit camera has a transmission / reception display unit of a measurement signal transmitter / receiver disposed in the cockpit and an image of a work site composed of video signals transmitted from the work site to the cockpit on one screen. Shoot to capture,
The video delay time measuring device is a video transmitted from the work site based on a delay time of a signal between the measurement signal transceivers and a display state of each transmission / reception display unit photographed by the cockpit camera. A video transmission system for remote control of a construction machine, characterized by measuring a signal transmission delay time. A remote control video transmission system for photographing a remote controlable construction machine at a work site and its surroundings with a camera for the work site and transmitting the video signal to a remote cockpit,
The construction machine or its work site and the cockpit, respectively, a measurement signal transceiver for transmitting and receiving signals for measuring communication delay time,
A video delay time measuring device for measuring a transmission delay time of a video signal transmitted from the work site to the cockpit on the basis of a delay time of the measurement signal between the measurement signal transceivers;
Wherein provided in the cockpit, and a remote control monitor for displaying an image of the work site from the video signal transmitted from the work site,
The video delay time measuring apparatus displays the measured transmission delay time of a video signal transmitted from the work site on the remote control monitor. In the video transmission system for remote control of the construction machine according to claim 1 ,
A remote control monitor for displaying a work site image from a video signal transmitted from the work site in the cockpit;
The video delay time measuring apparatus displays the measured transmission delay time of a video signal transmitted from the work site on the remote control monitor. In the video transmission system for remote control according to claim 2 or 3 ,
The video delay time measuring device divides the measured transmission delay time of the video signal transmitted from the work site into a plurality of levels according to the size, and displays the level on the remote control monitor. A video transmission system for remote control of construction machines. In the video transmission system for remote control of the construction machine according to any one of claims 1 to 4,
The work site camera and the measurement signal transmitter / receiver disposed on the work site are integrated, and the monitor for displaying the work site image and the signal transmitter / receiver disposed in the cockpit are integrated. A video transmission system for remote control of construction machines.

Images