WO1998021631A1

WO1998021631A1 - Remote controller for mobile station

Info

Publication number: WO1998021631A1
Application number: PCT/JP1997/004083
Authority: WO
Inventors: Seiichi Mizui
Original assignee: Komatsu Ltd.
Priority date: 1996-11-11
Filing date: 1997-11-10
Publication date: 1998-05-22
Also published as: JPH10275015A; AU4886797A

Abstract

A remote controller for a mobile station, whose operability is remarkably improved by predicting an image between two consecutive images transmitted to a monitor station from the mobile station with extremely high accuracy, inserting the predicted image between the two images and displaying a real-time image at the monitor station while compensating the low transmission ability. The distance to an object outside the mobile station from the reference position of the mobile station is measured and a distance image of the object is generated. In addition, the positional variation of the mobile station is measured. The distance image and positional variation are transmitted to the monitor station. At the monitor station, the distance of movement of the mobile station until a prescribed period of time shorter than a fixed period of time elapses from the point of time when the distance image is generated. Then, a predicted distance image until the fixed period of time elapses from the point of time when the distance image is generated by changing the distance data on each picture element of the distance image by the distance of movement. The predicted distance image thus generated is inserted between the distance image and the previous distance image transmitted from the mobile station and displayed.

Description

Specification

Mobile remote control device

Technical field

The present invention relates to a remote control device for remotely controlling a moving body such as an unmanned dump truck, and more particularly to a device for performing a remote control while detecting an obstacle existing along a traveling road surface in front of the moving body.

Background art

Since unmanned vehicles such as unmanned dump trucks cannot be driven completely unattended in terms of safety, etc., obstacles existing on the road in front of the vehicles are detected and detected. The results are sent to a monitoring station, and the monitoring station remotely controls the mobile unit with the intervention of an operator.

Conventional obstacle detection devices use ultrasonic sensors, laser radars, and millimeter-wave sensors to detect obstacles ahead of the moving object in the traveling direction. It is difficult to detect obstacles in a wide area because it is narrow.

Therefore, as compared with the obstacle detection devices using the ultrasonic sensor, laser radar, and millimeter wave sensor, there is an advantage that the amount of information that can be obtained is large, the viewing angle is wide, and obstacles can be detected over a wide range. Devices that detect an obstacle by capturing an image in front of a moving object and processing the captured image have been used.

Image information is very effective in realizing remote control with high operability, and ideally, a monitoring station can obtain continuous image information from a remote location in real time, such as a television broadcast.

However, mounting large-scale broadcasting stations on mobiles only for the purpose of remotely controlling and monitoring mobiles such as construction equipment is not only space-saving but also cost-effective. Is impossible, and it is impossible in practice. For this reason, an image transmitter that balances space and cost is mounted on a mobile object, and the image transmitter is used to transmit an image to a monitoring station. In recent years, spread spectrum communication systems and the like have been adopted.

However, the image transmission capabilities of image transmitters that can be mounted on such moving objects are currently low, and they have not been able to transmit images in real time, and the number of images (frames) that can be transmitted per second is limited. . For this reason, the operator had to rely on images like frame-by-frame for remote control, and the operability was severely impaired.

The present invention has been made in view of such a situation, and predicts, with extremely high accuracy, the previous image transmitted from the mobile unit to the monitoring station and the image in the middle of the next transmitted image. By inserting an image between the previous image and the next image to be transmitted, real-time images can be displayed at the monitoring station while compensating for the low transmission capacity, and the operability of remote operation is improved. Its primary purpose is to improve it dramatically.

Furthermore, since the image transmission from the mobile unit to the monitoring station requires time, the image is displayed at the monitoring station with a delay corresponding to the transmission delay time.

A second object of the present invention is to compensate for the delay time of image transmission and display the current image as a predicted image.

In addition, to improve the remote operability of unmanned dump trucks, it is necessary to freely change the viewpoint of the camera, which is the imaging device, according to the driving situation so that the monitoring station can display the image of the viewpoint according to the driving situation. Is desirable.

Also, when remotely controlling a power shovel, if the camera viewpoint can be moved in the horizontal direction of the work surface according to the work situation and an image corresponding to it can be displayed, it can be used for slope work, etc. Improvement of workability can be expected.

However, the camera is usually fixed at a fixed position, and displaying a free-viewpoint image by moving the camera position requires a complicated mechanism. This is often difficult to achieve.

A third object of the present invention is to enable an image of a desired viewpoint to be displayed only by simple processing without providing a complicated mechanism.

Disclosure of the invention

Therefore, in order to achieve the first object, in the main invention of the first invention of the present invention, a captured image of an imaging device mounted on a moving object and imaging the outside of the moving object is monitored at regular intervals. A remote control device for the mobile unit, which transmits to the station and remotely controls the mobile unit based on the captured image received by the monitoring station;

As the imaging device, a distance image generating unit that measures a distance from a reference position to an object outside the moving object and generates a distance image of the object is provided,

A measuring means for measuring a change in position of the moving body; and a transmitting means for transmitting a measurement result of the measuring means to the monitoring station, further comprising: a measuring result of the measuring means in the monitoring station. Calculating means for calculating the moving distance of the moving body from the time when the previous distance image is generated by the distance image generating means until a predetermined time smaller than the predetermined time elapses, based on

By changing the distance data of each pixel of the distance image by the moving distance based on the calculation result of the calculation means, the prediction of the elapse of the predetermined time from the time when the previous distance image was generated is performed. Predicted distance image generating means for generating a distance image;

Display means for inserting and displaying the predicted distance image generated by the predicted distance image generation means between the previous distance image and the distance image transmitted from the moving object. .

That is, according to the configuration of the present invention, as shown in FIGS. 1, 3, 4, and 5, the distance from the reference position of the moving body 10 to the object outside the moving body 10 is measured. A distance image 40 of the object is generated. In addition, the position change V of the moving body 10 is measured. These are transmitted to the monitoring station 20. In the monitoring station 20, based on the transmitted content, the moving distance of the moving body 10 from the time when the previous distance image 40 was generated until a predetermined time t smaller than the predetermined time Δt has elapsed. L 1 = V * t is calculated. Then, by changing the distance data d of each pixel 50 of the distance image 40 by the moving distance L1 (d−L1), the distance data d is changed for a fixed time Δ from the time when the previous distance image 40 was generated. Predicted distance images 42 (43, 44, 45, 46) during the passage of t are generated. Predicted distance image 4 2 (4 3, 4 4, 4 5, 4 6) force generated in this way Force between the previous distance image 40 and the next distance image 41 transmitted from the moving object 10 force It is inserted and displayed between them.

Further, according to another aspect of the first invention of the present invention, by utilizing the information indicating the position / posture change of the moving body 10 in each direction, as shown in FIG. 7 and FIG. The three-dimensional coordinate position of each pixel 50 of the distance image 60 of the vehicle body coordinate system X—Y—Z at the time when the time t has elapsed is obtained, and a predicted distance image 62 is generated based on the three-dimensional coordinate position. .

As described above, according to the present invention, based on the movement information of a moving object, image processing for changing the distance data or the three-dimensional coordinate position of the pixel of the previous distance image by an amount corresponding to the movement information is performed. As a result, the previous distance image (the latest distance image) and the predicted distance image in the middle of the next transmitted distance image are generated extremely accurately and at high speed. The predicted distance image generated in this way is inserted between the previous distance image and the next transmitted distance image, thereby compensating for the low image transmission capability and real-time image (eg, video rate). , 30 frames / sec) are displayed on the monitoring station. Remote control based on this real-time image dramatically improves remote controllability.

Further, in order to achieve the second object, according to the second invention of the present invention, in the configuration of the first invention, as shown in FIGS. 1 and 10, a distance from the mobile object 10 to the monitoring station 20 is increased. When the transmission delay time required to transmit the image 60 is Td and the time when the previous distance image 60 was generated is t1, only the time t from the time t1 that allows for the transmission delay time Td The predicted distance image 6 2 (63, 64, 65) at the elapsed time t 1 + t is generated. In this case, the predicted distance image 62 at the time t 1 + t is displayed when the time t−Td has elapsed since the previous distance image 60 was displayed on the display means 23 of the monitoring station 20 at the time t 1 + Td. ing.

In this way, the delay time of the image transmission from the mobile unit 10 to the monitoring station 20 is compensated, and the image at the current time t1 + t can be displayed as the predicted distance image 62 (63, 64, 65).

Further, in order to achieve a third object, according to a second aspect of the present invention, in the second aspect of the first aspect, the origin position of the vehicle body coordinate system XYZ is shifted by a predetermined amount in a predetermined coordinate axis direction. By performing a changing process (for example, changing by H in the vertical Y-axis direction), the viewpoint of the distance image displayed on the display means 23 is moved upward by H0.

As described above, an image of a desired viewpoint can be displayed only by simple processing without providing a complicated mechanism for moving the position of the camera. BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a block diagram showing a configuration example of a mobile remote control device according to an embodiment of the present invention.

FIG. 2 is a flowchart showing a procedure of processing executed by the monitoring station shown in FIG. FIG. 3 is a diagram schematically showing a relationship between a distance image transmitted from a moving object and a predicted distance image.

FIG. 4 is a diagram showing a state in which the moving body travels from above.

5 (a), 5 (b) and 5 (c) are diagrams showing a distance image and a predicted distance image obtained when the moving object travels on the traveling path shown in FIG.

FIG. 6 is a diagram showing a relationship between a vehicle body coordinate system at a certain time and a vehicle body coordinate system at a time when a predetermined time has elapsed from this time.

FIG. 7 is a diagram showing a state in which the moving object travels from above.

Figs. 8 (a), (b) and (c) are obtained when the moving object travels on the travel path shown in Fig. 7. FIG. 5 is a diagram showing a distance image and a predicted distance image to be obtained.

FIG. 9 is a flowchart showing the procedure of another process executed by the monitoring station shown in FIG.

FIG. 10 is a diagram used to explain a process of displaying a current image as a predicted distance image at a monitoring station. BEST MODE FOR CARRYING OUT THE INVENTION

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

In the present embodiment, as shown in FIG. 4, when a moving object 10 such as an unmanned dump truck travels on the traveling path 31, a tree 32, a house 33, and other vehicles 34 seen in front of the traveling object 31. It is assumed that the travel of the mobile unit 10 is remotely controlled by generating a distance image composed of the following, and determining the presence or the like of an obstacle by the monitoring station operator based on the distance image.

Note that, in the present invention, the content of the remote operation is optional, not only when the traveling of the moving body is remotely controlled, but also when the moving body that travels while performing a predetermined work with a predetermined work machine is remotely controlled. Is also applicable. For example, when excavating an object with a power shovel, applying the present invention makes it possible to obtain information on the distance to the excavated object during and after movement, its shape, and its shape in real time. The remote controllability of the machine operation lever can be dramatically improved.

In FIG. 6, X—Y—Z indicates a vehicle body coordinate system that moves together with the moving body 10. X is a coordinate axis corresponding to the vehicle width direction of the moving body 10, Z is a coordinate axis corresponding to the traveling direction of the moving body 10 (the traveling path 31), and Y is a vertical coordinate axis. FIG. 1 shows a configuration of a remote control device for a mobile object according to an embodiment of the present invention. As shown in FIG. 1, this remote device is roughly composed of an on-board device mounted on a mobile object 10 and a monitoring station 20.

The on-board device of the moving body 10 is configured to move from the reference position (reference plane) on the moving body 10 to each object in front of the moving body 10 in the traveling direction based on the imaging result of the camera mounted on the moving body 10. The distance image generation unit 1 measures the distance at a distance, and generates a three-dimensional distance image in which the distance data d indicating the distance from the reference position and the two-dimensional coordinate position data (i, j) are associated with each pixel. 1, an image transmitter 13 that wirelessly transmits the distance image generated by the distance image generation unit 11 to the monitoring station 20 via the antenna 15 at regular time intervals Δt, and a mobile unit 1 A measuring unit 12 that measures the position change or the position and posture change of the moving object 10 and monitors the measurement results of the measuring unit 12 via the antenna 16. And a sensor signal transmitter 14 for wirelessly transmitting to the station 20.

On the other hand, the monitoring station 20 receives the distance image and the sensor signal transmitted from the on-board device of the mobile object 10 via the antenna 24, and based on the received content, the previous distance image was generated. The moving object 10 moves from the time t1 until a predetermined time t smaller than the fixed time Δt, which is the interval of the transmission of the distance image, elapses. The force that performs the process of calculating the moving distance L1 that changes in the distance direction indicated as the distance data d, or the vehicle body coordinate system X—Y_Z at the time t1 when the previous distance image was generated, is calculated as the time t1 Is converted to the body coordinate system X—Y—Z at the time t 1 + t when the predetermined time t that is smaller than the fixed time Δt has elapsed, and the body coordinate system at the time t 1 + t at which the predetermined time t has passed X _ Y — 3D coordinate position of each pixel in the previous range image in Z A calculation unit 21 that performs a process of calculating the distance image d of each pixel of the distance image by the moving distance L1 based on the calculation result, thereby obtaining a time tl at which the previous distance image was generated. Generates a predicted distance image at time t1 + t during the image transmission interval Δt elapses from A predicted distance image generation unit 22 that generates a predicted distance image at a time t 1 + t during which the image transmission interval Δt elapses from the time t 1 at which the previous distance image was generated based on the three-dimensional coordinate position; A display unit 23 such as a CRT display that inserts and displays the generated predicted distance image between the previous distance image and the next distance image transmitted from the moving object 10 is displayed. Have been. In the monitoring station 20, based on the distance image displayed on the display unit 23 or based on a further image processing of the distance image, the operator determines whether there is an obstacle present in front of the moving object 10 and the like, and the drive control is performed. By transmitting the signal via the antenna 24, the mobile unit 10 is remotely controlled, and the traveling of the mobile unit 10 is drive-controlled. Here, the measurement unit 12 can be configured by a speed sensor that detects the speed V of the moving body 10.

In another embodiment, the measuring unit 12 detects a three-dimensional position of the moving body 10 such as a global positioning sensor (GPS) or a three-dimensional velocity of the moving body 10. Speed sensor and an angular speed sensor such as a gyro. For example, it can be composed of a gyroscope that detects the tilting angle of the moving body 10 in one direction and two inclinometers that detect the pitching angle and the rolling angle of the vehicle. As shown in Fig. 6, the rotation of the coordinate axes of the body coordinate system X—Y_Z at time t1 + t at time t1 after time t1 with respect to the coordinate axes of the body coordinate system X—Y—Z at time t1 The rotation angle (RX0, RY0, RZ0) of the vehicle body indicating the angle is output. The origin position (Vx · t, Vy · t, Vz · t) of the body coordinate system X-YZ at time t1 + t as viewed in body coordinate system X-YZ at time tl is the speed sensor at time t1 + t Output V

(Vx, Vy, Vz). It can also be obtained as the output of a position sensor such as GPS.

In the distance image generation unit 11, for example, a distance image 40 composed of a traveling road surface 31 as shown in FIG. 5A and a tree 32, a house 33, and other vehicles 34 existing along the traveling road surface 31. Is generated. Each pixel 50 of the distance image 40 has three dimensions: i, j; a two-dimensional coordinate position (i, j) in the two-dimensional coordinate system, and a distance d from the reference position (reference plane) of the moving object 10. The data (i, j, d) are associated with each other, and the pixel of each position j of the distance image 40 has brightness corresponding to the distance d. The brightness of the pixel 50 increases as the distance d decreases, and the brightness increases (the screen becomes whiter). As a method of distance measurement for generating such a three-dimensional distance image, for example, Japanese Patent Application No. Hei 7—2009 A method using a multi-lens lens (multi-eye lens) described in No. 9 can be used.

Hereinafter, the processing performed by the calculation unit 21 and the predicted distance image generation unit 22 of the monitoring station 20 will be described with reference to the flowchart of FIG.

• First Embodiment

In this embodiment, as shown in FIG. 4, it is assumed that the moving body 10 is traveling straight on a straight running path 31. In particular, this embodiment is suitable when the vehicle is traveling straight ahead, and can generate a predicted distance image by a simple process with a simple configuration. In this embodiment, the measuring unit 12 can be constituted only by the speed sensor. As shown in FIG. 5, the monitoring station 20 receives a distance image 40 (referred to as a previous distance image) at time t1. At time t2 when a time Δt has elapsed from time t1, the next distance image 41 is received (step 101: see FIGS. 5 (a) and 5 (b)).

Next, based on the received sensor signal, that is, based on the output V of the speed sensor, the moving object 10 from the time when the previous distance image 40 was generated until a predetermined time t (ΔΔt) elapses. Is calculated by the following equation.

L 1 = Vt (5)

(Step 10 2) The distance image when the time t has elapsed from the previous distance image 40 should be closer to the entire distance L 1 than the previous distance image 40 as a whole.

Therefore, a process of subtracting the calculated moving distance L1 from the distance data d of the pixel 50 of the previous distance image 40 is performed, and the distance data of the pixel 50 is changed from d to d-L1. You. This subtraction process is performed for all pixels. When the distance data changes to a negative value, the distance data may be set to 0. As a result, as shown in FIG. 5 (c), a predicted distance image 42 at a time t1 + t at which the image transmission interval Δt elapses from the time t1 when the previous distance image 40 was generated is generated. Be done

0 3). The predicted distance image 42 may be generated not only as one frame but also as two or more frames at different times t as the predicted distance images 42, 43, 44,.

The predicted distance image generated in this way, for example, a predicted distance image composed of 5 frames 4 2, 4 3, 4 4, 4 5, 4 6 Force As shown in Fig. 3, the previous distance image 40 and then the moving object It is inserted between the distance image 41 transmitted from 10 and the distance image 41 and displayed on the display unit 23.

As a result, a real-time image of, for example, a video rate of 30 frames / second is displayed on the display unit 23, which can sufficiently compensate for the low image transmission capability (image transmission at At intervals). . Then, the remote operation by the operator based on this real-time image greatly improves the remote operability of the mobile unit 10.

• Second embodiment

In this embodiment, as shown in FIG. 7, it is assumed that a moving body 10 is traveling on a traveling path 31 which is a curved road.

In this embodiment, the measurement unit 12 is configured by a sensor that detects a position change in each direction and a posture change in each direction, such as a speed sensor (or a position sensor) and an angular velocity sensor. Hereinafter, description will be made with reference to the flowchart shown in FIG.

As shown in FIG. 8, the monitoring station 20 receives the distance image 60 at time t1. At the time t2 when a time Δt has elapsed from the time t1, the next distance image 61 is received (step 201: see FIGS. 8 (a) and 8 (b)).

Hereinafter, based on the received sensor signal, the vehicle body coordinate system X—Y—Z at the time t1 when the previous distance image 60 was generated is calculated as the time t1 + t at which the predetermined time t has elapsed from the time. Is converted to the body coordinate system X—Y—Z at, and the three-dimensional coordinate position of each pixel of the previous distance image 60 in the body coordinate system X—Y—Z at the time t 1 + t at which the predetermined time t has elapsed Is calculated based on the three-dimensional coordinate position of each pixel of the previous distance image 60 in the body coordinate system X—Y—Z at this time t 1 + t. A process of generating the predicted distance image 62 at the time t1 + t during the elapse of the image transmission interval Δt from the time t1 is executed. First, since the three-dimensional information of (i, j, d) is associated with each pixel 50 of the distance image 60 as described above, this pixel is represented by the distance image data (i, j, d). Each pixel 50 to be moved is moved together with the moving body 10 as shown in FIGS. 6 and 7, and the three-dimensional coordinate position data (Χ) on the vehicle body coordinate system X—Y—Z having the predetermined position of the moving body 10 as the origin. , Υ, Ζ) can be converted into each pixel. As a result, each pixel 50 (point 画像 in FIG. 6) of the distance image 60 is represented by a three-dimensional coordinate position (XP (t 1), YP (t 1) on the body coordinate system X—Υ—Z at time t1. , ZP (t1)) (step 202).

Further, based on the received sensor signal, the rotation angle (RX0, RY0, RZO) of the vehicle body from time t1 to time t1 + t described above, and the time t1 viewed from the vehicle body coordinate system at time t1. The origin position (Vx * t, Vy · Vz · t) of the body coordinate system X—Y—Z at + t is obtained. The amount of change (Vx-t, Vyt, Vzt) between the time t1 and the time t1 + t is the output V (Vx, Vy, Vz) of the speed sensor at the time t1 + t. ) May be calculated based on the position at time t1 and the position at time t1 + t output from a position sensor such as GPS (step 203).

Next, the rotation angle (RXO, RY0, RZO) of the vehicle body between time t1 and time t1 + t, the vehicle body coordinate system X at time t1 + t viewed from the vehicle body coordinate system at time t1 Using the origin position of Y—Z (Vx.t, Vy-t, Vz-t), as shown in Fig. 6, the coordinate position of pixel 50 (point P) on the body coordinate system at time t1 (XP (tl), YP (tl), ZP (tl)) and the coordinate position (XP (t l + t), YP (t l + t) of the same pixel 50 (point P) in the vehicle body coordinate system at time t 1+ t , ZP (tl + t)) is obtained as in the following equation (1).

(0

However, in the above equation (1), MRO is a rotation matrix of the vehicle body coordinate system, and is represented by the following equation (2) using the rotation angles (RX0, RY0, RZ0) of the vehicle body.

COS (RXO) SIN (RXO) 0 COS (RYO) 0 -SIN (RYO)

MRO = -SIN (RXO) COS (RXO) 0 0 1 0

0 0 1 SIN (RYO) 0 COS (RYO)

Therefore, from the above equation (1), the coordinate position (XP (tl), YP (tl), ZP (tl)) of the pixel 50 (point P) in the vehicle body coordinate system at the time t1 is calculated at the time tl + t. It can be converted into the coordinate position of the body coordinate system (XP (t l + t), YP (tl + t), ZP (t 1 + t)) as follows.

(3)

The above conversion is performed for all the pixels of the range image 60 (Step 204). Next, the three-dimensional coordinate positions (XP (t l + t), YP (t l + t), ZP (t l + t)) of the vehicle body coordinate system at the time t l + t obtained above are converted into the distance image data (i, The process of converting it back to j and d) is performed. In this case, only the three-dimensional coordinates existing within the angle of view of the camera are selected, and the selected three-dimensional coordinates are returned to the distance image data. In this way, the three-dimensional coordinate position of each pixel 50 is converted into the distance d and the coordinates i and j on the screen. As a result, as shown in FIG. 8 (c), the time t when the previous distance image 60 was generated A predicted distance image 62 is generated at time t 1 + t in the middle of the elapse of the image transmission interval Δt from 1 (step 205).

The predicted distance image 62 may be generated not only as one frame but also as two or more frames at different times t as the predicted distance images 62, 63, 64,. The predicted distance image generated in this way, for example, a predicted distance image composed of 5 frames 62, 63, 64, 65, 66 As shown in Fig. 3, the previous distance image 60 and the moving object It is inserted between the distance image 61 transmitted from 10 and the distance image 61 and displayed on the display unit 23.

As a result, a real-time image of, for example, a video rate of 30 frames / second is displayed on the display unit 23, and the low image transmission capability (image transmission at the At interval) can be sufficiently compensated. . Then, the remote operation by the operator based on this real-time image greatly improves the remote operability of the mobile unit 10.

By the way, as shown in FIG. 10, when a distance image is transmitted from the mobile object 10 to the monitoring station 20, there is actually a transmission delay time Td required for transmission. Therefore, the image at the present time can be displayed on the display unit 23 as the predicted distance image in consideration of the delay time Td.

That is, as shown in FIG. 10, assuming that the previous distance image 60 was generated at time t1 by the moving object 10, based on the distance image 60, The monitoring station 20 generates a predicted distance image 62 at a time t 1 + t at which a time t has elapsed from the time t 1 in consideration of the transmission delay time Td. Then, when the time t-Td has elapsed since the previous distance image 60 was displayed on the display unit 23 of the monitoring station 20 at time tij + td, the predicted distance image at the above time t1 + t 6 Display 2. The other predicted distance images 63, 64, and 65 are displayed in the same manner. As a result, the current distance images 62, 63, 64, 65 are displayed at the monitoring station 20, and the remote control of the moving object 10 by the operator is further improved.

Note that this method may be applied to the above-described first embodiment.

It is also possible to change the viewpoint of the distance image displayed on the display unit 23 of the monitoring station 20 to a desired viewpoint.

In this case, a process of changing the origin position of the vehicle body coordinate system XY at time t 1 + t before conversion into the distance image 62 by a desired amount in the desired coordinate axis direction may be performed. Specifically, the viewpoint of the distance image displayed on the display unit 23 is raised by Η, and the If you want to raise the point of view of the data by H, the following expression (4) should be added to the right side of the above expression (3).

By changing the origin position of the vehicle body coordinate system X—Y—Z at time t 1 + t by に in the vertical Υ-axis direction, the viewpoint of the distance image displayed on the display unit 23, that is, the operator Is moved upward by Η.

If this method is applied, an image of a desired viewpoint can be displayed with simple processing without providing a complicated mechanism for moving the position of the camera. Therefore, the remote operability of the moving body 10 by the operator is further improved.

Claims

The scope of the claims

1. An image captured by an imaging device mounted on the moving body and capturing an image of the outside of the moving body is transmitted to the monitoring station at regular time intervals, and the moving body is remotely controlled based on the captured image received by the monitoring station. In a remote control device of a moving body to be operated,

Display means for inserting and displaying the predicted distance image generated by the predicted distance image generation means between the previous distance image and the distance image transmitted from the moving object. Remote control of the body.

2. An image captured by an imaging device that is mounted on the moving body and captures an image of the outside of the moving body is transmitted to the monitoring station at regular intervals, and the moving body is remotely controlled based on the captured image received by the monitoring station. In a remote control device of a moving body to be operated,

The imaging device measures a distance from a reference position to an object outside a moving object, and generates a distance image in which each pixel is associated with a three-dimensional coordinate position of a vehicle body coordinate system that moves with the moving object. In addition to providing image generation means,

The moving body is provided with measuring means for measuring a change in position and orientation in each direction of the moving body, and transmitting means for transmitting a measurement result of the measuring means to the monitoring station. Based on the measurement result of the measuring means, the vehicle body coordinate system at the time when the previous distance image was generated by the distance image generating means is changed to the vehicle body coordinate at the time when a predetermined time smaller than the predetermined time has elapsed from the time. Calculating means for calculating the three-dimensional coordinate position of each pixel of the previous distance image in the vehicle body coordinate system at the time when the predetermined time has elapsed;

Based on the three-dimensional coordinate position of each pixel of the previous distance image in the vehicle body coordinate system at the time when the predetermined time calculated by the calculating means has elapsed, the predetermined time from the time when the last distance image was generated Predicted distance image generating means for generating a predicted distance image on the way of passing;

3. When the transmission delay time required to transmit the distance image from the moving object to the monitoring station is Td, and the time at which the previous distance image was generated is t1, the time t that allows for the transmission delay time Td A predicted distance image at time t 1 + t, which has elapsed from time 1 by time t, is generated, and time t1-T d has elapsed since the previous distance image was displayed at time t 1 + Td on the display means of the monitoring station. 3. The remote control device for a moving object according to claim 1, wherein a predicted distance image at a time t1 + t is displayed at a point in time.

4. The remote control device for a moving object according to claim 1, wherein the measuring means is a speed sensor for detecting a speed of the moving object.

5. The measuring means is a combination of a position sensor for detecting the position of the moving body or a speed sensor for detecting the speed of the moving body, and a posture angle sensor for detecting the posture angle of the moving body. A remote control device for a mobile object according to claim 1.

6. The method according to claim 2, wherein the viewpoint of the distance image displayed on the display unit is moved by performing a process of changing the origin position of the vehicle body coordinate system by a predetermined amount in a predetermined coordinate axis direction. Remote control device for mobile objects.