JP5473304B2 - Remote location image display device, remote control device, vehicle control device, remote control system, remote control method, remote control program, vehicle control program, remote location image display method, remote location image display program - Google Patents

Description

  The present invention includes, for example, a remote image display device for operating an unmanned vehicle at a remote location, a remote control device, a vehicle control device, a remote control system, a remote control method, a remote control program, a vehicle control program, a remote location image display method, The present invention relates to a remote image display program.

A conventional remote control device sends an image captured by a camera attached to an unmanned vehicle to a remote control terminal, and the operator operates the control terminal while watching the video sent to the unmanned vehicle. Is remotely controlled (Patent Document 1).
JP-A-8-84375

  With a conventional remote control device, a remote pilot can only see the image of the shooting direction of the camera attached to the unmanned vehicle, and the pilot controls the unmanned vehicle with blind spots in the surrounding information of the unmanned vehicle. There was a problem that had to be done.

  The present invention has been made, for example, in order to solve such a problem, and an object of the present invention is to make it possible to visually provide surrounding information of an unmanned vehicle to a remote operator.

  The remote image display device of the present invention includes a vehicle information receiving unit that receives position information of the vehicle transmitted from a vehicle traveling in a remote location using a communication device, and the vehicle received by the vehicle information receiving unit. A map data acquisition unit that acquires map data around the vehicle from a map database device that stores map data of a travel region of the vehicle, and map data acquired by the map data acquisition unit A peripheral image generation unit that generates a peripheral image representing a map of the periphery of the vehicle based on a CPU (Central Processing Unit), and a peripheral image display that displays the peripheral image generated by the peripheral image generation unit on a monitor device A part.

  According to the present invention, for example, by generating and displaying a surrounding image of a vehicle, it is possible to visually provide surrounding information of the unmanned vehicle to a remote operator.

Embodiment 1 FIG.
A remote control system that controls a vehicle from a remote control device will be described.

FIG. 1 is a schematic diagram of a remote control system 100 according to the first embodiment.
An overview of the remote control system 100 according to the first embodiment will be described below with reference to FIG.

The remote control system 100 includes a remote control device 300 that transmits a control command from a remote place, and a vehicle on which a vehicle control device 209 that controls the vehicle based on the control command transmitted from the remote control device 300 is mounted.
The vehicle may be unmanned or manned, and travels by automatic control (autonomous operation) by the vehicle control device 209.
Hereinafter, the vehicle on which the vehicle control device 209 is mounted is referred to as an “unmanned vehicle 200”.

The unmanned vehicle 200 is provided with a GPS receiving system 201 (GPS: Global Positioning System), an antenna 202 and a gyro 203.
The GPS reception system 201 is a positioning device that receives a positioning signal from a positioning satellite and measures the current location of the unmanned vehicle 200 based on the reception result of the positioning signal. A conventional GPS receiver can be used as the GPS receiving system 201.
The antenna 202 is a communication device that transmits and receives a radio wave including a positioning result of the GPS receiving system 201 and a steering command from the remote control device 300.
The gyro 203 is a sensor device that measures a posture angle (roll angle, pitch angle, yaw angle) indicating a traveling direction of the unmanned vehicle 200.

First, the vehicle control device 209 includes position / posture information 210a (an example of position information) indicating the current position of the unmanned vehicle 200 measured using the GPS receiving system 201 and the posture angle of the unmanned vehicle 200 measured using the gyro 203. Is transmitted using the antenna 202 (S100: vehicle information transmission step).
Next, the remote control device 300 generates a three-dimensional peripheral image 313a (an example of a peripheral image) representing a three-dimensional map around the unmanned vehicle 200 based on the position and orientation information 210a transmitted by the vehicle control device 209. The image is displayed on the display device 901 (also referred to as a monitor device) (S200: vehicle periphery image display step).
Next, the remote control device 300 uses the operation command designated by the operator who has viewed the peripheral image displayed on the display device 901 by operating the input device such as the keyboard 902 and the mouse 903 as the movement destination position information 322a. (S300: Steering command transmission step).
Then, the vehicle control device 209 determines the unmanned vehicle 200 as a CPU (Central Processing) based on the destination position information 322a transmitted by the remote control device 300.
Control (steering, driving) using Unit) (S400: automatic steering process).
The detail of each process (S100-S400) is mentioned later.

FIG. 2 is a functional configuration diagram of the vehicle control device 209 according to the first embodiment.
A functional configuration of the vehicle control device 209 according to the first embodiment will be described below with reference to FIG.

  The vehicle control device 209 includes a position and orientation determination unit 210 (an example of a position information generation unit), a vehicle transmission unit 211 (an example of a vehicle information transmission unit), a vehicle reception unit 212 (an example of a steering command reception unit), and a vehicle control unit 213. And a vehicle storage unit 290.

  The position and orientation determination unit 210 measures the current position of the unmanned vehicle 200 using the GPS reception system 201 and measures the posture angle of the unmanned vehicle 200 using the gyro 203, and indicates the current position and posture angle of the unmanned vehicle 200. The position / orientation information 210a is generated using the CPU.

  The vehicle transmission unit 211 transmits the position and orientation information 210 a generated by the position and orientation determination unit 210 to the remote control device 300 using the antenna 202.

  The vehicle receiving unit 212 receives the destination position information 322a transmitted by the remote control device 300 using a communication device.

  The vehicle control unit 213 controls (steers and drives) the unmanned vehicle 200 using the CPU based on the destination position information 322a received by the vehicle reception unit 212.

  The vehicle storage unit 290 stores data used by the vehicle control device 209 using a storage medium. For example, the vehicle storage unit 290 stores the position / orientation information 210 a generated by the position / orientation determination unit 210 and the destination position information 322 a received by the vehicle reception unit 212.

FIG. 3 is a functional configuration diagram of the remote control device 300 according to the first embodiment.
The functional configuration of the remote control device 300 in the first embodiment will be described below with reference to FIG.

The remote control device 300 includes a remote location image display device 310, a vehicle control device 320, and a device storage unit 390.
The remote location image display device 310 generates a three-dimensional peripheral image 313a of the unmanned vehicle 200 based on the position and orientation information 210a transmitted by the vehicle control device 209 of the unmanned vehicle 200, and displays the generated three-dimensional peripheral image 313a as a display device. 901 is displayed.
The vehicle maneuvering device 320 transmits a maneuver command designated by the operator who has viewed the three-dimensional peripheral image 313a displayed on the display device 901 by operating an input device such as the keyboard 902 or the mouse 903 as the destination position information 322a. 302 is used for transmission.
Device storage unit 390 stores data used by remote image display device 310 and vehicle control device 320 using a storage medium. For example, the device storage unit 390 stores position / orientation information 210a, a three-dimensional peripheral image 313a, and destination position information 322a.

  The remote location image display device 310 includes a device reception unit 311 (an example of a vehicle information reception unit), a map data acquisition unit 312, a three-dimensional peripheral image generation unit 313 (an example of a peripheral image generation unit), and a three-dimensional peripheral image display unit 314. And a three-dimensional map database 391.

  The three-dimensional map database 391 is a device that stores and manages the three-dimensional map data 391a of the travel area of the unmanned vehicle 200 using a storage medium. The three-dimensional map data 391a indicates information on roads located in the area and information on features other than roads. For example, road information is information such as roads, lanes, white lines, signs, etc. (three-dimensional coordinates, shape, size, color, type, etc.), and information on features other than roads are buildings, facilities, Information on tunnels, mountains, rivers, etc. For example, the shape and size are represented by the three-dimensional coordinates of each of a plurality of end points forming the contour of the feature. As the three-dimensional map database 391, a GIS (Geographical Information System) database can be used.

  The device receiving unit 311 receives the position and orientation information 210a of the unmanned vehicle 200 transmitted from the vehicle control device 209 of the unmanned vehicle 200 traveling in a remote place using the reception antenna 301.

  The map data acquisition unit 312 acquires the 3D map data 391a around the unmanned vehicle 200 from the 3D map database 391 based on the position and orientation information 210a of the unmanned vehicle 200 received by the device reception unit 311.

  The three-dimensional peripheral image generation unit 313 generates a three-dimensional peripheral image 313a of the unmanned vehicle 200 using the CPU based on the three-dimensional map data 391a acquired by the map data acquisition unit 312.

  The three-dimensional peripheral image display unit 314 displays the three-dimensional peripheral image 313 a generated by the three-dimensional peripheral image generation unit 313 on the display device 901.

  The vehicle control device 320 includes a control information input unit 321, a movement position generation unit 322 (an example of a control command generation unit), and a device transmission unit 323 (an example of a control command transmission unit).

The maneuver information input unit 321 uses the keyboard 902 and the mouse 903 as maneuver information 321a designated by the user of the remote manipulator 300 (hereinafter referred to as “maneuver” or “remote maneuver”) as information for maneuvering the unmanned vehicle 200. Input from the input device.
The movement position generation unit 322 generates the movement destination position information 322a using the CPU based on the operation information 321a input by the operation information input unit 321.
The device transmission unit 323 controls the unmanned vehicle 200 by transmitting the movement destination position information 322 a generated by the movement position generation unit 322 to the vehicle control device 209 of the unmanned vehicle 200 using the transmission antenna 302.

FIG. 4 is a diagram illustrating an example of hardware resources of the remote control device 300 and the vehicle control device 209 according to the first embodiment.
In FIG. 4, the remote control device 300 and the vehicle control device 209 include a CPU 911 (also referred to as a central processing unit, a central processing unit, a processing unit, an arithmetic unit, a microprocessor, a microcomputer, and a processor) that executes a program. Yes. The CPU 911 is connected to the ROM 913, the RAM 914, the communication board 915, and the magnetic disk device 920 via the bus 912. Further, the CPU 911 of the remote control device 300 is connected to input / output devices such as a display device 901, a keyboard 902, and a mouse 903. Further, the CPU 911 of the vehicle control device 209 is connected to a drive device (device for controlling the speed and traveling direction) of the unmanned vehicle 200. The CPU 911 controls these hardware devices. Instead of the magnetic disk device 920, another storage device (for example, a semiconductor memory such as a RAM or a flash memory) may be used.
The RAM 914 is an example of a volatile memory. The storage medium of the ROM 913 and the magnetic disk device 920 is an example of a nonvolatile memory. These are examples of a storage device, a storage device, or a storage unit. A storage device in which input data is stored is an example of an input device, an input device, or an input unit, and a storage device in which output data is stored is an example of an output device, an output device, or an output unit.
The communication board 915 is an example of an input / output device, an input / output device, or an input / output unit.

  The communication board 915 is wired or wirelessly connected to a communication network such as a LAN (Local Area Network), the Internet, a WAN (Wide Area Network) such as ISDN, and a telephone line.

  The magnetic disk device 920 stores an OS 921 (operating system), a program group 923, and a file group 924. The programs in the program group 923 are executed by the CPU 911 and the OS 921.

  The program group 923 stores a program for executing a function described as “˜unit” in the embodiment. The program is read and executed by the CPU 911.

In the file group 924, in the embodiment, result data such as “determination result”, “calculation result of”, “processing result of” when executing the function of “to part”, “to part” The data to be passed between programs that execute the function “,” other information, data, signal values, variable values, and parameters are stored as items “˜file” and “˜database”.
The “˜file” and “˜database” are stored in a recording medium such as a disk or a memory. Information, data, signal values, variable values, and parameters stored in a storage medium such as a disk or memory are read out to the main memory or cache memory by the CPU 911 via a read / write circuit, and extracted, searched, referenced, compared, and calculated. Used for CPU operations such as calculation, processing, output, printing, and display. Information, data, signal values, variable values, and parameters are temporarily stored in the main memory, cache memory, and buffer memory during the CPU operations of extraction, search, reference, comparison, operation, calculation, processing, output, printing, and display. Is remembered.
In addition, arrows in the flowcharts described in the embodiments mainly indicate input / output of data and signals, and the data and signal values are recorded in the memory of the RAM 914, the magnetic disk of the magnetic disk device 920, and other recording media. . Data and signal values are transmitted online via a bus 912, signal lines, cables, or other transmission media.

  In addition, what is described as “˜unit” in the embodiment may be “˜circuit”, “˜device”, “˜device”, and “˜step”, “˜procedure”, “˜”. Processing ". That is, what is described as “˜unit” may be realized by firmware stored in the ROM 913. Alternatively, it may be implemented only by software, or only by hardware such as elements, devices, substrates, and wirings, by a combination of software and hardware, or by a combination of firmware. Firmware and software are stored as programs on a magnetic disk or other recording medium. The program is read by the CPU 911 and executed by the CPU 911. That is, the program causes the computer to function as “to part”. Alternatively, the procedure or method of “to part” is executed by a computer.

FIG. 5 is a flowchart showing a remote control method of remote control system 100 in the first embodiment.
A remote control method of remote control system 100 in the first embodiment will be described below with reference to FIG.

<S100: Vehicle information transmission process>
The vehicle control device 209 of the unmanned vehicle 200 measures the current position of the unmanned vehicle 200 and measures the posture angle of the unmanned vehicle 200 to obtain position and orientation information 210a indicating the current position and posture angle of the unmanned vehicle 200 as a remote control device 300. Send to.

<S200: Vehicle Peripheral Image Display Process (Remote Location Image Display Method)>
The remote control device 300 receives the position / orientation information 210a from the vehicle control device 209, generates a three-dimensional peripheral image 313a of the unmanned vehicle 200 based on the received position / orientation information 210a, and displays the generated three-dimensional peripheral image 313a. It is displayed on the device 901.

<S300: Steering command transmission process>
The remote control device 300 inputs the control information 321a designated by the operator who has viewed the three-dimensional peripheral image 313a displayed on the display device 901 by operating the input device such as the keyboard 902 and the mouse 903, and the input control information. The destination position information 322a is generated based on 321a, and the generated destination position information 322a is transmitted to the vehicle control device 209 of the unmanned vehicle 200.

<S400: Autopilot process>
The vehicle control device 209 of the unmanned vehicle 200 receives the destination location information 322a from the remote control device 300, and controls the unmanned vehicle 200 based on the received destination location information 322a.

  Below, the detail of each process (S100-S400) is demonstrated.

FIG. 6 is a flowchart of the vehicle information transmission step (S100) in the first embodiment.
Details of the vehicle information transmission step (S100) in the first embodiment will be described below based on FIG.
Each “˜ part” of the vehicle control device 209 of the unmanned vehicle 200 executes processing described below using a CPU.

<S110: Location Information Generation Processing>
The position / orientation determination unit 210 measures the current location of the unmanned vehicle 200 using the GPS reception system 201 and inputs a positioning result from the GPS reception system 201. Further, the position / orientation locating unit 210 measures the attitude angle of the unmanned vehicle 200 using the gyro 203 and inputs the measurement result from the gyro 203. The position and orientation determination unit 210 generates position and orientation information 210 a including the positioning result input from the GPS reception system 201 and the measurement result input from the gyro 203, and stores the generated position and orientation information 210 a in the vehicle storage unit 290. .
After S110, the process proceeds to S120.

<S120: Vehicle information transmission process>
The vehicle transmission unit 211 inputs the position / orientation information 210a generated in S110 from the vehicle storage unit 290, and transmits the input position / orientation information 210a to the remote control device 300 from the antenna 202 by radio waves.
After S120, the vehicle information transmission step (S100) ends.

  The vehicle information transmission step (S100) is repeatedly executed periodically (for example, every second).

FIG. 7 is a flowchart of the vehicle periphery image display step (S200) in the first embodiment.
Details of the vehicle periphery image display step (S200) in the first embodiment will be described below with reference to FIG.
Each “˜ unit” of the remote control device 300 executes processing described below using a CPU.

<S210: Vehicle information reception process>
The device receiving unit 311 receives the position / orientation information 210 a transmitted by the vehicle control device 209 of the unmanned vehicle 200 using the receiving antenna 301, and stores the received position / orientation information 210 a in the device storage unit 390.
After S210, the process proceeds to S220.

<S220: Map information acquisition process>
The map data acquisition unit 312 receives the position / orientation information 210a received in S210 from the device storage unit 390, and generates 3D map data 391a representing a map around the unmanned vehicle 200 based on the input position / orientation information 210a. Obtained from the dimensional map database 391.
For example, the map data acquisition unit 312 identifies the current location of the unmanned vehicle 200 from the position and orientation information 210a, calculates a predetermined range (for example, a radius of 30 m) centered on the current location of the unmanned vehicle 200, and the calculated range 3D map data 391a indicating information (three-dimensional coordinates, shape, size, color, etc.) of features (roads, buildings, etc.) for which coordinate values are set is acquired from the three-dimensional map database 391.
The map data acquisition unit 312 stores the 3D map data 391 a acquired from the 3D map database 391 in the device storage unit 390.
After S220, the process proceeds to S230.

<S230: Surrounding Image Generation Processing>
The 3D peripheral image generation unit 313 inputs the position / orientation information 210a received in S210 and the 3D map data 391a acquired in S220 from the device storage unit 390, and the input position / orientation information 210a and 3D map data are input. Based on 391a, a three-dimensional peripheral image 313a of the unmanned vehicle 200 is generated.
For example, the three-dimensional peripheral image generation unit 313 generates a predetermined gaze direction based on information such as the three-dimensional coordinates, shape, size, and color of each feature (road, building, etc.) indicated in the three-dimensional map data 391a. A three-dimensional peripheral image 313a showing a three-dimensional picture (figures) of the viewed various features is generated. Further, the three-dimensional peripheral image generation unit 313 identifies the posture angle of the unmanned vehicle 200 from the position / posture information 210a, and a three-dimensional picture representing the unmanned vehicle 200 facing the specified posture angle in the three-dimensional peripheral image 313a. include.

FIG. 8 is a diagram showing a three-dimensional peripheral image 313a in the first embodiment.
In the peripheral image generation process (S230), for example, the three-dimensional peripheral image generation unit 313 generates a three-dimensional peripheral image 313a representing the periphery of the unmanned vehicle 200 as viewed from above the unmanned vehicle 200 as shown in FIG. Generate.

In S230 of FIG. 7, the three-dimensional peripheral image generation unit 313 stores the generated three-dimensional peripheral image 313a in the device storage unit 390.
After S230, the process proceeds to S240.

<S240: Surrounding Image Display Processing>
The three-dimensional peripheral image display unit 314 inputs the three-dimensional peripheral image 313a generated in S230 from the device storage unit 390, and displays the input three-dimensional peripheral image 313a on the display device 901.
For example, the three-dimensional peripheral image display unit 314 displays a three-dimensional peripheral image 313a on the display device 901 as shown in FIG.
After S240, the vehicle periphery image display step (S200) ends.

The vehicle periphery image display step (S200) is executed every time the position and orientation information 210a is transmitted from the vehicle control device 209 of the unmanned vehicle 200.
As a result, the display device 901 of the remote control device 300 displays the three-dimensional peripheral image 313a updated as the unmanned vehicle 200 moves as a moving image.

  A remote operator who operates the remote control device 300 to remotely control the unmanned vehicle 200 determines the speed and traveling direction of the unmanned vehicle 200 by looking at the three-dimensional peripheral image 313a displayed on the display device 901. The direction of travel is input to the remote control device 300 as control information 321a by operating the keyboard 902 or the mouse 903.

FIG. 9 is a flowchart of the steering command transmission step (S300) in the first embodiment.
Details of the steering command transmission step (S300) in the first embodiment will be described below based on FIG.
Each “˜ unit” of the remote control device 300 executes processing described below using a CPU.

<S310: Maneuvering information input process>
The maneuver information input unit 321 inputs and inputs maneuver information 321a input by a remote pilot who has viewed the three-dimensional peripheral image 313a displayed on the display device 901 by operating an input device such as a keyboard 902 or a mouse 903. The operation information 321a is stored in the device storage unit 390.
After S310, the process proceeds to S320.

<S320: Maneuvering command generation process>
The movement position generation unit 322 inputs the operation information 321a input in S310 from the apparatus storage unit 390, generates the movement destination position information 322a based on the input operation information 321a, and uses the generated movement destination position information 322a as the apparatus. Store in the storage unit 390.
For example, the destination position information 322a is calculated by calculating the three-dimensional coordinates of the point where the unmanned vehicle 200 should be located after a predetermined time (for example, after one second) based on the speed and the traveling direction indicated in the operation information 321a. The destination position information 322a is generated including the three-dimensional coordinates.
After S320, the process proceeds to S330.

<S330: Steering command transmission process>
The device transmission unit 323 inputs the destination position information 322a generated in S320 from the device storage unit 390, and transmits the input destination position information 322a from the transmission antenna 302 to the vehicle control device 209 of the unmanned vehicle 200 by radio waves. To do.
After S330, the operation command transmission step (S300) ends.

  The control command transmission step (S300) is executed every time the control information 321a is input to the remote control device 300 by the remote operator.

FIG. 10 is a flowchart of the autopilot process (S400) in the first embodiment.
Details of the autopilot process (S400) in the first embodiment will be described below with reference to FIG.
Each “˜ part” of the vehicle control device 209 of the unmanned vehicle 200 executes processing described below using a CPU.

<S410: Steering command reception processing>
The vehicle receiving unit 212 receives the destination position information 322a transmitted by the remote control device 300 using the antenna 202, and stores the received destination position information 322a in the vehicle storage unit 290.
After S410, the process proceeds to S420.

<S420: Vehicle control processing>
The vehicle control unit 213 inputs the destination location information 322a received in S410, and controls the unmanned vehicle 200 based on the input destination location information 322a.
For example, the vehicle control unit 213 controls the driving device of the unmanned vehicle 200 to change the acceleration / deceleration and the traveling direction so that the unmanned vehicle 200 is located at a point indicated by the destination position information 322a after a predetermined time. The unmanned vehicle 200 is caused to travel.
After S410, the autopilot process (S400) ends.

The automatic pilot process (S400) is executed every time the movement destination position information 322a is transmitted from the remote control device 300.
Thereby, the unmanned vehicle 200 travels automatically instead of manually.

The vehicle control program causes the vehicle control device 209 (an example of a computer) to execute a vehicle information transmission step (S100) and an automatic control step (S400).
The remote control program causes the remote control device 300 (an example of a computer) to execute a vehicle periphery image display step (S200) and a control command transmission step (S300).
The remote location image display program causes the remote control device 300 to execute the vehicle periphery image display step (S200).

In the first embodiment, the following remote control system 100 has been described.
The remote control device 300 is equipped with a three-dimensional map database 391 in which three-dimensional information (three-dimensional map data 391a) around the unmanned vehicle acquired in advance is installed, and is adjusted to the self-position and posture of the unmanned vehicle 200 based on the information. The peripheral information of the unmanned vehicle 200 is displayed objectively and easily.

  In the remote control device 300, the map data acquisition unit 312 acquires peripheral information (three-dimensional map data 391a) of the unmanned vehicle 200 from the three-dimensional map database 391. The three-dimensional peripheral image generation unit 313 and the three-dimensional peripheral image display unit 314 display the feature as viewed from the arbitrary viewpoint direction three-dimensionally based on the peripheral information, and the unmanned vehicle 200 is displayed on the display. Process to superimpose. The remote operator can visually confirm the peripheral information of the unmanned vehicle 200 by visually confirming the result of this processing with the display device 901, and can clearly give the direction of movement to the unmanned vehicle 200.

FIG. 11 is an image diagram of the camera image 199 when the camera is taken from the unmanned vehicle 200.
When the camera image 199 photographed by the camera installed in the unmanned vehicle 200 is displayed on the display device 901 of the remote control device 300, as shown in FIG. However, since the camera image is not a bird's-eye view image including the unmanned vehicle 200, the situation of the front intersection and the situation of the left and right of the unmanned vehicle 200 cannot be grasped. For this reason, it is not easy for the remote operator to determine the operation information 321a by looking at the camera image.
On the other hand, as described in the first embodiment, based on the position / orientation information 210a of the unmanned vehicle 200 and the three-dimensional map database 291, the 3D peripheral image 313a of the unmanned vehicle 200 is seen from the upper rear side of the unmanned vehicle 200 in a bird's eye view. When displayed from the viewpoint (see FIG. 8), the remote operator can easily determine the operation information 321a as compared with the case of viewing the camera image because it is easy to check the front intersection and the left and right situation of the unmanned vehicle 200. be able to.

In addition, since the camera image has a large amount of data, it is necessary to provide a large-capacity communication band when the camera image is transmitted from the unmanned vehicle 200 to the remote control device 300.
On the other hand, in the first embodiment, since the data transmitted from the unmanned vehicle 200 to the remote control device 300 is the position and orientation information 210a with a small data amount, a large-capacity communication band is unnecessary, and the communication band that can be secured is limited. Can also be used if

The remote control of the vehicle using the remote control system 100 allows the passenger to arrive at the destination without driving the vehicle. For example, the remote control system 100 can be used in place of a driving service such as a taxi or substitute driving.
Moreover, the remote control system 100 can drive an unmanned vehicle by remote control. For example, the remote control system 100 is used for restoration activities such as driving a vehicle equipped with various sensors in a disaster area / risk area to acquire various information (images, temperature, toxic substance concentration, etc.) in the area. be able to.

  The unmanned vehicle 200 described in the embodiment only needs to be a movable body that can be driven and controlled, and may be in any form such as a four-wheeled vehicle, a two-wheeled vehicle, a large vehicle, and a small vehicle.

  The remote control device 300 may not include the 3D map database 391, and the map data acquisition unit 312 may acquire 3D map data 391a from a 3D map database 391 provided outside via a network.

Embodiment 2. FIG.
A mode of displaying the three-dimensional peripheral image 313a viewed from the line-of-sight direction designated by the remote pilot will be described.
Hereinafter, items different from the first embodiment will be mainly described, and items omitted will be the same as those of the first embodiment.

FIG. 12 is a functional configuration diagram of the remote control device 300 according to the second embodiment.
The functional configuration of the remote control device 300 in the second embodiment will be described below with reference to FIG.

The remote control device 300 according to the second embodiment includes a line-of-sight direction input unit 315 in addition to the “˜units” described in the first embodiment (see FIG. 3).
Hereinafter, the line-of-sight direction input unit 315 will be described, and description of other components will be omitted.

  The line-of-sight direction input unit 315 inputs the line-of-sight direction 315a designated by the remote pilot as the line-of-sight direction of the map represented by the three-dimensional peripheral image 313a from the input device.

  The functional configuration of the vehicle control device 209 in the second embodiment is the same as that in the first embodiment (see FIG. 2).

FIG. 13 is a flowchart of the vehicle periphery image display step (S200) in the second embodiment.
Details of the vehicle periphery image display step (S200) in the second embodiment will be described below with reference to FIG.

In the vehicle periphery image display step (S200) in the second embodiment, the line-of-sight direction input process (S250) is added to the process described in the first embodiment (see FIG. 7), and the peripheral image generation process (S230) is partially changed. This processing is performed (S231).
Hereinafter, the line-of-sight direction input process (S250) and the peripheral image generation process (S231) will be described, and description of other processes will be omitted.

<S250: Line-of-Sight Direction Input Processing>
The line-of-sight direction input unit 315 inputs the line-of-sight direction 315a specified by the remote operator by operating an input device such as the keyboard 902 or the mouse 903, and stores the input line-of-sight direction 315a in the device storage unit 390.
For example, the remote operator can easily remotely control the unmanned vehicle 200 such as a direction seen from the driver's seat of the unmanned vehicle 200, a direction looking down from behind the unmanned vehicle 200, and a direction seen from directly above the unmanned vehicle 200. The direction 315a is determined and input to the remote control device 300.
After S250, the process proceeds to S231.

<S231: Surrounding Image Generation Processing>
The three-dimensional peripheral image generation unit 313 inputs the position / orientation information 210a received in S210, the three-dimensional map data 391a acquired in S220 and the line-of-sight direction 315a input in S250 from the device storage unit 390, and the input position A three-dimensional peripheral image 313a of the unmanned vehicle 200 is generated based on the posture information 210a, the three-dimensional map data 391a, and the three-dimensional peripheral image 313a.
For example, when the line-of-sight direction 315a indicates “the direction seen from the driver's seat of the unmanned vehicle 200”, the three-dimensional peripheral image generation unit 313 generates an image as illustrated in FIG. 11 as the three-dimensional peripheral image 313a, and the line-of-sight direction When 315a indicates “the direction seen from the rear sky of the unmanned vehicle 200”, an image as illustrated in FIG. 8 is generated as the three-dimensional peripheral image 313a.
After S231, the process proceeds to S240, and the three-dimensional peripheral image 313a is displayed on the display device 901 in S240.

  The vehicle information transmission step (S100), the steering command transmission step (S300), and the automatic steering step (S400) are the same as those in the first embodiment.

The remote control system 100 according to the second embodiment can make remote control easier by displaying the three-dimensional peripheral image 313a viewed from the line-of-sight direction designated by the remote operator.
Further, the remote control device 300 may divide and display a plurality of three-dimensional peripheral images 313a viewed from different line-of-sight directions on the display device 901. For example, the remote control device 300 may display four three-dimensional peripheral images 313a viewed from four directions by dividing the screen into four parts vertically and horizontally.

Embodiment 3 FIG.
A mode of displaying the three-dimensional peripheral image 313a including an obstacle located around the vehicle will be described.
Hereinafter, items different from the first embodiment will be mainly described, and items omitted will be the same as those of the first embodiment.

FIG. 14 is a functional configuration diagram of the vehicle control device 209 according to the third embodiment.
A functional configuration of the vehicle control device 209 according to the third embodiment will be described below with reference to FIG.

The vehicle control apparatus 209 according to the third embodiment includes an obstacle sensor 204 and an obstacle detection unit 220 in addition to each “˜unit” described in the first embodiment (see FIG. 2).
Hereinafter, the obstacle sensor 204 and the obstacle detection unit 220 will be described, and descriptions of other configurations will be omitted.

The obstacle sensor 204 is a sensor device used for detecting a feature such as a stereo camera or an LRF (laser range finder).
LRF irradiates a laser toward each azimuth and each height, and observes the laser that has been reflected back to the feature. Then, the LRF specifies the distance from the feature based on the time until the laser returns, and specifies the direction in which the laser is irradiated as the direction in which the feature is located. LRF is also called a laser radar or a laser scanner.
Stereo cameras shoot the same subject at the same time from two different directions. From each of the two images taken at the same time, the pixel range in which the subject is shown is specified based on the color and brightness of each pixel, and the subject is triangulated based on the two shooting directions and the pixel range of the subject. And the distance is specified. Further, the position of the subject is specified based on the distance to the subject and the shooting direction, and the size of the subject is specified based on the distance to the subject and the pixel range.

The obstacle detection unit 220 (an example of the obstacle information generation unit) detects an obstacle located around the vehicle by using the obstacle sensor 204, and detects information on the detected obstacle (hereinafter referred to as “obstacle information 220a”). Is generated using a CPU. The obstacle information 220a indicates the position (three-dimensional coordinates) and size of the obstacle as the obstacle information.
For example, the obstacle detection unit 220 calculates the position and size of the feature based on the distance from the feature specified by the LRF (an example of the obstacle sensor 204) and the direction in which the feature is located. Obstacle information 220a is generated using the position and size of the feature as obstacle information. The obstacle detection unit 220 calculates, as the position of the feature (obstacle), a point that is separated from the LRF by the specified distance in the specified direction, and determines the laser irradiation range that approximates the specified distance and direction. Calculate as the size of the object.
Further, for example, the obstacle detection unit 220 performs image processing on two images simultaneously captured by a stereo camera (an example of the obstacle sensor 204), calculates the position and size of the feature in the image, Obstacle information 220a is generated using the calculated position and size of the feature as obstacle information.

FIG. 15 is a functional configuration diagram of remote control device 300 according to the third embodiment.
A functional configuration of remote control device 300 according to Embodiment 3 will be described below with reference to FIG.

In the remote control device 300 according to the third embodiment, the three-dimensional peripheral image generation unit 313 described in the first embodiment (see FIG. 3) includes a map formation unit 313A and an obstacle formation unit 313B.
Hereinafter, the map forming unit 313A and the obstacle forming unit 313B will be described, and description of other configurations will be omitted.

The map forming unit 313A is a 3D map around the unmanned vehicle 200 based on the 3D map data 391a acquired by the map data acquiring unit 312 like the 3D peripheral image generating unit 313 described in the first embodiment. Is generated.
Based on the obstacle information 220a transmitted from the vehicle control device 209, the obstacle formation unit 313B generates a three-dimensional peripheral image 313a by superimposing an obstacle picture on the three-dimensional map generated by the map formation unit 313A. To do.

FIG. 16 is a flowchart of the vehicle information transmission step (S100) in the third embodiment.
Details of the vehicle information transmission step (S100) in the third embodiment will be described below with reference to FIG.

In the vehicle information transmission process (S100) in the third embodiment, the obstacle information generation process (S130) is added to the process described in the first embodiment (see FIG. 6), and the vehicle information transmission process (S120) is partially changed. This processing is performed (S121).
Hereinafter, the obstacle information generation process (S130) and the vehicle information transmission process (S121) will be described, and description of other processes will be omitted.

<S130: Obstacle Information Generation Processing>
The obstacle detection unit 220 detects an obstacle located around the unmanned vehicle 200 using the obstacle sensor 204, generates information on the detected obstacle (obstacle information 220a), and generates the generated obstacle information 220a. Is stored in the vehicle storage unit 290.
After S130, the process proceeds to S121.

<S121: Vehicle information transmission process>
The vehicle transmission unit 211 inputs the position / orientation information 210a generated in S110 and the obstacle information 220a generated in S130 from the vehicle storage unit 290, and wirelessly transmits the input position / orientation information 210a and obstacle information 220a from the antenna 202. It transmits to the remote control device 300 by radio waves.

FIG. 17 is a flowchart of the vehicle periphery image display step (S200) in the third embodiment.
Details of the vehicle periphery image display step (S200) in the third embodiment will be described below with reference to FIG.

The vehicle periphery image display process (S200) in the third embodiment is a process (S211) in which the vehicle information reception process (S210) described in the first embodiment (see FIG. 7) is partially changed, and a peripheral image generation process ( S230) is changed to peripheral image generation processing A (S232) and peripheral image generation processing B (S233).
Hereinafter, the vehicle information reception process (S211), the peripheral image generation process A (S232), and the peripheral image generation process B (S233) will be described, and description of other processes will be omitted.

<S211: Vehicle information reception process>
The device receiving unit 311 receives the position / orientation information 210a and the obstacle information 220a transmitted by the vehicle control device 209 of the unmanned vehicle 200 using the receiving antenna 301, and receives the received position / orientation information 210a and the obstacle information 220a. Store in the storage unit 390.

<S232: Surrounding Image Generation Processing A>
The map formation unit 313A inputs the position / orientation information 210a received in S211 and the 3D map data 391a acquired in S220 from the device storage unit 390, and inputs the input position in the same manner as in the first embodiment (S230). A three-dimensional map around the vehicle is generated based on the posture information 210a and the three-dimensional map data 391a. The map forming unit 313A stores the generated three-dimensional map in the device storage unit 390.
After S232, the process proceeds to S233.

<S233: Surrounding Image Generation Processing B>
The obstacle formation unit 313B inputs the obstacle information 220a received in S211 and the three-dimensional map generated in S232 from the device storage unit 390, and displays a picture of the obstacle based on the obstacle information 220a. To generate a three-dimensional peripheral image 313a.
For example, the obstacle formation unit 313B creates a rectangle representing the obstacle based on the position and size of the obstacle indicated in the obstacle information 220a, and superimposes the created rectangle on the three-dimensional map to create a three-dimensional periphery. An image 313a is generated.

18 and 19 are diagrams showing a three-dimensional peripheral image 313a in the third embodiment.
In the peripheral image generation process B (S233), the obstacle forming unit 313B superimposes the obstacles (313b1 to 313b3) represented by squares on the three-dimensional map as shown in FIG. 18, for example, as shown in FIG. Is generated.
In FIG. 18, an obstacle 313b1 indicates the presence of the wall surface of the building also included in the 3D map data 391a, and the obstacle 313b2 and the obstacle 313b3 are not included in the 3D map data 391a. Vehicle).

The obstacle formation unit 313B may display a picture of each obstacle on the three-dimensional peripheral image 313a with a different color depending on the distance to each obstacle.
For example, in FIG. 18, the nearest obstacle 313b2 from the unmanned vehicle 200 is represented by “red”, the next nearest obstacle 313b3 is represented by “yellow”, and each obstacle 313b1 located farther away is represented by “blue”.

  As shown in FIG. 19, the obstacle formation unit 313B specifies a direction in which the unmanned vehicle 200 travels and the obstacle 313b1 does not exist, and displays a notation 313c (for example, an arrow) indicating the specified direction as navigation information. You may include in the dimension periphery image 313a.

In S233 of FIG. 17, the obstacle forming unit 313B stores the generated three-dimensional peripheral image 313a in the device storage unit 390.
After S233, the process proceeds to S240. In S240, the three-dimensional peripheral image 313a is displayed on the display device 901 as shown in FIGS.

  The steering command transmission step (S300) and the automatic piloting step (S400) are the same as those in the first embodiment.

  The remote control system 100 according to the third embodiment displays a three-dimensional peripheral image 313a including obstacles located around the vehicle, thereby indicating to the remote operator the presence of a moving body that is not registered in the three-dimensional map database 291. You can be informed and safer to remotely control.

In the third embodiment, the following remote control system 100 has been described.
When the unmanned vehicle 200 is equipped with the obstacle sensor 204 and detects the obstacle information 220a, the vehicle control device 209 sends the position and size to the remote control device 300, and the remote control device 300 provides the peripheral information of the unmanned vehicle 200. Obstacles are superimposed on the content that is displayed in an objectively understandable manner.

  In the remote control device 300, the map data acquisition unit 312 acquires peripheral information about the unmanned vehicle 200 from the three-dimensional map database 391. The three-dimensional peripheral image generation unit 313 and the three-dimensional peripheral image display unit 314 display the feature as viewed from the arbitrary viewpoint direction three-dimensionally based on the peripheral information, and the unmanned vehicle 200 and the unmanned vehicle 200 are displayed. Are displayed on the display device 901 in a superimposed manner. By visually confirming this display, the remote operator can objectively see the peripheral information of the unmanned vehicle 200 and can clearly give the direction of movement to the unmanned vehicle 200.

  As in the second embodiment, the remote control device 300 according to the third embodiment inputs the line-of-sight direction 315a designated by the remote pilot and generates a three-dimensional peripheral image 313a viewed from the designated line-of-sight direction 315a. It may be displayed.

Embodiment 4 FIG.
A mode of displaying a three-dimensional peripheral image 313a in which only a moving object is superposed as a picture among obstacles around the vehicle will be described.

  Hereinafter, items different from the third embodiment will be mainly described, and items that will not be described are the same as those of the third embodiment.

FIG. 20 is a functional configuration diagram of the vehicle control device 209 according to the fourth embodiment.
A functional configuration of the vehicle control device 209 according to the fourth embodiment will be described below with reference to FIG.

The vehicle control apparatus 209 according to the fourth embodiment includes a map data acquisition unit 230 (an example of a map data vehicle acquisition unit), a moving obstacle selection, in addition to each “˜unit” described in the third embodiment (see FIG. 14). A unit 231 (an example of an obstacle vehicle specifying unit) and a three-dimensional map database 291 are provided.
Hereinafter, the map data acquisition unit 230, the moving obstacle selection unit 231 and the 3D map database 291 will be described, and description of other configurations will be omitted.

  The 3D map database 291 is a device that stores and manages the same data (3D map data 291a) as the 3D map data 391a of the 3D map database 391 provided in the remote control device 300 using a storage medium.

  The map data acquisition unit 230 acquires, from the 3D map database 291, 3D map data 291 a including information on features located around the unmanned vehicle 200 based on the position and orientation information 210 a.

  The moving obstacle sorting unit 231 has information on obstacles that does not overlap with the feature information included in the three-dimensional map data 291a acquired by the map data acquisition unit 230 from the obstacle information 220a generated by the obstacle detection unit 220. Is identified as the moving obstacle information 231a.

  The functional configuration of the remote control device 300 in the fourth embodiment is the same as that in the third embodiment (see FIG. 15).

FIG. 21 is a flowchart of the vehicle information transmission step (S100) in the fourth embodiment.
Details of the vehicle information transmission step (S100) in the fourth embodiment will be described below based on FIG.

In the vehicle information transmission process (S100) in the fourth embodiment, a map information acquisition process (S140) and an obstacle identification process (S150) are added to the processes described in the third embodiment (see FIG. 16), and a vehicle information transmission process is performed. This is a process (S122) in which (S121) is partially changed.
Hereinafter, the map information acquisition process (S140), the obstacle identification process (S150), and the vehicle information transmission process (S122) will be described, and description of other processes will be omitted.

<S140: Map information acquisition process>
The map data acquisition unit 230 inputs the position / orientation information 210a generated in S110 from the vehicle storage unit 290, and, similar to the map data acquisition unit 312 of the remote control device 300, based on the input position / orientation information 210a, 3D map data 291a is acquired from the 3D map database 291. The map data acquisition unit 230 stores the acquired three-dimensional map data 291a in the vehicle storage unit 290.
After S140, the process proceeds to S150.

<S150: Obstacle identification processing>
The moving obstacle sorting unit 231 inputs the obstacle information 220a generated in S130 and the 3D map data 291a acquired in S140 from the vehicle storage unit 290, and the obstacle information 220a that does not overlap with the 3D map data 291a. Is identified as the moving obstacle information 231a.
For example, the moving obstacle selection unit 231 compares the three-dimensional coordinates of each obstacle indicated in the obstacle information 220a with the three-dimensional coordinates of each feature indicated in the three-dimensional map data 291a, and is indicated by the three-dimensional coordinates. Obstacle information 220a whose existence range does not overlap with any of the features indicated in the 3D map data 291a is specified as moving obstacle information 231a.
In FIG. 18, the information on the obstacle 313b1 (wall surface of the building) is information that overlaps with the wall surface of the building shown in the three-dimensional map data 291a, so it is not specified as the moving obstacle information 231a, and the obstacle 313b2 (pedestrian) And the information on the obstacle 313b3 (oncoming vehicle) are information that does not overlap with any of the features (roads, sidewalks, buildings) shown in the three-dimensional map data 291a, and thus are specified as the moving obstacle information 231a.
The moving obstacle sorting unit 231 stores the specified moving obstacle information 231a in the vehicle storage unit 290.
After S150, the process proceeds to S122.

<S122: Vehicle information transmission process>
The vehicle transmission unit 211 inputs the position / orientation information 210a generated in S110 and the moving obstacle information 231a specified in S150 from the vehicle storage unit 290, and receives the input position / orientation information 210a and the moving obstacle information 231a as an antenna. 202 transmits to the remote control device 300 by radio waves.

FIG. 22 is a flowchart of the vehicle periphery image display step (S200) in the fourth embodiment.
Details of the vehicle periphery image display step (S200) in the fourth embodiment will be described below with reference to FIG.

The vehicle periphery image display step (S200) in the fourth embodiment is a process in which the vehicle information reception process (S211) and the peripheral image generation process B (S233) described in the third embodiment (see FIG. 17) are partially changed ( S212, S234).
Hereinafter, the vehicle information reception process (S212) and the peripheral image generation process B (S234) will be described, and description of other processes will be omitted.

<S212: Vehicle information reception process>
The device receiving unit 311 receives the position / orientation information 210a and the moving obstacle information 231a transmitted by the vehicle control device 209 of the unmanned vehicle 200 using the receiving antenna 301, and receives the received position / orientation information 210a and the moving obstacle information 231a. Is stored in the device storage unit 390.

<S234: Surrounding Image Generation Processing B>
The obstacle forming unit 313B inputs the moving obstacle information 231a received in S212 and the three-dimensional map generated in S232 from the device storage unit 390, and the moving obstacle information is similar to S233 in the third embodiment. A three-dimensional peripheral image 313a is generated by superimposing a moving obstacle picture on the three-dimensional map based on 231a.
For example, data obtained by removing each obstacle 313b1 from the three-dimensional peripheral image 313a shown in FIG. 18 is generated as the three-dimensional peripheral image 313a.
The obstacle formation unit 313B stores the generated three-dimensional peripheral image 313a in the device storage unit 390.
After S234, the process proceeds to S240, where the three-dimensional peripheral image 313a is displayed on the display device 901 as shown in FIG. 18 (except for each obstacle 313b1).

  The steering command transmission step (S300) and the automatic piloting step (S400) are the same as those in the third embodiment.

  The remote control system 100 according to the fourth embodiment displays the three-dimensional peripheral image 313a in which only the moving object is superimposed on the obstacles around the vehicle, thereby clearly indicating the presence of the moving object to the remote operator. You can be informed and safer to remotely control.

In the fourth embodiment, the following remote control system 100 has been described.
The unmanned vehicle 200 is equipped with a 3D map database 291, and the vehicle control device 209 is a feature (electric pole, sign, building, etc.) in which the obstacle detected by the obstacle sensor 204 is registered in the 3D map database 291. If there is an unregistered object, it is sent to the remote control device 300 as a moving obstacle, and the remote control device 300 superimposes the moving obstacle information on the contents that display the peripheral information of the unmanned vehicle 200 in an objectively understandable manner. To display.
The remote control device 300 does not superimpose features and obstacles (buildings etc.) already registered in the 3D map database 391 and does not display them on the display device 901. Since only unregistered obstacles are displayed in a superimposed manner, the remote operator visually confirms this display, so that the peripheral information of the unmanned vehicle 200 can be viewed objectively and the direction of movement to the unmanned vehicle 200 is clearly indicated. Can be done.

Embodiment 5 FIG.
A mode in which the remote control device 300 displays a three-dimensional peripheral image 313a in which a moving obstacle is specified and a picture of the moving obstacle is superimposed will be described.

  Hereinafter, items different from the third embodiment will be mainly described, and items that will not be described are the same as those of the third embodiment.

FIG. 23 is a functional configuration diagram of remote control device 300 according to the fifth embodiment.
A functional configuration of the remote control device 300 according to the fifth embodiment will be described below with reference to FIG.

The remote control device 300 according to the fifth embodiment includes a moving obstacle sorting unit 316 (an example of an obstacle specifying unit) in addition to each “˜unit” described in the third embodiment (see FIG. 15).
Hereinafter, the moving obstacle sorting unit 316 will be described, and description of other components will be omitted.

  The moving obstacle sorting unit 316 displays obstacle information that does not overlap with the feature information included in the three-dimensional map data 391a acquired by the map data acquisition unit 312 from the obstacle information 220a received by the device receiving unit 311. The moving obstacle information 316a is specified.

  The functional configuration of the vehicle control device 209 in the fifth embodiment is the same as that in the third embodiment (see FIG. 20).

FIG. 24 is a flowchart of the vehicle periphery image display step (S200) in the fifth embodiment.
Details of the vehicle periphery image display step (S200) in the fifth embodiment will be described below based on FIG.

In the vehicle periphery image display step (S200) in the fifth embodiment, the obstacle specifying process (S250) is added to the process described in the third embodiment (see FIG. 17), and the peripheral image generation process B (S233) is partially performed. This is the changed processing (S235).
Hereinafter, the obstacle identification process (S250) and the peripheral image generation process B (S235) will be described, and description of other processes will be omitted.

<S250: Obstacle identification processing>
The moving obstacle sorting unit 316 inputs the obstacle information 220a received in S211 and the 3D map data 391a acquired in S220 from the device storage unit 390, and moves the obstacles according to the fourth embodiment (S150). Similarly to the unit 231, obstacle information 220a that does not overlap with the three-dimensional map data 391a is specified as the moving obstacle information 316a.
The moving obstacle sorting unit 316 stores the specified moving obstacle information 316a in the device storage unit 390.

<S235: Surrounding Image Generation Processing B>
The obstacle formation unit 313B inputs the moving obstacle information 316a specified in S250 and the three-dimensional map generated in S232 from the device storage unit 390, and the moving obstacle information is the same as S233 in the third embodiment. Based on 316a, a picture of a moving obstacle is superimposed on a three-dimensional map to generate a three-dimensional peripheral image 313a.
For example, data obtained by removing each obstacle 313b1 from the three-dimensional peripheral image 313a shown in FIG. 18 is generated as the three-dimensional peripheral image 313a.
The obstacle formation unit 313B stores the generated three-dimensional peripheral image 313a in the device storage unit 390.
After S235, the process proceeds to S240. In S240, the three-dimensional peripheral image 313a is displayed on the display device 901 as shown in FIG. 18 (excluding each obstacle 313b1).

  The steering command transmission step (S300) and the automatic piloting step (S400) are the same as those in the third embodiment.

  The remote control system 100 according to the fifth embodiment displays the three-dimensional peripheral image 313a in which only the moving object is superimposed on the obstacles around the vehicle, thereby clearly indicating the presence of the moving object to the remote operator. You can be informed and safer to remotely control.

1 is a schematic diagram of a remote control system 100 according to Embodiment 1. FIG. 2 is a functional configuration diagram of a vehicle control device 209 according to Embodiment 1. FIG. 2 is a functional configuration diagram of a remote control device 300 according to Embodiment 1. FIG. FIG. 2 is a diagram illustrating an example of hardware resources of a remote control device 300 and a vehicle control device 209 according to Embodiment 1. 3 is a flowchart showing a remote control method of remote control system 100 in the first embodiment. The flowchart of the vehicle information transmission process (S100) in Embodiment 1. FIG. The flowchart of the vehicle periphery image display process (S200) in Embodiment 1. FIG. FIG. 4 shows a three-dimensional peripheral image 313a in Embodiment 1. The flowchart of the steering command transmission process (S300) in Embodiment 1. FIG. 5 is a flowchart of an automatic pilot process (S400) in the first embodiment. The image figure of the camera image 199 at the time of camera photography from the unmanned vehicle 200. FIG. The function block diagram of the remote control apparatus 300 in Embodiment 2. FIG. The flowchart of the vehicle periphery image display process (S200) in Embodiment 2. FIG. FIG. 10 is a functional configuration diagram of a vehicle control device 209 according to a third embodiment. FIG. 10 is a functional configuration diagram of a remote control device 300 according to a third embodiment. The vehicle information transmission process (S100) in Embodiment 3 is a flowchart. The flowchart of the vehicle periphery image display process (S200) in Embodiment 3. FIG. 10 shows a three-dimensional peripheral image 313a according to Embodiment 3. FIG. 10 shows a three-dimensional peripheral image 313a according to Embodiment 3. FIG. 10 is a functional configuration diagram of a vehicle control device 209 according to a fourth embodiment. The flowchart of the vehicle information transmission process (S100) in Embodiment 4. The flowchart of the vehicle periphery image display process (S200) in Embodiment 4. FIG. 10 is a functional configuration diagram of a remote control device 300 according to a fifth embodiment. The flowchart of the vehicle periphery image display process (S200) in Embodiment 5.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 100 Remote control system, 199 Camera image, 200 Unmanned vehicle, 201 GPS receiving system, 202 Antenna, 203 Gyro, 204 Obstacle sensor, 209 Vehicle control apparatus, 210 Position and orientation determination unit, 210a Position and orientation information, 211 Vehicle transmission unit, 212 vehicle receiving unit, 213 vehicle control unit, 220 obstacle detection unit, 220a obstacle information, 230 map data acquisition unit, 231 moving obstacle selection unit, 231a moving obstacle information, 290 vehicle storage unit, 291 3D map database , 291a 3D map data, 300 remote control device, 301 reception antenna, 302 transmission antenna, 310 remote location image display device, 311 device reception unit, 312 map data acquisition unit, 313 3D peripheral image generation unit, 313a 3D periphery Image, 313b 1,313b2, 313b3 Obstacle, 313c Notation, 313A Map forming unit, 313B Obstacle forming unit, 314 Three-dimensional peripheral image display unit, 315 Gaze direction input unit, 315a Gaze direction, 316 Moving obstacle sorting unit, 316a Moving obstacle Object information, 320 Vehicle control device, 321 Control information input unit, 321a Control information, 322 Movement position generation unit, 322a Destination position information, 323 Device transmission unit, 390 Device storage unit, 391 3D map database, 391a 3D map Data, 901 display device, 902 keyboard, 903 mouse, 911 CPU, 912 bus, 913 ROM, 914 RAM, 915 communication board, 920 magnetic disk device, 921 OS, 923 program group, 924 file group.

Claims (14)

A vehicle information receiving unit that receives position information of the vehicle transmitted from a vehicle traveling in a remote place using a communication device;
Map data acquisition for acquiring three-dimensional map data around the vehicle from a map database device storing three-dimensional map data of the travel region of the vehicle based on the vehicle position information received by the vehicle information receiving unit And
A peripheral image generation unit that generates a three-dimensional peripheral image representing a map around the vehicle based on the three-dimensional map data acquired by the map data acquisition unit using a CPU (Central Processing Unit);
A peripheral image display unit that displays a three-dimensional peripheral image generated by the peripheral image generation unit on a monitor device;
The vehicle information receiving unit receives, from the vehicle, information on obstacles located around the vehicle detected by a sensor installed in the vehicle, in addition to the position information of the vehicle.
The peripheral image generation unit, on the basis of the information received obstacle by the vehicle information receiving unit, the obstacle and notation indicating the obstacle to the map is superposed a notation indicating the direction in which no 3 remote image display device and generating a dimensional images as the three-dimensional peripheral image. The vehicle information receiving unit receives information including a distance between the vehicle and the obstacle as information on the obstacle,
The remote image display device according to claim 1, wherein the peripheral image generation unit colors the notation indicating the obstacle according to the distance included in the obstacle information. A vehicle information receiving unit that receives position information of the vehicle transmitted from a vehicle traveling in a remote place using a communication device;
Map data acquisition for acquiring three-dimensional map data around the vehicle from a map database device storing three-dimensional map data of the travel region of the vehicle based on the vehicle position information received by the vehicle information receiving unit And
A peripheral image generation unit that generates a three-dimensional peripheral image representing a map around the vehicle based on the three-dimensional map data acquired by the map data acquisition unit using a CPU (Central Processing Unit);
A peripheral image display unit that displays a three-dimensional peripheral image generated by the peripheral image generation unit on a monitor device;
The vehicle information receiving unit receives, from the vehicle, information including the distance between the vehicle and an obstacle located around the vehicle as the obstacle information, in addition to the vehicle position information.
The surrounding image generation unit superimposes a notation indicating the obstacle on the map based on the obstacle information received by the vehicle information reception unit, and responds to the distance included in the obstacle information. A remote location image display device that generates an image colored in a notation indicating the obstacle as the three-dimensional peripheral image. The map data acquisition unit acquires three-dimensional map data including information on features located around the vehicle from the map database device;
The remote location image display device displays obstacle information that does not overlap with feature information included in the three-dimensional map data acquired by the map data acquisition unit from the obstacle information received by the vehicle information reception unit. Has an obstacle identification part to identify,
The peripheral image generating unit generates an image in which the obstacle is superimposed on the map as the three-dimensional peripheral image based on information on the obstacle specified by the obstacle specifying unit. The remote place image display apparatus in any one of Claims 1-3. The remote area according to any one of claims 1 to 4, wherein the peripheral image generation unit generates a three-dimensional overhead map around the vehicle including the vehicle as the three- dimensional peripheral image. Image display device. The remote image display device includes a gaze direction input unit that inputs a gaze direction designated by the user as a gaze direction of the map represented by the three-dimensional peripheral image from the input device,
The said surrounding image generation part produces | generates the image showing the said map seen from the gaze direction input by the said gaze direction input part as said three-dimensional surrounding image. The remote place image display apparatus of description. The remote image display device according to any one of claims 1 to 6,
A remote control device comprising: a vehicle control device that transmits a control command for controlling the vehicle to the vehicle to control the vehicle. The vehicle control device is
A maneuvering information input unit for inputting maneuvering information designated by the user as information for maneuvering the vehicle from an input device;
A maneuver command generator for generating the maneuver command using a CPU based on the maneuver information input by the maneuver information input unit;
The remote control device according to claim 7, further comprising a control command transmission unit that controls the vehicle by transmitting the control command generated by the control command generation unit to the vehicle using a communication device. The remote control device according to claim 7 or claim 8,
A vehicle control device mounted on a vehicle to control the vehicle,
The vehicle control device includes:
A position information generating unit that measures a current position of the vehicle using a positioning device, and generates position information indicating the current position of the vehicle using a CPU (Central Processing Unit);
An obstacle information generating unit that detects an obstacle located around the vehicle using a sensor and generates information of the detected obstacle using a CPU;
A vehicle information transmitter for transmitting the position information generated by the position information generator and the information of the obstacle generated by the obstacle information generator to the remote control device using a communication device;
A steering command receiver for receiving the steering command transmitted by the remote control device using a communication device;
A remote control system comprising: a vehicle control unit that controls the vehicle using a CPU based on a control command received by the control command receiving unit. A vehicle control device mounted on the vehicle measures the current position of the vehicle, generates position information of the vehicle indicating the measured current position, detects an obstacle located around the vehicle, and detects the detected obstacle Generate information on the object, send the vehicle position information and the obstacle information,
A remote control device installed in a remote location receives the position information of the vehicle and the information of the obstacle,
The remote control device acquires three-dimensional map data around the vehicle based on the position information of the vehicle, and creates a map around the vehicle based on the acquired three-dimensional map data and the obstacle information. wherein generating an image obtained by superimposing a notation indicating the direction in which the obstacle is not present with the notation indicating the obstacle as a three-dimensional peripheral image of the vehicle, three-dimensional peripheral image of the generated the vehicle is displayed, and
The remote control device generates a control command to control the vehicle, and transmits the generated control command;
The remote control method, wherein the vehicle control device receives the control command and controls the vehicle based on the received control command. Vehicle information reception processing for receiving, using a communication device, position information of the vehicle transmitted from a vehicle traveling in a remote place and information on obstacles located around the vehicle;
Map data acquisition for acquiring three-dimensional map data around the vehicle from a map database device storing three-dimensional map data of the travel region of the vehicle based on the position information of the vehicle received by the vehicle information reception process Processing,
Based on the three-dimensional map data acquired by the map data acquisition process and the information on the obstacle received by the vehicle information reception process, the notation indicating the obstacle on the map around the vehicle and the obstacle Peripheral image generation processing for generating an image superimposed with a notation indicating a direction in which there is no image using a CPU (Central Processing Unit) as a three-dimensional peripheral image;
A peripheral image display process for displaying a three-dimensional peripheral image generated by the peripheral image generation process on a monitor device;
Maneuvering information input processing for inputting maneuvering information designated by the user as information for maneuvering the vehicle from an input device;
A steering command generation process for generating a steering command for steering the vehicle based on the steering information input by the steering information input process using a CPU;
A remote control program for causing a computer to execute a control command transmission process for controlling the vehicle by transmitting the control command generated by the control command generation process to the vehicle using a communication device. A vehicle information reception unit performs vehicle information reception processing for receiving the position information of the vehicle transmitted from a vehicle traveling in a remote place and the information of an obstacle located around the vehicle using a communication device,
Map data acquisition unit, the vehicle information based on the position information of the vehicle received by the receiving unit, the map database apparatus 3D map data around the vehicle that stores the three-dimensional map data of the running region of the vehicle Perform map data acquisition processing acquired from
Based on the 3D map data acquired by the map data acquisition unit and the information on the obstacle received by the vehicle information reception unit, a peripheral image generation unit adds the obstacle to the map around the vehicle. A peripheral image generation process is performed to generate a three-dimensional peripheral image using a CPU (Central Processing Unit) as an image superimposed with a description indicating the direction in which the obstacle does not exist,
A remote image display method for a remote image display device, wherein the peripheral image display unit performs a peripheral image display process for displaying the three-dimensional peripheral image generated by the peripheral image generator on a monitor device. A vehicle information receiving unit receives position information of the vehicle transmitted from a vehicle traveling in a remote place using a communication device, and includes information including a distance between the vehicle and an obstacle located around the vehicle. Vehicle information receiving process for receiving as obstacle information,
Map data acquisition unit, the vehicle information based on the position information of the vehicle received by the receiving unit, the map database apparatus 3D map data around the vehicle that stores the three-dimensional map data of the running region of the vehicle Perform map data acquisition processing acquired from
The peripheral image generation unit superimposes a notation indicating the obstacle on a map around the vehicle based on the three-dimensional map data acquired by the map data acquisition unit, and the distance included in the obstacle information The peripheral image generation process for generating an image colored according to the notation indicating the obstacle as a three-dimensional peripheral image using a CPU (Central Processing Unit),
A remote image display method for a remote image display device, wherein the peripheral image display unit performs a peripheral image display process for displaying the three-dimensional peripheral image generated by the peripheral image generator on a monitor device.   A remote image display program for causing a computer to execute the remote image display method according to claim 12 or 13.

Images