JP2020027606A - Information processing method, information processing device, and information processing system - Google Patents

Abstract

To provide an information processing method capable of urgently stopping a vehicle safely.SOLUTION: An information processing method of an information processing device 140 for performing a remote control operation of a vehicle 200 via a communication network, includes: a step (S10) of acquiring vehicle information of the vehicle 200; a step (S20) of acquiring a delay time of the communication network; a step (S30) of calculating a stop position when the vehicle 200 is urgently stopped on the basis of the vehicle information and the delay time; and a step (S40) of outputting stop position information indicating the calculated stop position.SELECTED DRAWING: Figure 4

Description

  The present disclosure relates to an information processing method, an information processing device, and an information processing system of an information processing device that remotely controls a vehicle.

  Vehicle control in which an operator at a remote location indirectly drives and steers a vehicle that the driver does not board or does not operate using a wireless communication such as a wireless LAN (Local Area Network) or a mobile phone line. There is a system.

  In the case of such a vehicle control system, a sensor mounted on the vehicle (for example, a millimeter-wave radar, a laser radar, and a camera) detects a sensing result obtained by observing the surroundings of the vehicle from the vehicle via a communication network. The operator remotely steers the vehicle by transmitting the control information on the travel of the vehicle to the vehicle from the operator.

  Patent Document 1 discloses a remote operation system (vehicle control system) for a semi-autonomous unmanned vehicle (operated vehicle). The remote control system disclosed in Patent Document 1 obtains distance measurement data in a travel area of the own vehicle, performs autonomous traveling based on the distance measurement data, and is transmitted from a remote control device by an operator at a remote location. A semi-autonomous vehicle that performs remote control based on the control information (control information). In this remote control system, a semi-autonomous vehicle can be remotely controlled.

Japanese Patent No. 5366703

  By the way, the operator may stop the vehicle in an emergency. In this case, the operator needs to safely stop the vehicle in an emergency. According to the technology described in Patent Literature 1, an image captured by an imaging unit included in a vehicle is displayed on a display unit, and an operator remotely operates the vehicle by checking the image displayed on the display unit. The method may not be able to safely stop the vehicle in an emergency.

  Therefore, an object of the present disclosure is to provide an information processing method, an information processing apparatus, and an information processing system that can safely stop a vehicle in an emergency.

  An information processing method according to an aspect of the present disclosure is an information processing method of an information processing device for remotely operating a vehicle via a communication network, the method including: acquiring vehicle information of the vehicle; Obtaining a delay time; calculating a stop position when the vehicle is urgently stopped based on the vehicle information and the delay time; and outputting stop position information indicating the calculated stop position. Steps.

  An information processing device according to an aspect of the present disclosure is an information processing device for remotely operating a vehicle via a communication network, and includes a vehicle information acquisition unit that acquires vehicle information of the vehicle, and a delay of the communication network. A delay time acquisition unit that acquires time, a control unit that calculates a stop position when the vehicle is emergency stopped based on the vehicle information and the delay time, and stop position information that indicates the calculated stop position And an output unit that outputs

  An information processing system according to an aspect of the present disclosure includes the information processing device described above, and a display device that displays a video based on the stop position information output by the information processing device.

  According to the information processing method, the information processing device, and the information processing system according to an embodiment of the present disclosure, the vehicle can be safely and emergency stopped.

FIG. 1 is a diagram showing a schematic configuration of the vehicle control system according to the first embodiment. FIG. 2 is a diagram illustrating a functional configuration of the remote operation system according to the first embodiment. FIG. 3 is a diagram illustrating a functional configuration of the operated vehicle according to the first embodiment. FIG. 4 is a flowchart showing an operation of the remote operation system according to the first embodiment. FIG. 5A is a sequence diagram illustrating an example of an operation of acquiring a delay time of the NW according to Embodiment 1. FIG. 5B is a sequence diagram showing another example of the operation of acquiring the delay time of the NW according to Embodiment 1. FIG. 6 is a diagram illustrating a display example of a stop position of the operated vehicle according to the first embodiment. FIG. 7 is a diagram illustrating a functional configuration of the remote operation system according to the second embodiment. FIG. 8 is a flowchart showing the operation of the remote operation system according to the second embodiment. FIG. 9 is a diagram illustrating a display example of a stop position of the operated vehicle according to the second embodiment. FIG. 10 is a flowchart showing the operation of the remote operation system according to Embodiment 3. FIG. 11 is a sequence diagram illustrating an operation of calculating a packet loss rate between the remote operation device and the operated vehicle according to the third embodiment. FIG. 12A is a diagram showing a display example of a stop position of the operated vehicle according to Embodiment 3. FIG. 12B is a diagram showing another display example of the stop position of the operated vehicle according to Embodiment 3. FIG. 12C is a diagram showing still another display example of the stop position of the operated vehicle according to Embodiment 3. FIG. 13 is a sequence diagram showing an operation of transmitting and receiving a stop instruction between the remote operation device according to Embodiment 3 and the operated vehicle. FIG. 14 is a flowchart showing the operation of the remote operation system according to Embodiment 4. FIG. 15A is a diagram showing a display example of the stop position of the operated vehicle according to Embodiment 4. FIG. 15B is a diagram showing another display example of the stop position of the operated vehicle according to Embodiment 4. FIG. 16 is a flowchart showing the operation of the remote operation system according to Embodiment 5. FIG. 17A is a diagram illustrating a stop position in a case where the stop distance is short in the operated vehicle according to Embodiment 5. FIG. 17B is a diagram showing a stop position in a case where the stop distance is long in the operated vehicle according to Embodiment 5. FIG. 17C is a diagram showing a display example of the stop range of the operated vehicle according to Embodiment 5.

(Knowledge underlying the present disclosure)
As described above, in the technology described in Patent Literature 1, an image captured by an imaging unit included in a vehicle is displayed on a display unit, and an operator remotely operates the vehicle by checking the image displayed on the display unit. . That is, in Patent Literature 1, only the image captured by the imaging unit is displayed on the display unit. In this case, even if the operator issues an instruction to stop the vehicle in an emergency, the position at which the vehicle stops is unknown. In other words, when an emergency stop is performed, the operator cannot explicitly know at which point the vehicle stops. Therefore, in the method of Patent Document 1, the vehicle may not be able to safely perform an emergency stop.

  Therefore, an information processing method according to an aspect of the present disclosure is an information processing method of an information processing device for remotely operating a vehicle via a communication network, wherein the step of obtaining vehicle information of the vehicle includes the steps of: Obtaining a network delay time, calculating a stop position when the vehicle is emergency stopped based on the vehicle information and the delay time, and stopping position information indicating the calculated stop position. Outputting.

  Thus, the operator can confirm the position where the vehicle stops when the vehicle is emergency stopped by checking the image (the image displaying the stop position) in which the stop position information output from the information processing device is displayed. It can be grasped explicitly. That is, the operator can perform an emergency stop operation in consideration of the stop position when the vehicle is to be emergency stopped. Therefore, according to the information processing method according to an aspect of the present disclosure, the vehicle can be safely emergency stopped.

  Further, for example, a plurality of deceleration patterns for emergency stopping of the vehicle are set in the vehicle, and the stop position is calculated in each of the plurality of deceleration patterns.

  Accordingly, the operator can emergency stop the vehicle more safely from the displayed plurality of stop positions by checking the video in which the plurality of stop positions calculated in each of the plurality of deceleration patterns are displayed. A deceleration pattern can be selected. Therefore, according to the information processing method according to an aspect of the present disclosure, the vehicle can be emergency stopped more safely.

  Further, for example, the method further includes a step of calculating the number of transmissions N (N ≧ 2) for repeatedly transmitting control information indicating an emergency stop based on a packet loss rate in the communication network, It is calculated using the number of times.

  By calculating the stop position using the number of transmissions based on the packet loss rate, even when a packet loss occurs in the communication between the vehicle and the information processing device, the packet loss is considered. Thus, the stop position can be calculated. Therefore, according to the information processing method according to an aspect of the present disclosure, the vehicle can be safely emergency-stopped even when a packet loss occurs.

  In addition, for example, the stop position includes a first stop position in a case where the vehicle is emergency stopped by the control information transmitted Nth time.

  Thereby, the operator can grasp the farthest first stop position among the stop positions of the vehicle in consideration of the packet loss rate. That is, the operator can determine whether the vehicle can safely stop by checking the first stop position. Therefore, according to the information processing method according to an aspect of the present disclosure, the vehicle can be safely emergency-stopped even when a packet loss occurs.

  In addition, for example, the stop position further includes a second stop position when the vehicle is urgently stopped by the control information transmitted for the first time, and the stop position information includes the first stop position and the second stop position. The first stop range based on the stop position is included.

  Thereby, the operator can grasp the stop range where the vehicle may stop due to the occurrence of the packet loss. That is, the operator can determine whether the vehicle can safely stop by checking the first stop range. Therefore, according to the information processing method according to an aspect of the present disclosure, even when a packet loss occurs, the vehicle can be more safely emergency-stopped.

  Further, for example, furthermore, obtaining obstacle information including a position of an obstacle around the vehicle, and the vehicle and the obstacle collide based on the stop position and the position of the obstacle. Determining whether there is a risk of performing the vehicle, and in the outputting step, when it is determined that there is a risk of the vehicle colliding with the obstacle, the vehicle and the obstacle further Outputs alert information for notifying that there is a risk of collision.

  Thus, when there is a risk that the vehicle collides with the obstacle, it is possible to notify the operator of the risk of collision by alert information before the vehicle collides with the obstacle. Therefore, the operator can stop the vehicle (for example, emergency stop) before the vehicle collides with the obstacle. Further, even when the vehicle collides with the obstacle, an emergency stop operation can be performed in advance before the collision, so that the impact of the collision can be reduced. For example, when there is an obstacle in the traveling direction of the vehicle, and the vehicle has not performed a driving action (for example, a stopping action) to avoid a collision with the obstacle, the operator, based on the alert information, The vehicle can be remotely controlled.

  Also, for example, the stop position includes a third stop position calculated based on the vehicle information and a fourth stop position farther than the third stop position, and the stop position information includes the third stop position and the third stop position. A second stop range based on the fourth stop position is included.

  Thereby, the operator can operate the vehicle based on the second stop range. That is, since the operator can operate the vehicle in consideration of the stop position where the vehicle may stop, the operator can operate the vehicle more appropriately.

  Also, for example, the third stop position is calculated based on a first speed lower than the speed of the vehicle included in the vehicle information, and the fourth stop position is calculated based on a second speed higher than the speed. Is done.

  This makes it possible to calculate the second stop range based on the speed at which the vehicle information has a large effect on the stop position. Therefore, the second stop position can be calculated more accurately.

  Also, for example, the vehicle information includes information indicating a current position and a speed of the vehicle.

  As a result, the stop position can be calculated using information acquired by a sensor or the like mounted on the vehicle.

  In addition, for example, the vehicle information further includes at least one of an acceleration, a steering angle, an angular velocity, and an angular acceleration of the vehicle.

  Thereby, the stop position can be calculated with higher accuracy.

  Further, for example, a remote operation device according to an aspect of the present disclosure is an information processing device for remotely operating a vehicle via a communication network, and a vehicle information acquisition unit that acquires vehicle information of the vehicle, A delay time obtaining unit that obtains a delay time of a communication network, and a control unit that calculates a stop position when the vehicle is urgently stopped based on the vehicle information and the delay time, and the calculated stop position. And an output unit for outputting the stop position information.

  Thereby, the same effects as those of the above information processing method can be obtained. That is, the information processing device outputs the stop position information indicating the stop position, so that the operator can confirm the image in which the output stop position information is displayed (the image in which the stop position is displayed). The operator can clearly grasp the position where the vehicle stops when the vehicle is stopped urgently. That is, the operator can perform an emergency stop operation in consideration of the stop position when the vehicle is to be emergency stopped. Therefore, according to the information processing device according to an aspect of the present disclosure, the vehicle can be safely emergency stopped.

  Further, for example, a remote operation system according to an aspect of the present disclosure includes the information processing device described above, and a display device that displays a video based on the stop position information output by the information processing device.

  Thereby, the operator can check the video (video in which the stop position is displayed) in which the stop position information output from the information processing device is displayed on the display device. The operator can clearly grasp the position where the vehicle stops when the vehicle is stopped urgently. That is, the operator can perform an emergency stop operation in consideration of the stop position when the vehicle is to be emergency stopped. Therefore, according to the information processing system according to an aspect of the present disclosure, the vehicle can be safely emergency stopped.

  Note that these general or specific aspects may be realized by a system, a method, an integrated circuit, a non-transitory recording medium such as a computer program or a computer-readable CD-ROM, and the like. The present invention may be realized by any combination of circuits, computer programs, or recording media. The program may be stored in a recording medium in advance, or may be supplied to the recording medium via a wide area communication network including the Internet or the like.

  Hereinafter, embodiments will be specifically described with reference to the drawings.

  Each of the embodiments described below shows a comprehensive or specific example. Numerical values, shapes, materials, constituent elements, arrangement positions and connection forms of constituent elements, steps, order of steps, and the like shown in the following embodiments are merely examples, and do not limit the present disclosure. In addition, among the components in the following embodiments, components not described in the independent claims indicating the highest concept are described as arbitrary components.

  Each drawing is a schematic diagram, and is not necessarily strictly illustrated. In each of the drawings, substantially the same components are denoted by the same reference numerals, and redundant description may be omitted or simplified.

  Further, in this specification, terms indicating the shape of elements such as rectangles, and numerical values and numerical ranges are not expressions expressing only strict meanings, but are substantially equivalent ranges, for example, differences of about several%. Is an expression that means also including.

(Embodiment 1)
Hereinafter, an information processing method and the like of the remote control device according to the present embodiment will be described with reference to FIGS.

[1-1. Configuration of vehicle control system]
First, the configuration of a vehicle control system including a remote control device will be described with reference to FIGS. FIG. 1 is a diagram showing a schematic configuration of a vehicle control system according to the present embodiment.

  As shown in FIG. 1, the vehicle control system 10 includes a controlled vehicle 200 and a remote control system 100 (specifically, a remote control device) via a wireless base station 310 such as a wireless LAN or a communication terminal and a network 300. 140) so that they can communicate with each other. The wireless base station 310 and the network 300 are examples of a communication network. The operated vehicle 200 is an example of a vehicle that is remotely operated by the operator H. In the present specification, a vehicle is, for example, an automatic driving vehicle that controls the driving of the vehicle without requiring the operation of a driver, but can be switched to either automatic driving or manual driving to travel. Vehicle. The vehicles include not only those generally called vehicles such as automobiles, trains and buses, but also ships such as ferries and aircraft such as airplanes.

  The remote operation system 100 will be described in detail with reference to FIG. FIG. 2 is a diagram showing a functional configuration of the remote operation system 100 according to the present embodiment.

  As shown in FIGS. 1 and 2, the remote operation system 100 includes a display device 110, an operation input device 120, an emergency stop device 130, and a remote operation device 140.

  The display device 110 is a monitor that is connected to the remote operation device 140 and displays an image related to the operated vehicle 200. The display device 110 can cause the operator H to recognize the stop position of the operated vehicle 200 by displaying the stop position when the operated vehicle 200 is urgently stopped. Further, the display device 110 can cause the operator H to recognize the operated vehicle 200 and the obstacle state by displaying the operated vehicle 200 and the state of the obstacle around the operated vehicle 200 to the operator H. It may be.

  A plurality of display devices 110 may be connected to the remote control device 140. For example, the display device 110 for displaying the stop position when the operated vehicle 200 is stopped in an emergency, and the display device 110 for displaying the operated vehicle 200 and the state of the obstacle are connected to the remote operation device 140. You may. Note that a video includes a moving image and a still image. The obstacle is a vehicle other than the operated vehicle 200, a person, or the like, and mainly means a moving body that becomes an obstacle when the operated vehicle 200 travels. The obstacle may be a real estate fixed on the ground.

  The operation input device 120 is a device that is connected to the remote operation device 140 and receives a remote operation of the operator H. The operation input device 120 is, for example, a steering wheel, a foot pedal (for example, an accelerator pedal and a brake pedal), and is a device for operating the operated vehicle 200. The operation input device 120 outputs the input vehicle operation information to the remote operation device 140.

  The emergency stop device 130 is a device that is connected to the remote control device 140 and to which a remote control of the operator H is input. The emergency stop device 130 is, for example, an emergency stop button or the like, and is a device for urgently stopping the operated vehicle 200. The emergency stop device 130 outputs the input emergency stop information to the remote control device 140.

  The remote control device 140 is a device for the operator H in a remote place to remotely control the operated vehicle 200 via a communication network. In the present embodiment, the remote control device 140 further displays a stop position of the operated vehicle 200 when the operator H makes an emergency stop of the operated vehicle 200. As illustrated in FIG. 2, the remote operation device 140 includes a control unit 141, a communication unit 142, and a storage unit 143.

  The control unit 141 is a control device that controls various components of the remote control device 140. In the present embodiment, for example, the control unit 141 urgently stops the operated vehicle 200 based on the vehicle information of the operated vehicle 200 received via the communication unit 142 and the delay time of the communication network. The stop position at the time is calculated. The control unit 141 causes the display device 110 to display the stop position by outputting information for displaying the stop position on the display device 110.

  The vehicle information is information on the traveling of the operated vehicle 200 that the operated vehicle 200 itself has. The vehicle information includes the speed of the operated vehicle 200 itself and the current position. Further, the vehicle information may further include at least one of the acceleration, the steering angle, the angular velocity, and the angular acceleration of the operated vehicle 200 itself. Thereby, the control unit 141 can calculate the stop position of the operated vehicle 200 with higher accuracy.

  The delay time is a time required for transmitting information between the remote control device 140 and the operated vehicle 200. When the delay time becomes longer, that is, when the delay of the communication network becomes longer, the time required from the transmission of the operation instruction to the operated vehicle 200 to the start of the operation of the operated vehicle in accordance with the operation instruction becomes longer. For example, when the operation instruction is an instruction to perform an emergency stop, the time required from the transmission of the instruction to perform an emergency stop to the start of the stop operation of the operated vehicle 200 increases. As a result, the distance from the transmission of the emergency stop instruction to the operated vehicle 200 to the stop of the operated vehicle 200 increases, and the risk of an accident increases. Therefore, the operator H needs to know at which position the operated vehicle 200 stops when the operated vehicle 200 is stopped urgently. Therefore, as described above, the control unit 141 calculates the stop position using the delay time of the communication network.

  Further, the control unit 141 generates an image necessary for the operator H to operate the operated vehicle 200 based on image information around the operated vehicle 200 received via the communication unit 142, and generates the generated image. Is output to the display device 110. At this time, when obstacle position information including the current position of the obstacle is received from the operated vehicle 200, a video may be generated using the received obstacle position information. For example, the color of an obstacle close to the operated vehicle 200 shown in the video may be changed, or an obstacle close to the operated vehicle 200 shown in the video may be blinked and displayed.

  The control unit 141 may cause the display device 110 to superimpose the stop position on the video received from the operated vehicle 200 via the communication unit 142, for example. Further, the control unit 141 may cause the display device 110 to display an overhead view showing the stop position. For example, the control unit 141 acquires map information around the operated vehicle 200 from the current position of the operated vehicle 200 included in the vehicle information, and superimposes a stop position on the obtained map information, thereby obtaining an overhead view. May be generated.

  The control unit 141 may further have a real-time clock function for measuring the current date and time. Alternatively, the control unit 141 may use a time specified based on a GPS signal that is a signal from a GPS satellite received via the communication unit 142 as an accurate GPS time. The control unit 141 receives GPS signals at predetermined time intervals.

  The communication unit 142 is a wireless communication module for performing wireless communication with the operated vehicle 200 via a communication network. The communication unit 142 receives the vehicle information of the operated vehicle 200 and the information (specifically, the RTT measurement response packet) for calculating the delay time of the communication network via the communication network. Further, the communication unit 142 transmits control information for controlling traveling of the operated vehicle 200 to the operated vehicle 200 via the communication network under the control of the control unit 141. The control information includes vehicle control information based on the vehicle operation information and emergency stop control information based on the emergency stop information.

  The storage unit 143 is a storage device that stores a control program executed by the control unit 141. The storage unit 143 may store the vehicle information acquired via the communication unit 142, a delay time, and the like. The storage unit 143 is realized by, for example, a semiconductor memory.

  The remote operation device 140 as described above is an example of an information processing device for remotely operating the operated vehicle 200 via a communication network. The remote operation system 100 is an example of an information processing system.

  Although not shown, the remote control system 100 is connected to the remote control device 140 and outputs a warning sound to the operator H in order to warn the user about an obstacle. A sound output device (for example, a speaker) or the like for recognizing the user may be provided. Thereby, the operator H can know that the operated vehicle 200 is in a situation that requires an emergency stop.

  The remote control device 140 may cause the display device 110 to display a stop position at the time of an emergency stop when there is a possibility that the operated vehicle 200 cannot safely travel. For example, when the distance between the operated vehicle 200 and the obstacle is equal to or less than a predetermined distance, the remote operation device 140 may display the stop position when the operated vehicle 200 is emergency stopped on the display device 110. Good. Alternatively, the remote control device 140 may cause the display device 110 to display a stop position when the operated vehicle 200 is emergency stopped, for example, when information for calling attention to an obstacle is displayed or output. . Alternatively, when a predetermined operation is performed on the emergency stop device 130, the remote control device 140 may cause the display device 110 to display a stop position when the operated vehicle 200 is emergency stopped. The predetermined operation is, for example, when the emergency stop device 130 is an emergency stop button, the operator H presses the emergency stop button by a predetermined amount. In addition, when the remote control device 140 includes a sound pickup device, when the sound pickup device obtains a sound indicating that the stop position is displayed, the remote control device 140 displays a stop position when the operated vehicle 200 is emergency stopped. The information may be displayed on the device 110.

  Next, the operated vehicle 200 will be described in detail with reference to FIG. FIG. 3 is a diagram showing a functional configuration of the operated vehicle 200 according to the present embodiment.

  As shown in FIG. 3, the operated vehicle 200 includes a sensor unit 210, an imaging unit 220, a vehicle information acquisition unit 230, a control unit 240, a communication unit 250, and a storage unit 260. The operated vehicle 200 is a vehicle capable of performing autonomous traveling based on traveling data held in advance by the vehicle and traveling data generated based on information detected by the sensor unit 210 and the like, The vehicle is capable of performing automatic driving, which is remotely operated by an operator H at a remote location under predetermined conditions.

  The sensor unit 210 is a device that detects a situation around the operated vehicle 200. The sensor unit 210 detects, for example, at least one of the position, speed, and size of an obstacle (for example, another vehicle, a person, or the like) existing around the operated vehicle 200. The sensor unit 210 is realized by LIDAR (Light Detection and Ranging), radar (for example, millimeter wave radar), or a combination thereof.

  The imaging unit 220 is a camera that captures an image around the operated vehicle 200. The imaging unit 220 is provided, for example, at a position where photographing in the front, rear, left, and right directions from the operated vehicle 200 is possible. That is, the imaging unit 220 is provided in the operated vehicle 200 so that the surroundings of the operated vehicle 200 can be photographed. The imaging unit 220 may be composed of, for example, a plurality of cameras. Further, the imaging unit 220 may be a camera mounted on a drive recorder.

  A plurality of sensor units 210 and imaging units 220 may be installed in operated vehicle 200. That is, the operated vehicle 200 may include one or more sensor units 210 and one or more imaging units 220.

  The vehicle information acquisition unit 230 acquires information related to traveling of the operated vehicle 200 from various sensors mounted on the operated vehicle 200 via an in-vehicle network such as a CAN (Control Area Network). The various sensors include a speed sensor that detects the speed of the operated vehicle 200, and a GPS (Global Positioning System) sensor that detects the current position of the operated vehicle 200. That is, the vehicle information acquisition unit 230 acquires the speed of the operated vehicle 200 from the speed sensor, and acquires the current position of the operated vehicle 200 from the GPS sensor. Various sensors include a steering angle sensor that detects a steering angle of the operated vehicle 200, a brake sensor that detects a degree of braking, an accelerator sensor that detects a degree of accelerator (hereinafter, also referred to as an accelerator opening), and And a blinker sensor for detecting the direction indicated by the blinker.

  The control unit 240 is a control device that controls various components of the operated vehicle 200. In the present embodiment, the control unit 240 receives the vehicle information including the speed and the current position of the operated vehicle 200 acquired via the vehicle information acquisition unit 230, and from at least one of the sensor unit 210 and the imaging unit 220. The obtained sensing information of the operated vehicle 200 (for example, video information captured by the imaging unit 220) is transmitted to the remote operation system 100 via the communication unit 250. Further, control unit 240 outputs control information obtained from remote control system 100 via communication unit 250 to a travel control unit (not shown) that controls traveling of the own vehicle. The traveling control unit operates a speed control unit (for example, an ECU (engine control unit)) that controls the speed by operating an accelerator, a brake, and a shift lever of the operated vehicle 200, and operates the steering of the operated vehicle 200. And a steering control unit for controlling the traveling direction of the operated vehicle 200.

  The control unit 240 detects an obstacle around the vehicle of the operated vehicle 200 based on the sensing information obtained from each of the sensor unit 210 and the imaging unit 220, and generates obstacle position information indicating the current position of the obstacle. May be. The obstacle position information includes at least one of the current position of the obstacle, the speed of the obstacle, the acceleration of the obstacle, the traveling direction of the obstacle, the size of the obstacle, and the type of the obstacle. You may go out. The type of the obstacle is, for example, a type that distinguishes a pedestrian, a motorcycle, a car, and the like. The current position of the obstacle indicates the position of the obstacle when each sensor senses the obstacle.

  The control unit 240 generates video information around the operated vehicle 200 based on the sensing information obtained from the imaging unit 220. The image information around the operated vehicle 200 may be individually generated as image information in each of the front, rear, left, and right directions of the operated vehicle 200, and the front, rear, left, and right directions of the operated vehicle 200 may be combined into one image. It may be generated by combining as information.

  The control unit 240 may have a real-time clock function for measuring the current date and time. Alternatively, control unit 240 may use a time specified based on a GPS signal that is a signal from a GPS satellite acquired via communication unit 250 as an accurate GPS time. The control unit 240 receives GPS signals at predetermined time intervals.

  The communication unit 250 is a wireless communication module for performing wireless communication with the remote control device 140 via the wireless base station 310 and the network 300. The communication unit 250 transmits vehicle information, video information, and obstacle position information to the remote control device 140 via the wireless base station 310 and the network 300 under the control of the control unit 240. In addition, the communication unit 250 receives control information regarding the travel of the operated vehicle 200 via the wireless base station 310 and the network 300.

  The storage unit 260 is a storage device that stores a control program executed by the control unit 240. The storage unit 260 may store control information acquired via the communication unit 250, sensing information obtained from the sensor unit 210 and the imaging unit 220, and the like. The storage unit 260 is realized by, for example, a semiconductor memory or the like.

[1-2. Operation of Remote Control System]
Next, the operation of the remote operation system 100 will be described with reference to FIGS. FIG. 4 is a flowchart showing the operation of the remote operation system 100 according to the present embodiment. The operations in steps S10 to S40 shown in FIG. 4 are operations performed in remote control device 140.

  As shown in FIG. 4, the remote control device 140 acquires vehicle information of the operated vehicle 200 (S10). Specifically, control unit 141 receives the vehicle information of operated vehicle 200 via communication unit 142. The communication unit 142 functions as a vehicle information acquisition unit that acquires vehicle information of the operated vehicle 200. Further, the remote control device 140 acquires the delay time of the NW (communication network) (S20).

  Here, the acquisition of the delay time of the communication network will be described with reference to FIGS. 5A and 5B. 5A and 5B show a case where a round trip delay time is calculated as an example of the delay time. FIG. 5A is a sequence diagram showing an example of an operation of acquiring a delay time of an NW (communication network) according to the present embodiment. FIG. 5A illustrates acquisition of a delay time when time is synchronized between remote control device 140 and operated vehicle 200. The time is synchronized when, for example, the remote control device 140 and the operated vehicle 200 use the GPS time as the time, or when they have a radio clock. Hereinafter, a case where the remote control device 140 and the operated vehicle 200 have a GPS sensor (not shown) will be described. The GPS time means time information included in a radio wave received by the GPS sensor from a satellite.

  As shown in FIG. 5A, first, the remote control device 140 adds a time stamp (timestamp_1) to an RTT (Round Trip Time) measurement packet (S110). The control unit 141 acquires the time at which the RTT measurement packet is transmitted as timestamp_1 from the GPS sensor and writes the time in the RTT measurement packet. Then, the control unit 141 transmits an RTT measurement packet to the operated vehicle 200 via the communication unit 142 (S120).

  The operated vehicle 200 receives the RTT measurement packet (S130). The control unit 240 acquires a time stamp (timestamp_2) at the time of receiving the RTT measurement packet from the GPS sensor (S140). In addition, the control unit 240 adds a time stamp (timestamp_1 to timestamp_3) to the RTT measurement response packet which is a response corresponding to the RTT measurement packet (S150). The control unit 240 acquires the time at which the RTT measurement response packet is transmitted as the timestamp_3 from the GPS sensor, and writes the time in the RTT measurement response packet. Then, the control unit 240 transmits an RTT measurement response packet to the remote operation device 140 via the communication unit 250 (S160).

  The control unit 141 of the remote control device 140 receives the RTT measurement response packet via the communication unit 142 (S170). The control unit 141 acquires a time stamp (timestamp_4) at the time of receiving the RTT measurement response packet (S180). The control unit 141 acquires the timestamp_4 from the GPS sensor. Then, the control unit 141 calculates the NW round trip delay time using the time stamp (timestamp_1 to timestamp_4) (S190). The control unit 141 calculates the NW reciprocation delay time by subtracting the difference between timestamp_2 and timestamp_3 from the difference between timestamp_1 and timestamp_4, for example. The control unit 141 functions as a delay time acquisition unit that acquires the delay time of the communication network.

  Next, the acquisition of the delay time when the time is not synchronized between the remote control device 140 and the operated vehicle 200 will be described with reference to FIG. 5B. FIG. 5B is a sequence diagram showing another example of the operation for acquiring the delay time of the NW according to the present embodiment.

  Steps S210 to S230 shown in FIG. 5B are the same as steps S110 to S130 shown in FIG. 5A, and a description thereof will be omitted.

  As shown in FIG. 5B, the operated vehicle 200 adds timestamp_1 and duration time to the RTT measurement response packet (S240). The duration time is the time from when the RTT measurement packet is received until when the RTT measurement response packet is transmitted, and is calculated by the real-time clock function of the control unit 240. Then, control unit 240 transmits an RTT measurement response packet to remote control device 140 via communication unit 250 (S250).

  The control unit 141 of the remote control device 140 receives the RTT measurement response packet via the communication unit 142 (S260). The control unit 141 acquires a time stamp (timestamp_4) at the time of receiving the RTT measurement response packet (S270). Then, the control unit 141 calculates the NW round trip delay time using the time stamp (timestamp_1 and timestamp_4) and the duration time (S280). The control unit 141 calculates the NW round trip delay time by subtracting the duration time from the difference between the timestamps_1 and timestamps_4, for example.

  Note that, in the above description, an example is shown in which the processing time in the operated vehicle 200 (for example, the difference between timestamp_2 and timestamp_3 shown in FIG. 5A or the duration time shown in FIG. 5B) is subtracted to calculate the round trip delay time. , But is not limited to this. The round-trip delay time may be calculated, for example, from the difference between timestamps_1 and timestamps_4.

  In the above description, an example in which the delay time is a round-trip delay time has been described. However, a one-way delay time (for example, a time from when an RTT measurement packet is transmitted by the remote operation system 100 to when it is received by the operated vehicle 200). ).

  Note that the control unit 141 may store the transmission time in the storage unit 143 in association with the sequence number of the packet, without writing the transmission time directly in the packet. Then, in step S190 or S280, control unit 141 may read the transmission time from storage unit 143, and calculate the round trip delay time using the read transmission time and the reception time when the RTT measurement response packet was received. .

  The measurement of the delay time is repeatedly executed at predetermined time intervals. The control unit 141 may store the calculated delay time in the storage unit 143, for example. Then, in step S20, the control unit 141 may acquire the delay time by reading out the latest delay time from the storage unit 143 when the vehicle information is acquired.

により算出される。第２項目は、オペレータＨが緊急停止装置１３０を操作してから、被操作車両２００が実際に緊急停止動作（以降、停止動作とも記載する）を開始するまでに被操作車両２００が進む距離（いわゆる空走距離）を示す。第３項目は、被操作車両２００が停止動作を開始してから実際に停止するまでに被操作車両２００が進む距離（いわゆる制動距離）を示す。減速度ａは、被操作車両２００を緊急停止させるときの減速度であり、例えば、０．１Ｇ〜０．３Ｇ（Ｇ：重力加速度）などである。被操作車両２００は、例えば、停止動作として、一定の減速度ａで減速する。 Referring to FIG. 4 again, control unit 141 calculates the stop position of operated vehicle 200 (S30). The stop position is p, the current position of the operated vehicle 200, v is the speed of the operated vehicle 200, a is the deceleration, t_nw is the delay time of the communication network, and t_sys is the system delay time.

Is calculated by The second item is a distance traveled by the operated vehicle 200 from when the operator H operates the emergency stop device 130 to when the operated vehicle 200 actually starts the emergency stop operation (hereinafter also referred to as a stop operation) ( So-called idle running distance). The third item indicates the distance traveled by the operated vehicle 200 from the start of the stop operation of the operated vehicle 200 to the actual stop (so-called braking distance). The deceleration a is a deceleration when the operated vehicle 200 is emergency stopped, and is, for example, 0.1 G to 0.3 G (G: gravitational acceleration). The operated vehicle 200 decelerates at a constant deceleration a, for example, as a stop operation.

  Note that the system delay time includes at least one of a delay time generated in the remote operation system 100 and a delay time generated in the operated vehicle 200. The delay time generated in the remote operation system 100 is a time from when the operator H operates the emergency stop device 130 to when the emergency stop control information is transmitted by the communication unit 142. The delay time generated in the operated vehicle 200 is a time from when the emergency stop control information is received to when the stop operation is started. The system delay time may be a predetermined constant.

  Then, the control unit 141 outputs the stop position of the operated vehicle 200 to the display device 110 (S40). Specifically, control unit 141 outputs stop position information indicating the stop position of operated vehicle 200. The control unit 141 outputs to the display device 110, for example, video information including the stop position of the operated vehicle 200 or video information for displaying an overhead view including the stop position of the operated vehicle 200. The control unit 141 functions as an output unit that outputs stop position information via a communication module for communicating with the display device 110.

  When acquiring the stop position of the operated vehicle 200 from the remote control device 140, the display device 110 displays an image including the stop position (S50). An example of the stop position displayed by the display device 110 will be described with reference to FIG. FIG. 6 is a diagram illustrating a display example of the stop position of the operated vehicle 200 according to the present embodiment. Roads 401 and 402 of the image M1 illustrated in FIG. 6 indicate roads on which the operated vehicle 200 travels. FIG. 6 shows the current position 200p (corresponding to the current position 200p shown in Expression 1) and the stop position 200q of the operated vehicle 200. The arrow shown in FIG. 6 indicates the traveling direction of the operated vehicle 200.

  As shown in FIG. 6, when the operated vehicle 200 performs an emergency stop at the current position 200p, a part of the operated vehicle 200 intersects the road 401 and the road 402 as shown in the stop position 200q. It can be seen that the vehicle enters the existing intersection. By checking the image M1, the operator H can easily recognize that if an emergency stop is performed at the current position 200p, there is a risk of collision with another vehicle traveling on the road 401. . Therefore, the operator H can appropriately determine the timing of the emergency stop of the operated vehicle 200 in accordance with the stop position 200q displayed on the display device 110. The operator H can safely stop the operated vehicle 200 by operating the emergency stop device 130 when the stop position 200q displayed on the display device 110 does not collide with an obstacle or the like.

  In FIG. 6, the current position 200p and the stop position 200q of the operated vehicle 200 are shown by the shape of the vehicle, but the shapes of the current position 200p and the stop position 200q are not particularly limited. Further, at least the stop position 200q of the current position 200p and the stop position 200q of the operated vehicle 200 may be displayed on the image M1.

  As described above, the information processing method of remote control device 140 according to the present embodiment is an information processing method for remotely controlling operated vehicle 200 via network 300 and wireless base station 310. The information processing method is based on the step of acquiring vehicle information of the operated vehicle 200 (S10), the step of acquiring delay time of the network 300 and the wireless base station 310 (S20), and the vehicle information and the delay time. It includes a step (S30) of calculating a stop position when the operated vehicle 200 is emergency stopped, and a step (S40) of outputting stop position information indicating the calculated stop position.

  Thereby, the operator H confirms the video (for example, the video M1) on which the stop position information output from the remote control device 140 is displayed, and when the operated vehicle 200 is emergency stopped, the operated vehicle 200 The position where the vehicle stops can be clearly grasped. In other words, the operator H can perform an emergency stop operation in consideration of the stop position when the operated vehicle 200 is emergency stopped. Therefore, according to the information processing method according to the present embodiment, operated vehicle 200 can be safely emergency-stopped. According to the information processing method according to the present embodiment, for example, when the operated vehicle 200 is emergency stopped, the operated vehicle 200 stops at an intersection or the like, and there is a risk that an accident occurs at the stopped position. Can be reduced.

(Embodiment 2)
Hereinafter, an information processing method and the like of the remote control device 140 according to the present embodiment will be described with reference to FIGS. In the present embodiment, a description will be given focusing on differences from the first embodiment. The remote operation system 100a according to the present embodiment includes a first emergency stop device 130a, a second emergency stop device 130b, and a third emergency stop device 130, instead of the emergency stop device 130 shown in the remote operation system 100 according to the first embodiment. And an emergency stop device 130c. The details will be described below.

[2-1. Configuration of remote control system]
First, the configuration of the remote control system 100a including the remote control device 140 will be described with reference to FIG. FIG. 7 is a diagram showing a functional configuration of a remote operation system 100a according to the present embodiment.

  As shown in FIG. 7, the remote control system 100a includes a first emergency stop device 130a, a second emergency stop device 130b, and a third emergency stop device 130c. The first emergency stop device 130a, the second emergency stop device 130b, and the third emergency stop device 130c have different decelerations when the operated vehicle 200 is emergency stopped. That is, in the remote control system 100a, a plurality of deceleration patterns for emergency stop of the operated vehicle 200 are set. The deceleration is preset in each of the first emergency stop device 130a to the third emergency stop device 130c. In the present embodiment, the deceleration (hereinafter, also referred to as the first deceleration) corresponding to the first emergency stop device 130a among the first emergency stop device 130a to the third emergency stop device 130c is the highest, and the third It is assumed that the deceleration corresponding to the emergency stop device 130c (hereinafter also referred to as a third deceleration) is the lowest. Further, the deceleration corresponding to the second emergency stop device 130b is also described as a second deceleration. Note that the number of emergency stop devices provided in the remote control system 100a is not limited to three, and may be two or more.

[2-2. Operation of remote control system]
Next, the operation of the remote control system 100a will be described with reference to FIGS. FIG. 8 is a flowchart showing the operation of the remote operation system 100a according to the present embodiment. Steps S310 and S320 in FIG. 8 are the same as S10 and S20 shown in FIG. 4 of the first embodiment, and description thereof will be omitted.

  As shown in FIG. 8, the remote control device 140 calculates the stop position of the operated vehicle 200 for each of the plurality of decelerations (S330). In the present embodiment, control unit 141 calculates a stop position for each of the first to third decelerations corresponding to each of first to third emergency stop devices 130a to 130c. Specifically, the control unit 141 calculates three stop positions by changing the value of the deceleration a in Expression 1.

  Then, the control unit 141 outputs the plurality of stop positions in the operated vehicle 200 to the display device 110 (S340). Specifically, control unit 141 outputs a plurality of pieces of stop position information indicating a stop position of operated vehicle 200. The control unit 141 outputs to the display device 110, for example, video information including a plurality of stop positions in the operated vehicle 200 or video information for displaying an overhead view including a plurality of stop positions in the operated vehicle 200. .

  When acquiring a plurality of stop positions in the operated vehicle 200 from the remote control device 140, the display device 110 displays an image including the plurality of stop positions (S350). An example of the plurality of stop positions displayed by the display device 110 will be described with reference to FIG. FIG. 9 is a diagram showing a display example of the stop position of the operated vehicle 200 according to the present embodiment. FIG. 9 shows a stop position 200a which is a stop position when the first emergency stop device 130a is operated, a stop position 200b which is a stop position when the second emergency stop device 130b is operated, and a third emergency stop device 130c. The stop position 200c, which is the stop position when is operated, shows the image M2 displayed on the same screen. Here, displaying on the same screen means that the stop positions 200a to 200c are simultaneously displayed on the screen displayed by the display device 110.

  As shown in FIG. 9, when an emergency stop is performed at the current position 200p, the operated vehicle 200 stops at one of the stop positions 200a to 200c according to the deceleration. By checking the image M2, the operator H can know the deceleration at which the operated vehicle 200 can be emergency stopped at a safe position. That is, by confirming the image M2, the operator H can safely stop the operated vehicle 200 more safely by operating any one of the first emergency stop device 130a to the third emergency stop device 130c. You can determine if you can.

  As shown in the stop position 200a, when the operated vehicle 200 is emergency stopped by operating the first emergency stop device 130a (in other words, at the first deceleration), a part of the operated vehicle 200 enters the intersection. Stop in the state of doing. As shown in the stop position 200b, when the operated vehicle 200 is emergency-stopped by operating the second emergency stop device 130b (in other words, at the second deceleration), immediately after the operated vehicle 200 passes the intersection. Stop at the position. In other words, it can be understood that when the operated vehicle 200 is emergency stopped by operating the first emergency stop device 130a and the second emergency stop device 130b, there is a high risk of collision with another vehicle. Also, as shown in the stop position 200c, when the operated vehicle 200 is emergency stopped by operating the third emergency stop device 130c (in other words, at the third deceleration), the operated vehicle 200 moves away from the intersection. Stop at the position. That is, when the operated vehicle 200 is urgently stopped by operating the third emergency stop device 130c, the operation with the other vehicles or the like is smaller than when the first emergency stop device 130a and the second emergency stop device 130b are operated. It can be seen that the danger of collision is low. Therefore, the operator H can safely stop the operated vehicle 200 by operating the third emergency stop device 130c to stop the operated vehicle 200 in an emergency.

  In the above description, an example has been described in which the operated vehicle 200 decelerates at a constant deceleration (for example, first to third decelerations), but the invention is not limited to this. For example, at least one of the first to third decelerations may be set so that the value of the deceleration changes according to the elapsed time from the start of deceleration. That is, the value of the deceleration of at least one of the first to third decelerations may change over time. For example, the value of the deceleration may be set so that the deceleration gradually increases after the start of the deceleration. For example, each of the first to third decelerations may have a different degree of change over time in the value of the deceleration. That is, in the remote operation system 100a, a plurality of deceleration patterns having different degrees of change in the deceleration in the elapsed time from the start of the stop operation for the emergency stop of the operated vehicle 200 may be set.

  As described above, a plurality of deceleration patterns for emergency stop of the operated vehicle 200 are set in the operated vehicle 200 (for example, first to third decelerations). The stop position is calculated in each of the plurality of deceleration patterns (for example, stop positions 200a to 200c).

  Thereby, when the plurality of stop positions calculated in each of the plurality of deceleration patterns are displayed on the display device 110, the operator H can more safely operate the operated vehicle 200 from the plurality of stop positions displayed on the display device 110. Can be selected as a deceleration pattern that can cause an emergency stop. Therefore, according to remote control device 140 according to the present embodiment, operated vehicle 200 can be emergency stopped more safely.

  In the above description, the example in which the operator H selects the deceleration pattern from a plurality of emergency stop devices (for example, the first emergency stop device 130a to the third emergency stop device 130c) has been described, but the present invention is not limited to this. As long as a method of selecting a deceleration pattern for urgently stopping the operated vehicle 200 by the operation of the operator H can be selected, a method other than the above may be used to select the deceleration pattern. For example, the operator H may select a deceleration pattern at the time of an emergency stop by changing the degree of depression of the brake pedal of the operation input device 120. For example, the deceleration at the time of an emergency stop may be set to be large by increasing the degree of depression of the brake pedal. Further, when the remote operation system 100a includes a sound collecting device (for example, a microphone or the like) that collects the voice of the operator H, the operator H may specify the deceleration pattern by the voice.

(Embodiment 3)
Hereinafter, an information processing method and the like of the remote control device 140 according to the present embodiment will be described with reference to FIGS. In the present embodiment, a description will be given focusing on differences from the first embodiment. The configuration of the remote operation system according to the present embodiment is the same as that of remote operation system 100 according to the first embodiment, and thus description thereof will be omitted. The present embodiment is characterized in that the remote operation device 140 calculates the stop position of the operated vehicle 200 in consideration of the packet loss rate of the communication network. That is, the stop position calculated by the control unit 141 may include a stop position when packet loss occurs (at least one of a first stop position and a second stop position described below). The details will be described below.

[3-1. Operation of remote control system]
The operation of the remote control system will be described with reference to FIGS. FIG. 10 is a flowchart showing the operation of the remote operation system according to the present embodiment. Step S410 in FIG. 10 is the same as S10 shown in FIG. 4 of the first embodiment, and a description thereof will not be repeated.

  As illustrated in FIG. 10, the remote control device 140 acquires a delay time of a network (NW) and a packet loss rate (S420). The delay time of the NW is the same as in the first embodiment, and the description is omitted. The packet loss rate is a rate of a packet loss that occurs suddenly due to the influence of noise or the like.

  Here, the packet loss rate in the communication network will be described with reference to FIG. FIG. 11 is a sequence diagram showing an operation of calculating a packet loss rate between remote control device 140 and operated vehicle 200 according to the present embodiment. The remote control device 140 transmits an RTT measurement packet every first period in order to calculate a packet loss rate between the remote control device 140 and the operated vehicle 200, that is, a communication network. A different sequence number is assigned to each of the RTT measurement packets. FIG. 11 shows an example in which RTT measurement packets to which sequence numbers seq1 to seq10 are added are transmitted. Further, upon receiving the RTT measurement packet from the remote operation device 140, the operated vehicle 200 transmits to the remote operation device 140 an RTT measurement response packet to which the sequence number included in the RTT measurement packet is added. After transmitting the RTT measurement packet to which the sequence number (for example, seq1) is added to the operated vehicle 200, the remote control device 140 transmits an RTT measurement response packet including the sequence number (for example, seq1) within the second period. If reception is not possible, it is determined that a packet loss has occurred. Note that the second period may be, for example, a period shorter than the first period. The first period is, for example, one second.

  The control unit 141 of the remote control device 140 calculates a packet loss rate from the reception status of the RTT measurement response packet during a predetermined period. Specifically, the control unit 141 determines the number of times the RTT measurement packet has been transmitted in a predetermined period (hereinafter also referred to as the number of packet transmissions), and after transmitting the RTT measurement packet, the RTT measurement packet is transmitted within a second period. The packet loss rate is calculated from the number of times that the RTT measurement response packet corresponding to the measurement packet has not been received (hereinafter also referred to as the number of packet non-receptions). More specifically, the control unit 141 calculates the packet loss rate by dividing the number of untransmitted packets by the number of transmitted packets. The packet loss rate is represented by, for example, a percentage.

  In the example of FIG. 11, the number of packet transmissions is 10, and the number of packet non-receptions is 1 (specifically, one of the RTT measurement response packets seq5), so that the packet loss rate is 10%. Note that the control unit 141 may calculate the packet loss rate from the latest packet transmission count and the packet non-reception count at the time when the vehicle information of the operated vehicle 200 is obtained in step S410. For example, the control unit 141 may calculate the packet loss rate from the number of packet transmissions and the number of packet non-receptions during a period from the time when the vehicle information is acquired to the time before the third period. Alternatively, the control unit 141 determines the packet loss from the number of unreceived packets at the latest predetermined number of packet transmissions (for example, 10) when the vehicle information of the operated vehicle 200 is acquired in the past in step S410. The rate may be calculated.

  As described above, the control unit 141 acquires the packet loss rate in the communication network. Note that the control unit 141 may obtain the packet loss rate by receiving the packet loss rate in the communication network from another device via the communication unit 142. The control unit 141 may receive the packet loss rate from the operated vehicle 200, for example.

  Referring again to FIG. 10, based on the packet loss rate acquired in step S <b> 420, control unit 141 sets the emergency stop control information (an example of control information) indicating that emergency stop of operated vehicle 200 is performed on the operated vehicle. The number of transmissions N (N ≧ 2) for repeatedly transmitting the emergency stop control information to reach 200 is calculated (S430). The control unit 141 calculates the number of transmissions N such that, for example, a value obtained by multiplying the number of transmissions N by the packet loss rate is 100% or more. In the example of FIG. 11, the control unit 141 sets the number of transmissions N to two, for example. The number of transmissions N is not the number of times that the remote control device 140 actually transmits the emergency stop control information to the operated vehicle 200. If the remote control device 140 transmits N times, the emergency stop of the operated vehicle 200 is performed even if a packet loss occurs. This is the estimated number of times that control information is estimated to arrive.

により算出される。２項目は、オペレータＨが緊急停止装置１３０を操作してから、被操作車両２００が実際に緊急停止動作（以降では、停止動作とも記載する）を開始するまでに被操作車両が進む距離（いわゆる空走距離）を示す。式２は、式１に加え、制御情報を繰り返す送信する時間間隔Δｔを含む。例えば、被操作車両２００が１回目の制御情報で停止動作を開始した場合、Δｔ×（ｎ−１）はゼロとなり、式１と等しい停止位置が算出される。 The control unit 141 calculates the stop position of the operated vehicle 200 using the number of transmissions N calculated in step S430 (S440). The present embodiment is characterized in that the number of transmissions N at which the remote control device 140 transmits control information for emergency stop is used to calculate the stop position of the operated vehicle 200. The current position of the operated vehicle 200 is p, the speed of the operated vehicle 200 is v, the deceleration is a, the delay time of the communication network is t_nw, the system delay time is t_sys, and the time interval for repeatedly transmitting control information is Δt ( Assuming the time interval Δt in FIG. 13), the stop position when the operated vehicle 200 is emergency stopped by the n-th (n ≦ N) control information is:

Is calculated by The two items are the distance traveled by the operated vehicle (so-called “stop operation”) from when the operator H operates the emergency stop device 130 to when the operated vehicle 200 actually starts the emergency stop operation (hereinafter, also referred to as a stop operation). Idle running distance). Equation 2 includes, in addition to Equation 1, a time interval Δt at which control information is repeatedly transmitted. For example, when the operated vehicle 200 starts the stop operation based on the first control information, Δt × (n−1) becomes zero, and the stop position equal to Expression 1 is calculated.

  For example, the control unit 141 may calculate only the stop position when the operated vehicle 200 is emergency stopped based on the control information at the Nth time (n = N). That is, the control unit 141 may calculate only the stop position when the operated vehicle 200 is emergency stopped at the position farthest from the current position 200p. In addition, the control unit 141 uses, for example, the control information at the first time (n = 1), the stop position at the time when the operated vehicle 200 is emergency stopped, and the control information at the Nth time (n = N). The stop position at the time when the operated vehicle 200 stops in an emergency may be calculated. That is, the control unit 141 determines the stop position of the operated vehicle 200 at the position closest to the current position 200p shown in Expression 2 when the emergency stop is performed, and the position of the operated vehicle 200 farthest from the current position 200p shown in Expression 2. The stop position at the time of emergency stop may be calculated. Thereby, the range of the stop position where the operated vehicle 200 may stop when the emergency stop is performed can be known.

  Then, the control unit 141 outputs the stop position of the operated vehicle 200 to the display device 110 (S450). Specifically, control unit 141 outputs image information including the stop position of operated vehicle 200 or image information for displaying an overhead view including the stop position of operated vehicle 200 to display device 110.

  When acquiring the stop position of the operated vehicle 200 from the remote control device 140, the display device 110 displays an image including the stop position (S460). An example of the stop position displayed by the display device 110 will be described with reference to FIGS. 12A to 12C. FIGS. 12A to 12C are diagrams illustrating stop positions of the operated vehicle 200 when a packet loss occurs. FIG. 12A is a diagram illustrating a display example of a stop position of operated vehicle 200 according to the present embodiment. A video M3 illustrated in FIG. 12A illustrates a stop range 201 which is a range in which the operated vehicle 200 may be emergency stopped due to a packet loss. The stop range 201 is an example of a first stop range.

  As shown in FIG. 12A, control unit 141 may cause display device 110 to display the range of the stop position of operated vehicle 200. The control unit 141 determines the operated position from the stop position 200d when the number of transmissions is the first time (an example of a second stop position) and the stop position 200e when the number of transmissions is the Nth (an example of the first stop position). The stop range 201 of the vehicle 200 may be displayed. In other words, the control unit 141 may output video information including the stop range 201 based on the stop positions 200d and 200e as the stop positions to the display device 110.

  This allows the operator H to know the range in which the operated vehicle 200 may be emergency-stopped when the operated vehicle 200 is emergency-stopped in a situation where packet loss has occurred. That is, the operator H can determine whether or not the operated vehicle 200 can be safely emergency-stopped by checking the range in which the operated vehicle 200 may be emergency-stopped. For example, since the stop range 201 shown in FIG. 12A is a range including an intersection, the operator H can determine that an emergency stop is dangerous.

  The control unit 141 may, for example, use map information around the current position 200p of the operated vehicle 200 or an image such as an intersection in the traveling direction of the operated vehicle 200 based on an image captured by the imaging unit 220 of the operated vehicle 200. 12A may be displayed on the display device 110 when there is a range in which there is a risk of collision with the display device 110. Note that the existence of a range where there is a risk of collision in the traveling direction means that the region exists within a first distance from the current position 200p. The first distance is determined by the speed of the operated vehicle 200 and the like, and is, for example, several meters or tens of meters.

  Although FIG. 12A shows an example in which the stop range 201 is displayed in a rectangular shape, the stop range 201 may be elliptical, or may have any other shape as long as it includes the stop positions 200d and 200e. Good. Further, the stop range 201 may be displayed by blinking or the like. Further, the stop range 201 shows an example in which the stop range 201 is displayed in a frame shape by a solid line, but may be displayed by being filled with a predetermined color (for example, red) or the like.

  Next, another example of the stop position displayed by the display device 110 will be described with reference to FIG. 12B. FIG. 12B is a diagram showing another display example of the stop position of the operated vehicle 200 according to the present embodiment. Image M4 shown in FIG. 12B shows stop position 200f when operated vehicle 200 stops at the position closest to current position 200p.

  As illustrated in FIG. 12B, the control unit 141 determines the stop position 200f that is the closest to the current position 200p among the stop positions where the operated vehicle 200 may stop when a packet loss occurs in the display device 110. May be displayed. The control unit 141 may display the image M4 including the stop position 200f (an example of the second stop position) when the operated vehicle 200 is emergency stopped based on the emergency stop control information when the number of transmissions is the first time. In other words, the control unit 141 may output video information including the stop position 200f as the stop position to the display device 110.

  The control unit 141 is, for example, map information around the current position 200p of the operated vehicle 200, or from the video imaged by the imaging unit 220 of the operated vehicle 200, the operated vehicle 200 has entered the intersection, or 12B may be displayed on the display device 110 when the operated vehicle 200 is about to enter the intersection. Immediately before entering an intersection means, for example, that the current position 200p is a position within a second distance from the intersection. The second distance is, for example, several meters or several tens of meters.

  Next, still another example of the stop position displayed by the display device 110 will be described with reference to FIG. 12C. FIG. 12C is a diagram showing still another display example of the stop position of the operated vehicle 200 according to the present embodiment. Image M5 shown in FIG. 12C shows stop position 200g when operated vehicle 200 stops at the position farthest from current position 200p.

  As shown in FIG. 12C, the control unit 141 determines that the stop position 200g furthest to the current position 200p among stop positions where the operated vehicle 200 may stop when a packet loss occurs in the display device 110. May be displayed. The control unit 141 may display the image M5 including the stop position 200g (an example of the first stop position) when the operated vehicle 200 is emergency stopped based on the emergency stop control information when the number of transmissions is N. In other words, the control unit 141 may output video information including the stop position 200g as the stop position to the display device 110.

  The control unit 141 may, for example, use map information around the current position 200p of the operated vehicle 200 or an image such as an intersection in the traveling direction of the operated vehicle 200 based on an image captured by the imaging unit 220 of the operated vehicle 200. If there is no area that is likely to collide with the image M5, the image M5 illustrated in FIG. 12C may be displayed on the display device 110. It should be noted that the absence of an area at risk of collision in the traveling direction means that the area does not exist within a third distance from the current position 200p. The third distance is, for example, several tens of meters or several hundreds of meters.

  As described above, according to the information processing method of remote control device 140 according to the present embodiment, the number of transmissions N (N ≧ 2) in which control information indicating emergency stop is repeatedly transmitted based on the packet loss rate in the communication network. ) Is further calculated (S430), and the stop position is further calculated using the number of transmissions N.

  Accordingly, by calculating the stop position using the number of transmissions N based on the packet loss rate, even if a packet loss occurs in the communication between the operated vehicle 200 and the remote operation device 140, The stop position can be calculated in consideration of the packet loss. For example, the operator H can safely stop the operated vehicle 200 by grasping the farthest first stop position (for example, the stop position 200g) among the stop positions of the operated vehicle 200 in consideration of the packet loss rate. Can be determined. Therefore, according to the information processing method of remote control device 140 according to the present embodiment, it is possible to safely stop an operated vehicle even when a packet loss occurs.

  Further, the operator H can more accurately determine whether the operated vehicle 200 can safely stop by knowing the stop range 201 where the operated vehicle 200 may stop due to the occurrence of packet loss. You can judge. The stop range 201 includes a stop position 200d and a stop position 200e. Thus, even when a packet loss occurs, the operated vehicle 200 can be emergency stopped more safely.

  Here, an operation in which the remote control device 140 transmits emergency stop control information (described as a stop instruction in FIG. 13) when a packet loss has occurred will be described with reference to FIG. FIG. 13 is a sequence diagram showing an operation of transmitting and receiving a stop instruction between remote control device 140 and operated vehicle 200 according to the present embodiment. The operation shown in FIG. 13 is performed irrespective of the number of transmissions N calculated above.

  As shown in FIG. 13, first, the remote control device 140 acquires a stop instruction for urgently stopping the operated vehicle 200 from the operator H (S510). Specifically, the control unit 141 acquires a stop instruction by acquiring emergency stop information from the emergency stop device 130. When acquiring the emergency stop information, the control unit 141 starts transmitting a stop instruction including an emergency stop of the operated vehicle 200 to the operated vehicle 200 (S520). Specifically, control unit 141 transmits emergency stop control information based on the emergency stop information to operated vehicle 200 via communication unit 142.

  The control unit 141 transmits a second stop instruction when a stop instruction response, which is a response to the transmission of the first stop instruction, is not received from the operated vehicle 200. The control unit 141 repeatedly transmits a stop instruction at time intervals Δt until a stop instruction response is obtained from the operated vehicle 200. FIG. 13 illustrates a case where the stop instruction transmitted from the first to fifth times by the control unit 141 has not reached the operated vehicle 200 due to the influence of the packet loss. In other words, the control unit 141 has not received a stop instruction response that is a response to the first to fifth stop instructions.

  The operated vehicle 200 receives the stop instruction transmitted from the remote operation device 140 (S530). FIG. 13 illustrates an example in which the operated vehicle 200 receives the stop instruction transmitted by the remote control device 140 for the sixth time. Upon receiving the stop instruction from the remote operation device 140, the operated vehicle 200 transmits a stop instruction response corresponding to the stop instruction to the remote operation device 140 (S540). Then, the operated vehicle 200 performs a stop operation based on the stop instruction (S550). Upon receiving the stop instruction response from the operated vehicle 200 (S560), the remote operation device 140 stops transmitting the stop instruction (S570).

  As described above, the remote control device 140 repeatedly transmits the stop instruction until a stop instruction response is received from the operated vehicle 200, so that even when a packet loss occurs in the communication network, the operated vehicle 200 can be emergency stopped.

(Embodiment 4)
Hereinafter, the operation of the remote operation system 100 will be described with reference to FIGS. 14 to 15B. FIG. 14 is a flowchart showing the operation of the remote operation system 100 according to the present embodiment. The configuration of the remote operation system according to the present embodiment will be described with respect to an example similar to the remote operation system 100 according to the first embodiment, but is similar to the remote operation system 100a according to the second embodiment. You may.

  As shown in FIG. 14, the remote control device 140 acquires the vehicle information of the operated vehicle 200 and the obstacle information (obstacle position information) (S610). Specifically, the control unit 141 receives the vehicle information of the operated vehicle 200 from the operated vehicle 200 via the communication unit 142. In addition, the control unit 141 acquires obstacle information on obstacles existing around the operated vehicle 200 via the communication unit 142. The obstacle information is, for example, information including a current position of an obstacle existing around the operated vehicle 200 detected by at least one of the sensor unit 210 and the imaging unit 220 included in the operated vehicle 200. Further, the obstacle information may further include at least one of information on the speed of the obstacle, the acceleration of the obstacle, the traveling direction of the obstacle, the size of the obstacle, and the type of the obstacle. The type of the obstacle is, for example, a type that distinguishes a pedestrian, a motorcycle, a car, and the like. The current position of the obstacle indicates the position of the obstacle when each sensor senses the obstacle. The obstacle information is, for example, information on an obstacle existing in the traveling direction of the operated vehicle 200. In the present embodiment, the communication unit 142 also functions as an obstacle information acquisition unit that acquires obstacle information. The obstacle information may be stored in the storage unit 143.

  In step S610, the remote control device 140 is not limited to acquiring the obstacle information from the operated vehicle 200. The remote control device 140 may acquire, for example, obstacle information on an obstacle existing around the operated vehicle 200 from an imaging device, a sensor, or the like installed on a road on which the operated vehicle 200 runs.

  The processing in steps S620 and S630 is the same as the processing in steps S20 and S30 shown in FIG. 4, and a description thereof will be omitted.

  Then, the control unit 141 determines whether or not the stop position of the operated vehicle 200 calculated in step S630 and the obstacle based on the obstacle information acquired in step S610 overlap (S640). That is, the control unit 141 determines whether there is a danger of the operated vehicle 200 colliding with the obstacle. As described above, in the present embodiment, the control unit 141 also functions as a collision risk determination unit that determines whether there is a risk that the operated vehicle 200 will collide with an obstacle. The control unit 141 determines whether or not the stop position of the operated vehicle 200 and the obstacle overlap with each other, for example, using intersection determination.

  The stop position is, for example, a position at which the operated vehicle 200 stops when the operator H emergency-stops the operated vehicle 200 located at the current position (for example, when the emergency stop device 130 is operated). Note that the stop position is a position where the operated vehicle 200 stops when the operator H normally stops the operated vehicle 200 located at the current position (for example, when the brake pedal of the operation input device 120 is operated). There may be.

  FIG. 15A is a diagram illustrating a display example of the stop position P1 of the operated vehicle 200 according to the present embodiment. In FIG. 15A, the stop position P1 of the operated vehicle 200 and the obstacle O do not overlap on the video M6. In this case, the control unit 141 determines that the stop position P1 of the operated vehicle 200 does not overlap with the obstacle O (No in S640), that is, that the operated vehicle 200 does not collide with the obstacle O.

  The control unit 141 performs the above-described determination, for example, using a frame (for example, a rectangular frame) surrounding the operated vehicle 200 located at the stop position 200h as the stop position P1 of the operated vehicle 200. It is not limited to. The control unit 141 may determine whether there is a risk that the operated vehicle 200 will collide with the obstacle O, for example, based on whether the stop position 200h and the obstacle O overlap. In addition, the frame indicating the stop position P1 includes the operated vehicle 200 and is larger than the operated vehicle 200 in a plan view. For example, the size of the frame for the operated vehicle 200 is set in advance. By setting the frame indicating the stop position P1 to be large, the operated vehicle 200 can travel more safely. Note that the stop position P1 is an example of a third stop position.

  Then, the control unit 141 outputs the stop position P1 of the operated vehicle 200 and the obstacle position to the display device 110 (S650). The control unit 141 outputs, for example, stop position information indicating the stop position P1 of the operated vehicle 200 and obstacle information (obstacle position information) including the position where the obstacle O exists. The control unit 141 displays, for example, video information including the stop position P1 of the operated vehicle 200 and the obstacle position, or an overhead view including the stop position P1 of the operated vehicle 200 and the obstacle position. Is output to the display device 110. For example, if No in step S640, the control unit 141 prohibits the output of the alert information.

  When the display device 110 acquires the stop position P1 and the obstacle position of the operated vehicle 200 from the remote control device 140, an image including the stop position P1 and the obstacle position (for example, the image M6 illustrated in FIG. 15A). ) Is displayed (S660).

  As shown in FIG. 15A, it can be seen that when the operated vehicle 200 performs an emergency stop at the current position 200p, it can be safely stopped without colliding with the obstacle O. By checking the image M6, the operator H can easily recognize that if the emergency stop is performed at the current position 200p, the operated vehicle 200 can be safely emergency stopped.

  Further, since the operator H can recognize that the obstacle O exists on the traveling route of the operated vehicle 200, the operator H performs a driving action to avoid the collision with the obstacle O. It can be observed whether or not. Then, when the operated vehicle 200 is not performing a driving action for avoiding a collision with the obstacle O, the operator H can urgently stop the operated vehicle 200 at an appropriate timing while viewing the image M6. it can.

  FIG. 15B is a diagram showing another display example of the stop position P2 of the operated vehicle 200 according to the present embodiment. In FIG. 15B, the stop position P2 of the operated vehicle 200 and the obstacle O overlap on the video M7. In this case, the control unit 141 determines that the stop position P2 of the operated vehicle 200 and the obstacle O overlap (Yes in S640), that is, there is a risk that the operated vehicle 200 and the obstacle O may collide. In addition, when the distance between the stop position P2 and the obstacle O is less than or equal to a predetermined distance on the video M7, the control unit 141 determines that there is a risk that the operated vehicle 200 and the obstacle O may collide. You may. The stop position P2 is a frame (for example, a rectangular frame) surrounding the operated vehicle 200 located at the stop position 200i.

  Then, the control unit 141 outputs the stop position P2 of the operated vehicle 200, the obstacle position, and the alert information to the display device 110 (S670). The control unit 141 may, for example, stop position information indicating the stop position P2 of the operated vehicle 200, obstacle information indicating the position of the obstacle O (obstacle position information), and the operated vehicle 200 Alert information for notifying the operator H that there is a danger of collision is output. The control unit 141 may control the stop position P2 of the operated vehicle 200, the obstacle position, and the video information including the alert A based on the alert information, or the stop position P2 of the operated vehicle 200, the obstacle position, and the like. Video information for displaying an overhead view including alert A based on the alert information is output to display device 110.

  When acquiring the stop position P2, the obstacle position, and the alert information of the operated vehicle 200 from the remote control device 140, the display device 110 outputs the stop position P2, the obstacle position, and the alert (S680). In the present embodiment, display device 110 outputs an alert by displaying a video including alert A (for example, video M7 shown in FIG. 15B).

  As shown in FIG. 15B, in the video M7, an example is shown in which the character “Caution for collision” is displayed as the alert A. Thus, the operator H can recognize that there is a risk of the operated vehicle 200 colliding with the obstacle O by simply looking at the alert A displayed on the video M7. For example, if the operated vehicle 200 travels in the state of automatic driving as it is, there is a risk of colliding with the obstacle O, or the operated vehicle 200 will stop despite the obstacle O is present in the traveling direction. If not, the operator H can stop the operated vehicle 200 urgently based on the image M7 shown in FIG. 15B.

  Further, as described above, since the remote control device 140 causes the display device 110 to display the alert A, the operator H can easily notice that there is a risk of the operated vehicle 200 colliding. The alert A may be displayed, for example, in the video M7 near a position where the operated vehicle 200 and the obstacle O may collide with each other and do not overlap with the road 402.

  Note that the alert information is not limited to information indicating that a character calling attention is displayed on the video M7, and may be information indicating that the display mode of the video M7 is changed. The alert information is, for example, information indicating that at least one of the size, the color, and the shape of the frame of the stop position P2 of the operated vehicle 200 is changed from when there is no danger of the operated vehicle 200 colliding. Alternatively, the information may indicate that the video M7 is to be blinked.

  Further, the alert information is not limited to changing the display mode. For example, the alert information may be information indicating that there is a risk of the operated vehicle 200 colliding with the operator H by a sound (for example, a warning sound). When the display device 110 includes a speaker, the sound may be output from the display device 110 or may be output from a device other than the display device 110. Further, the alert information may be information including giving a stimulus to the user by vibration (vibration of a desk, chair, or the like). Further, the alert information may be a combination thereof.

  As shown in FIG. 15B, when an emergency stop is performed at the current position 200p, it is understood that the operated vehicle 200 may collide with the obstacle O as shown at the position of the obstacle O and the stop position P2. . The operator H can recognize that it is necessary to stop the operated vehicle 200 urgently by checking the image M7.

  15A and 15B, an example in which the current position 200p and the stop positions (for example, the stop positions P1 and P2) of the operated vehicle 200 and the position of the obstacle O are displayed on the image. However, at least the stop position and the position of the obstacle O may be displayed.

  As described above, the information processing method in remote control device 140 according to the present embodiment includes a step of acquiring vehicle information of operated vehicle 200 (S610) and a step of acquiring delay time of the communication network (S620). Calculating a stop position P1 or P2 when the operated vehicle 200 is emergency stopped based on the vehicle information and the delay time (S630), and outputting stop position information indicating the calculated stop position P1 or P2. (S650, S670). Then, the information processing method further obtains obstacle information including the position of the obstacle O around the operated vehicle 200 (S610), and based on the stop position P1 or P2 and the position of the obstacle O. Determining whether there is a risk that the operated vehicle 200 and the obstacle O collide (S640).

  Then, in the determining step, when it is determined that there is a risk of collision between the stop position P2 of the operated vehicle 200 and the obstacle O (Yes in S640), the control unit 141 determines the stop position P2 in the outputting step. In addition to the indicated stop position information, obstacle information including the position of the obstacle O and alert information are output (S670). If the control unit 141 determines in the determining step that there is no danger of collision between the stop position P1 of the operated vehicle 200 and the obstacle O (No in S640), the control unit 141 outputs the stop position P1 in the output step. Obstacle information including the position of the obstacle O is output in addition to the indicated stop position information (S650).

  Accordingly, when there is a risk that the operated vehicle 200 collides with the obstacle O, the operator A can be notified of the danger of collision by the alert A before the operated vehicle 200 collides with the obstacle O. it can. That is, the operator H can stop the operated vehicle 200 (for example, emergency stop) before the operated vehicle 200 collides with the obstacle O. Further, even when the operated vehicle 200 and the obstacle O collide, the emergency stop operation can be performed in advance before the collision, so that the impact of the collision can be reduced.

(Embodiment 5)
Hereinafter, the operation of the remote operation system 100 will be described with reference to FIGS. 16 to 17C. FIG. 16 is a flowchart showing the operation of the remote operation system 100 according to the present embodiment. The configuration of the remote operation system according to the present embodiment will be described with respect to an example similar to the remote operation system 100 according to the first embodiment, but is similar to the remote operation system 100a according to the second embodiment. You may. Further, the processing of steps S710 and S720 is the same as the processing of steps S10 and S20 shown in FIG. 4 (or the processing of S310 and S320 shown in FIG. 8), and a description thereof will be omitted.

  As shown in FIG. 16, the control unit 141 calculates the range of the stop position of the operated vehicle 200 (S730). The control unit 141 calculates the range of the stop position based on changes in vehicle information (for example, speed, steering, and the like) of the operated vehicle 200. The range of the stop position is a range in which the operated vehicle 200 stops or has a high possibility of stopping while the vehicle information changes (for example, while the speed, steering, and the like of the operated vehicle 200 change). When the vehicle information is a speed, for example, the control unit 141 calculates a stop position in each case based on an upper limit value and a lower limit value of the speed, and a range including the calculated two stop positions. Is the range of the stop position. For example, the range of the stop position is formed by two stop positions and an area sandwiched between the two stop positions.

により算出される。また、例えば、速度の上限値ｖ＿ｍａｘは、現在の速度をｖとすると、

により算出される。なお、ｖ＿ｔｈｒｅｓｈｏｌｄは、速度の変化量を示しており、定数であってもよいし、加速度又は時間等に基づいて算出されてもよい。速度の変化量ｖ＿ｔｈｒｅｓｈｏｌｄが定数である場合、例えば、速度の変化量ｖ＿ｔｈｒｅｓｈｏｌｄは、１０ｋｍ／ｓ〜２０ｋｍ／ｓなどである。なお、速度の下限値ｖ＿ｍｉｎ及び上限値ｖ＿ｍａｘを算出するための情報（例えば、式３、式４、変化量ｖ＿ｔｈｒｅｓｈｏｌｄなど）は、記憶部１４３に記憶されている。 For example, assuming that the current speed is v, the lower limit value v_min of the speed is

Is calculated by Further, for example, the upper limit value v_max of the speed is as follows, where v is the current speed.

Is calculated by Note that v_threshold indicates the amount of change in speed, and may be a constant, or may be calculated based on acceleration, time, or the like. When the speed change amount v_threshold is a constant, for example, the speed change amount v_threshold is 10 km / s to 20 km / s. The information for calculating the lower limit value v_min and the upper limit value v_max of the speed (for example, Equations 3, 4 and the change amount v_threshold) is stored in the storage unit 143.

  Then, the control unit 141 calculates the minimum distance and the maximum stop distance of the operated vehicle 200 based on the lower limit value v_min and the upper limit value v_max of the speed calculated by Expressions 3 and 4. The minimum distance is a distance that minimizes the distance from the current position 200p of the operated vehicle 200 to the stop position. The minimum distance may be a distance estimated to be the minimum. The maximum distance is the distance at which the distance from the current position 200p of the operated vehicle 200 to the stop position is the maximum. The maximum distance may be a distance estimated to be the maximum.

により算出される。 For example, the minimum distance of the stop position, that is, the minimum value of the stop distance, is p, the current position of the operated vehicle 200, v_min is the lower limit of the speed of the operated vehicle 200, a is deceleration, and t_nw is the delay time of the communication network. , And the system delay time is t_sys,

Is calculated by

  The second item is the distance traveled by the operated vehicle 200 from when the operator H operates the emergency stop device 130 to when the operated vehicle 200 actually starts the emergency stop operation (hereinafter, also referred to as a stop operation). Indicates the minimum value (the so-called minimum value of the idling distance). The third item indicates the minimum value of the distance traveled by the operated vehicle 200 from the start of the stop operation of the operated vehicle 200 to the actual stop (the so-called minimum value of the braking distance). The deceleration a is a deceleration when the operated vehicle 200 is emergency stopped, and is, for example, 0.1 G to 0.3 G (G: gravitational acceleration). The operated vehicle 200 decelerates at a constant deceleration a, for example, as a stop operation.

により算出される。 Also, for example, the maximum distance of the stop position, that is, the maximum value of the stop distance, is p at the current position of the operated vehicle 200, v_max at the upper limit of the speed of the operated vehicle 200, a at deceleration, and delay time of the communication network. Is t_nw and the system delay time is t_sys,

Is calculated by

  The second item is the maximum value of the distance traveled by the operated vehicle 200 from the time when the operator H operates the emergency stop device 130 to the time when the operated vehicle 200 actually starts the stop operation (the so-called maximum value of the idle running distance). ). The third item indicates the maximum value of the distance traveled by the operated vehicle 200 from the start of the stop operation of the operated vehicle 200 to the actual stop (the so-called maximum value of the braking distance).

  Then, the control unit 141 calculates the range of the stop position of the operated vehicle 200 using the two stop positions. For example, it is assumed that the stop position calculated based on Expression 5 is a stop position P3 illustrated in FIG. 17A. The stop position P3 is a frame (for example, a rectangular frame) surrounding the operated vehicle 200 located at the stop position 200j. Further, for example, it is assumed that the stop position calculated based on Expression 6 is the stop position P4 illustrated in FIG. 17B. The stop position P4 is a frame (for example, a rectangular frame) surrounding the operated vehicle 200 located at the stop position 200k.

  The stop position P4 is a position farther from the current position 200p of the operated vehicle 200 than the stop position P3. FIG. 17A is a diagram illustrating a stop position P3 when the stop distance is short in the operated vehicle 200 according to the present embodiment. FIG. 17B is a diagram illustrating a stop position P4 when the stop distance is long in the operated vehicle 200 according to the present embodiment. The stop position P3 is an example of a third stop position, and the stop position P4 is an example of a fourth stop position.

  As illustrated in FIG. 17C, the control unit 141 determines a range including the stop positions P3 and P4 as the range of the stop position (stop range 202). The stop range 202 includes, for example, stop positions P3 and P4, and an area sandwiched between the stop positions P3 and P4. Thereby, even while the vehicle information of the operated vehicle 200 is changing, the stop range 202 where the operated vehicle 200 may stop can be calculated. FIG. 17C is a diagram showing a display example of the stop range 202 of the operated vehicle 200 according to the present embodiment. The stop range 202 is an example of a second stop range.

  Note that, in the above description, an example in which the stop range 202 is calculated based on the maximum distance and the minimum distance has been described. The stop range 202 may be calculated using each of a far stop position and a near stop position as compared to a stop position when the speed change amount v_threshold is not considered. The stop range 202 may be calculated using, for example, a stop position calculated using a speed lower and a speed higher than the current speed v of the operated vehicle 200. Further, the stop range 202 may be calculated using a stop position when the speed change amount v_threshold is not taken into consideration and one of a stop position farther from the stop position and a stop position closer to the stop position. The stop range 202 may be calculated, for example, using a stop position calculated using the current speed v of the operated vehicle 200 and one of a speed lower than and higher than the current speed v. .

  Then, the control unit 141 outputs the range of the stop position (the stop range 202) including the stop positions P3 and P4 of the operated vehicle 200 to the display device 110 (S740). Specifically, control unit 141 outputs stop position information including stop range 202 of operated vehicle 200. The control unit 141 outputs to the display device 110, for example, video information including the stop range 202 of the operated vehicle 200 or video information for displaying an overhead view including the stop range 202 of the operated vehicle 200.

  When acquiring the stop range 202 of the operated vehicle 200 from the remote operation device 140, the display device 110 displays an image (for example, the image M8 shown in FIG. 17C) including the stop range 202 (S750).

  As described above, the information processing method in remote control device 140 according to the present embodiment includes a step of acquiring vehicle information of operated vehicle 200 (S710) and a step of acquiring delay time of the communication network (S720). And a step (S730) of calculating a stop range 202 when the operated vehicle 200 is emergency stopped based on the vehicle information and the delay time, and a step of outputting the calculated stop range 202 (S770).

  Stop positions for calculating the stop range 202 include a third stop position calculated based on the vehicle information and a fourth stop position farther from the third stop position. The third stop position is, for example, a stop position P3 calculated based on a first speed lower than the current speed v of the operated vehicle 200 included in the vehicle information. The fourth stop position is, for example, a stop position P4 calculated based on a second speed higher than the current speed v of the operated vehicle 200 included in the vehicle information.

  In the present embodiment, in the calculation step, control unit 141 calculates based on stop position P3 calculated based on lower limit value v_min of the speed of operated vehicle 200 and upper limit value v_max of the speed of operated vehicle 200. The stop range 202 is calculated based on the stop position P4 thus set. The lower limit value v_min of the speed is an example of a first speed lower than the current speed v, and the upper limit value v_max of the speed is an example of a second speed higher than the current speed v.

  Then, in the outputting step, the control unit 141 outputs stop position information including the stop position P3 and the stop position P4 to the display device 110.

  Thereby, the operator H can operate the operated vehicle 200 based on the stop range 202 shown in the image M8 of FIG. 17C. Since the operator H can operate the operated vehicle 200 in consideration of the stop position where the operated vehicle 200 may stop, the operator H can operate the operated vehicle 200 more appropriately.

(Other embodiments)
As described above, the present disclosure has been described based on Embodiments 1 to 5 (hereinafter, also referred to as embodiments), but the present disclosure is not limited to the above embodiments.

  For example, in each of the above embodiments, the example has been described in which the control unit of the remote control device causes the display device to display an image including the current position of the operated vehicle, but the present disclosure is not limited thereto. The control unit of the remote operation device may cause the display device to display a position at which the operated vehicle starts the stop operation instead of the current position of the operated vehicle. In this case, for example, the stop position may be calculated using the stop operation start position instead of the current position 200p in Expressions 1 and 2. In this case, a one-way delay time may be used as the delay time.

  Further, in each of the above-described embodiments, an example has been described in which the remote control device displays the stop position at the time of an emergency stop on the display device when there is a possibility that the operated vehicle cannot safely travel. , But is not limited to this. The remote control device may always display the stop position at the time of the emergency stop on the display device.

  Further, the communication method between the devices included in the remote operation system in each of the above embodiments is not particularly limited. Wireless communication or wired communication may be performed between the devices. In addition, wireless communication and wired communication may be combined between devices.

  Further, in each of the above embodiments, the example has been described in which the operated vehicle transmits the obstacle position information to the remote control device. However, the operated vehicle does not have to transmit the obstacle position information.

  Further, the processing units (for example, the control unit) of the remote control device and the operated vehicle according to each of the above embodiments are typically realized as an LSI which is an integrated circuit. These may be individually integrated into one chip, or may be integrated into one chip so as to include some or all of them.

  Further, the integrated circuit is not limited to the LSI, and may be realized by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) that can be programmed after manufacturing the LSI, or a reconfigurable processor that can reconfigure the connection and setting of circuit cells inside the LSI may be used.

  In each of the above embodiments, each component may be configured by dedicated hardware, or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory.

  Further, the numbers used above are all examples for specifically describing the present disclosure, and each embodiment of the present disclosure is not limited to the illustrated numbers.

  The division of functional blocks in the block diagram is merely an example, and a plurality of functional blocks can be implemented as one functional block, one functional block can be divided into a plurality of functional blocks, and some functions can be transferred to other functional blocks. You may. Also, the functions of a plurality of functional blocks having similar functions may be processed by a single piece of hardware or software in parallel or time division.

  Further, the order in which the steps in the flowchart are executed is merely an example for specifically describing the present disclosure, and may be an order other than the above. Also, some of the above steps may be performed simultaneously (in parallel) with other steps.

  In addition, a form obtained by applying various modifications that can be conceived by those skilled in the art to the above embodiments and the like, and realized by arbitrarily combining the components and functions in the embodiments and the like without departing from the spirit of the present disclosure. Embodiments are also included in the present disclosure.

  The information processing method and the like according to an aspect of the present disclosure are effective for a remote control device that allows an operator at a remote location to remotely control a vehicle via a communication network.

10 Vehicle control system 100, 100a Remote operation system (information processing system)
110 display device 120 operation input device 130 emergency stop device 130a first emergency stop device 130b second emergency stop device 130c third emergency stop device 140 remote control device (information processing device)
141 control unit (delay time acquisition unit, output unit)
142 communication unit (vehicle information acquisition unit)
143 storage unit 200 operated vehicle (vehicle)
200p Current position 200a to 200c, 200h to 200k, 200q Stop position 200d, 200f Stop position (second stop position)
200e, 200g Stop position (first stop position)
201 Stop range (first stop range)
202 Stop range (second stop range)
210 sensor unit 220 imaging unit 230 vehicle information acquisition unit 240 control unit 250 communication unit 260 storage unit 300 network 310 wireless base station 401, 402 road A alert M1 to M8 image O obstacle P1, P2 stop position P3 stop position (third position) Stop position)
P4 Stop position (fourth stop position)

Claims (12)

An information processing method of an information processing device for remotely controlling a vehicle via a communication network,
Obtaining vehicle information of the vehicle;
Obtaining a delay time of the communication network;
Based on the vehicle information and the delay time, calculating a stop position when the vehicle is emergency stop,
Outputting stop position information indicating the calculated stop position. In the vehicle, a plurality of deceleration patterns when emergency stopping the vehicle is set,
The information processing method according to claim 1, wherein the stop position is calculated in each of a plurality of deceleration patterns. A step of calculating a number of transmissions N (N ≧ 2) for repeatedly transmitting control information indicating an emergency stop based on a packet loss rate in the communication network;
The information processing method according to claim 1, wherein the stop position is calculated further using the number of transmissions. 4. The information processing method according to claim 3, wherein the stop position includes a first stop position in a case where the vehicle is urgently stopped based on the control information transmitted Nth time. 5. The stop position further includes a second stop position in the case where the vehicle is emergency stopped by the control information transmitted for the first time,
The information processing method according to claim 4, wherein the stop position information includes a first stop range based on the first stop position and the second stop position. further,
Obtaining obstacle information including a position of an obstacle around the vehicle;
Based on the stop position and the position of the obstacle, determining whether there is a risk of collision between the vehicle and the obstacle,
In the outputting, when it is determined that there is a risk of collision between the vehicle and the obstacle, further, alert information for notifying that there is a risk of collision between the vehicle and the obstacle is output. The information processing method according to any one of claims 1 to 5. The stop position includes a third stop position calculated based on the vehicle information and a fourth stop position farther than the third stop position,
The information processing method according to any one of claims 1 to 6, wherein the stop position information includes a second stop range based on the third stop position and the fourth stop position. The third stop position is calculated based on a first speed lower than the speed of the vehicle included in the vehicle information,
The information processing method according to claim 7, wherein the fourth stop position is calculated based on a second speed higher than the speed. The information processing method according to claim 1, wherein the vehicle information includes information indicating a current position and a speed of the vehicle. The information processing method according to claim 9, wherein the vehicle information further includes at least one of an acceleration, a steering angle, an angular velocity, and an angular acceleration of the vehicle. An information processing device for remotely controlling a vehicle via a communication network,
A vehicle information acquisition unit that acquires vehicle information of the vehicle,
A delay time obtaining unit for obtaining a delay time of the communication network,
Based on the vehicle information and the delay time, a control unit that calculates a stop position when the vehicle is emergency stop,
An output unit configured to output stop position information indicating the calculated stop position. An information processing apparatus according to claim 11,
A display device for displaying an image based on the stop position information output by the information processing device.

Images