JP7357820B1 - Remote control device, mobile remote control system, mobile control device, image display device, mobile object and remote control method - Google Patents

Abstract

The present disclosure relates to a remote control device that remotely controls a moving body based on control information from an operator, and which includes output from a moving body control device that outputs a moving body control amount for controlling the moving body. a first receiving unit that receives a first wave variable obtained by converting the state quantity of the moving object into a wave variable via a transmission path; and a first receiving unit that converts a control amount based on the first wave variable and the control information into a wave variable, and converts each of them. It has a first wave variable converting section that outputs the post-state quantity and the second wave variable, and a first transmitting section that transmits the second wave variable to the mobile object control device via the transmission path. There is.

Description

The present disclosure relates to a mobile object remote control system that allows an operator to remotely control a mobile object located in a remote location via a network.

In recent years, with the development of communication technology, the development of systems for remotely controlling moving objects such as automobiles and mobility robots has progressed. As a result, remote autonomous driving systems are being developed that provide backup and assistance to automobile autonomous driving systems. Furthermore, in the field of mobility robots, there are high expectations for the realization of telepresence systems that allow facility guidance services from remote locations and the experience of being in a remote location as if you were there.

International Publication No. 2019/225118

G. Niemeyer and J. E. Slotine, "Stable Adaptive Teleoperation," in IEEE Journal of Oceanic Engineering, vol. 16, no. 1, pp. 152-162, Jan. 1991, doi:10.1109/48.64895. Y. Yokokohji, T. Tsujioka and T. Yoshikawa, "Bilateral control with time-varying delay including communication blackout," Proceedings 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. HAPTICS 2002,2002,pp.285-292,doi :10.1109/HAPTIC.2002.998970.

In a mobile remote control system in which an operator controls a mobile object from a remote location, images and operation data are transmitted and received between the mobile object and the operator. In this case, a network including wireless communication is used. Generally, communication delays occur when transmitting and receiving data over a network. As a result, images and maneuvering data arrive at the pilot with a delay, which can lead to deterioration in maneuverability due to a decrease in the pilot's sense of presence, and a delay in situational judgment, making the behavior of the moving object unstable. There was sex.

A prior example disclosed in Patent Document 1 proposes a method of suppressing instability by reducing the operating speed of the working machine when communication delay is large in a system for remotely controlling a working machine from a remote location. has been done. This makes it possible to provide a remote control device that can remotely control working machines with high precision and efficiency. However, in the method of the prior example, if a period of time with a large communication delay occurs continuously, there is a problem that the work becomes time consuming and inefficient.

The present disclosure has been made to solve the above-mentioned problems, and is a mobile remote control system that can improve work efficiency by suppressing unstable operation even when communication delays occur in the network. The purpose is to provide a maneuvering system.

A remote control device according to the present disclosure is a remote control device that remotely controls a moving body based on control information from an operator, and is a mobile body control device that outputs a moving body control amount for controlling the moving body. a first receiving unit that receives, via a transmission path , a first wave variable obtained by wave variable conversion of the state quantity of the moving body and image information around the moving body, which is output from the device;
a first wave variable conversion unit that converts the first wave variable and the control amount based on the control information into wave variables, and outputs the converted state quantity and second wave variable, respectively;
a first transmitter that transmits the second wave variable to the mobile body control device via the transmission path; the image information; and the converted state quantity output from the first wave variable converter. and an image generation unit that generates an image to be displayed on an image display device based on the above .

According to the remote control device according to the present disclosure, by performing wave variable conversion on the passivity defined by the state quantity and the control amount of the moving body, instability of the control is suppressed and communication delay occurs in the transmission path. A mobile remote control system with high work efficiency can be realized even in the case of a remote control system.

1 is a diagram showing the configuration of a mobile remote control system according to a first embodiment; FIG. 1 is a block diagram showing the configuration of a remote control device according to Embodiment 1. FIG. 1 is a block diagram showing the configuration of a latitude control device according to a first embodiment; FIG. It is a figure which shows an example of a structure when a moving object is a car. FIG. 2 is a diagram showing a bilateral control system in which an operator remotely controls a work arm located at a remote location. FIG. 1 is a block diagram showing a bilateral control system. 1 is a block diagram showing a mobile remote control system according to a first embodiment; FIG. It is a figure explaining the dynamic characteristic when a moving object is a car. FIG. 3 is a diagram illustrating a method of rotating image information. FIG. 2 is a block diagram showing a mobile remote control system according to a second embodiment. FIG. 3 is a diagram schematically showing distortion of wave variables when communication delay is not constant. FIG. 2 is a block diagram showing the configuration of a remote control device according to a second embodiment. FIG. 2 is a block diagram showing the configuration of a latitude control device according to a second embodiment. FIG. 3 is a block diagram showing the configuration of a remote control device according to a third embodiment. FIG. 3 is a diagram illustrating movement prediction of left and right front wheels when the moving object is a car. 7 is a diagram illustrating an example of an image generated by an image generation unit of a remote control device according to a third embodiment. FIG. 1 is a diagram showing a hardware configuration for realizing a remote control device and a moving object control device according to Embodiments 1 to 3. FIG. 1 is a diagram showing a hardware configuration for realizing a remote control device and a moving object control device according to Embodiments 1 to 3. FIG.

<Embodiment 1>
<Mobile remote control system>
FIG. 1 is a diagram showing the configuration of a mobile object remote control system 1000 according to Embodiment 1 of the present disclosure. As shown in FIG. 1, the mobile object remote control system 1000 is a system in which an operator OP remotely controls a mobile object MV from a remote location, and is composed of a control system SS, a mobile system MS, and a network NW. Note that FIG. 1 is also used as a mobile object remote control system 2000 according to Embodiment 2 of the present disclosure.

The network NW connects a plurality of components to each other using cables, radio waves, etc., and is capable of transmitting and receiving data. There are various types of networks, such as LAN (Local Area Network), WAN (Wide Area Network), the Internet, telephone lines, and wireless communication. In the present disclosure, the network is not limited to these, and may be any medium that can transmit and receive information between the maneuvering system SS and the mobile system MS.

The mobile system MS includes a mobile body MV, a camera CM mounted on the mobile body MV, a sensor SN, a mobile body control unit MD, and a mobile body control device FD. Note that although in FIG. 1, the mobile body control device FD is shown as being provided outside the mobile body MV for convenience, it is actually mounted on the mobile body MV.

The mobile object MV is, for example, a car, a flying object, a drone, a probe, a mobility robot that moves on wheels or legs, an agricultural machine, etc., and if it is a moving artificial object, it is a target of remote control.

The camera CM has a function of acquiring image data necessary for remote control of the mobile object MV and transmitting it to the mobile object control device FD. The camera CM is installed at a position where it can photograph not only the front but also the sides, rear, or all directions of the moving object. Usually, a digital camera composed of a lens, an image sensor, a shutter, etc. is used, and can capture images of the surroundings of the mobile object MV. Additionally, by installing an infrared camera that detects infrared images in addition to the digital camera, remote control will be possible even at night. Furthermore, by installing LiDAR (Light Detection and Ranging), depth data can also be acquired at the same time. In this way, the camera is not limited to digital cameras as long as it can acquire image data necessary for remote control of a moving body.

The sensor SN has a function of acquiring sensor data, mainly angular velocity and translational velocity, required when remotely controlling the mobile body MV, and transmitting it to the mobile body control device FD. When the mobile object MV is a car, the sensor SN includes, for example, a vehicle speed sensor, an IMU (Inertial Measurement Unit) sensor, a steering angle sensor, and the like.

The mobile body control unit MD generates a command value for an actuator (not shown) installed in the mobile body MV by referring to the mobile body control amount input from the mobile body control device FD, and actually controls the actuator. Has a function.

The control system SS is a system in which an operator OP controls a mobile object MV located at a remote location, and includes an image display device ID, a control device ST, and a remote control device RD.

The operator OP is a person who operates the mobile MV from a remote location, determines the operation of the mobile MV by looking at the image displayed on the image display device ID, and actually controls the mobile MV. Achieve the determined action.

The operation by the operator OP is detected by the remote control device RD via the control device ST. Note that maneuvering in the present disclosure refers to controlling the angular velocity and translational velocity of the mobile body MV via the maneuvering device ST.

A control amount is obtained by the operation of the operator OP, and the control amount is a value that is a reference value or a direct control amount when controlling the angular velocity and translational velocity of the mobile object MV. For example, if the mobile object MV is a car In this case, the steering angle of the steering wheel is referenced as the control amount for controlling the angular velocity, and the amount of depression of the accelerator pedal and the brake pedal is referenced as the control amount for controlling the translational speed. If the amount of control is the steering angle, and the input to the electric motor that rotates the steering shaft directly corresponds to the steering angle command, it is a "direct control amount" and the input to the electric motor corresponds to the steering angle command. If not, the steering angle is used as a "control reference value." In this disclosure, maneuvers related to angular velocity are unified under the term "steering."

Note that the steering device ST is not limited to a steering wheel, a pedal, and a brake pedal. In other words, it may be a joystick, a cross key used in a gaming controller, a keyboard, a mouse, etc., and is selected in consideration of the characteristics of the mobile object MV, ease of operation, etc.

The image display device ID has a function of displaying an image generated by the remote control device RD and presenting it to the operator OP. Although a single display is normally used in the image display device ID, it is also possible to display a wide range of images using multiple displays. Furthermore, using a head-mounted display, the viewpoint of the image can be changed in accordance with the movement of the operator OP's head. In this way, the image display device is not particularly limited as long as it allows the operator OP to recognize the surroundings of the mobile object MV. Note that images here include not only still images of landscapes but also moving images that change over time.

<Remote control device>
FIG. 2 is a block diagram showing the configuration of remote control device RD according to the first embodiment. The remote control device RD has a wave variable conversion section R1, a reception section R2 (first reception section), a transmission section R3 (first transmission section), an image generation section R4, and a maneuver detection section R5. It is connected to the display device ID and the control device ST, and can send and receive data to and from the network NW.

The receiving unit R2 receives the image information, wave variable _vm , and sensor information transmitted from the mobile object control device FD (FIG. 1) via the network NW, and outputs the image information and sensor information to the image generating unit R4. , has a function of outputting the wave variable v _m to the wave variable converter R1.

The transmitter R3 has a function of transmitting the wave variable _um generated by the wave variable converter R1 to the mobile object control device FD via the network NW.

The wave variable conversion unit R1 generates speed information (post-conversion state quantity) and wave variable u _m based on the wave variable v _m input from the reception unit R2 and the maneuver amount input from the maneuver detection unit R5, Each has a function of outputting to the image generation section R4 and the transmission section R3. Note that wave variable conversion and its effects will be described later. Further, in this disclosure, it is assumed that the velocity information is either angular velocity or translational velocity, or both.

The image generation unit R4 uses the image information input from the reception unit R2 and the speed information input from the wave variable conversion unit R1 to generate a presentation image to be presented to the operator OP (FIG. 1), and displays the image. It has a function to output a presentation image to the device ID. It also has a function to superimpose information included in the sensor information to assist remote control on the image. Details of the image generation unit R4 will be described later.

The maneuver detection section R5 has a function of decoding the maneuver data input from the maneuver device ST (FIG. 1) and outputting it as a maneuver amount to the wave variable converting section R1.

<Mobile object control device>
FIG. 3 is a block diagram showing the configuration of the mobile object control device FD according to the first embodiment. The mobile object control device FD includes a wave variable conversion section F1 (second wave variable conversion section), a reception section F2 (second reception section), a transmission section F3 (second transmission section), an image acquisition section F4, and a sensor. It has an acquisition section F5, is connected to the camera CM, sensor SN, and mobile object control section MD, and can transmit and receive data with the network NW.

The receiving unit F2 has a function of receiving the wave variable _us transmitted from the remote control device RD (FIG. 1) via the network NW, and outputting it to the wave variable converting unit F1.

The transmitting unit F3 transmits the wave variable _vs generated by the wave variable converting unit F1, the image information input from the image acquiring unit, and the sensor information acquired by the sensor acquiring unit F5 to the remote control device RD via the network NW. It has the ability to send.

The image acquisition unit F4 has a function of acquiring image data from the camera CM and outputting it as image information to the transmission unit F3. Note that since image data is usually large in data size, it is encoded for compression. Common encoding formats include jpeg, png, H. 264 and H. There are various methods such as H.265. The image acquisition unit F4 also has a function of encoding image data if necessary. Note that in recent years, some cameras have an encoding function themselves, and in that case, there is no need to encode in the image acquisition unit F4.

The sensor acquisition unit F5 has a function of decoding sensor data from the sensor SN, extracting velocity information, that is, angular velocity and translational velocity, and outputting it to the wave variable conversion unit F1. It also has a function of outputting sensor data to the transmitter F3 as sensor information.

The wave variable conversion unit F1 generates a moving body control amount and a wave variable v _s using the wave variable us inputted from the receiving unit F2 and the speed information inputted from the _sensor acquisition unit F5, and performs mobile body control, respectively. It has a function of outputting to the section MD and the transmitting section F3. Note that wave variable conversion and its effects will be described later.

<Mobile object>
FIG. 4 is a diagram showing the configuration of the mobile object MV in detail. Note that FIG. 4 is a diagram showing an example of a configuration when the mobile object MV is a car.

A steering wheel 1 installed for a driver to operate the mobile body MV is engaged with a steering shaft 2. The steering shaft 2 is engaged with a pinion shaft 13 of a rack and pinion mechanism 4. The rack shaft 14 of the rack and pinion mechanism 4 is capable of reciprocating in accordance with the rotation of the pinion shaft 13, and a front knuckle 6 is connected to both left and right ends of the rack shaft 14 via tie rods 5. The front knuckle 6 rotatably supports a front wheel 15 as a steered wheel, and is rotatably supported by the vehicle body frame.

The torque generated by the operator's operation of the steering wheel 1 rotates the steering shaft 2, and the rack and pinion mechanism 4 moves the rack shaft 14 in the left-right direction in accordance with the rotation of the steering shaft 2. The movement of the rack shaft 14 causes the front knuckle 6 to rotate around a kingpin shaft (not shown), thereby steering the front wheels 15 in the left-right direction. Therefore, the operator can change the amount of lateral movement of the mobile body MV by operating the steering wheel 1 when the vehicle moves forward and backward.

Although the word "pilot" is used here to explain the mobile body MV, in a remote control system, the pilot is located in a remote location and therefore does not directly operate the steering wheel 1 of the mobile body MV. Furthermore, in the case of a non-boarded moving object such as a fully self-driving car or a drone, a component such as the steering wheel 1 for operator control is not required.

A vehicle speed sensor 20, an IMU sensor 21, a steering angle sensor 22, a steering torque sensor 23, and the like are installed in the mobile body MV as sensors SN for recognizing the running state of the mobile body MV.

The movable body MV is equipped with actuators such as an electric motor 3 for realizing lateral movement of the movable body MV, a vehicle drive device 7 and a brake control device 10 for controlling the longitudinal movement of the movable body MV. has been done.

Acceleration/deceleration control device 9 controls vehicle drive device 7 and brake control device 10 , and steering control device 12 controls electric motor 3 .

The electric motor 3 is generally composed of a motor and a gear, and can freely rotate the steering shaft 2 by applying torque to the steering shaft 2. That is, the electric motor 3 can freely steer the front wheels 15 independently of the operator's operation of the steering wheel 1.

The vehicle drive device 7 is an actuator for driving the moving body MV in the front-back direction. The vehicle drive device 7 rotates the front wheels 15 and the rear wheels 16 using driving force obtained from a drive source such as an engine or a motor via a transmission and a shaft (not shown). Thereby, the vehicle drive device 7 can freely control the driving force of the mobile body MV.

On the other hand, the brake control device 10 is an actuator for braking the moving body MV, and controls the amount of braking of the brakes 11 installed on each of the front wheels 15 and rear wheels 16 of the moving body MV. A typical brake generates braking force by using hydraulic pressure to press a pad against a disc rotor that rotates together with the front wheels 15 and rear wheels 16 .

As described above, the camera CM is installed at a position where it can photograph not only the front but also the sides, rear, or all directions of the mobile body MV.

The mobile body control unit MD converts the control amount into a current value or the like based on the information from the sensor SN and the mobile body control amount input from the mobile body control device FD, and outputs it to the actuator.

In this case, the moving body control unit MD calculates the current value to be supplied to the electric motor 3 in order to cause the steering of the moving body MV to follow the target steering amount, and outputs the calculation result to the electric motor 3. In addition, the moving object control unit MD calculates the driving force and braking force necessary to make the acceleration of the moving object MV follow the target acceleration/deceleration amount, and outputs the calculation results to the vehicle drive device 7 and the brake control device 10. .

The above-mentioned sensor and a plurality of other devices including the mobile object control device FD constitute an in-vehicle network using a CAN (Controller Area Network) or a LAN (Local Area Network) in the mobile object MV. Each device in the mobile MV shown in FIG. 4 can obtain its own information via the network. Moreover, the sensors SN can mutually transmit and receive data via the network.

The configuration in which the mobile body MV is a car has been described above, but the sensor, actuator, etc. can have a similar configuration even if the mobile body MV is other than a car.

<Bilateral control>
Wave variable transformation, which is important in the present disclosure, will be described below. First, a force feedback bilateral control system involving communication of force and velocity, which is a general method of using bilateral control, will be described with reference to FIGS. 5 and 6.

FIG. 5 shows a bilateral control system in which an operator remotely controls a work arm located at a remote location. In this system, the operator OP applies a force F _op to the control arm OA, which imitates the work arm WA, thereby causing the control arm OA to generate a velocity V _o . The steering arm control device OAD detects the speed _Vo and transmits it to the work arm control device WAD via the transmission path TL. The work arm control device WAD controls the position of the work arm WA at a speed V _a based on the speed V _o obtained from the control arm control device OAD. At this time, the force _Fa applied to the work arm WA is detected. The force F _a includes the inertial force of the work arm WA, and the reaction force when the work arm WA is in contact with the environment EV such as a wall. The work arm control device WAD transmits the detected force _Fa to the control arm control device OAD via the transmission path TL. Based on this information, the steering arm control device OAD performs force control on the steering arm OA to generate a force F _o . In this way, a bilateral control system is a system that transmits information from the operator side and the work side in both directions for control. In particular, the system shown in FIG. 5 is called a force feedback bilateral control system because it feeds back force information to the operator OP. Note that although it is conventionally expressed as a transmission path TL here, in this disclosure, it is used to have the same meaning as a network.

However, it is known that in bilateral control systems, the control system becomes unstable due to communication delays that exist in the transmission path. The reason for this will be explained using FIG. 6. FIG. 6 is a diagram depicting the system of FIG. 5 as a block diagram. In FIG. 6, the bilateral control system is composed of an operator OP, a control arm system OAS, a transmission line TL, a work arm system WAS, and an environment EV. In this description, in order to avoid complexity, the control arm and the control arm control device will be referred to as the control arm system OAS, and the work arm and the work arm control device will be referred to as the work arm system WAS. The transmission path TL also includes a communication delay element DL1 and a communication delay element DL2 that delay the signal by a certain period of time T.

As shown in Figure 6, the bilateral control system includes elements such as operator OP and environment EV that are difficult to mathematically model and predict. Internal stability analysis is difficult. Therefore, such systems utilize a stability index called passivity.

Passivity is one indicator of input/output stability, and if a system has passivity, it can be said to have some kind of stability. Note that a system with passivity is called a passive system.

More specifically, the input and output to a certain system are represented by vectors x and y, and the product of the input and output is defined as a kind of power. Let E be the energy function and P be the power rate. Assume that there exists a non-negative function P _diss representing the dissipation rate for dissipating energy, and a system is said to be passive if it satisfies the following equation (1). Here, the superscript T represents the transposition of the matrix.

The above formula (1) can be rewritten as the following formula (2).

From equation (2), it can be understood that the energy output by the system to the outside by time t does not exceed the initial energy. That is, energy is consumed due to the existence of the dissipation rate P _diss . Note that when P _diss =0, it is said that there is no loss. Note that the power P and energy function E here do not necessarily have physical meaning, and may be artificial power or energy. If we can define the product of input and output for which we want to claim passivity as virtual power and virtual energy, and show that the system is passive with respect to virtual energy, we can say that the system is stable in the sense of virtual energy. In the case of force feedback type bilateral control, the input is force, the output is velocity, and the product of these is defined as the power.

Passivity is characterized by the fact that the series combination of passive systems is also passive, and in FIG. If it is passive, it can be said to be stable.

Typically, the operator OP and environment EV can be considered passive. Furthermore, the control arm system OAS and the work arm system WAS are also kept passive by performing control that consumes energy. However, it is known that when force and velocity are directly transmitted and received as in a force feedback bilateral control system, the transmission line TL is not necessarily passive and may become unstable in some cases.

Therefore, in order to ensure the passivity of the transmission path, a transformation called wave variable transformation is used. Wave variable conversion is a conversion that makes the transmission path passive, and the passivity of the transmission path is guaranteed. Below, an outline of wave variable conversion will be described based on Non-Patent Document 1. In the wave variable conversion, conversions as shown in the following equations (3) and (4) are performed.

The above equation (3) represents the wave variable u _mo and the wave variable u _sa when information is sent from the control arm system OAS to the work arm system WAS.

The above equation (4) represents the wave variable v _mo and the wave variable v _sa when information is sent from the work arm system WAS to the control arm system OAS.

Furthermore, in Equations (3) and (4), b is a parameter called characteristic impedance, and is set as a positive real number. Furthermore, it is assumed that F _o , V _o , F _a , and V _a are vectors, assuming the case of arms having multiple degrees of freedom.

Here, due to the communication delay of the delay time T, the wave variable v _mo and the wave variable v _sa , and the wave variable u _ma and the wave variable u _so have a relationship expressed by the following equation (5).

When wave variable conversion is performed, the inflow and outflow of energy at time t in the transmission path is expressed by the following equation (6), and is passive (P≧0) and lossless.

Note that the integration in the second line of Equation (6) above represents integration from time tT to time t with τ as a time variable.

By using wave variable conversion in this way, the transmission path can be made passive, and the system shown in FIG. 6 can be stabilized. Furthermore, from equation (6), it can be seen that when wave variables are used, the passivity of the transmission path does not depend on the delay time T of communication delay. That is, by making use of this characteristic, it is possible to suppress instability of the remote control system even when communication delay is large. However, the example presented here is of a bilateral control system using force and velocity. In order to apply this to a remote control system for a mobile object, the following processing is performed.

<Wave variable conversion section>
The processing contents and effects of the wave variable conversion section of the present disclosure will be described below. As mentioned above, in the case of a mobile remote control system, in an environment with communication delays, images and control data arrive late, resulting in a decrease in maneuverability and a delay in judgment due to a decrease in the remote operator's sense of presence. There was a possibility that the behavior of the moving object would become unstable. Such an unstable phenomenon will be explained by stability evaluation using passivity, as in the case of a bilateral control system. Note that in the bilateral control system, power defined as the product of force and velocity was introduced, but in this disclosure, virtual power and virtual energy defined as the product of steering and angular velocity are introduced. do. In the following, virtual power and virtual energy will be referred to as virtual power and virtual energy, respectively.

The reason for introducing such virtual energy is that delays in the movement of a mobile object and delays in recognizing the surrounding situation due to communication delays are considered to be one of the factors that cause instability. In other words, if it is possible to guarantee passivity with the input and output of the angular velocity indicating the rotational motion and the velocity indicating the translational motion of the moving object, as well as its control amount, steering, acceleration, and deceleration, it is possible to avoid system instability. it is conceivable that.

In the following explanation, virtual power will be defined as the product of steering and angular velocity, but it is also possible to define similar virtual power for translational motion. In other words, the product of the translational velocity indicating the translational motion of the moving body and the acceleration operation and deceleration operation that are the amount of control thereof may be defined as the virtual power. Therefore, since the same result is obtained for translational motion, only the angular velocity and the passivity due to steering will be explained below.

FIG. 7 is a block diagram showing the mobile object remote control system 1000 shown in FIG. In FIG. 7, the steering of the operator OP is δ _m , the angular velocity is ω _m , the wave variables are um , v _{m ,} us , _vs , _the moving body control amount is δ _s , and the angular velocity of the moving body MV is ω _s There is. It is also assumed that the transmission path TL includes communication delay elements DL1 and DL2 that delay the signal by a certain delay time T. In addition, in FIG. 7, the mobile body MV and the environment in which the mobile body MV is placed are represented by a block MEV as a mobile body+environment.

As in the case of a bilateral system, the operator OP believes that the virtual energy will not increase as long as he performs common-sense steering, such as steering that does not increase the angular velocity. It will be done. In other words, it can be considered passive.

Furthermore, it is assumed that the moving body MV normally has a characteristic in which the angular velocity is the output in response to the steering input.

FIG. 8 is an explanatory diagram regarding dynamic characteristics when the mobile object MV is a car. In FIG. 8, the model is simplified by replacing the left and right front wheels and the left and right rear wheels of the automobile with one wheel 15 and one wheel 16, respectively, and is called a two-wheel model. In this case, the dynamic characteristics of the vehicle can be expressed by the following equation (7).

In the above formula (7), δ is the steering angle, V is the speed of the center of gravity of the moving body, ω is the angular velocity, β is the side slip angle around the center of gravity of the moving body, m is the mass, I is the moment of inertia around the center of gravity, l _f and l _r are distances from the center of gravity to the front wheels 15 and rear wheels 16, respectively, and C _f and C _r are cornering stiffnesses. Furthermore, in FIG. 8, the inertial coordinate system is represented by XY, and the moving object coordinate system fixed to the moving object is represented by xy. ψ is the orientation of the moving body coordinate system with respect to the X-axis of the inertial coordinate system.

Cornering stiffness is a value determined by the condition between the tire and the contact surface, and is dependent on the environment. In this way, the moving object is always influenced by the environment, so in FIG. 7, the moving object + environment is represented by a block MEV.

The steering angle δ is set in the moving body control unit MD (FIG. 3) so as to realize the moving body control amount δ _s input from the wave variable conversion unit F1 (FIG. 3) of the moving body control device FD (FIG. 3). This is a control amount for controlling the mobile object MV. Normally, the moving body control unit MD performs control so that the angular velocity ω _s does not become unstable. Furthermore, when considering interaction with the environment, it is thought that virtual energy is usually controlled within a range where it becomes passive. For example, if the mobile object MV is a car, it may be moving on an icy road or snowy road, and in such a case, the virtual energy will not increase because skid prevention control etc. will be activated. it is conceivable that. Therefore, the mobile MV and the environment can be considered passive with respect to virtual energy. Similarly, the remote control device RD and the mobile object control device FD are also controlled within a range that satisfies the passivity of virtual energy.

From these, the only element for which passivity to virtual energy is not satisfied is the transmission line TL, but as in the case of bilateral systems, this is caused by wave variable transformation as shown in equations (8) and (9) below. Passivation is possible by

The above equation (8) represents the wave variable u _m and the wave _{variable us} when information is sent from the remote control device RD to the mobile object control device FD.

The above equation (9) represents the wave variable v _m and the wave variable v _s when information is sent from the remote control device RD to the mobile object control device FD.

In this case as well, the power P can be expressed by the following equation (10) by calculation similar to equation (6), which indicates that the transmission path TL is passive.

By executing the wave variable conversion described above in the wave variable converter R1 of the remote control device RD and the wave variable converter F1 of the mobile object control device FD, the transmission path TL can be made passive. In this manner, the mobile remote control system of FIG. 7 can maintain stability because all blocks are passive due to the wave variable conversion units R1 and F1. In particular, the passivity of the transmission line TL does not depend on the delay time T, so even if a large communication delay occurs, it is always passive and stability can be guaranteed.

The processing in the wave variable converters R1 and F1 will be explained in more detail. First, the processing in the wave variable conversion unit R1 of the remote control device RD will be explained. The wave variable conversion unit R1 first receives the wave variable v _m from the mobile object control device FD via the transmission path TL. The wave variable v _m is a signal obtained by delaying the wave variable v _s output by the mobile object control device FD by the delay time T, and the wave variable v _m (t) at time t is expressed by the following formula (11). be able to.

Next, the angular velocity ω _m of the image to be presented to the operator OP is determined from the steering angle δ _m of the operator OP and the wave variable v _m . The angular velocity ω _m (t) at time t can be expressed by the following equation (12).

From Equations (11) and (12), the wave variable u _m (t) at time t can be expressed by Equation (13) below.

The wave variable u _m is transmitted from the remote control device RD to the mobile object control device FD via the transmission path TL.

Next, the processing in the wave variable conversion unit F1 of the mobile object control device FD will be explained. The wave variable converter F1 first receives the wave variable _us from the remote control device RD via the transmission line TL. The wave variable u _s is a signal that is delayed by the delay time T from the wave variable u _m output by the remote control device RD, and the wave variable u _m (t) at time t is expressed by the following equation (14). I can do it.

Next, the wave variable conversion unit F1 updates the steering angle δ _s using the angular velocity ω _s of the mobile body MV obtained from the sensor acquisition unit F5 of the mobile body control device FD. The angular velocity ω _s (t) at time t can be expressed by the following equation (15).

From Equations (14) and (15), the wave variable v _s (t) at time t can be expressed by Equation (16) below.

The wave variable _vs is transmitted from the mobile control device FD to the remote control device RD via the transmission path TL.

The wave variable conversion unit R1 of the remote control device RD and the wave variable conversion unit F1 of the mobile object control device FD can perform wave variable conversion and make the transmission path TL passive by performing the above processing. .

<Image generation section>
Next, returning to the explanation of FIG. 2, the image generation section R4 will be explained. As shown in FIG. 7, the angular velocity ω _m is input to the operator OP, which means that the angular velocity ω _m obtained by wave variable conversion is converted into the rotation angle θ _m of the image presented to the operator OP. It means to convert. In other words, this means that the image information input from the image generation unit R4 to the image display device ID is rotated by an angle θ _m .

Here, the rotation angle θ _m (t) at time t can be expressed by the following equation (17).

Image information including communication delay inputted from the receiving unit R2, that is, past image information, is rotated by a rotation angle θ _m . Here, the time when image information is received is set to 0, and the time during which the same image information is used is set to t. In this way, the image rotated by the rotation angle θ _m corresponding to the angular velocity ω _m is presented to the operator OP, and the operator OP performs steering by looking at the image, so that the steering angle δ _m is determined by the rotation angle θ m corresponding to the angular velocity ω m. It can be considered as an output. Moreover, as an actual effect, the operator OP can feel as if his/her own steering is changing the image in real time, and can perform stable operation.

Next, an example of a method for rotating image information by a rotation angle _θm will be described using FIG. 9. In FIG. 9, a virtual camera is placed on the XYZ three-dimensional virtual space, and image information acquired from the receiving unit R2 is pasted in the visible range at the virtual camera position VCP. In the example of FIG. 9, image information II including left and right LN of the road surface and a horizontal line HL is shown.

This image information is image information obtained via the network NW, and is past image information around the mobile object MV including delay time. By rotating the posture of the camera at the virtual camera position VCP by the rotation angle θ _m , the image presented to the operator OP can be changed from the past image at time 0 to the current image at time t.

This method of generating an image is just one example, and there are various methods of rotating the image by the rotation angle θ _m . For example, with depth data that can be acquired by LiDAR, the same effect as changing the viewpoint can be obtained by simply transforming the coordinates of a point group.

Note that when the product of the translational speed and the acceleration or deceleration operation is defined as the virtual power, the same effect can be obtained by enlarging and reducing the image or translating the image using the translational speed.

In addition, as a process of the image generation unit R4, the sensor information from the reception unit R2 is superimposed on the image converted using the wave variable as a numerical value or an object, and is presented to the operator, thereby improving maneuverability. can be done. For example, when the moving body MV is a car, if a steering angle is obtained as the sensor information, a method of drawing an object imitating a steering wheel by rotating it by the steering angle may be considered. In this case, the operator OP can recognize what kind of steering angle the mobile object MV actually has, and can reflect this information in the operation. As another example, if the speed of the moving object MV can be acquired as sensor information, it is possible to express the speed numerically and superimpose it on the image. In this case, the operator OP can recognize at what speed the mobile object MV is currently moving, and can reflect this information in the operation. As another example, if the sensor information includes information on lane markings on the road surface, if the marking lines are drawn as lines in the image, the driver can easily recognize the lanes.

By performing wave variable conversion and image generation in this way, the stability of the mobile remote control system is guaranteed, and the operator OP can remotely control the mobile MV as if there were no delays due to communication delays. becomes. Since this effect does not depend on the magnitude of the delay time T, it is possible to suppress instability of remote control even when there is a large communication delay.

Note that in the above description, the wave variable converter converts the speed information into wave variables, but the present invention is not limited to this. For example, acceleration information (angular acceleration, translational acceleration) and position information (angular information, translational information) can also be converted into wave variables. These are generalized as "state quantities of a moving body." When these are used, instability of remote control can also be suppressed.

<Embodiment 2>
In the mobile remote control system 1000 of the first embodiment described above, the delay time T of the communication delay is assumed to be constant. On the other hand, in the case of remote control via a network such as the Internet, communication delays are rarely constant and vary. Such a system is illustrated as shown in FIG.

FIG. 10 is a block diagram of a mobile remote control system 2000 according to the second embodiment. FIG. 10 shows that the delay time of the communication delay from the remote control device RD to the mobile control device FD is T ₁ (t), and the delay time of the communication delay from the mobile control device FD to the remote control device RD is T ₂ (t). This means that the communication delay on the transmission path TL between the remote control device RD and the mobile object control device FD is not a constant delay, but depends on time t and varies depending on the direction of communication. In such a case, the integral of the wave variable becomes a different value on the transmitting side and the receiving side, causing a drift, and when remotely controlling the mobile object MV, the maneuverability decreases. Furthermore, there may be cases where the passivity of the transmission path TL is not satisfied.

In the second embodiment, a mobile remote control system 2000 that solves these problems will be described with reference to Non-Patent Document 2. The difference between the present disclosure and Non-Patent Document 2 is that Non-Patent Document 2 is a wave variable conversion for passivity defined by force and speed, whereas in the present disclosure, it is defined by speed information and the amount of maneuver. The difference is that the definition of passivity is different.

FIG. 11 schematically shows how the wave variable is distorted after passing through the communication delay when the communication delay is not constant. That is, the left diagram in FIG _. 11 is a waveform showing the time change of the wave variable u _m output by the remote control device RD, and the right diagram in FIG. This is a waveform showing a temporal change in the wave variable u _s delayed by the delay time T ₁ (t) of the element DL1.

From Figure 11, when the communication delay fluctuates, the integral value of the input signal wave on the transmitting side, shown in the left diagram, does not match the integral value of the output signal wave, shown on the right diagram, on the receiving side, which causes drift. I understand. In such a case, compensation can be achieved by transmitting or receiving compensation information for the wave variables at the same time as the wave variables.

FIG. 12 is a block diagram showing the configuration of the remote control device RD1 of the mobile object remote control system 2000 (FIG. 1) of the second embodiment, and FIG. 13 is a block diagram showing the configuration of the mobile object control device FD1. .

<Remote control device>
The remote control device RD1 shown in FIG. 12 includes a wave variable compensator R6 (first wave variable compensator) in addition to the components of the remote control device RD1 shown in FIG. 2. Furthermore, the wave variable converter R1 has a function of outputting, in addition to the wave variable _um , leader compensation information for compensating for distortion of the wave variable on the mobile object control device FD1 side.

The transmitter R3 has a function of transmitting the wave variable _um and the leader compensation information from the wave variable converter R1 to the network NW.

The receiving unit R2 has a function of receiving a wave variable v _m and follower compensation information for compensating the wave variable v _m from the network NW, and outputting it to the wave variable compensating unit R6.

The wave variable compensation unit R6 has a function of compensating the wave variable using the wave variable v _m from the receiving unit R2 and the follower compensation information, and outputting it as a compensation wave variable v _mc (first compensation wave variable). ing.

In addition, the same components as those of the remote control device RD explained using FIG. 2 are given the same reference numerals, and redundant explanation will be omitted.

<Mobile object control device>
The mobile body control device FD1 shown in FIG. 13 includes a wave variable compensator F6 (second wave variable compensator) in addition to the components of the mobile body control device FD1 shown in FIG. In addition to the wave variable _vs , the wave variable converter F1 has a function of outputting follower compensation information for compensating for distortion of the wave variable on the remote control device RD1 side.

The transmitter F3 has a function of transmitting the wave variable _vs and follower compensation information from the wave variable converter F1 to the network NW.

The receiving unit F2 has a function of receiving the wave variable _us transmitted from the remote control device RD1 via the network and the leader compensation information for compensating the wave variable _us , and outputting it to the wave variable compensating unit F6. are doing.

The wave variable compensation unit F6 has a function of compensating the wave variable using the wave variable _us and the reader compensation information from the receiving unit F2, and outputting it as a compensation wave variable _usc (second compensation wave variable). ing.

Note that other components that are the same as those of the mobile object control device FD explained using FIG. 3 are given the same reference numerals, and redundant explanation will be omitted.

<Operation of wave variable compensation section>
Next, the operation of the wave variable compensator will be explained. The wave variable compensator compensates the wave variables in order to guarantee the passivity of the system and reduce the effects of drift even if the communication delay varies.

Since the wave variable compensator F6 of the mobile body control device FD1 and the wave variable compensator R6 of the remote control device RD1 perform similar operations, the operation of the wave variable compensator F6 in the mobile body control device FD1 will be described below.

Consider restoring the wave variable us transmitted from the remote control device RD1 and received by the mobile object control device FD1 to _{a wave variable ust without} distortion. That is, consider a wave variable u _st that satisfies the following equation (18).

The wave variable compensator F6 of the mobile object control device FD1 compensates the wave variable u _s using the leader compensation information acquired from the receiver F2 so that the received wave variable u _s approaches the wave variable u _st as much as possible. do. There are multiple compensation methods, but when the remote control device RD1 transmits the transmission time t _m,last of the wave variable u _m as leader compensation information, and the wave variable u _s (t) received at time t, the following formula is used. By performing compensation using (19), the compensation wave variable u _sc (t) at time t can be obtained.

In the above equation (19), K _m is a gain having a positive value. This makes it possible to obtain distortion-free wave variables even when the communication delay varies, and as a result, the influence of drift can be reduced.

In addition, as a compensation method other than the above-mentioned time stamp, the value expressed by the following formula (20) is calculated by integrating the wave variable u _m in advance with the remote control device RD1, and the value is transmitted as the leader compensation information. .

The mobile object control device FD1 performs compensation using a formula similar to formula (19). In this way, by using the reader compensation information, it is possible to compensate for the influence of drift, and even when communication delay fluctuates, it is possible to avoid system instability.

In the above, the processing in the wave variable compensator F6 of the mobile object control device FD1 has been explained, but the wave variable compensator R6 of the remote control device RD1 similarly compensates the wave variable from the mobile object control device FD1, The influence of drift on the remote control device RD1 side can be reduced.

Note that if the communication delay fluctuates, passivity may not be guaranteed in the transmission path even when wave variable conversion is used. For example, if communication is interrupted, the last received wave variable continues to be output, resulting in infinite energy generation and instability.

The wave variable compensator monitors the passivity of the system, and if the passivity of the system is not maintained, processing is performed to output the wave variable as 0. A determination as to whether passivity is maintained is made by monitoring the balance of virtual energy with reference to Non-Patent Document 2. Below, the operation of the wave variable compensator will be explained based on the virtual energy balance monitor.

In order for the system shown in FIG. 10 to be passive, according to equation (10), the virtual energy stored in the system must satisfy the following equation (21).

In the above formula (21), E _total (0) is the virtual energy stored in the system in the initial state. When wave variables are used, the virtual energy E s (t) at time t accumulated in the wave variable compensation unit F6 of the mobile object control device FD1, and the virtual energy E _s (t) accumulated in the wave variable compensation unit R6 of the remote control device RD1. The virtual energy E _m (t) at time t can be calculated using the following equations (22) and (23), respectively.

In Equation (23), t _s,last is the transmission time of the wave variable vs _s in the mobile object control device FD1. The wave variable compensator F6 and the wave variable compensator R6 monitor the virtual energy E _s (t) and the virtual energy E _m (t), respectively, and set the upper limit of the virtual energy E _s,lim and the upper limit E _{m of the virtual energy, respectively.} If the value exceeds _lim , the compensation wave variable u _sc and the compensation wave variable v _mc are output as 0 to prevent energy from being accumulated indefinitely.

That is, the wave variable compensator F6 and the wave variable compensator R6 perform calculations expressed by the following equations (24) and (25), respectively.

Note that the upper limits of virtual energy E _s,lim and E _m,lim need to be set within a range where the system satisfies passivity.

At this time, the virtual energy E _total accumulated in the system is expressed by the following equation (26), so the passivity of the system is satisfied and stable.

By using the wave variable compensator as shown in this embodiment, it is possible to realize an efficient remote control system for a mobile object that does not become unstable even when communication delay fluctuates.

Note that by introducing a buffer into the receiving section and making the communication delay pseudo-constant, it is relatively easy to cope with the case where the communication delay fluctuates even in the remote control device RD and mobile control device FD of the first embodiment. It is also possible to do so. That is, a buffer is provided in the receiving section R2 of the remote control device RD, and the image information and wave variables from the mobile object control device FD are held for a predetermined time T and output after T seconds. Similarly, a buffer is also provided in the receiving section F2 of the mobile object control device FD.

Although such a method can easily cope with variations in communication delay, performance may degrade with the longest time delay. Note that even if a buffer is introduced in the receiving section, communication may be interrupted, so it is necessary to provide a wave variable compensation section that can monitor virtual energy.

<Embodiment 3>
In the mobile object remote control systems 1000 and 2000 of the first and second embodiments described above, by performing wave variable conversion and wave variable compensation, the operator OP can move as if there were no delays due to communication delays. It becomes possible to remotely control the body MV. However, by further presenting auxiliary information regarding the movement of the mobile object MV to the operator OP, it becomes possible to provide a more accurate mobile object remote control system. Below, as a mobile object remote control system of Embodiment 3, a configuration will be described in which movement prediction of a mobile object MV is performed and the information is superimposed on a presented image.

FIG. 14 is a block diagram showing the configuration of remote control device RD2 in mobile remote control system 3000 (FIG. 1) according to the third embodiment. Compared to the remote control device RD shown in FIG. 2, a movement prediction unit R7 is added. Other components that are the same as those of the remote control device RD explained using FIG. 2 are designated by the same reference numerals, and redundant explanation will be omitted. Note that the configuration of the mobile object control device FD is the same as the mobile object remote control system 1000 (FIG. 1) of the first embodiment.

<Remote control device>
The movement prediction unit R7 has a function of predicting the movement of the mobile body MV using the sensor information from the reception unit R2 and a prediction model that predicts the future state of the mobile body MV, and outputting the result to the image generation unit R4. has.

The image generation unit R4 uses the image information from the reception unit R2, the speed information from the wave variable conversion unit R1, and the movement prediction from the movement prediction unit R7 to generate a presentation image to be presented to the operator OP, It has a function of outputting the presented image to the image display device ID.

<Operation of movement prediction unit>
The operation of the movement prediction unit R7 will be explained in more detail below. When the mobile object MV is a car, by indicating to the operator OP what positions the front wheels and rear wheels will pass in the future, the operator OP can perform more precise maneuvering.

FIG. 15 is a diagram illustrating movement prediction of the left and right front wheels when the mobile object MV is a car. The movement prediction unit R7 performs movement prediction by using a prediction model that predicts the future state of the mobile object MV. In the case of a car, movement prediction can also be made using Equation (7) representing the dynamic characteristics of a car based on a two-wheel model, but a simpler prediction method using a kinematics model will be described below.

Below, a kinematic model using the two-wheel model shown in FIG. 8 will be explained. The kinematics model of a car is expressed by the following equation (27).

In the above equation (27), X _v and Y _v are the positions of the center of gravity of the vehicle in the inertial coordinate system. The meanings of other symbols are explained in Equation (7), so the explanation will be omitted. Further, the left side of Equation (27) represents the time differentiation of X _v , Y _v , ψ, and β.

By integrating Equation (27) for a predetermined prediction time _Tf , it is possible to predict the position and orientation of the center of gravity. Since the positions of the left and right front wheels relative to the center of gravity are fixed, if the future position and orientation of the center of gravity are obtained, the positions of the left and right front wheels can be predicted by appropriate coordinate transformation.

FIG. 16 is a diagram showing an example of an image generated by the image generation unit R4 in the remote control device RD2 of the third embodiment. In FIG. 16, a virtual camera is placed on the XYZ three-dimensional virtual space, and image information acquired from the receiving unit R2 is pasted in the visible range at the virtual camera position VCP. In the example of FIG. 16, the predicted trajectories LPT and RPT of the left and right front wheels obtained by the movement prediction unit R7 are superimposed on the image information II including the left and right lanes LN and the horizontal line HL of the road surface.

By rotating the posture of the camera at the virtual camera position VCP by the rotation angle θ _m , the image presented to the operator OP can be changed from the past image at time 0 to the current image at time t. is the same as in Embodiment 1 described using FIG. 10, and by adding information on the predicted trajectories LPT and RPT to this, the operator OP can perform remote control with higher accuracy.

In addition, although the method of predicting the movement of the left and right front wheels has been described above, it is also possible to similarly predict the movement of the left and right rear wheels and superimpose it on the image. Further, the state predicted by the movement prediction unit R7 can be speed or angular velocity, and the predicted information can also be superimposed on the image. In addition, by making predictions that improve maneuverability, highly accurate remote control becomes possible.

In addition, when the moving object is not a car, for example, in the case of an airplane, the movement prediction unit R7 predicts the position of the tip of the wing, and the movement prediction result can be superimposed on the image with high accuracy. This allows for easy maneuvering. By predicting the movement of a moving object in this way, highly accurate remote control becomes possible.

<Hardware configuration>
The components of the remote control devices RD to RD2 and the mobile object control devices FD and FD1 in the first to third embodiments described above can be configured using a computer, and can be configured by the computer executing a program. Realized. That is, the remote control devices RD to RD2 and the mobile object control devices FD and FD1 are realized by the processing circuit 60 shown in FIG. 17, for example. A processor such as a CPU (Central Processing Unit) or a DSP (Digital Signal Processor) is applied to the processing circuit 60, and the functions of each part are realized by executing a program stored in a storage device.

Note that dedicated hardware may be applied to the processing circuit 60. When the processing circuit 60 is dedicated hardware, the processing circuit 60 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), or an FPGA (Field-Programmable Circuit). Gate Array), or a combination of these.

The functions of each component of the remote control devices RD to RD2 and the mobile object control devices FD and FD1 can be realized by separate processing circuits, or these functions can be realized collectively by a single processing circuit. .

Further, FIG. 18 shows a hardware configuration in the case where the processing circuit 60 is configured using a processor. In this case, the functions of each part of the remote control devices RD to RD2 and the mobile control devices FD and FD1 are realized by a combination of software and the like (software, firmware, or software and firmware). Software etc. are written as programs and stored in the memory 62. A processor 61 functioning as a processing circuit 60 realizes the functions of each part by reading and executing a program stored in a memory 62 (storage device). That is, it can be said that this program causes the computer to execute the procedures and methods for operating the components of the remote control devices RD to RD2 and the mobile object control devices FD and FD1.

Here, the memory 62 includes, for example, nonvolatile or volatile semiconductor memory such as RAM, ROM, flash memory, EPROM (Erasable Programmable Read Only Memory), and EEPROM (Electrically Erasable Programmable Read Only Memory), and HDD (Hard Disk Memory). The storage medium may be a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, a DVD (Digital Versatile Disc) and its drive device, or any storage medium that will be used in the future.

The above describes the configuration in which the functions of each component of the remote control devices RD to RD2 and the mobile object control devices FD and FD1 are realized by either hardware, software, or the like. However, it is not limited to this, and some components of the remote control devices RD to RD2 and mobile object control devices FD and FD1 can be realized with dedicated hardware, and some other components can be realized with software, etc. You can also. For example, for some components, the functions are realized by the processing circuit 60 as dedicated hardware, and for some other components, the processing circuit 60 as the processor 61 executes the program stored in the memory 62. The function can be realized by reading and executing it.

As described above, the remote control devices RD to RD2 and the mobile control devices FD and FD1 can implement the above-mentioned functions using hardware, software, etc., or a combination thereof.

Although the present disclosure has been described in detail, the above description is illustrative in all aspects, and the present disclosure is not limited thereto. It is understood that countless variations not illustrated can be envisioned without departing from the scope of this disclosure.

Note that, within the scope of the disclosure, the embodiments of the present disclosure can be freely combined, and the embodiments can be modified or omitted as appropriate.

Claims (8)

A remote control device that remotely controls a mobile object based on control information from a pilot,
A first wave variable obtained by converting the state quantity of the moving object into a wave variable, which is output from a moving object control device that outputs a moving object control amount for controlling the moving object, and image information around the moving object. a first receiving unit that receives the information via the transmission path;
a first wave variable conversion unit that converts the first wave variable and the control amount based on the control information into wave variables, and outputs the converted state quantity and second wave variable, respectively;
a first transmitter that transmits the second wave variable to the mobile body control device via the transmission path;
and an image generation unit that generates an image to be displayed on an image display device based on the image information and the converted state quantity output from the first wave variable conversion unit. Device. The state quantity is
The remote control device according to claim 1 , comprising angular velocity, translational velocity, angular acceleration, translational acceleration, or position information of the moving body. a first wave variable compensator that compensates for a drift in which an integral value differs between an input signal wave input to the transmission path and an output signal wave output from the transmission path due to variations in communication delay in the transmission path; has
The first wave variable compensator includes:
Compensating the first wave variable and outputting it as a first compensated wave variable;
The first wave variable conversion section is
The remote control device according to claim 1 or 2 , wherein the first compensation wave variable is converted into a wave variable to obtain the converted state quantity. A mobile remote control system comprising the remote control device according to claim 1,
The mobile body control device includes:
a second receiving unit that receives the second wave variable output from the remote control device via the transmission path;
a second wave variable conversion unit that converts the second wave variable and the state quantity of the moving object into wave variables, and outputs the converted object as the moving object control amount and the first wave variable, respectively;
a second transmitter that transmits the first wave variable to the remote control device via the transmission path,
The mobile body is
A mobile object remote control system, comprising: a mobile object control section that controls driving of the mobile object using the mobile object control amount output from the second wave variable conversion section. A mobile body control device communicating with the remote control device according to claim 1, comprising:
a second receiving unit that receives the second wave variable output from the remote control device via the transmission path;
a second wave variable conversion unit that converts the second wave variable and the state quantity of the moving object into wave variables, and outputs the converted object as the moving object control amount and the first wave variable, respectively;
A mobile body control device, comprising: a second transmitter that transmits the first wave variable to the remote control device via the transmission path. An image display device that displays the image generated by the image generation unit in the remote control device according to claim 1 . A movable body, comprising a movable body control unit that controls driving of the movable body using the movable body control amount output from the second wave variable conversion unit in the movable body control device according to claim 5 . A remote control method for remotely controlling a mobile object based on control information from a pilot, the method comprising:
A first wave variable obtained by converting the state quantity of the moving object into a wave variable, which is output from a moving object control device that outputs a moving object control amount for controlling the moving object, and image information around the moving object. and received via the transmission path,
converting the first wave variable and the control amount based on the control information into wave variables, and using the converted state variables and second wave variables, respectively;
transmitting the second wave variable to the mobile body control device via the transmission path;
A remote control method that generates an image to be displayed on an image display device based on the image information and the converted state quantity.

Images