KR20020096323A - Teleoperaion system with streaming buffer - Google Patents

Abstract

PURPOSE: A system for a remote operation using a streaming buffer is provided to minimize a control input cycle by maintaining a system time delay constantly using a buffer and to make a streaming control possible by increasing a control band width. CONSTITUTION: In an Internet remote operating system comprising a master unit(100) and a slave unit(300), a control-input device(120) inputs a reference control. An Internet connection unit(140) transmits a control signal inputted in the control input device(120) to packet data through the Internet. A connection socket(330) and a connection manager(340) receive the packet data being transmitted through the Internet(200). A streaming buffer unit(320) recreates the received packet data as a control command having a predetermined time delay. A local controller(310) controls a slave robot(350) in accordance with the recreated control command. The slave robot(350) performs a feedback of an operating state to the local controller(310).

Description

Remote operation system with streaming buffer {Teleoperaion system with streaming buffer}

The present invention relates to an Internet remote control system, and in particular, to keep the system time delay constant using a buffer, thereby minimizing the control input period and improving the control bandwidth to stream. The present invention relates to a remote control system using a streaming buffer that enables streaming control.

Recently, attention has been focused on the development of a teleoperation system that operates at a remote location to perform a specific task. In many areas, such as telesurgery, remote sensing and remote maintenance / repair, applications are increasing. Therefore, there is a need for a remote control system that can operate accurately and quickly at a remote location.

However, the use of a private line as a command transmission medium to a remote location has caused a large production and maintenance cost, requiring a remote operation system that can accurately and safely use remote operation at a competitive cost. There is a demand for the development of a remote control system using the public line, the Internet, wireless Internet, and Bluetooth.

Research on remote operation using the Internet has been actively conducted since 1999. When using the Internet as a transmission medium, there are various advantages in addition to the above-mentioned costs, but the biggest difficulty in developing a remote control system using the Internet is first, there is a time delay, and secondly, The variation in time delay is due to the characteristics of the Internet. Many conventional studies have considered these two features as one, and have tried to reduce their properties in many ways, with considerable advancement in their own right, but have not fundamentally solved these two problems, and have completed the actual operating instructions. It was difficult to carry out. Therefore, due to this performance problem, the application range of the remote control system using the Internet is reduced.

Figure 1a, Figure 1b is a block diagram showing a conventional remote control system using the Internet, showing two representative prior art using the Internet as a transmission medium.

1A is a block diagram illustrating a command-based teleoperation system. This is a simple task-based remote operation that allows the user to execute a simple task command that is predefined in consideration of two characteristics of the transmission medium (the existence of a time delay and the variability of the time delay). When the work command input by the remote operator 10 is transmitted to the work manager 30 on the receiving side through the transmission medium 20, the work manager 30 transmits the work command to the first controller 40. The robot 50 is driven. Naturally, the control of the work is inconvenient, and the next work command is transmitted only when each work is completed. As a result, it is difficult to execute continuous commands and the control bandwidth is narrow. Therefore, there is an advantage in that a very simple system can be configured in an environment that is not significantly affected by the time delay, but due to its performance, there is a narrow application range.

1B is a block diagram illustrating an event-based teleoperation system. Unlike the remote control system based on the command, the system is configured to enable continuous remote operation. When the reference input for the control command is made by the remote operator 60, the control command is transmitted to the second control unit 80 through the transmission medium 20, and the second control unit 80 according to the transmitted control command. The robot 90 is driven. When each control command (position information, speed information, etc.) is performed, it is notified to the remote operator 60, and after that, the next control command is sent to be performed. However, due to the inconsistent time intervals between the control commands, chattering occurs in the remote control and has difficulty in accurate control.

The two systems described above also had the following problems.

First, there is still a time delay with the remote.

Second, because the time delay is still variable, the system becomes unstable, vibration occurs, and malfunctions due to incorrect inputs occur.

Third, due to the second feature, it is difficult to combine robust control and predictive control to improve the first feature.

Fourth, a new controller must be newly created for each new remote control system.

An object of the present invention for solving the above problems is to use a buffer to maintain a constant system time delay (Constant), thereby minimizing the control input period, improve the control bandwidth streaming control It is to provide a remote control system using a streaming buffer that enables (Streaming Control).

Figure 1a, 1b is a block diagram showing a remote control system using a conventional Internet.

Figure 2 is a block diagram showing a remote control system according to the present invention.

3 is a flowchart illustrating a buffer management process of a buffer manager according to the present invention.

4 is a block diagram illustrating a remote control system for feedback in accordance with the present invention.

5a to 5f are diagrams showing the test results of the remote control system according to the present invention.

<Description of Symbols for Major Parts of Drawings>

10, 60: remote operator 20: transmission medium

30: job manager 40: first control unit

50, 90: robot 80: second control unit

100: master unit 110: operator

120: control input device 130, 330: connection socket

140, 340: access manager 160, 360: Internet connection

200: transmission medium 300: slave unit

310: local controller 320: streaming buffer unit

321: buffer manager 322: buffer

323: control timer 324: instruction regeneration unit

A feature of the present invention for achieving the above object is the Internet remote control system consisting of a master unit and a slave unit, the master unit is a control input device for the reference control input, the control signal input from the control input device via the Internet Internet connection unit for transmitting the packet data, the slave unit Internet connection unit for receiving the packet data of the master unit transmitted through the Internet, and regenerating the packet data received from the Internet connection unit with a control command having a certain time delay A streaming buffer unit, a local controller controlling the controlled device according to the control command regenerated by the streaming buffer unit, and a feedback operation condition fed back to the local controller while operating according to a control command of the local controller. It includes a control device In the remote control system using the streaming buffer which.

Prior to the detailed description of the present invention, in this specification, a time delay of a packet on a network is defined as a packet time delay, and the time between each packet is referred to as jitter (hereinafter referred to as jitter). define. In addition, the time delay of the entire system between the master unit and the slave unit is referred to as a system time delay to prevent mixing.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

2 is a block diagram illustrating a remote control system according to the present invention, and is mainly composed of a master unit, a slave unit, and the internet as a data transmission medium therebetween.

The master unit 100 includes an operator 110, a control input device (master robot) 120, a connection manager 140, which is an internet connection unit, and a connection socket 130, and the slave unit 300. ) Is composed of an internet connection unit 330, 340, a streaming buffer unit 320, a local controller 310, and a slave robot 350.

An asynchronous socket or a synchronous socket may be used as the connection sockets 130 and 330 of the master unit 100 and the slave unit 300.

The master unit 100 is operated by the operator 110, the control input device 120 is operated, the connection based (Connection-Oriented) protocol (TCP / IP) based connection for stable and fast delivery of control signals The operation command data is packetized using the socket 130 and transferred to the slave unit 300. The connection manager 140 is responsible for network related functions such as data transmission and reception, connection management, and user authentication.

The slave unit 300 transmits the data received from the Internet connection unit 330, 340 to the streaming buffer unit 320. The data is reconstructed into a control input having a predetermined interval by the streaming buffer unit 320 and then processed as a reference input of the local controller 310 to control the slave robot 350.

The slave robot 350 is a controlled device that finally receives the control command of the master unit. In the embodiments of FIGS. 2 and 4, the slave robot 350 is a robot, but any device capable of receiving and operating the control command of the master unit may be used. You may use it. For example, if a sensor is used instead of a robot, it can be easily applied to the telemetry field.

The local controller 310 may use various conventionally developed local controllers (PID controller, sliding mode controller, etc.), and the control input device 120 and the internet connection unit 130, 140 of the master unit 100. The command transmission media () may be used for the Internet, the wireless Internet, Bluetooth, and the like.

The streaming buffer 320 may include a buffer 322 for storing packet data transmitted from the master unit 100; A control timer 323 for generating a real time clock to maintain a predetermined time delay with the master unit 100; A command reproduction unit 324 for regenerating data stored in the buffer 322 into a control command according to the control timer 323; And a buffer manager 321 that monitors the buffer 322 and controls the buffer initialization, definition of a predetermined time delay, real-time buffer resizing, and the instruction regeneration unit 324. It consists of).

Looking at the function of the buffer manager 321 among the modules, it is possible to define the initial buffer size to determine the length of a constant time delay according to the size of the defined buffer and the control input period.

(One)

In addition, as shown in Equation (1), when the sum of the amount of change in time delay (jitter _{transmitting side} -jitter _{receiving side} ) is within a range that the buffer can take (current buffer size x buffer period), stable operation can be ensured. If it is out of this range, the buffer size is resized in real time to allow stable remote operation while minimizing chattering.

3 is a flowchart illustrating a buffer management process of a buffer manager according to the present invention.

If the network situation suddenly worsens and is out of the range that the buffer can compensate for, the size of the buffer becomes zero and the equation (1) cannot be satisfied. In such a case, the slave unit 300 may cause a fatal congestion or an Amp Fault as in the existing system, and chattering may occur even if the network condition improves within a short time. Since this chattering prevents accurate reference input tracking, a buffer manager is needed to reliably increase the buffer size when the buffer is exceeded.

The buffer management process of the buffer manager 321 first initializes the buffer, receives packet data from the Internet connection unit (S400), and analyzes the packet data (Pharsing) (S410). If the data of the analysis result control command is inserted into the buffer 322, the data from the connection socket 330 (S420), and performs a buffer preparation test (S430). The buffer preparation test S430 is an operation for filling the buffer until the initial buffer size is reached, and is performed only when increasing the data in the initial buffer.

When the buffer is prepared, the validity of the buffer is determined using Equation (1) (S440). If the buffer is valid, the latest data of the current buffer is used as the control reference input (S450). If it is not valid, the data of the previous step is copied because it is outside the limit of the buffer and used as a control reference input (S460). When the control command is generated in the command regenerator 324 according to the control reference input and the control of the slave robot 350 is performed by the local controller 310 (S470), the current buffer is deleted (S480).

If the buffer is invalid, new data will enter the buffer while controlling the robot by duplicating the data from the previous step. This method effectively increases the size of the buffer with little chattering, and the jitter's elasticity allows the buffer to recover. The buffer validity determination S440 is performed at each control period differently from the buffer preparation test S430.

4 is a block diagram showing a remote control system for feedback according to the present invention.

The streaming buffer unit 320a of the illustrated master unit 100 cross-connects the streaming buffer unit 320 of FIG. 2 to the Internet connection unit 160 of the master unit 100, and controls commands of the master unit 100. When the reflection data of the slave robot 350 according to the present invention is transmitted to the master unit 100, by having a predetermined time delay, it is to implement a feedback without deterioration of workability and stability. The echo data refers to status, force, acceleration information, etc. collected from various sensors of the slave robot 350. This enables the realization of the telepresence concept. At this time, the control input device 120 of the master unit 100 needs to be provided with a separate device for implementing the feedback.

5A to 5F are diagrams showing the test results of the remote control system according to the present invention. FIG. 5A is a graph showing a reference curve and an actual curve, FIG. 5B is a packet delay graph, and FIG. 5C is a real angular velocity graph. 5D shows an actual output (voltage) graph, FIG. 5E shows a buffer size without a buffer manager, and FIG. 5F shows a buffer size with a buffer manager.

5 is an experiment for long-distance remote operation, the distance between the master unit and the slave unit is about 15,000km. The control input period was 10ms, the initial buffer size was 10 (100ms), and 6 degrees of freedom slave robot was used.

The control input period and the initial buffer size are determined according to the distance between the master unit and the slave unit and the network conditions. In general, the control input period is 10ms and the initial buffer size 5 (50ms) in the short and short distances. Control input period 10ms, initial buffer size 10 (100ms). In addition, when using the wireless Internet as a command transmission medium, a value of a control input period (20 ms) and an initial buffer size of 5 (100 ms) is given.

The system according to the invention is also capable of short-range and short-range remote control, in which case it shows better performance than the long-distance remote operation. This is because the change in jitter value is small due to the short network distance, which is obvious. Therefore, in the following description, only the case of long distance remote operation will be described.

As shown in FIG. 5A, the initial packet time delay was 122ms, and the successful control was possible with a constant time delay as large as the buffer size (100ms). As a result, continuous control of the slave robot was possible while keeping the overall system time delay constant at about 0.22 sec (222 msec).

As shown in FIG. 5B, the packet time delay exceeded the buffer limit at about 23 sec. However, the buffer manager effectively increased the size of the buffer four times, enabling continuous stable control. The total system delay time after 23 sec was about 0.26 sec.

As can be seen in FIGS. 5C and 5D, chattering hardly occurs, and the buffer size can be effectively increased.

In the case of FIG. 5E, only the buffer is used without the buffer manager 321. The buffer state when the buffer is exhausted is shown. In FIG. 5F, the buffer situation when the buffer manager 321 is introduced is shown. have. As shown in FIG. 5E, in the absence of the buffer manager 321, if an unexpected level of sudden traffic occurs, the buffer size may be 0 or less (about 22.8 sec or less). In this case, since the meaning of the buffer becomes meaningless, chattering occurs as in the conventional remote control system. However, when the buffer manager 321 is introduced, the size of the buffer is effectively increased, so that the buffer size is maintained at a constant level as in about 23 sec after FIG. 5F.

As described above, the remote control system according to the present invention has a constant system time delay, but has a relatively large value of about 0.2 sec. To overcome this problem, if you apply Robust Control and Predictive Control, you can expect a better effect.

As described above, the present invention has the following effects.

First, the streaming buffer is introduced to have a constant control input period. The system time delay is fixed, so that remote operation can be performed smoothly and accurately, and robust control and predictive control can be performed.

Second, the variable system delay time can be constantly converted by using the streaming buffer, thereby minimizing the reference input period. That is, a new operation can be performed in a shorter time, and accurate reference input can be followed.

Third, the previous two features simplify the system and significantly improve the control bandwidth.

Fourth, it is possible to use the Internet, wireless Internet, Bluetooth, etc. as a command transmission medium, and various existing local controllers (PID Controller, Sliding Mode Controller, etc.) can be used. There is this.

Fifth, if the streaming buffer is also cross-configured on the master side, a stable feedback system can be easily realized.

Claims (7)

In the Internet remote control system consisting of a master unit and a slave unit, The master unit is composed of a control input device for the reference control input, and an Internet connection unit for transmitting the control signal input from the control input device as packet data through the Internet, The slave unit includes an internet connection unit for receiving packet data of the master unit transmitted through the internet, a streaming buffer unit for regenerating packet data received from the internet connection unit with a control command having a predetermined time delay, and in the streaming buffer unit. A streaming buffer comprising a local controller controlling a controlled device according to a regenerated control command, and a controlled device operating according to a control command of the local controller and feeding back an operation status to the local controller. Remote control system using. The apparatus of claim 1, wherein the streaming buffer unit comprises: a buffer configured to store packet data transmitted from the master unit; A control timer for generating a real time clock to maintain a predetermined time delay with the master unit; A command regenerator for regenerating data stored in the buffer into a control command according to the control timer; And And a buffer manager for controlling the initialization of the buffer, the definition of a predetermined time delay, the real-time buffer size adjustment, and the instruction regeneration unit while monitoring the status of the buffer. The buffer manager of claim 2, wherein the buffer manager initializes the buffer first, and fills the data from the Internet connection until the initial buffer size is reached. When the buffer is ready, the buffer manager determines the validity of the buffer. A remote control system using a streaming buffer, characterized in that the latest data is used as a control reference input, and if not valid, the data of the previous step is copied and used as a control reference input. 4. The buffer validity of claim 3, wherein the buffer validity of the buffer manager is Remote control system using a streaming buffer, characterized in that determined by. here, Is the amount of time change between each packet. 4. The remote control system of claim 3, wherein the initial buffer size is determined according to a distance between the master unit and the slave unit and a network condition. The remote control system of claim 1, further comprising a streaming buffer unit configured to add a time delay of feedback data in addition to the Internet connection unit of the master unit. The remote control system using a streaming buffer of claim 1, wherein the command transmission medium between the control input device and the Internet connection unit is one selected from the Internet, wireless Internet, and Bluetooth.