JPH1190872A - Remote control device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve operability by transforming the type or content of the transmission data with a transmission data transforming means in response to a work mode judged by a work mode judging means. SOLUTION: The slave side controller 16 of a remote control device has a work mode judging function judging the work mode being presently executed by a slave arm 2 and a data transforming function and transforms the type or content of user data 10 in response to the judged result. For the judgment of the work mode, a threshold value is set for the size of the positional drift quantity between the target position of a master arm 6 and the observed position of the slave arm 2, for example. If the size is the threshold value or below, the movement is judged as a space shift. If the size exceeds the threshold value, the movement is judged as a restrained motion. When the work mode is judged as the restrained motion work, a standard user data structure is used. When the work mode is judged as the space shift work, a user data structure suppressing the number of effective digits of control/ observation data and increasing the ratio of image data that much is used.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a remote control device for remotely controlling a work execution device such as a slave arm disposed in a work environment by a control device such as a master arm disposed in a control environment. BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a remote control device which enables the use of a normal telephone line network for data transmission between a control environment and a work environment, and can expand the range of application.

[0002]

2. Description of the Related Art The expansion of information communication networks and the development of computer network technologies have not only greatly affected the progress of multimedia technologies, but are also giving new possibilities to remote control of robots.

For example, recently, a master arm and a slave arm are connected to a large-capacity optical fiber communication cable,
Attempts have been made to remotely control a slave arm located at a distance of several km or more by connecting via a communication line such as Ethernet or the Internet.

[0004] If such technology becomes available, robots working in environments that are dangerous for humans to work directly, such as nuclear power plants, space, and oceans, as well as robots,
By assuming a remote control system using a commercial line, the system can be applied to a so-called human coexistence robot in which a human living space and its operation range directly intersect, and can be applied to the field of medical care and welfare.

By the way, there are various communication lines which can be used for transmitting and receiving signals between the above-mentioned master arm and slave arm.

For example, Broadband ISDN (B-ISD) expected as a next-generation network
Since N) has a communication capacity of 150 Mbps or more, there is no problem with time delay due to communication or limitation of communication capacity. However, at present, such networks are not widespread, and cannot take advantage of the wide area of the networks.

[0007] The Internet, which has become the most popular computer network, is the World Wide Network.
Web (World Wide Web: WWW) and Browser (Brow)
By using the ser) interface, an intuitive and easily operable interface can be realized even in remote operation. However, in the Internet, there are problems such as loss of information and instability of communication speed.

On the other hand, ISDN (Integrated Services Digital Network)
Is spreading as a multipurpose information transmission means including multimedia.

[0009] However, the ISDN has a communication capacity of about 1.5 Mbps for the primary group speed interface, but has problems such as the necessity of laying an optical fiber cable and a high communication fee.

[0010] From the above, it would be very convenient if existing telephone network equipment (telephone pair conductor) having low communication cost could be used as a signal transmission path of a remote control device.

However, the existing telephone network has a communication capacity of 128 kbps or less. For this reason, the reality is that the appearance of a remote control technique capable of overcoming the low communication capacity described above without lowering the operability is desired.

[0012]

As described above, even if any communication line is used as a signal transmission line for remote control, the communication capacity is limited, and there is a possibility that operability may be reduced.

Accordingly, the present invention provides a remote control device capable of stably and remotely controlling a work execution device such as a slave arm using a control device such as a master arm under the condition of a limited communication capacity. It is intended to provide.

[0014]

In order to achieve the above object, a remote control device according to the present invention is provided with a work execution device arranged in a work environment to execute work, and a remote control device arranged in a control environment to execute the work. A control device for remotely controlling the device, an image acquisition device arranged in the work environment to capture a work state of the work execution device as an image, and an image arranged in the control environment and acquired by the image acquisition device. An image presenting device, control / observation information necessary for steering the work executing device, and image presentation to the image presenting device, which is provided between the work executing device and the image acquiring device and the steering device and the image presenting device. A data transmission device for transmitting image information necessary for the operation, a work mode determination unit for determining a work mode executed by the work execution device, and a work mode determined by the work mode determination unit In response and a transmission data deforming means for deforming the format or content of the data to be transmitted via the data transmission device.

The work form determining means determines that the work form is a spatial movement work form when the deviation between the position / posture data of the work execution device and the position / posture data of the control device is equal to or less than a predetermined value. It is also possible to determine that it is the work form of the constrained motion when it is larger than a predetermined value, or to determine that it is the work form of spatial movement when the force / moment data at the tip of the work execution device is less than a predetermined value. However, when the value is larger than the predetermined value, it may be determined that the operation mode is the work mode of the restricted motion.

Further, the transmission data transforming means is adapted to determine the size of the control / observation information area and the image information occupied in the data transmitted / received at one time in accordance with the work mode determined by the work mode determination means. The size of the area for the image information may be changed by changing the size of the entire data to be transmitted and received at one time according to the work mode determined by the work mode determination unit. It may change.

Further, the transmission data transforming means selects the control / observation information item in the data transmitted / received at one time according to the work mode determined by the work mode determination means, thereby obtaining the image information. The size of the area for the image may be changed.

Further, the transmission data transforming means includes:
According to the work mode determined by the work mode determination means,
By changing the size of the image acquired by the image acquisition device, the size of the image data for one image may be changed, or according to the work mode determined by the work mode determination unit, The size of the image data for one image may be changed by changing the data compression ratio for the image obtained by the image obtaining apparatus.

In the remote operation device according to the present invention, there is provided a work mode determining means for determining the work mode being executed by the work execution apparatus, and the form or content of the data transmitted through the data transmission device is determined. There is provided transmission data deforming means for deforming according to the work mode determined in (1).

The form of work performed by the work execution device can be divided into several forms. In the present invention, for example, space movement and restraint movement are divided.

Spatial movement refers to carrying an object along a trajectory in three-dimensional free space. When performing this spatial movement work, the movement accuracy of the tip of the work execution device is not so required. That is, in the case of this spatial movement work, image information of where the transported object is currently moving is required from the tip movement information of the work execution device. Therefore, in order to realize good operability during this space movement operation, the control / observation period between the operation execution device and the control device may be long, but it is important to shorten the image update period. .

On the other hand, the restraining movement refers to a state in which the transported object is restrained by an external environment. Typical examples of such a state include an operation of inserting a rod into a hole and assembly of a puzzle. In this restraint work, it is necessary to control the mutual forces acting between the object and the external environment. That is, it is necessary to transmit the mutual force between the object carried by the work execution device and the external environment to the operator who operates the control device with a sense of realism. Therefore,
To achieve good operability during this restraint work,
The image update cycle may be long, but it is important to shorten the control / observation cycle between the work execution device and the control device.

From such a viewpoint, the remote operation device according to the present invention is provided with a work mode determination means for determining the work mode being executed by the work execution apparatus, and the determination result of the work mode determination means is used to transform the transmission data. In this way, the form and contents of the data transmitted through the data transmission device are dynamically changed by this modification means, so that the limited communication capacity of the communication line can be effectively used, and the stable and operable A good remote control system has been realized.

Here, changing the format and contents of data (hereinafter, referred to as user data) transmitted and received between the work environment and the control environment means adjusting the control / observation cycle and the image update cycle. That means.

When shortening the image update cycle, one of the following methods is specifically adopted. (1) By reducing the number of significant figures for control and observation information,
The area for image information in user data transmitted and received at a time is increased.

(2) By increasing the size of the entire user data, the area for image information in the user data transmitted and received at one time is increased.

(3) The control / observation information is divided into a plurality of pieces, and user data is transmitted and received a plurality of times to make one control cycle, so that an area for image information in the user data transmitted and received at one time is increased. I do.

(4) By reducing the video size of one image obtained by the image acquisition device, the size of image data to be transmitted and received is reduced.

(5) The size of image data to be transmitted and received is reduced by increasing the image compression ratio for one image obtained by the image acquisition device.

The work mode of the work execution device is determined by
It is recognized from the operation of the operator who operates the control device and the behavior of the work execution device.

Specifically, for example, one of the following methods is adopted.

(1) A threshold value is set for the magnitude of the amount of displacement between the target position of the control device and the observation position of the work execution device,
If it is less than the threshold value, it is determined to be a spatial movement, and if it exceeds the threshold value, it is determined to be a restricted motion.

(2) When the force / moment data at the tip of the work execution device is equal to or less than a predetermined value, it is determined that the movement is spatial, and when it is greater than the predetermined value, it is determined that the movement is restricted.

[0034]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a conceptual diagram of a remote control device according to one embodiment of the present invention. In FIG. 1, a portion surrounded by a two-dot chain line 1 indicates a work environment. The work environment 1 includes a slave arm 2 as a work execution device.
A video camera 3 as an image acquisition device that captures the working state of the slave arm 2 as an image is provided. Further, the work environment 1 includes a data transmission device, inputs and outputs control data and observation data necessary for controlling the slave arm 2 by, for example, a force feedback type, and outputs an image signal obtained by the video camera 3. A slave controller 4 for processing is also arranged.

On the other hand, in FIG. 1, a portion surrounded by a two-dot chain line 5 indicates a control environment set at a position far away from the work environment 1. This operating environment 5 includes slaves
A master as a control device for controlling the arm 2
A video monitor 7 is provided as an image presenting device that presents images acquired by the arm 6 and the video camera 3 to the operator. The control environment 5 includes a data transmission device, a master control device 8 for inputting and outputting control data and observation data necessary for controlling the slave arm 2 by force feedback, and controlling the video monitor 7. Is arranged.

The data transmission device built in the slave side control device 4 and the data transmission device built in the master side control device 8 correspond to an existing communication network, in this example, an ISDN (Integrated Services Digital Network).
9 is connected.

As can be seen from the above configuration, the remote control device according to this example is constituted by a force feedback type master / slave manipulator that operates the master arm 6 while watching the video monitor 7 using the ISDN 9 as a signal transmission path. Have been. That is, on the master side, the output of the master arm 6 is given as control data (target value) of the slave arm 2, and the observation data (current value) of the slave arm 2 and the image data of the video camera 3 are received. I have to.

At this time, the size of the data that can be transmitted and received between the master side and the slave side, that is, the size of the user data is limited. In this example, the format and contents of the transmitted and received data are modified. I have. That is, as for the control / observation data, the latest information is always transmitted and received to secure the control cycle required for the master / slave operation. As for image data, an image obtained by the video camera 3 is defined as a still image, and this is referred to as a JPEG (Joint
Photographic Coding Experts Group) compressed image data. Then, the image data is divided into a plurality of pieces and transmitted and received little by little so that the monitor image is updated, for example, every few seconds.

FIG. 2 shows the contents of data transmitted and received between the slave controller 4 and the master controller 8. In this example, as shown in FIG. 3, the control / observation data 11 and the image data 12 are mixed in one user data 10, and transmitted and received between the master and the slave.

FIGS. 4A and 4B show an example of user data regarded as a standard specification. The size of the data is 100 bytes. The first 80 bytes are allocated as an area for control / observation data, and the last 20 bytes are allocated as an area for image data.

The contents of the user data are as follows. That is, the control / observation data is ID, S1,
S2, θ1, θ2, θ3, θ4, θ5, θ6, θ7 or ID, S1, S2, Px, Py, Pz, Dx, Dy,
Dz, each of which is separated by a space.

The ID is a header of data and represents information on user data. Specifically, it indicates a division number for one image data, and the first is 000 and the last is 99.
9 and others indicate the number of data obtained by dividing one piece of image data. The master reads this ID information, detects the beginning and end of the image sent from the slave, and reconstructs the data. ASCII integer
Described in code, the size is 3 bytes.

S1 is an information identifier (1). Specifically, it indicates whether the control / observation information is fingertip information, joint information, position information, speed information, force information, and the like. Integers are described in ASCII code and have a size of 1 byte. S2 is an information identifier (2). Specifically, it shows control / observation information for opening and closing the tip gripper of the slave arm. Integers are described in ASCII code and have a size of 1 byte.

Θ1, θ2, θ3, θ4, θ5, θ6, θ
7 or Px, Py, Pz, Dx, Dy, Dz
Observation information. Depending on whether the information is the tip information or the joint information, and whether the information is the position information, the speed information, or the force information, seven bytes of floating-point data of seven bytes are secured. It is written in ASCII code, and each value is separated by a space. The size is 56 bytes.

The image data is transmitted to the slave side by JPE
It is the data amount divided by the area for the image data in the user data in the image data of one sheet compressed in the G format. The master collects these and reconstructs them,
The image is expanded and displayed on the video monitor 7 as an image.

The 100-byte user data is ISD
Through N, for example, a round trip can be made between Tsukuba City, Ibaraki Prefecture and Kawasaki City, Kanagawa Prefecture in about 92 milliseconds. That is, a control cycle of about 11 Hz can be secured between the master and the slave. On the other hand, as for an image, if image data for displaying one image is 4000 bytes,
Since the user data is transmitted and received by 20 bytes each time it makes one round trip, the image displayed on the video monitor 7 is updated every 200 times, that is, about every 18 milliseconds.

By the way, when the slave arm 2 is caused to perform a predetermined operation by following the master arm 6, the operation form performed by the slave arm 2 can be divided into several types. For example, as shown in FIG. 5, when the operation of inserting the rod 15 into the hole 14 is performed by the slave arm 2 as shown in FIG. 5, a track in a three-dimensional free space as shown in FIG. The operation of transporting an object along the line to the vicinity of the target position or the operation of simply moving away from the target position as shown in FIG.

In the case of this spatial movement operation, the movement accuracy of the tip of the slave arm 2 is not so required, but rather, the image information is important to know where the carrying object or the arm tip is currently moving. . In other words, in order to realize good operability in this spatial movement operation, the control / observation period between the master and the slave may be long, but it is preferable to shorten the image update period. To achieve this under a constant user data length condition, the ratio of the image data area to the user data length should be increased.

On the other hand, as shown in FIG.
The operation of inserting the rod 15 into the hole 14 by the arm 2 can be referred to as a constrained exercise operation because the object to be conveyed, that is, the rod 15 is constrained by the external environment.

In such a restraining motion work, it is necessary to control a mutual force acting between the object and the external environment. In other words, in order to realize good operability in this restraining exercise work, the mutual force between the object carried by the slave arm 2 and the external environment is given to the operator who operates the master arm 6 with a sense of realism. Communication is important.

Therefore, during the restraining exercise work, even if the image update cycle is long, the control
It is preferable to shorten the observation period. In order to achieve this under a constant user data length condition, the ratio of the control / observation data area to the user data length should be increased.

From this point of view, in the remote control device according to this example, the controller 1 on the slave side shown in FIG.
6 has a work form determination function and a data transformation function for determining the work form currently being executed by the slave arm 2, and changes the format or content of the user data 10 according to the determination result.

For the determination of the work mode, for example, a threshold value is set for the amount of displacement between the target position of the master arm 6 and the observation position of the slave arm 2. When the threshold value is exceeded, it is determined that the exercise is a restricted exercise. As another determination method, a method may be adopted in which when the force / moment data at the tip of the slave arm 2 is equal to or less than a predetermined value, it is determined that the movement is spatial, and when it is larger than the predetermined value, it is determined that the movement is restrained.

Then, in this example, when it is determined that the work form is the restricted exercise work, as shown in FIG.
The data length is 100 bytes and a standard user data structure is used. When the work mode is determined to be a space movement work, the data length is 100 by, as shown in FIG.
te, but compared to the standard user data structure,
The number of significant digits of the control / observation data is reduced from 7 to 5 digits, and a user data configuration is used in which the proportion of the image data is increased accordingly. In this example, the image data area is increased to 35 bytes during the constraint exercise work.

As described above, in this example, it is determined whether the operation mode currently executed by the slave arm 2 is the constrained movement work or the space movement work. In the case of the constrained movement work, the data length is 100 bytes and the standard operation is performed. Using a simple user data structure,
In the case of the spatial movement work, the data length is 100 bytes, but the user data configuration in which the proportion of the image data occupies is increased as compared with the standard user data configuration.

Therefore, at the time of the space movement work, although the accuracy for controlling the trajectory of the slave arm 2 is reduced,
The control cycle between the master and the slave is maintained at about 11 Hz. On the other hand, if the image data area is
4000 bytes required for one image as it increases to bytes
Minute image data can be transferred 115 times of transmission and reception, and the image update cycle can be reduced from about 18 seconds to about 11 seconds.

Therefore, it is possible to improve the operability at the time of the space movement work without increasing the user data length.

In the example shown in FIG. 1, the work form determining means determines whether the work form currently being executed by the slave arm 2 is a restricted movement work or a space movement work. The operator operating the device 6 may determine the work mode as needed, and may be able to change the format or content of the data to be transmitted by a method such as operating a switch.

Further, in the example shown in FIG. 1, during the space movement operation, the user data length is increased without increasing the user data length.
The operability is improved by increasing the ratio of the image data area to the data area, but is not necessarily limited to this method.

For example, as shown in FIG. 7, the size of the entire user data may be increased at the time of spatial movement work, and the increased size may be used for image data.

In the example of FIG. 7, the size of the user data is increased from the standard specification of 100 bytes to 120 bytes and the area allocated to the image data is expanded from 20 bytes to 40 bytes at the time of the spatial movement operation.

When the above-described user data configuration is employed during the spatial movement operation, 4000 by required for one image
Image data for te can be transferred by 100 transmissions / receptions, and the image update cycle can be reduced from about every 18 seconds to about 9.2 seconds.

The 120 bytes of user data are
Since the shuttle between Tsukuba and Kawasaki takes about 108 milliseconds, the control cycle between the master and slave is about 11 Hz to about 9.
Although the frequency is reduced to 3 Hz, it does not pose a problem in work in free space such as carrying an object.

Also, as shown in FIG. 8, it is effective to divide the control / observation data into two parts at the time of the space movement operation and transmit / receive the user data twice to make one control cycle. In the example shown in FIG. 8, the user data length is standard 100 bytes, but the first user data 20
Then, the ID, S1, the first half of the control / observation data are transmitted and received,
The second user data 21 transmits and receives the ID, S2, and the latter half of the control / observation data. As a result, the area required for control / observation data can be reduced from 80 bytes to 50 by.
te, and the area allocated to image data per user data is expanded from 20 bytes to 50 bytes.

When the above-described user data configuration is employed during the spatial movement operation, 4000 by required for one image
The image data for te can be transferred by 80 transmissions / receptions, and the image update cycle can be doubled or increased from about every 18 seconds to about every 7.4 seconds.

However, the control / observation data is two users
The control cycle between the master and the slave is halved from about 11 Hz to about 5.4 Hz since the control cycle is updated by data transmission and reception. However, this is not a problem when working in free space such as carrying objects.

In each of the above examples, the image data obtained by the monitor television 3 is not processed, but the size of the image data is reduced by reducing the number of dots per image during the spatial movement operation. Is also effective.

FIG. 9A shows how the size of an image file changes due to the difference between the video size and the compression ratio when a certain image is compressed in the JPEG format. . FIG. 9B shows the time until the image is updated when this is transmitted / received with the user data of the standard specification.

For example, when the slave arm 2 is performing a constraint exercise, the video size is 320 × 24.
Assuming that the image data is 0 and the compression ratio is about 1/40, the image is updated about every 13.1 seconds as can be seen from FIG. 9B. If the video size of the image data is switched to 240 × 180 when the space moving work state is changed, the control cycle between the master and the slave is about 11H.
The image update period can be increased to about every 8.8 seconds while being kept at z. Such a data transformation method may be adopted.

It is also effective to change the JPEG compression ratio for an image obtained by the video camera 3.

For example, when the slave arm 2 is performing a constraint exercise, the video size 240 × 1
Assuming that image data of 80 and a compression ratio of about 1/25 are used, the image is updated about every 14.5 seconds as can be seen from FIG. 9B. Then, at the same time as changing to the spatial motion work, if the compression ratio of the image data is switched to about 1/40,
The image update cycle can be increased to about every 8.8 seconds while the control cycle between the master and the slave is kept at about 11 Hz.

As shown in FIG. 10, it is also effective to change both the video size and the user data size in accordance with the spatial movement work and the constraint movement work.

For example, when the operation is changed to the constraint exercise operation, both the video size and the user data size are reduced. By doing so, it is possible to control and control the restraining exercise work without significantly reducing the image update cycle.
The observation cycle can be increased, and the force control at the time of the constraint exercise work can be improved.

[0075]

As described above, according to the present invention,
It is possible to realize remote control that secures a control / observation cycle that can be controlled stably and an image update cycle that can be easily operated while effectively using the limited communication capacity of the communication line.

[Brief description of the drawings]

FIG. 1 is a conceptual diagram of a remote control device according to an embodiment of the present invention.

FIG. 2 is a diagram showing data contents on a slave side and a master side of the remote control device.

FIG. 3 is a diagram showing a data flow between a slave side and a master side of the remote control device.

FIG. 4 is a diagram showing the contents of user data used in the restraining exercise work in the remote control device.

FIG. 5 is a diagram for explaining the definition of a restraint exercise task and a space movement task.

FIG. 6 is a view showing contents of user data used in the space movement work in the remote control device.

FIG. 7 is a diagram showing a modified example of user data used at the time of spatial movement work.

FIG. 8 is a diagram showing another modification of the user data used at the time of the space movement work.

FIG. 9 shows the relationship between video size, JPEG format compression ratio, and image data file size, and video size and J.
Diagram showing the relationship between PEG format compression ratio and image update time

FIG. 10 is a view for explaining an example in which both the video size and the compression ratio are changed during the space movement work and the constraint movement work.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... Work environment 2 ... Slave arm 3 ... Monitor television 4 ... Slave side control device 5 ... Operation environment 6 ... Master arm 7 ... Video monitor 8 ... Master side control device 9 ... ISDN 10 ... User data 11 ... Observation data 12 image data 14 hole 15 rod 16 controller 20 first user data 21 second user data

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Takafumi Matsumaru 1st station, Komukai Toshiba-cho, Saiwai-ku, Kawasaki-shi, Kanagawa Inside Toshiba R & D Center

Claims (8)

[Claims] A work execution device arranged in a work environment to execute a work; a control device arranged in a control environment to remotely control the work execution device; and a control device arranged in the work environment to perform the work. An image acquisition device that captures a work state as an image, an image presentation device that is arranged in the steering environment and presents an image acquired by the image acquisition device, the work execution device, the image acquisition device, the steering device, and the image presentation device A data transmission device that is provided between and transmits control / observation information necessary for operation of the work execution device and image information necessary for image presentation to the image presentation device; A work mode determining means for determining a work mode in which the data is transmitted via the data transmission device according to the work mode determined by the work mode determining means. Remote control apparatus characterized by comprising; and a transmission data modification means for. 2. The work form determining means determines that the work form is a spatial movement work form when the amount of deviation between the position / posture data of the work execution device and the position / posture data of the control device is equal to or less than a predetermined value. The remote control device according to claim 1, wherein the determination is made, and when the value is larger than a predetermined value, it is determined that the operation mode is the work mode of the restraint exercise. 3. The work form determination means determines that the force / moment data at the tip of the work execution device is equal to or less than a predetermined value, the work form is a spatial movement work form, and when the force / moment data is larger than the predetermined value, the work form of the constrained movement is determined. The remote control device according to claim 1, wherein the remote control device has been determined to be: 4. The transmission data transformation means according to the work mode determined by the work mode determination means, for the size of the control / observation information area and the image information occupied in data transmitted and received at one time. 2. The remote control device according to claim 1, wherein the size of the area is changed. 5. The transmission data transforming means changes the size of the whole data to be transmitted / received at one time according to the work form determined by the work form determination means to change the size of the area for the image information. The remote control device according to claim 1, wherein the remote control device is changed. 6. The image information processing apparatus according to claim 1, wherein the transmission data transformation unit selects the control / observation information item in the data transmitted and received at a time according to the work mode determined by the work mode determination unit. 2. The remote control device according to claim 1, wherein the size of the area for the remote control is changed. 7. An image for one image by changing the size of an image acquired by the image acquisition device according to the work mode determined by the work mode determination unit. The remote control device according to claim 1, wherein the size of the data is changed. 8. The apparatus according to claim 1, wherein the transmission data deforming unit changes a data compression ratio of the image acquired by the image acquiring device according to the work mode determined by the work mode determination unit. 2. The remote control device according to claim 1, wherein the size of the image data is changed.