KR101484269B1 - Remote Handshake System and Control Method thereof - Google Patents

Abstract

The present invention relates to an operation device, which is provided in the shape of a hand of a human body and is in contact with a user for shaking hands; A haptic device connected to the manipulator for receiving position information from a manipulator and receiving a stimulus generated from a difference in position information in response to the manipulation; And a communication unit for mutually transmitting and receiving information between the haptic device and the haptic device, wherein the haptic device calculates a force transmitting the user based on the stimulus generated from the difference between the positional information input from the manipulator and the opposite positional information transmitted through the communication unit And the controller is controlled in a range in which the contact stability is ensured so as to control the manipulator so as to improve the contact stability with the manipulator when shaking hands for more practical and various emotional expressions, There is an effect that the actual sensibility can be improved by executing the adaptive control.

Description

Remote Handshake System and Control Method Therefor,

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a remote handshake system and a control method thereof. More particularly, the present invention relates to a remote handshake system and a control method thereof, And a control method thereof.

A handshake is a conscious act in which two people face each other and shake their hands together up and down to express the meaning of greetings, gratitude, dear, reconciliation. The remote handshake system is a teleconferencing system or a social network Service (SNS), etc., by making a handshake with a person far away can increase the bond with the other side.

Stability and transparency are important issues in remote handshake systems. Stability is a very important issue in relation to the safety of the operator who shakes hands, and transparency is a very important issue because the operator must be given the feeling of actually shaking hands.

In 1995, Kunii and Hashimoto proposed a remote handshake system for the first time, considering only the grip strength (Yasuharu Kunii and Hideki Hashimoto, "Tele-handshake using Handshake Device", Proceedings of the IEEE 21 ^st International Conference on Industrial Electronics , Control , and Instrumentation , 1995.) A one-degree-of-freedom hand robot capable of expressing grip strength is constructed, and each operator grasps a hand robot to measure grip strength and then transmits the grip strength to the opponent operator. To ensure the stabilization of both robots, both ends are connected with a stable range of virtual dynamics.

Meanwhile, Wang et al. Demonstrated a remote handshake system using two haptic devices, despite the 300 ms time delay between Geneva, Switzerland and Ottawa, Canada. In this case, we use the wave variable method proposed by Neimeyer and Slotine for stabilization (G. Niemeyer and JJ, Slotine, Stable adaptive teleoperation, Proceedings of the IEEE Journal of Oceanic Engineering, Vol. 152-162, 1991.)

They assume that both the master and the slave are passive, and that if only the communication channel modeled as two-port satisfies the passivity, the stability of the entire system can be ensured by securing the passivity of the whole system. Therefore, in order to satisfy the passivity of the communication channel, we define wave variables collectively known as U and V and replace the physical quantities such as position / velocity and force with wave variables and transmit them to secure the passivity of the entire two- .

In addition, Kwon and Park proposed a remote meeting system that provides image information to a remote handshake system by adding a digital video transport system to the above system (Y.-M. Kwon and J.-W Park, "Tangible Tele-Meeting System with DV-ARPN (Augmented Reality Peripheral Network)", Proceedings of the International Conference on Computational Science and its Applications, 2005.) The system acquires the operator and the haptic device by using the image acquisition device, and sends it to each other, and can perform the remote handshake with the partner using the haptic device.

However, these conventional techniques have problems in that an end-effector is used as a metal rod or a robot hand made of a metal is used to provide a feel of a real human hand such as warmth, texture, and grip force . In addition, the wave-variables used as the manipulation control method of the handshake system cause position-drift problem, which poses a problem in terms of accurate position-tracking.

SUMMARY OF THE INVENTION Accordingly, the present invention has been made to solve the above problems, and it is an object of the present invention to provide a remote handshake capable of enhancing the contact stability with the manipulator when shaking hands with a remote person, and performing adaptive control of catching lightly or strongly, And a control method thereof.

According to an aspect of the present invention, there is provided an operation device comprising: a manipulator provided in the shape of a human body for making contact with a user for shaking; A haptic device connected to the manipulator for receiving position information from the manipulator and receiving the stimulus generated from the difference in position information in response to the manipulation; And a communication unit for mutually transmitting and receiving the information between the haptic devices, wherein the haptic device transmits to the user based on the stimulus generated from the difference between the position information input from the manipulator and the opposite position information transmitted through the communication unit And the manipulator is controlled by adjusting the range so that the force is calculated and the contact stability is ensured.

The haptic device includes a haptic interface unit connected to the manipulator for inputting positional information by a user from the manipulator and transferring a stimulus generated from a difference in positional information to a user in response to an operation; A controller for calculating a force transmitted to the user based on the stimulus generated from the difference between the position information transmitted from the haptic interface unit and the opposite position information transmitted from the communication unit, And a sample / hold unit that converts the analog signal output from the haptic interface unit to a digital signal and transmits the digital signal to the controller, converts the digital signal output from the controller into an analog signal, and transmits the analog signal to the haptic interface unit.

The controller may include: a position controller for controlling a position of the manipulator through the haptic interface; And an EBA controller that adjusts the force to be transmitted to the user to a range in which the contact stability is ensured by applying an Energy-Bounding Algorithm (EBA), which is a stabilization technique based on passivity theory, and transmits the adjusted power to the haptic interface unit.

The position controller may be a proportional-differential controller.

The control unit may adjust the control gain value according to the grip force value of the counterpart user.

The manipulator may be provided with alginate and skin silicon may be attached to the outer surface.

The manipulator may include a pressure sensor which is provided at a portion where the user grasps for shaking and measures the grip force of the user.

The manipulator may include a heating element that is provided in an area held by the user for shaking and generates heat.

The remote handshake system includes a photographing unit for photographing a picture of a user of the other party; And an image display unit for enhancing the image acquired through the photographing unit to the manipulator.

The control unit estimates a moving frequency of a counterpart user based on the image acquired through the photographing unit, and synchronizes the motion of the manipulating unit and the image enhanced by the image display unit according to the estimated frequency.

According to another aspect of the present invention, there is provided a method for controlling a mobile terminal, Receiving counterpart position information of a counterpart user via a communication unit; Calculating a force to be transmitted to the user based on the stimulus generated from the difference between the position information input from the manipulator and the opposite position information transmitted through the communication unit; Adjusting a force to be transmitted to a user to a range in which contact stability is ensured; And transmitting the adjusted transmission force to the user via the manipulator.

In the step of adjusting the force to be transmitted to the user to a range in which the contact stability is ensured, the force to be transmitted to the user is applied to the range in which the contact stability is ensured by applying the Energy-Bounding Algorithm (EBA) .

The control method of the remote handshake system may further include a step of receiving grip strength information of a counterpart user through the communication unit and a step of adjusting a gain of the control unit according to input grip strength information.

The control method of the remote shaking water system includes the steps of: photographing the image of the other user; Receiving a photographed image through the communication unit; And augmenting the transmitted image to the manipulator.

The control method of the remote handshake system may further include estimating a moving frequency of a counterpart user based on the transmitted image and synchronizing the motion of the manipulator and the augmented image according to the estimated frequency.

The control method of the remote shaking water system may further include a step of operating the heating element provided in the manipulator at a predetermined temperature.

According to the remote handshake system of the present invention, it is possible to improve the contact stability with the manipulator when shaking hands for more practical and various emotional expressions, and to improve the sensibility by performing adaptive control of catching lightly or strongly have.

In addition, there is an effect that a feeling of warmth, grip strength, touch and the like can be transmitted for the real feeling improvement.

1 is a block diagram of a remote handshake system according to an embodiment of the present invention.
Figure 2 is a mechanical modeling of the shaking motion.
Figure 3 is a block diagram of the equivalent system of Figure 2;
4 is a view showing the shape of a robot hand.
FIG. 5 is a block diagram for explaining a method of enhancing an image of the remote handshake system of FIG. 1;
6 is a flowchart of a control method of a remote handshake system according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same reference numerals are used to designate the same or similar components throughout the drawings. In addition, the preferred embodiments of the present invention will be described below, but it is needless to say that the technical idea of the present invention is not limited thereto and can be variously modified by those skilled in the art.

2 is a block diagram of the remote shaking system according to an embodiment of the present invention, Fig. 2 is a mechanical modeling of the shaking motion, Fig. 3 is a block diagram of the equivalent system of Fig. 2, FIG. 5 is a diagram illustrating a method of enhancing an image of a remote handshake system of FIG. 1; FIG.

Referring to FIG. 1, a remote handshake system 1 according to an embodiment of the present invention includes an operation device 100 provided in the shape of a human body to be in contact with a user for shaking hands, The haptic device 200 includes a haptic device 200 for inputting positional information by a user from the haptic device 200 and transmitting the stimulus generated from the difference of the positional information to the user in response to the operation, 300).

A manipulator (100, robot hand) is a component that a user touches in place of the hand of another user to shake hands with a remote party, and is provided in the shape of a human hand for the purpose of improving the real feeling.

The manipulator 100 is made of alginate in accordance with the hand shape of the human body, and skin silicon is attached to the outer surface of the manipulator 100 to improve the sensibility.

In the present embodiment, the manipulating device 100 inserts and drives a tendon-driven artificial skeleton therein. The tendon drive method has the advantage of lowering the accuracy and strength of the joint angle but lowering the cost and volume compared to the motor-driven method. Also, it can be implemented similar to human body, and real feeling can be improved.

5, the manipulator 100 includes a pressure sensor 110 provided at a portion for a user to grip for shaking and measuring a gripping force of a user, a heating element 120 provided in an area held by the user for shaking and generating heat, .

The pressure sensor 110 transmits the detected pressure value to the controller 230 of the haptic device 200 to measure the user's gripping force. The control unit 230 controls the servomotor connected to the wire of each node of the actuator 100 according to the transmitted pressure value, thereby realizing an appropriate grip force necessary for the handshake.

The heating element 120 is formed of a peltier or a thermal pad, and is inserted into the palm of the manipulator 100. In order to quickly transfer the heat generated from the heat generating element 120 to the entirety of the manipulator 100, a pad having a high thermal conductivity is inserted into the palm of the manipulator 100 or the whole of the manipulator. The temperature of the heating element 120 fixes or varies the current supplied to the peltier or the heating pad so that a temperature similar to the body temperature is maintained.

The haptic device 200 receives location information from the operation device 100 and calculates a force to be transmitted to the user in response to a user operation so as to control the operation device 100 so that the user can receive a feeling of shaking hands with the actual user do.

The haptic device 200 implements the interaction between the manipulator 100 and the user through mechanical modeling of the actual handshake system.

The modeling of the handshake consciousness can be represented as shown in FIG. In FIG. 2, m1arm and m1h are the arms and hands of the user (Human 1), and m2arm and m2h are the arms and hands of the other person (Human 2). The arms and hands of the user (Human 1) and the other person (Human 2) are connected to the equivalent wrist by spring (k) and damping (c). The user (Human 1) is a system that holds the m2h of the other person (Human 2) and shakes the mass m2arm connected by the spring (k2) and the damping (c2).

At this time, m2arm is moved by the other person (Human 2). Likewise, the counterpart (Human 2) can equate to a system that grasps m1h of the user (Human 1) and shakes the mass (m1arm) connected by the spring (k1) and damping (c1).

Each operator (Human 1 and Human 2) can express emotion through handshake to the opponent's operator by applying force to the wrist or arm. In the system of FIG. 2, the force is controlled by the wrist, The mouse can shake hands. Here, when the operator adjusts the force of the wrist, it means that the opponent operator feels hard or soft by adjusting the value of the stiffness (k) or the damping (c).

The handshake system modeled mechanically as shown in FIG. 2 can be represented by a system block as shown in FIG.

Referring to FIG. 3, the haptic device 200 is connected to the manipulator 100, receives position information from the manipulator 100 by a user, receives stimuli generated from differences in position information in response to the manipulation, The haptic interface unit 210 that transmits the haptic interface unit 210 and the haptic interface unit 210 and the opposite position information transmitted from the communication unit 300, The haptic interface unit 210 converts the analog signal output from the haptic interface unit 210 into a digital signal and transmits the digital signal to the controller 230. The controller 230 converts the digital signal output from the controller 230 into an analog signal And a sample / hold unit 220 for converting the signal to a haptic interface unit 210.

The haptic interface is connected to the manipulator 100 to receive the position information of the manipulator 100. The haptic interface receives the stimulus generated from the difference of the position information in response to the manipulation of the manipulator 100 by the user, .

The sample / hold unit 220 converts an analog signal, which is a continuous signal output from the haptic interface, into a digital signal and provides the analog signal to the control unit 230. The digital signal, which is a discrete signal output from the control unit 230, And provides it to the haptic interface.

The control unit 230 receives information from the haptic interface unit 210 and exchanges information with the haptic device 200 of the other user through the communication unit 300. Based on the collected information, (100).

First, the control unit 230 calculates a force transmitting the stimulus generated from the difference between the position information transmitted from the haptic interface unit 210 and the opposite position information transmitted through the communication unit 300, .

The control unit 230 includes a position controller 231 for controlling the position of the manipulator 100 through the haptic interface unit 210 and an energy limiting algorithm (EBA), which is a stabilization technique based on passivity theory, And an EBA controller 232 that adjusts the contact stability to a range in which the contact stability is guaranteed by applying the Energy-Bounding Algorithm to the haptic interface unit 210.

The position controller 231 controls the position of the manipulator 100 according to the position information of the other party transmitted through the communication unit 300. The position controller 231 uses a proportional-differential controller to be modeled with the stiffness (k) and damping (c) shown in Fig.

The EBA controller 232 is provided to ensure contact stability. That is, since the user's motion includes various uncertainties, it is difficult to accurately model the energy. In the course of sampling and holding in the sample / hold unit 220, energy is necessarily generated, and in the communication unit 300, The EBA controller 232 is included in the controller 230 for the stability of the remote operation system.

The EBA controller 232 has an EBA (Energy-Bounding Algorithm, Ryu Jae-ha, Kim Jong-Pil, "A Haptic Interface Stabilization Method and a Haptic System Using the Same" 10-2005-0101745, 2005) to each controller with a position-position architecture.

In this embodiment, the EBA controller 232 enables the reference position of the other party to be stably followed through the position controller 231. [ At this time, the hardness or softness of the system can be controlled by controlling the gain of the controller 230 by the position controller 231 and the EBA controller 232.

That is, the control unit 230 adjusts the control gain value according to the grip force value of the counterpart user. The grip force value is determined according to the pressure value measured by the pressure sensor 110 provided in the operation device 100 as described above. By controlling the control gain value according to the grip force value, the user feels hard or soft when shaking hands through the manipulator 100, and enables emotional expression of the other user.

1 and 5, a remote handshake system 1 according to the present embodiment includes a photographing unit 400 for photographing a user of a counterpart user, an image acquisition unit 400 for acquiring an image acquired through the photographing unit 400, (Not shown).

The photographing unit 400 photographs an image of each user and transmits the captured image to the control unit 230 of the counterpart haptic device 200 through the communication unit 300. The control unit 230 displays the image obtained through the photographing unit 400 Estimates the moving frequency of the other user based on the frequency, and synchronizes the motion of the manipulator 100 and the image enhanced by the image display unit 500 according to the estimated frequency. Therefore, it is possible to eliminate the spatial error between the operation unit 100 and the image due to the time delay caused by the image transmission.

The image display unit 500 may be provided with a projector or the like.

As described above, the image of the user is strengthened through the photographing unit 400 and the image display unit 500, so that the user can feel the immersion feeling as if they are actually shaking hands with the other party.

As described above, according to the remote handshake system 1 of the present invention, it is possible to improve the contact stability with the operation device 100 when shaking hands for more practical and various emotional expressions, and to perform adaptive control of catching lightly or strongly, There is an effect that the degree of freedom can be improved. In addition, there is an effect that a feeling of warmth, grip strength, touch and the like can be transmitted for the real feeling improvement.

Hereinafter, a method of controlling a remote handshake system according to an embodiment of the present invention will be described with reference to the accompanying drawings. However, descriptions of the same things as those described in the remote handshake system 1 according to the embodiment of the present invention will be omitted.

6 is a flowchart of a control method of a remote handshake system according to an embodiment of the present invention.

6, a method for controlling a remote handshaking system according to an exemplary embodiment of the present invention includes a step S100 of inputting location information by a user from an operation device 100, (S200), and calculates a force to be transmitted to the user based on the stimulus generated from the difference between the position information input from the manipulator (100) and the opposite position information transmitted through the communication unit (300) A step S300 of adjusting a force to be transmitted to the user to a range in which contact stability is ensured in step S400 and a step S500 of transmitting the adjusted transmission force to the user through the manipulator 100. [

In step S400 of adjusting the force to be transmitted to the user to a range in which the contact stability is ensured, the energy to be transmitted to the user is applied by the Energy-Bounding Algorithm (EBA) .

The control method of the remote handshake system according to the present embodiment includes the steps of receiving the grip force information of the user of the other user through the communication unit 300 to transmit the emotion between the users when executing the remote handshake, 230) of the signal.

In addition, the control method of the remote handshake system of the present embodiment includes the steps of capturing an image of a user of the other user, receiving a photographed image through the communication unit 300, To the manipulator (100).

At this time, the motion frequency of the other user is estimated based on the transmitted image, and the motion of the manipulator 100 and the enhanced image are synchronized according to the estimated frequency, so that the manipulator 100 and the image It is possible to solve the spatial error between them.

In addition, the control method of the remote handshake system of the present embodiment further includes the step of operating the heating element 120 provided in the operation device 100 at a predetermined temperature so that the user can feel warmth in the remote handshake.

As described above, according to the control method of the remote handshaking system of the present invention, it is possible to improve the contact stability with the operation device 100 when shaking hands for more practical and various emotional expressions, and to perform adaptive control of catching lightly or strongly, There is an effect that the degree of freedom can be improved. In addition, there is an effect that a feeling of warmth, grip strength, touch and the like can be transmitted for the real feeling improvement.

It will be apparent to those skilled in the art that various modifications, substitutions and substitutions are possible, without departing from the scope and spirit of the invention as disclosed in the accompanying claims. will be. Therefore, the embodiments disclosed in the present invention and the accompanying drawings are intended to illustrate and not to limit the technical spirit of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments and the accompanying drawings . The scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of equivalents should be construed as falling within the scope of the present invention.

1: Remote handshake system
100: Actuator
110: Pressure sensor
120: heating element
200: Haptic equipment
210: haptic interface unit
220: sample / hold unit
230:
231: Position controller
232: EBA controller
300:
400:
500:

Claims (16)

A manipulator provided in the shape of a human body to contact the user for shaking;
A haptic device connected to the manipulator for receiving positional information from the manipulator by the user and controlling the manipulator; And
And a communication unit for mutually transmitting and receiving the information between the haptic devices,
The haptic device calculates a force transmitted to the user based on the stimulus generated from the difference between the position information by the user input from the manipulator and the opposite position information transmitted through the communication unit, So as to control the manipulator. The method according to claim 1,
In the haptic device,
A haptic interface unit connected to the manipulator;
A controller for calculating a force transmitted to the user based on the stimulus generated from the difference between the position information by the user transmitted from the haptic interface unit and the opposite position information transmitted from the communication unit, ; And
And a sample / hold unit for converting the analog signal output from the haptic interface unit into a digital signal and transmitting the digital signal to the controller, and converting the digital signal output from the controller into an analog signal and transmitting the analog signal to the haptic interface unit. The handshake system. 3. The method of claim 2,
Wherein,
A position controller for controlling the position of the manipulator through the haptic interface unit; And
A remote handshake system including an EBA controller that adjusts a force to be transmitted to a user by applying an Energy-Bounding Algorithm (EBA), which is a stabilization technique based on passivity theory, to a range in which contact stability is ensured and transfers the adjusted range to the haptic interface unit. The method of claim 3,
Wherein the position controller is a proportional-differential controller. 3. The method of claim 2,
Wherein,
And adjusts the control gain value according to the gripping force value of the counterpart user. The method according to claim 1,
Wherein the manipulator is provided with alginate and skin silicone is attached to the outer surface. The method according to claim 1,
Wherein the manipulator includes a pressure sensor provided at a portion that the user grasps for shaking to measure the grip force of the user. The method according to claim 1,
Wherein the manipulator comprises a heating element which is provided in an area held by the user for shaking to generate heat. 3. The method of claim 2,
A photographing unit for photographing the image of the other user; And
Further comprising: a video display unit for augmenting an image acquired through the photographing unit with the manipulator. 10. The method of claim 9,
Wherein,
And estimates a moving frequency of the other user based on the image acquired through the photographing unit, and synchronizes the motion of the manipulator and the image enhanced by the image display unit according to the estimated frequency. Receiving position information from a manipulator by a user;
Receiving counterpart position information of a counterpart user via a communication unit;
Calculating a force to be transmitted to the user based on the stimulus generated from the difference between the position information input from the manipulator and the opposite position information transmitted through the communication unit;
Adjusting a force to be transmitted to a user to a range in which contact stability is ensured; And
And transmitting the adjusted transmission force to the user via the manipulator. 12. The method of claim 11,
In the step of adjusting the force to be transmitted to the user to a range in which contact stability is ensured,
Wherein the power to be delivered to the user is adjusted to a range in which contact stability is guaranteed by applying an Energy-Bounding Algorithm (EBA) as a stabilization technique based on passivity theory. 12. The method of claim 11,
Receiving the grip force information of the counterpart user through the communication unit; And
And controlling the gain of the control unit according to the input grip strength information. 12. The method of claim 11,
Capturing an image of the other user;
Receiving a photographed image through the communication unit;
Further comprising the step of augmenting the transmitted image to the manipulator. 15. The method of claim 14,
Estimating a moving frequency of a counterpart user based on the transmitted image, and synchronizing the motion of the manipulator and an augmented image according to the estimated frequency. 12. The method of claim 11,
And operating the heating element provided in the manipulator at a predetermined temperature.

Images