KR100838181B1 - Hand interface device using miniaturized absolute position sensors, system for hand interface using it - Google Patents

Abstract

The present invention relates to a hand interface glove using an ultra-small absolute position sensor and a hand interface system using the same. The micro-displacement of a finger joint is measured using an ultra-small absolute position sensor for measuring an absolute position of a finger joint, and the finger measured as described above. By controlling the movement of the virtual hand model using the movement of the joint, it interacts with the objects in the virtual environment, and also allows the correction of the change distance that varies according to the user's body size by two simple operations. By combining the same high quality virtual reality technology, the product review process can be simulated similarly to the real world, so that product defects and errors can be found in advance, and the overall design and design process can be shortened in the production process. .

Hand Interface Hardware, Hand Interface Software, Miniature Absolute Position Sensors

Description

HAND INTERFACE DEVICE USING MINIATURIZED ABSOLUTE POSITION SENSORS, SYSTEM FOR HAND INTERFACE USING IT}

1 is a view showing the configuration of a hand interface system using a micro absolute position sensor according to the present invention,

2A is a diagram showing the configuration of a hand interface hardware according to the present invention;

2b is a view showing the configuration of a miniature absolute position sensor according to the present invention;

Figure 2c is a view showing the principle of operation of the miniature absolute position sensor according to the present invention,

3 is a view showing the configuration of a hand interface management unit according to the present invention;

4 is a diagram showing the configuration of a hand interface API according to the present invention;

<Explanation of symbols for the main parts of the drawings>

100: hand interface hardware 110: hand interface globe

120: sensor signal amplifier 130: data acquisition device

200: Hand interface software 210: Hand interface management

220: hand interface API 300: collision handling unit

400: virtual environment

The present invention relates to a system capable of interacting with a virtual environment using a hand interface, and more particularly, by tracking the movement of a hand in a real space to precisely and precisely control the virtual hand model of the virtual space to control the virtual environment. The present invention relates to a hand interface glove using an ultra-small absolute position sensor and a hand interface system using the same.

Hand interface technology refers to an interface technology relating to the interaction between the human hand and each object in the virtual environment. The hand interface technology is a human-oriented interface technology that provides a higher level of immersion by supplementing existing visual and auditory special effects, and is widely used in game and virtual medical fields. However, there are few applications worldwide in the field of manufactured goods, such as virtual product production.

In recent years, virtual reality technology has been applied to overcome high cost and low efficiency problems in manufacturing processes of automobiles and ships. However, the virtual reality technology developed so far has been a technology that provides simple visual information, and the technology that can interact with objects in the virtual space using the tactile organs is applied in the industrial field in terms of usability and precision. It was difficult.

Therefore, the development of virtual reality technology for industrial production processors should be precise and easy to use. To do this, the user must feel as if he is touching and handling an object, and moreover, he must be able to interact with the objects in the virtual space.

Therefore, the virtual reality technology for the production processor in the industrial field is a high-precision interaction technology using the tactile organ, which can accurately and precisely interact with the objects in the virtual space without causing any discomfort or inconvenience to the user. There is a need for the development of a realistic hand interface device which is a technology.

This hand interface technology, US Patent Publication No. US 2004/0164880 (Wearable Data Input Device Employing Wrist and Finger Movements) is a switch type for wrist and finger movement to replace the input of information using a common keyboard The built-in sensor can give the effect of locating the key keyboard, but it can not measure the multiple joints to support the precise movement of the finger, can not guarantee the high accuracy, and the calibration process is complicated.

US Patent Publication No. 6,937,033, entitled "Position Sensor With Resistive Element," describes a principle and method of implementing a displacement sensor by attaching a well-flexible resistive sensor to a finger or body joint to track movement. . However, there is a problem that can not be applied to an environment that requires a sense of reality, such as design, evaluation, assembly because it does not have a strong characteristic of changes in the length, position, humidity, temperature, etc. according to the user.

U.S. Patent Publication No. 6,701,296 (Strain-sensing Goniometers, Systems and Recognition Algorithms) uses a resistive sensor that bends against the finger to detect the resistance change that occurs when the finger is bent and applied to the globe interface. The patent on the method also has the disadvantage of being easily affected by the length, position, humidity, temperature, etc. of the human finger joint has the disadvantage of having to perform a frequent calibration process.

US Patent No. 7,012,593 (Glove-type Data Input Device and Sensing Method Thereof) discloses a change in the shape or position of a glove according to the finger movement of a user wearing the glove and thus detected at at least one contact surface. It is for inputting data of mobile phones or keyboards by detecting the degree of pressure, but it has the advantage of making it easier to input data of small terminal devices, but it does not support high-precision finger movements that can simulate the virtual design, evaluation and assembly of the product. I can't.

Accordingly, the present invention has been made to solve the above problems of the prior art, an object of the present invention is to create a virtual hand model of the virtual space using a signal measuring the absolute position of the finger joint using a small absolute position sensor By allowing users to control and interact with objects in the virtual space, the user's hand movements in the real space are precisely and accurately matched to the motions of the virtual hand model in the virtual space to support natural hand movements and visual consistency that are highly consistent with the real space. The present invention provides a hand interface globe using an ultra-small absolute position sensor and a hand interface system using the same.

On the other hand, another object of the present invention is to support a natural hand interface that is highly consistent with the real space to enable realistic review in an environment such as "car interior review scenario," the user grasps the steering wheel in the car interior space, instrument panel The present invention provides a hand interface globe using a miniature absolute position sensor and a hand interface system for realistic virtual evaluation such as pressing a button, manipulating a gearstick and opening a door handle.

On the other hand, another object of the present invention is to perform a correction operation to be made by a simple operation of the setting operation according to the user's body shape in the process of accurately tracking the user's hand using a micro absolute position sensor that measures the absolute position of the finger joint The present invention provides a hand interface globe using an ultra-compact absolute position sensor and a hand interface system using the same.

Hand interface globe using the ultra small absolute position sensor of the present invention for achieving the above object is a miniature absolute position sensor for tracking the absolute position of the finger and the glove device made of the shape of the hand to be mounted on the hand of the glove device The sensor unit detects the analog signal indicating the absolute position of the finger joint due to the movement of the finger joint by installing a plurality of positions in the position corresponding to the finger joint and amplifies the data by receiving the analog signal of the finger joint detected by the sensor unit. And a data acquisition device for converting the digital signal through filtering and outputting the digital signal.

At this time, the ultra-small absolute position sensor is characterized in that it detects a signal regarding the absolute position of the finger joint by the change in the length of the fine wire inserted into the embedded coil according to the absolute position of the finger joint, detachable on the surface of the glove device Characterized in that.

On the other hand, the hand interface system using the ultra small absolute position sensor of the present invention by installing a micro absolute position sensor for measuring the absolute position of the finger joint at the position corresponding to the finger joint of the globe mounted on the hand, the absolute position signal of each joint The control unit controls the movement of the virtual hand model by calculating the joint angle of each finger joint by using the hand interface hardware for detecting and transmitting the absolute position signal of each joint, and transmitting the interaction between the virtual hand model and the virtual environment object. And hand interface software for controlling the action.

In this case, the ultra-small absolute position sensor of the hand interface hardware detects a signal regarding the absolute position of the finger joint by the change of the length of the fine wire inserted into the embedded coil according to the absolute position of the finger joint. And a data acquisition device for converting and arranging absolute position signals of each joint into digital signals of each joint.

In addition, the hand interface software stores the maximum and minimum voltages of the respective finger joints and detects any joint angle of the user based on the stored maximum and minimum voltages, as well as of the displacement distance that depends on the user's body size. The motion of the virtual hand model is controlled by calculating a joint angle of each finger joint by using a hand interface management unit for correcting a change and the absolute position signal of each of the joints, and controlling the movement of the virtual hand model and the object of the virtual environment. Characterized in that it comprises a hand interface API for controlling the interaction.

Hereinafter, a hand interface globe using an ultra-small absolute position sensor of the present invention and a hand interface system using the same will be described in detail with reference to the accompanying drawings.

1 is a view showing the configuration of a hand interface system using a miniature absolute position sensor according to the present invention.

The hand interface system using an ultra-small absolute position sensor is a hand interface software that allows a user and an object in a virtual space to interact by using a hand interface hardware 100, a device for detecting a finger joint movement, and a detected finger joint movement. 200 is made.

The hand interface hardware 100 installs an ultra-small absolute position sensor for measuring the absolute position of the finger joint at a position corresponding to the finger joint of the globe mounted on the hand, and detects and transmits an absolute position signal of each joint.

The hand interface hardware 100 includes a hand interface glove 110, a sensor signal amplifier 120, and a data acquisition device 130.

The hand interface glove 110 consists of a glove in the form of a glove. The hand interface glove 110 is equipped with a miniature absolute position sensor that measures tracking information about a user's hand gesture. The hand interface globe 110 transmits the analog signal sensed by the ultra small absolute position sensor to the data acquisition device 130.

The sensor signal amplifier 120 amplifies the analog signal by the hand interface glove 110.

The data acquisition device 130 receives an analog signal sensed by the ultra small absolute position sensor. The input signal is input as a number of signals corresponding to the number of ultra-small absolute position sensors. The miniature absolute position sensor mounted on the hand interface glove 110 measures the movement of the joint of the finger and is installed at the positions of the first and second joints of the finger. Accordingly, the data acquisition device 130 is equipped with ten ultra-small absolute position sensors in one hand interface globe 110 to receive analog signals from a total of 20 sensors.

The data acquisition device 130 converts analog signals input from 20 sensors into digital signals having a desired level through amplification and filtering processes. The miniature absolute position sensor interfaces with the data acquisition device 130 through a connector.

In the computer, the sampling signal input using the data acquisition device 130 in the form of a card is displayed as each sensor data. The computer delivers data from each tiny absolute position sensor to hand interface software 200 running on the computer through an operating program designed in Windows-based Visual C ++.

The hand interface software 200 calculates the joint angle of each finger joint using the transmitted absolute position signals of each joint to control the movement of the virtual hand model and to control the interaction between the virtual hand model and the virtual environment object. .

The hand interface software 200 includes a hand interface management unit 210 and a hand interface API 220.

The hand interface management unit 210 stores the measured minimum voltage and the maximum voltage in order to linearly convert voltage values of the finger bendings of the plurality of ultra-absolute absolute position sensors installed in the hand interface globe 110, and stores the maximum bending area of each finger. Stores in the Windows registry.

The hand interface management unit 210 is software that exists independently of the hand interface API 220.

The hand interface management unit 210 is responsible for accurate correction function through the display of the hand interface, device information, status monitoring and the virtual hand model. It monitors status such as checking device connection status and status of each sensor.

The hand interface management unit 210 performs a function of correcting a change in displacement distance that varies depending on a user's body size. The calibration process is performed by two hand gestures to obtain the correct initial value of each finger during interface initialization. Two simple hand gestures are the thumb-only gesture with the fists folded and the thumb-only gesture with the palms fully open. These two motions provide the voltage values for the minimum and maximum bending range of the finger joint. The reason why the thumb and the other four fingers are bent at different times is that the exact minimum and maximum displacement values of the thumb cannot be obtained when the fist is held. After the first operation, all voltage values from 20 sensors and the second from 20 sensors are stored in the registry.

Based on the minimum and maximum voltage values of each finger joint stored in this way, the following equation (1) calculates the joint angle when the user bends the finger joint at an arbitrary angle.

Angle _t (i) = ( V _t (i)- V _min (i)) / ( V _max (i)- V _min (i)) x Angle _max (i) ..... (1)

Where Angle _t (i) is the joint angle when the user bends the i-th joint at an arbitrary angle, V _max (i) is the voltage value when the I-th joint is maximized, and V _min (i) is The voltage value when the I-th joint is bent to the maximum, and finally Angle _max (i) is the range of the minimum maximum joint angle of the I-th joint. Angle _max (i) allows the user to adjust his or her hand through the GUI of the hand interface management unit 210. The first and second joints of the fingers are mounted directly to determine the joint angle, and the third joint, without the sensors, to predict the third joint's angle from the ergonomic movement of the hand. Since the third joint moves together at an angle that is almost similar to the second joint, it can be predicted by mapping to the second angle. This has the advantage of reducing the number of sensors as much as possible to secure sufficient data bandwidth and lower the manufacturing cost.

In addition, the user can compare various joint angles by using a virtual hand model that shows smooth joints by applying variable skin deformation technique for realistic and accurate hand interface correction. I can adjust it.

The hand interface API 220 controls the movement of the virtual hand model by calculating the joint angle of each finger joint using the transmitted absolute position signal of each joint, and the interaction between the virtual hand model and the object of the virtual environment 400. Function to control the action.

The hand interface API 220 supports device initialization, device connection, and device input / output data streaming so that the hand interface device can be easily integrated in a virtual reality application. The hand interface API 220 is a PC-based library that can be easily integrated into various applications. The hand interface API 220 loads the sensor initial value and the finger joint range initial value from the registry and calculates the finger joint angle based on the voltage value of each finger joint measured by the data acquisition device 130. Provided by In addition, the hand interface API 220 is provided in the form of a programming library, and is linked to be used at compile time so that a function call can be used in a virtual environment 400 application corresponding to an application.

Figure 2a is a configuration of the hand interface hardware according to the present invention, Figure 2b is a configuration of a miniature absolute position sensor, Figure 2c is a view showing the principle of operation of the miniature absolute position sensor.

The hand interface hardware 100 is a hardware device capable of precisely tracking the shape of a hand by attaching a hand-shaped globe 10 and two small absolute position sensors 20 to one finger (10 sensors applied to one hand). to be. The miniature absolute position sensor 20 of the hand interface hardware 100 is a device for measuring the linear length change caused by the displacement. The principle is that the length of the fine wire 22 to be inserted into the fixing part 24 in which the coil is embedded varies according to the absolute position of the finger joint, and the pulse wave is applied in proportion to the insertion length of the wire 22. The waveform changes and measures it to determine the absolute displacement of the finger joint. An automatic principle in which the waveform of an applied pulse wave varies depending on the length of the wire 22 to be inserted is shown in the circuit of FIG. 2C.

In the ultra-small absolute position sensor 20 of FIG. 2B, the fixing part 24 and the moving part 21 are installed in the glove 10 with the joints interposed therebetween. One end of the moving part 21 is provided with a wire 22 inserted into a coil embedded in the fixing part 24 according to the absolute position of the joint. The fixed part 24 detects the absolute position of the joint by detecting a change in the waveform of an applied pulse wave when the coil 22 is embedded and the wire 22 is inserted into the embedded coil. The sensed analog signal is transmitted to the data acquisition device.

The miniature absolute position sensor used in the hand interface hardware 100 is a linear variable differential transducer (LVDT) type position sensor that converts mechanical displacement into an electrical signal. LVDT is a transducer that changes the magnetic flux induced from the primary coil to the secondary coil due to the movement of the core, that is, the mutual inductance. The LVDT is mechanically and electrically separated and moves in proportion to the displacement of the movable core. The electrical output is generated in proportion to the possible displacement of the core. The output of the LVDT has an inductive current in the internal coil depending on the input position of the core and detects the magnitude of the current and converts it into a strain, which is exactly linear.

The LVDT type absolute position sensor is widely used in inspection devices, semiconductor processing equipment, robots, medical devices, etc., but due to its size, no attempt has been made to measure displacement of a human finger joint by contact. If it is manufactured in a very small size so that it can measure the displacement of the finger joint of a human, it can overcome the deterioration in stability due to the change in the external environment, which is a disadvantage of the conventional resistance or optical method. That is, the micro absolute position sensor is directly used to measure the joint angle by simulating human finger joints and muscles, which can guarantee the uniformity of the measured values even if there is a difference in user body shape. ) Has the advantage of unnecessary work. In addition, LVDT type absolute position sensors with 4 kHz update rate and 12 bit resolution can be used for high precision hand interface.

Conventional sensing methods do not measure the movement of finger joints directly, so tracking accuracy and efficiency are inferior. Therefore, the correction process is required according to the non-intuitive adjustment of each joint during the correction process according to the user's body type difference. There is this.

However, the absolute position sensor can measure the small displacement according to the intuitive adjustment value, so it can theoretically be used for a long time with a single initial calibration process.

When the existing absolute position sensor is attached to the surface of the hand globe, the wire may break away when the maximum displacement of the fine wire is exceeded according to the maximum bending according to the size of the finger of the user. To compensate for these shortcomings of existing LVDT sensors, the flexible plastic resin guides can be extended to the ends in the pipe to minimize the possibility of departure.

If the joints of the fingers are measured by the contact method, the wire bends, and the continuous use may require replacement or repair due to deterioration of the sensor. If the sensor is fixedly attached to the surface of the hand glove, it can be difficult to remove for repair, thus overcoming this drawback by making the sensor a detachable module.

3 is a diagram illustrating a configuration of a hand interface management unit according to the present invention.

The hand interface management unit 210 may include a control unit 212, a sensor correction unit 211, a state monitoring unit 213, and a display unit 214.

The sensor corrector 211 corrects a displacement distance that varies depending on a user's body size. The correction is done by hand gestures to get the correct initial value of each finger during the initialization operation.

The state monitoring unit 213 monitors a state such as checking a device connection state and a state of each sensor.

The display unit 214 may help the virtual hand model to be displayed on a graphic screen so as to accurately match the actual hand interface.

The controller 212 controls the sensor correction unit 211, the state monitoring unit 213, and the display unit 214 while performing a function of controlling the hand interface device information, the state monitoring, and the correction process.

4 is a diagram illustrating a configuration of a hand interface API according to the present invention.

The hand interface API 220 supports initialization of a device, device connection, and device input / output data streaming to easily integrate and operate the hand interface hardware 100 in a virtual reality application, and can be easily integrated into various applications. It's a library.

The hand interface API 220 loads the sensor initial value and the finger joint range initial value from the registry and calculates the finger joint angle based on the voltage value of each finger joint measured by the data acquisition device 130. Provided by In addition, the hand interface API 220 is provided in the form of a programming library, and linked to be used at compile time so that a function call can be used in a virtual reality application corresponding to an application program.

Detailed description of the hand interface API is made up of the hand high level API 450 and the hand device API 430. The hand device API 430 performs the data acquisition device and the initialization of the sensor data to manage the device of the hand interface to ensure the correct starting of the device. The device control module 431 serves to fetch the hand interface sensor data from the data acquisition device from the buffer and to bring the desired upper and lower resistance values and the resolution.

The hand high level API 450 loads the virtual graphic model data into the scenegraph 451 structure to support hierarchical collision detection techniques for real time collisions.

The scene graph module 451 receives the hand tracking information from the hand device API 430 and updates the transformation matrix for controlling the virtual hand model and the virtual environment model. In the virtual hand model control, the correction of the ergonomic-based finger movement measurement value is performed, and the collision processing module performs the interaction processing between the real time collision processing and the hand gesture. At this time, the loaded object and the hand interface are mapped through the world coordinate system to have a single unit and receive the movement information of the user's hand every moment and update it with the graphic hand model. In order to increase the realism, calculation of collision events between the millions of polygon data models and virtual objects is handled in real time. For real-time collision processing with large data models, partial objects of interest are pre-classified to reduce computation time required for real-time work and hierarchical collision detection. Based on the collision processing information, the graphic update of the virtual hand model having the hand model and the movement as close as possible to the real user is performed.

Although the present invention has been described in more detail with reference to some embodiments, the present invention is not necessarily limited to these embodiments, and various modifications can be made without departing from the spirit of the present invention.

As described above, the hand interface glove using the ultra small absolute position sensor and the hand interface system using the same according to the present invention accurately track the user's hand movement by using the ultra small absolute position sensor and real-time three-dimensional virtual hand model in real time. By synchronizing to the virtual space, the user and the objects in the virtual space can be precisely interacted to provide a natural and intuitive hand interface in the virtual environment such as the virtual review and the virtual product production processor that are required in the industry. .

In addition, the present invention has an effect that the correction operation according to the user's body shape is made by a simple operation because it is measured using the absolute position of the finger joint and synchronized to the virtual space by the movement of the finger.

Claims (14)

A glove device having a shape of a hand to be mounted on the hand; A plurality of ultra-small absolute position sensors for tracking the absolute position of the finger joint at a position corresponding to the finger joint of the glove device, the sensor unit detecting an analog signal representing the absolute position of the finger joint by the movement of the finger joint; And a data acquisition device that receives an analog signal of a finger joint sensed by the sensor unit, converts the data into a digital signal through amplification and filtering of data, and outputs the digital signal. The method of claim 1, The micro absolute position sensor is a hand interface glove using a micro absolute position sensor, characterized in that for sensing the absolute position of the finger joint by the change in the length of the fine wire inserted into the coil embedded in accordance with the absolute position of the finger joint . The method of claim 1, The micro absolute position sensor is a hand interface glove using a micro absolute position sensor, characterized in that installed in the position of the second, third joint of each finger. The method of claim 1, The micro absolute position sensor is a hand interface glove using a micro absolute position sensor, characterized in that the detachable surface of the glove. A hand interface hardware for installing an ultra-small absolute position sensor for measuring an absolute position of a finger joint at a position corresponding to a finger joint of a globe mounted on a hand, for detecting and transmitting an absolute position signal of each joint; Hand interface software for controlling the movement of the virtual hand model by controlling the joint angle of each finger joint using the transmitted absolute position signal of each joint, and controls the interaction between the virtual hand model and the virtual environment object Hand interface system using a small absolute position sensor, characterized in that. The method of claim 5, The miniature absolute position sensor of the hand interface hardware detects a signal about the absolute position of the finger joint by a change in the length of the fine wire inserted into the coil embedded according to the absolute position of the finger joint. Hand interface system used. The method of claim 5, The hand interface hardware further comprises a data acquisition device for converting and arranging the detected absolute position signal of each joint into a digital signal of each joint. The method of claim 5 wherein the hand interface software is A hand interface that stores the maximum and minimum voltages of each finger joint, detects any joint angle of the user based on the stored maximum and minimum voltages, as well as corrects a change in displacement distance depending on the body size of the user. Management unit; Hand interface API for controlling the movement of the virtual hand model by calculating the joint angle of each finger joint using the transmitted absolute position signal of each joint, and controls the interaction between the virtual hand model and the object of the virtual environment Hand interface system using a small absolute position sensor, characterized in that comprises. The method of claim 8, The process of calibrating the hand interface management unit is performed by two simple operations of setting the maximum and minimum voltage of each finger joint. The method of claim 9, The two simple operations are for accurately measuring the distance of each finger displacement, and the hand interface system using an ultra-compact absolute position sensor, characterized in that the fist is in a state where the thumb is folded and the hand is in the state where the thumb is bent. . The method of claim 9, The step of calibrating the hand interface management unit is a hand interface system using an ultra-miniature absolute position sensor, characterized in that the user directly adjusts the joint angle by using a virtual hand model representing a smooth joint by applying a variable skin deformation technique. The method of claim 8, The interaction between the virtual hand model and the virtual environment object of the hand interface API is performed through collision processing between the virtual hand model and the virtual environment object. The method of claim 12, The collision processing is a hand interface system using a compact absolute position sensor, characterized in that by calculating the collision event between the millions of polygon data virtual hand model and the virtual environment object. The method of claim 12, And the collision processing is performed by a hierarchical collision detection technique.

Images