KR100960972B1 - system for Haptic representation of time-varying information by computer-based apparatus - Google Patents

Abstract

The present invention relates to a haptic expression system for haptic representation of time-based information of a computer-based device. The present invention collects target data to be converted into haptic information by time and receives an activation option signal for haptic information from a host computer. Receiving data collection unit; A data analyzer analyzing the collected target data; A haptic information converter for converting the analyzed target data into haptic information; And a haptic information output unit that outputs the haptic information, which has received the activation option signal, in a tactile form among the converted haptic information.

According to the present invention, since the user recognizes intangible information in the form of tactile form, the user can predict the amount of change of the information through the change of the haptic stimulus, and can easily check whether the system is normally operated or whether the portable device can be safely removed. It can be effective.

Haptic, Tactile, Tactile, Vibration, Tactile of Information

Description

System for Haptic representation of time-varying information by computer-based apparatus

The present invention relates to a haptic representation system for time-varying information of a computer-based device, and more particularly, to reproduce data or physical state information generated over time in a computer environment by using a haptic representation technique. The present invention relates to a haptic representation system for time-varying information of a computer-based device that can be delivered to a user so that the user can easily grasp data or physical state information.

In general, information related to a user in a computer environment is as follows.

a) Information generated for the user, including software indicating whether the drive is operating, or data flow information, including a hard disk drive or USB drive;

b) Internet-based network packets that are wired or wirelessly connected and generated by the user or the application program used by the user.

c) network packets between a computer and a computer-based device or computer, including information generated by the user or an application program used by the user, including serial or parallel communication over a wired or wireless connection.

Among the above information, a notification signal such as a) expresses an operation state by using an LED attached to a main body in the case of a hard disk drive and a built-in LED in the case of a small USB drive. This operating state relates to the read / write state or normal / abnormal state of the data of the system in the case of hard disk drives, and to the safe removal of USB from the computer by checking the read / write of data in the case of portable USB drives. give.

The information generated by the user or the application program used by the user may be, for example, an internet based network packet such as b). In general, the Internet packet allows the user to visually check the transmission or reception status information through the icon displayed in the notification area to predict the network status. In addition, in case of security-critical Internet banking, a separate window displays the amount of the received network, the amount of the network transmitted, or the total amount of the network. This is to maintain security by observing network traffic of the computer in use and predicting whether or not an unauthorized program is executed if the transmission or reception amount surges for no reason.

 c) refers to communication between computer-based devices or between a computer and a computer-based device. In this case, the data exchanged between the devices can be visually confirmed through a terminal program (eg, 'HyperTerminal' for the Microsoft Windows operating system). In the case of a device connected to a computer, such display is possible through a display device including an LED. This confirmation of the received data is used to check whether there is a connection or the data value to be exchanged.

However, since a), b), and c) all require visual attention, there is a problem of dispersing the user's attention.

In case of a), when controlling a remote computer (such as 'Remote Desktop Connection' in Microsoft Windows), it is impossible to check the physical operation status of the computer or computer-based device.

In case of doubt that the computer system has stopped working, the user detects whether the hard disk drive, which is a core device, is driven by detecting the vibration generated by the LED or the driving. Depending on the state of the system, even if the operation is stopped, if the hard disk is running, you can think of it as working, not down.

However, in the case of a remote computer it is impossible to detect by LED or vibration. In addition, conventional vibration-generating storage devices, such as mechanical rotation-based hard disk drives, are being developed in a direction that does not generate mechanical vibrations, such as USB memory, due to the development of technology. Although there is an advantage that the system itself is quiet because there is no vibration, it causes a problem that the user cannot predict information about the system.

In the case of b) and c), a separate window (window) must be opened to observe network traffic. In particular, in the case of c), in case of serial communication using a device having a limited display means, it is inconvenient to transmit a specific ASCII value promised for debugging to a host computer or to install a separate display device.

Disclosure of Invention Problems to be Solved by the Invention The present invention has been made to solve the above problems, by processing intangible information generated by a computer-based device and making it tactile information, thereby dispersing concentration by visual information, wasting resources on a screen device, and unnecessary. The present invention provides a haptic expression system and method for time-varying information of a computer-based device that maximizes user convenience by suppressing additional code.

The present invention relates to a haptic expression system for haptic representation of time-based information of a computer-based device. The present invention collects target data to be converted into haptic information by time and receives an activation option signal for haptic information from a host computer. Receiving data collection unit; A data analyzer analyzing the collected target data; A haptic information converter for converting the analyzed target data into haptic information; And a haptic information output unit that outputs the haptic information, which has received the activation option signal, in a tactile form among the converted haptic information.

Preferably, the target data includes physical rotation data of a storage device, LED operation data of a memory-based drive, data input / output amount data of a memory-based drive, temperature or heat generation data of a computer-based device, and network traffic of a memory-based drive. And at least one of data and character encoded data.

Also preferably, the intensity of the output haptic information is a function for physical rotation data of the storage device, a function for data input / output amount data of a memory-based drive, a function for temperature or calorific value data of a computer-based device, and for character encoding data. It is characterized in that any one or a combination of one or more of the functions.

In addition, the haptic information output unit is characterized in that the actuator (Actuator).

Also, preferably, the haptic information converter converts the target data into corresponding haptic information on the haptic lookup table by using a haptic lookup table to convert the haptic information into haptic information.

Also preferably, the mapping is performed by a pulse width modulated signal PWM.

Also, the mapping may be performed by mapping the case of the target data that has received the activation option signal and the case of the target data that has not been received to different haptic information.

Also preferably, the mapping may be performed by mapping to an analog output voltage using a digital-to-analog converter that outputs an analog input signal.

Also preferably, the data collection unit may include a port-specific observation function of a network connected to a host computer.

And preferably, the target data is characterized in that it comprises data about the host computer of the remote location collected by the data collector through the network.

On the other hand, the present invention relates to a haptic expression method for expressing the information over time in a haptic form of a computer-based device, (a) the data collector to collect the target data to be converted into haptic information by time, the host computer Receiving an activation option signal for haptic information from; (b) analyzing the collected target data by the data analyzer, and converting the analyzed target data into haptic information by the haptic information converter; And (c) outputting, by haptic information, the haptic information output unit receiving the activation option signal from among the converted haptic information in a tactile form.

Preferably, the target data includes physical rotation data of a storage device, LED operation data of a memory-based drive, data input / output amount data of a memory-based drive, temperature or heat generation data of a computer-based device, and network traffic of a memory-based drive. And at least one of data and character encoded data.

Also preferably, the intensity of the output haptic information is a function for physical rotation data of the storage device, a function for data input / output amount data of a memory-based drive, a function for temperature or calorific value data of a computer-based device, and for character encoding data. It is characterized in that any one or a combination of one or more of the functions.

Also preferably, the step (a) may include (a1) observing, by port, a network connected to the host computer by the data collection unit.

Also preferably, the step (a) includes (a2) the data collector collecting data on a remote host computer through a network.

And preferably, the step (b), (b1) using the haptic lookup table (haptic lookup table) comprising the step of mapping the target data with the corresponding haptic information on the haptic lookup table; .

According to the present invention, since the user recognizes intangible information in the form of tactile, the user can predict the amount of change of information through the change of the haptic stimulus, and can easily check whether the system is normally operated or whether the portable device is safely removed. It has an effect.

According to the present invention, since the operation related information of the computer-based device of the remote area as well as the computer-based device of the local area can be transmitted through the network, the user can recognize the information of the target computer of the remote place in the form of tactile sense. .

According to the present invention, since the user can recognize the traffic generated by the network through the tactile informatization of the packet, the unintended packet flow can be confirmed without dispersing additional concentration, thereby maximizing security.

According to the present invention, there is an effect that the user can recognize only the necessary information in the form of tactile through frequently used port setting.

According to the present invention, even in the case of communication with the computer-based device lacking the display means, there is an unnecessary effect of additional communication for confirmation.

According to the present invention, it is possible to accurately grasp whether or not data is received from a host device in a communication with a remote device such as a remote Bluetooth without adding a display device.

According to the present invention, by using the tactile modality of the information, there is also an effect that the user is not influenced by audio-visual computer applications.

Before describing the details for carrying out the present invention, it should be noted that configurations that are not directly related to the technical gist of the present invention are omitted within the scope of not distracting the technical gist of the present invention.

In addition, the terms or words used in the present specification and claims are consistent with the technical spirit of the present invention on the basis of the principle that the inventor can appropriately define the concept of the term in order to explain the invention in the best way. It should be interpreted as meaning and concept.

The present invention collects at a low level the physical value or the amount of data flow of a commonly used computer-based device and outputs it by tactile information so that a user can check various information of the computer-based device without visual confirmation.

Hereinafter, a haptic expression system for time-varying information of a computer-based device according to an exemplary embodiment of the present invention will be described with reference to FIGS. 1 to 4.

1 is an overall configuration diagram of a haptic representation system for time-varying information of a computer-based device according to an embodiment of the present invention, Figures 2a and 2b is an illustration of the interval and waveform of the vibration stimulation, Figures 3a to 3 3C is a conceptual diagram of an effective output voltage according to a change in a pulse width modulated signal, and FIG. 4 is an exemplary diagram of an application program showing traffic of a network connection.

As shown in FIG. 1, a haptic expression system for time-varying information of a computer-based device according to an exemplary embodiment of the present invention includes a data collector 100, a data analyzer 200, and a haptic information converter 300. And a haptic information output unit 400.

Hereinafter, the haptic information according to a preferred embodiment of the present invention will be described as a vibration stimulus, but the present invention is not limited thereto.

The data collector 100 collects target data to be converted into haptic information by time, and receives an activation option signal for the haptic information.

The target data to be converted may be collected from the host computer 1, the host computer 1 based device, or the remote host computer 2 connected to the network. Examples include hard disk drive operation (rotation) data, LED drive data of a USB drive, data input / output amount data of a USB drive, temperature or heat generation data of a computer-based device, amount data of network traffic, or character encoding data.

The activation option signal is received via an event using a host computer 1 or a host computer 2, i.e. a hotkey or mouse of a keyboard.

In addition, the data analysis unit 200 analyzes the collected target data.

In addition, the haptic information converter 300 converts the target data into haptic information based on the analyzed result.

The haptic information converter 300 may convert the target data into haptic information by mapping target data with corresponding haptic information on the haptic lookup table using a haptic lookup table.

An example of the specific conversion by the haptic information conversion unit 300 is as follows.

a) Hard Disk Drive Operation (Rotation)-> Conversion to Haptic Information

Operation information about the rotation of the hard disk drive is displayed through LEDs installed on the front of the computer via the motherboard. If the momentary operation of the computer is abnormal for a variety of reasons, the user can check the operation of the hard disk drive through the LED, it is not easy to check if the computer is in a difficult place, such as remote.

In this case, when the haptic information conversion unit 300 expresses the operation information of the hard disk drive as haptic information and transmits the information to the user, the haptic information conversion unit 300 can diagnose the computer state of the remote place while proceeding with the computer work where the user is located.

In this case, the haptic information converting unit 300 preferably sets a threshold to the value of the operation information of the hard disk drive and maps the stepwise haptic information. In addition, it is preferable that the haptic information conversion unit 300 is set as a microprocessor so that operation information of the hard disk drive can be converted into independent haptic information without passing through the main board of the computer.

b) LED operation of USB drive-> conversion to haptic information

As data moves between your computer and the USB drive, the LEDs on a typical USB drive will work. In addition or instead, the haptic information conversion unit 300 may convert information corresponding to the operation signal of the LED into haptic information. When the movement of the data is finished, the haptic notification state is set as additional haptic information so that the user who received it removes the USB drive from the computer. This eliminates the hassle of waiting for the user to check the LEDs because the user cannot remove the USB drive while moving information.

c) setting the haptic notification state as a function of the amount of network traffic

In the case of a network, various ports exist for each service, and thus, a user may recognize a tactile form by haptic information similarly to the case described in a) or b) by setting a port to be observed.

The haptic information can be a function not only of the amount of network traffic, but also of the derivative or integration of the amount of network traffic. That is, by using the derivative of the first or second order or more, the user can recognize the information related to the change amount of the network traffic in a tactile form. And by using the integral, the amount of accumulated network traffic can be confirmed. In this case, it is desirable to adjust the strength of the haptic information in proportion to the amount of network traffic accumulated in the short term.

d) in the case of representative character encodings using the English alphabet, such as ASCII or the American Standard Code for Information Interchange;

ASCII is used in many devices that use characters, such as computers and communication equipment, and most character encodings are based on ASCII. Since one particular character is a single byte, it can be changed from decimal 0 to 255 when replaced with an ASCII code.

Thus, taking an example of a device for serial communication with a host computer, the data is again transmitted with an ASCII code as a confirmation of the input from the computer, and the haptic information conversion unit 300 is as shown in Equation 1 below. Further communication may not be performed by adjusting the intensity I of the vibration stimulus, which is haptic information, as a function f for the ASCII value, or by using a pre-appointed haptic lookup table.

I = f (ascii code)

The haptic lookup table refers to a table used for reference of a value instead of using a "switch statement" or a "case statement" when processing a plurality of data. That is, when the haptic information output unit 400 includes an actuator, a signal for driving an actuator for inducing vibration stimulation corresponding to a signal desired by the user in advance is determined in a series of arrays, and the corresponding signal is determined according to a search condition. It means a table to generate a vibration stimulus through the output of.

The haptic lookup table according to the present invention is configured to generate a voltage value for configuring an output signal, a pulse width modulator (PWM) value, or a vibration stimulation pattern of a corresponding actuator. It can be set as a set of hourly parameters.

In addition, the haptic lookup table may map the case of the target data that receives the activation option signal and the case of the target data that has not been received into different haptic information, and output a analog input signal (D / A converter). An analog converter can also be used to map to an analog output voltage.

The haptic information output unit 400 outputs the converted haptic information in a form that physically expresses it. As a result, the user may recognize the data state of the computer-based device or the device connected thereto in a tactile form.

The haptic information output unit 400 may include an actuator, and may deliver haptic information to the user by driving the actuator.

The haptic information output unit 400 is capable of data communication inside or outside the wired or wireless, and may be set as a mouse, a keyboard, a speaker, or a haptic pad including a vibrating element.

The output haptic information includes tactile feedback that generates cutaneous stimuli or force feedback that produces kinesthetic stimuli.

Tactile feedback refers to the stimulus transmitted through human skin. The haptic information output unit 400 preferably transmits tactile feedback to a hand part including a human finger through a driving means such as a vibration motor, an ultrasonic motor, a solenoid, or a piezoelectric element. Cell phones with vibrating elements, small linear electronic actuators developed by CEA / LIST in France, and texturing mice using piezoelectric elements from KAIST can be used.

In the case of force feedback, it refers to haptic information that involves relatively macroscopic movements perceived by a user's muscle sense, that is, muscles or joints of a person, not skin. Force echo joysticks or US Sensitive Phantoms can be used.

When using such a force echo device, various haptic information can be output by using various object data as a control variable for generating the tactile sense of the force echo device. That is, the information on the target data can be output in one or more motions. If the user activates the haptic information while using the force echo device, the presently generated object data as a function of the movement of the force echo device can be recognized in the tactile form.

Both of the above-described tactile stimuli are preferably used to recognize the tactile stimulus as needed by utilizing an option activated by the user.

When the haptic information output unit 400 is based on the vibration element, the tactile stimulus described above in the tactile feedback corresponds to the operation of the vibration, and when the haptic information conversion unit 300 is set as a microprocessor, The operation of the vibration is that the digital output of the microprocessor to drive the vibration is output as integer type (0V, 1V, ..., 5V, ...) or real type (continuously changing voltage value, etc.) It means the state of doing.

In the case of a microprocessor that outputs a specific digital voltage value such as 5 V or 3.3 V, in addition to the on / off vibration stimulus, a pulse width modulated signal transmitted to an actuator that outputs the vibration stimulus to adjust the strength of the vibration stimulus is adjusted. do. 2A and 2B show an exemplary view of a section and a waveform of a vibration stimulus.

Pulse-width modulation modulates the on / off of digital outputs over a very short time (less than a few tens of milliseconds). The inertia of the actuator makes the effect of slow or fast rotation without physically turning on or off. It is a way of induction. As a result, it has the same effect as changing the driving voltage, but since it is easier to implement than changing the driving voltage, it is used to control a DC motor or the like. Since the pulse width modulation method is a conventional technique, a detailed description thereof will be omitted.

2A and 2B, the (Ta-Tb) section is set to ON and the (Tc-Tb) section is set to OFF in the specific time section Tc-Ta. In FIG. 2B, the duty ratio (Ta-Tb) / (Tc-Ta), which is a time ratio related to the output, is adjusted by adjusting Tb, which is a boundary between on and off, so that the vibration stimulus continuously becomes stronger and weaker. You can see that it is working. At this time, Tb moves in a range larger than Ta 'and smaller than Tc. Here Ta 'means the minimum time that the actuator can be driven.

3A to 3C illustrate the concept of the effective output voltage according to the change of the pulse width modulation signal. An example of 3A / 4 in FIG. 3B and A / 2 in FIG. 3C is illustrated based on A, which is an output effective voltage of FIG. 3A for outputting a specific section on.

The above-mentioned weak vibration and strong vibration can be obtained by adjusting the duty ratio of the applied voltage which is the period which is turned on in the pulse width modulation. In this case, the AD / DA board may be used to directly change the applied voltage without using the pulse width modulation method.

When the haptic information output unit 400 is based on the vibration element, the control is performed by one independent variable such as voltage, so that the stimulus is not varied. However, the user does not have to touch only a specific part to recognize the touch. There is an advantage.

When the haptic information output unit 400 is a tactile display device for concretely implementing a texture instead of a vibrating element, a stimulus is applied to a human finger in a braille form through a contactor. At this time, the movement amplitude and the frequency of the contactor are determined according to the type and control method of the actuator used. Unlike the case of the vibration element-based method, there are many independent control variables so that various touches can be realized. In the case of haptic on, the contactor is moving to a specific frequency, including 0 Hz, with an amplitude greater than 0 mm. 0 Hz is a condition where there is no movement of the contactor and statically maintains the amplitude.

In the haptic information output unit 400, when the control means of the vibrating element is described as an input voltage V applied to the vibrating element, the relationship between V and the signal over time can be summarized as follows.

A signal over time means a signal describing a state at an observation time. In general, the signal may be accumulated over time and compared with a previous state. Examples of such signals include information such as network traffic, amount of communication packets with peripheral devices, hard disk drive temperature, rotation RPM information, data input / output amount of storage devices including USB, CPU usage, etc. to be.

In the case of a hard disk drive, signals related to the passage of time can be visually identified through graphs through most applications. In other words, it is possible to confirm the current state absolutely / relatively through the amount of change of information according to the change of time.

In Figure 4, you can see the network connection information graph built into the Windows Task Manager of Microsoft Windows. By checking the network-specific traffic, the user can perform network condition diagnosis or security check.

In addition to the software-based confirmation, the LEDs can be simplified to provide information such as the amount of network traffic, the amount of communication packets with the peripheral device, the temperature of the hard disk drive, the rotation RPM information, and the amount of data input / output of the storage device including the USB. You can check it visually.

For example, the hard disk drive LED attached to the main body, the LED showing the input / output status mounted on the USB drive, and the RX / TX LED of the LAN card are typical. In this case, the magnitude of the continuous signal is confirmed by LED by simplifying S. In some cases, it is easier for the user to intuitively check whether the driver is driven or not, rather than showing the actual S value.

If such a signal is used to derive a control variable (for example, a voltage value) to the actuator, it can be set as a function relationship f as shown in Equation 2 below. The function f may be set as a linear function, a nonlinear function, a function in which a threshold is set for each section, a derivative function, or an integral function.

V = f (S)

By setting a threshold to simplify the signal processing (for example, specifying a representative value for each section of 2.5 and 2.8 as 2.0), it is easy to use a haptic lookup table. This has the advantage of minimizing signal processing and complex pattern analysis. Taking network traffic as an example, you can give users a predefined signal to recognize when a sudden increase in network traffic occurs.

Otherwise, when the function result value is used as the input of the actuator, when the D / A output or the D / O (low or high output) is used, the pulse width modulated signal may be implemented.

Hereinafter, a haptic expression method for time-varying information of a computer-based device according to an exemplary embodiment of the present invention will be described with reference to FIG. 5.

5 is a flowchart illustrating a haptic expression method for time-varying information of a computer-based device according to an exemplary embodiment of the present invention.

First, as shown in FIG. 5, the data collection unit 100 collects target data to be converted into haptic information by time, and receives an activation option signal for target data collected from a separate computer (S10). ).

Next, the data analysis unit 200 analyzes the target data receiving the activation option signal (S20).

Next, the haptic information converter 300 converts the target data into haptic information using a haptic lookup table based on the analysis result (S30).

Finally, the haptic information output unit 400 outputs the converted haptic information in a form that physically expresses it so that the user can recognize it in a tactile form (S40).

As described above and described with reference to a preferred embodiment for illustrating the technical idea of the present invention, the present invention is not limited to the configuration and operation as shown and described as described above, it is a deviation from the scope of the technical idea It will be understood by those skilled in the art that many changes or modifications can be made to the invention without departing from the scope of the invention. Accordingly, all such suitable changes, modifications, or equivalents should be considered to be within the scope of the present invention.

1 is an overall configuration diagram of a haptic representation system for information of a computer-based device according to an embodiment of the present invention.

2A and 2B are exemplary diagrams illustrating sections and waveforms of vibration stimuli.

3A to 3C are conceptual views of an effective output voltage according to a change in a pulse width modulated signal.

4 is an illustration of an application program showing traffic on a network connection.

5 is an overall flowchart of a haptic representation method for information of a computer-based device according to an embodiment of the present invention.

Claims (16)

In the haptic representation system for representing the time-based information of the computer-based device in the tactile form, A data collector 100 for collecting target data to be converted into haptic information by time and receiving an activation option signal for haptic information from a host computer; A data analyzer 200 analyzing the collected target data; A haptic information converter 300 for converting the analyzed target data into haptic information; And And a haptic information output unit 400 for outputting haptic information, which has received an activation option signal, from the converted haptic information in a tactile form. The haptic information converter 300 converts the target data into haptic information by mapping the target data with corresponding haptic information on the haptic lookup table using a haptic lookup table. The mapping is performed by mapping the case of the target data that has received the activation option signal and the case of the target data that has not been received to different haptic information. The data collector 100 is characterized in that it comprises a port-specific observation function of the network connected to the host computer, The target data is a haptic representation system for the time-varying information of the computer-based device, characterized in that it comprises data about the host computer of the remote location collected by the data collector 100 via a network. The method of claim 1, The target data may include physical rotation data of a storage device, LED operation data of a memory-based drive, data input / output amount data of a memory-based drive, temperature or heat generation data of a computer-based device, network traffic data and characters of a memory-based drive. A haptic representation system for time-varying information of a computer-based device comprising any one or more of the encoded data. The method of claim 2, The intensity of the output haptic information is one of a function for physical rotation data of a storage device, a function for data input / output amount data of a memory-based drive, a function for temperature or calorific value data of a computer-based device, and a function for character encoded data. Or a combination of any one or more of the haptic representation system for the time-varying information of the computer-based device. The method of claim 1, The haptic information output unit 400 is a haptic representation system for the time-varying information of the computer-based device, characterized in that the actuator (Actuator). delete The method of claim 1, And said mapping is performed by a pulse width modulated signal (PWM). delete The method of claim 1, The mapping is performed by mapping to an analog output voltage by using a digital-to-analog converter (D / A converter) for outputting an analog input signal. delete delete delete delete delete delete delete delete

Images