KR20050077190A - Device and method of computer interface using sensor and artificial intelligence module - Google Patents

Abstract

The present invention relates to a computer interface device, and more particularly, to a computer interface device for a disabled person who combines a sensor and an artificial intelligence module to enable a disabled person to use a computer.

 In particular, it is an object of the present invention to provide a device that can detect the movement of the head of the disabled using only one sensor and to control the movement and click of the computer mouse using an artificial intelligence module capable of learning by itself.

In order to achieve the above object, the present invention provides a computer interface device including a sensor unit 200, a microcomputer unit 100, and a main computer unit 600, wherein the sensor unit 200 uses a sensor 210. Measure the motion information and process the signal using the filtering circuit 220 including the capacitor discharge switching circuit for early stabilization of the filter output, and then transfer the signal to the microcomputer unit 100 and transmit the signal to the main computer unit 600. ) Selects a click signal by receiving the digital signal converted by the microcomputer unit 100 and selects a click signal so that the AI module 900 learns to detect the click signal. Simultaneous display of mouse position and click on Windows with a real-time mouse control unit 800 for simultaneously performing position control and controlling a click signal. It is characterized by.

In addition, the method using such a device measures the movement of the head information using a sensor and generating a motion signal using a filtering circuit including a switching circuit for the capacitor forced discharge for early stabilization of the filter output (S200), An A / D converter for converting the analog signal, which is the motion signal, into a digital signal, a microprocessor unit having a microprocessor for controlling the A / D converter and a communication module and a communication module for communicating with the main computer unit Transmitting the converted digital signal to the main computer unit (S600), receiving a digital signal from the microcomputer unit (S710), selecting a click signal (S720), and detecting a user click signal using the selected signal. Offline learning step (S700), including the step (S730) for learning the artificial intelligence module so that The real-time mouse control step (S800) and the real-time mouse, including the step of receiving the real-time digital signal from the microcomputer unit (S810) and controlling the position of the mouse cursor (S820) and simultaneously controlling the click signal (S830, S840) Characterized in that the step of displaying on the window at the same time using the mouse position control signal and the click signal output in the control step.

According to the present invention, by combining a sensor (for example, a gyroscope, etc.) and an artificial intelligence module, a user click pattern for each user, rather than forcing the user to acquire a predetermined mouse operation pattern like an existing interface (HCI) The effect is that the system can learn by itself and provide a user-friendly interface.

Description

Device and method of computer interface using sensor and artificial intelligence module

The present invention relates to a computer interface device, and more particularly, to a computer interface device for a disabled person who combines a sensor and an artificial intelligence module to enable a disabled person to use a computer.

As a related art related to the human computer interface (HCI) of paralyzed patients, a method using a sensor and a method using a biopotential have been used.

First, a method using a sensor may include a character input system of a handicapped person using a high sensitivity magnetic sensor. This method employs a magnetic receiver mounted on the head to input characters while scanning the keyboard drawn on the monitor with this movement. Polhemus FASTRAK sensor, which is one of the sensors used in the space tracking device used in this method, is convenient to use as a high sensitivity sensor, but it is very expensive, and both the transmitter and the receiver should be wired and not portable, which makes it difficult to use. In addition, since a magnetic transmitter and a receiver are used, there is a high possibility of malfunction in a situation in which disturbance of electromagnetic waves is increasing rapidly.

 Secondly, the tilt sensor is used to realize the up and down and left and right movements of the mouse, and the swelling of the cheek is detected by a touch switch mounted next to the ball. There is a way to implement Drag. In this method, since two sensors are used, there is a problem that the volume of the sensor and the signal processing unit is large and inflating the ball to manipulate the touch switch is not good.

Third, there is a method to implement a mouse using a gyroscope and an infrared sensor, but the gyroscope controls only the movement of the mouse up, down, left and right, and clicks to detect and intentionally blink an eye by using an infrared sensor. have. However, since the infrared transceiver needs to detect a change in skin shape with respect to the contraction of the eye muscles, the signal varies according to the attachment position, and the blinking of the eye itself and the sensor detecting the same have aesthetic problems.

Next, using HCI using biopotential, EEG (ElectroEncephaloGraph) digitally measures EEG and detects and modulates biopotential change (EOG) caused by eye movement by electrophysiological method. There is a method used.

First, there is an approach to measure EEG and control commands or emotions based on human thoughts directly from the brain, but it takes time and effort to attach electrodes, has limited accuracy, and can move the head to patients. There is no big advantage.

Another approach using biopotential is an attempt to detect eye movement from an eye potential (EOG) and use it as an interface means. In other words, by measuring the eye conduction accompanying the movement of the eye to estimate the movement and flicker of the eye, to implement the interface from it. This method can be said to be excellent in the convenience and economical, aesthetics, etc. in detachable, but there is a problem that the eyes are easily tired.

Therefore, the present invention is to solve the above problems, in the present invention based on the head movement information of the disabled using only one sensor, a user-friendly computer interface device that can simultaneously control the movement and click of the mouse of the computer And to provide a method.

It is still another object of the present invention to provide a user-friendly computer interface device and method by allowing a system to learn and recognize a click pattern of each user by using an artificial intelligence module.

Computer interface device according to the present invention for achieving the above object, in the computer interface device consisting of a sensor unit 200, a micro computer unit 100, a main computer unit 600, the sensor unit 200 is Measures the motion information of the head using the sensor 210, and processes the signal using the filtering circuit 220 including a capacitor discharge switching circuit for early stabilization of the filter output and then transfers the signal to the microcomputer 100 The main computer unit 600 receives the digital signal converted by the microcomputer unit 100 and selects a click signal so that the AI module 900 learns to detect the click signal. And a real-time mouse control unit 800 for controlling the click signal at the same time to perform the position control of the cursor of the mouse in real time. It is characterized by displaying the switch and the click at the same time.

In addition, the method using such a device measures the movement of the head information using a sensor and generating a motion signal using a filtering circuit including a switching circuit for the capacitor forced discharge for early stabilization of the filter output (S200), Through an A / D converter for converting the analog signal, which is the motion signal, into a digital signal, a microprocessor for controlling the A / D converter and a communication module, and a microcomputer unit having a communication module for communicating with the main computer unit. Transmitting the converted digital signal to the main computer unit (S600), receiving a digital signal from the microcomputer unit (S710), selecting a click signal (S720), and detecting a user click signal using the selected signal. Offline learning step (S700), including the step (S730) for learning the artificial intelligence module so that, The real-time mouse control step (S800) and the real-time mouse, including the step of receiving the real-time digital signal from the microcomputer unit (S810) and controlling the position of the mouse cursor (S820) and simultaneously controlling the click signal (S830, S840) Characterized in that the step of displaying on the window at the same time using the mouse position control signal and the click signal output in the control step.

Hereinafter, the present invention will be described in more detail with reference to the drawings.

1 is a block diagram showing an overall system according to the present invention.

Referring to the overall configuration and signal flow of the present invention as shown in Figure 1 as follows. The angular velocity of two degrees of freedom of the head is measured using the sensor unit 200 that can easily measure the head movement, and the measured angular velocity signal is an A / D converter (Analog to Digital Converter) 400 of the microcomputer unit 100. After converting the analog signal into a digital signal and transmitting the converted output signal to the communication module of the main computer unit 600 through the communication module 500, through the real-time mouse control unit 800 of the main computer unit of the mouse The interface is implemented by converting it into signals corresponding to movements and clicks and handing them to Windows. At this time, the signal corresponding to the click is implemented by detecting the nodding motion of the head using the artificial intelligence module 900 learned by the user click pattern using the offline learning unit 700. Passing these signals to Windows, Windows recognizes the patient's head movements as if the mouse is moving, and allows all manipulations of the computer, including the Internet Browser.

The microcomputer unit 100 will be described in more detail. The microcomputer unit 100 includes an A / D converter 400, a communication module 500, and a microprocessor 300 for controlling the communication modules. The voltage from 0V to + 5V, which is an analog signal input from the sensor, is sampled at 100 Hz and converted into a digital signal that can be recognized by the microprocessor 300. In communication, two channel signals generated from two sensors are transmitted using a time division multiplexing (TDM) method. The communication module for communication between the main computer unit and the microcomputer unit may be wired or wireless.

2 is a circuit diagram illustrating a sensor unit.

The interface device for implementing the hardware of the present invention first needs a sensor to measure the head movements of the paralyzed patient, and it is preferably used as the sensor 210 in consideration of aesthetics, performance and price Ceramic Gyro (Ceramic Gyro) It is recommended to use a sensor.

Attaching the sensor takes advantage of the small size and weight, so in the development of the actual product attached to the accessories such as glasses or hats, easy to remove and minimize the aesthetic problems. The sensors were mounted in two horizontal and vertical directions to detect the vertical movement of the head.

The filtering circuit of the sensor unit 200 passes the output signal of the sensor through the high pass filter to eliminate drift and offset due to temperature changes, and then removes the high frequency noise. Pass it through. This is because the signal of the sensor 210 due to the movement of the head is mainly located in a low frequency band, and thus, when the cutoff frequency of the high pass filter is not set very low, an artifact signal appears. This virtual signal makes the operation of the mouse cursor very difficult. As a result, the time constant of the high pass filter becomes very large, which causes a problem in that it takes a long time for the filter output to stabilize to a normal state when the power is applied. Therefore, as shown in FIG. Shortly after application, short this switch to discharge the capacitor. The switch operation for discharging may be a manual operation, but may preferably be controlled by a timing signal of a microprocessor of the microcomputer unit.

3 is a block diagram showing an overall method of processing according to the present invention.

The method using the device having the above configuration comprises measuring the movement of the head using a sensor and generating a motion signal using a filtering circuit including a switching circuit for a capacitor forced discharge for early stabilization of the filter output ( S200), an A / D converter for converting the analog signal which is the motion signal into a digital signal, a microprocessor having a microprocessor for controlling the A / D converter and a communication module and a communication module for communicating with the main computer unit In step S600, the digital signal converted through the computer unit is transferred to the main computer unit. The digital signal from the microcomputer unit is received (S710), the click signal is selected (S720), and the click signal for each user is selected using the selected signal. Offline learning comprising the step (S730) for learning the artificial intelligence module to be detected Real-time mouse control step comprising the step (S700), receiving the real-time digital signal from the microcomputer unit (S810) to control the position of the mouse cursor (S820) and simultaneously control the click signal (S830, S840) S800) and the mouse position control signal and the click signal output in the real-time mouse control step to display on the window at the same time through the mouse position and click according to the movement of the head on the window.

4 is a block diagram illustrating in more detail the relationship between an offline learning step and a real time mouse control step for detecting a click signal and realizing mouse position and click in real time.

 First, in the offline learning step (S700) to select a signal corresponding to the click from the digital signal entered through the communication module (S720) to train the artificial intelligence module (S730). In the real-time mouse control step (S800) to implement the up and down, left and right movements of the mouse cursor with a real-time digital signal coming through the communication module (S810) (S820), and also inputs the upper and lower signals to the artificial intelligence module to detect the click signal (S830, S840). The click signal detected through the above process and the signals corresponding to the up, down, left and right of the real-time mouse cursor are displayed on the window through the display step on the window.

5 is a block diagram illustrating a 3-layer perceptron as an example of an artificial intelligence module used in an offline learning step according to the present invention.

Three input patterns corresponding to the intentional nod of the subject and the input patterns when freely moved during the movement of the head are selected. In the former case, click is set as the teacher output signal, and in the latter case, the non-click is the teacher output signal. Set to. Each input pattern vector Is composed of time delay data as shown in equation (1). When the y (n) value is input to the neural network, offset removal and scaling is performed as in Equation 1 to prevent the neural network from saturation or the weight between neurons being too small. In addition, when a click occurs by the output of the neural network, the click generation position is recognized as the click position at which the click is desired, not as the cursor position at the time of recognition as the click, but as the cursor position where the click pattern started.

p _x , p _y : the x and y position of the cursor (pixels)

w _x , w _y : weighting factors

x (n), y (n): angular velocity in the x and y directions

The position control of the mouse cursor proceeds in parallel with the recognition of the click. As shown in Equation 2, the pixel unit position of the mouse is obtained by subtracting the sensor output offset from the digital signals x (n) and y (n) corresponding to the head movement (angular velocity) signals entering the communication module of the main computer unit. Convert to (p _x , p _y ). Therefore, as the angular velocity increases, the speed of the movement of the mouse cursor also increases accordingly. The weight of the integration (w _x , w _y ) is determined experimentally. In addition, in order to prevent the mouse cursor from shaking in response to the small amount of noise included in the sensor signal, the signal included in a certain range was considered as noise and excluded from the numerical integration.

What has been described above is only one embodiment according to the present invention, and the present invention is not limited to the above-described embodiment, and as claimed in the following claims, the present invention belongs without departing from the gist of the present invention. Anyone skilled in the art will have the technical spirit of the present invention to the extent that various modifications can be made.

According to the present invention, rather than forcing the user to acquire a predetermined mouse operation pattern like the existing HCI by combining the artificial intelligence module, the system can learn a user's click pattern by itself and provide a user-friendly interface. have.

As another effect of the present invention, by implementing a mouse interface using only one sensor, there is an effect that can be practical to solve the size, wearing, aesthetic problems.

Another effect of the present invention, unlike the method of using eye movements and flickering, since the position and click of the mouse can be implemented only by the movement of the head, there is an effect that does not cause eye fatigue even when used for a long time.

Therefore, this system is effective in providing the disabled with a new interface device that can communicate with society.

1 is a block diagram illustrating an overall system according to the present invention.

2 is a circuit diagram showing a sensor unit included;

3 is a flow diagram generally illustrating the method processing according to the present invention.

4 is a flowchart illustrating a relationship between an offline learning step and a real time mouse control step for detecting a click signal and realizing mouse position and click in real time.

Figure 5 is a block diagram showing a three-layer perceptron (3-Layer Perceptron) as an example of the artificial intelligence module used in the offline learning step according to the present invention.

<Description of Symbols for Main Drawings>

100: micro computer unit 200: sensor unit

300: microprocessor 400: A / D converter

500: communication module 600: main computer

700: offline learning unit 800: real-time mouse control unit

900: artificial intelligence module

210: sensor 220: filtering circuit

Claims (8)

In the computer interface device consisting of the sensor unit 200, the microcomputer unit 100, the main computer unit 600, The sensor unit 200 measures the head movement information using the sensor 210 and performs signal processing using a filtering circuit 220 including a capacitor discharge switching circuit for early stabilization of the filter output. Transfer to the computer unit 100, The main computer unit 600 receives the digital signal converted by the microcomputer unit 100 and selects a click signal so that the AI module 900 learns to detect the click signal. And a real-time mouse control unit 800 for controlling the click signal by simultaneously detecting the position of the mouse cursor and controlling the click signal at the same time. Device. The computer interface device according to claim 1, wherein the filtering circuit of the sensor unit is controlled by a timing signal of a microprocessor of the microcomputer unit, for which a capacitor forced discharge switching circuit for early stabilization of a filter output is controlled. In the computer interface method using a computer interface device consisting of a sensor unit 200, a microcomputer unit 100, the main computer unit 600, Measuring movement of the head using a sensor and generating a movement signal using a filtering circuit including a switching circuit for discharging a capacitor for early stabilization of the filter output (S200); Through an A / D converter for converting the analog signal, which is the motion signal, into a digital signal, a microprocessor for controlling the A / D converter and a communication module, and a microcomputer unit having a communication module for communicating with the main computer unit. Transmitting the converted digital signal to the main computer unit (S600), And receiving a digital signal from the microcomputer unit (S710), selecting a click signal (S720), and learning an artificial intelligence module to detect a click signal for each user using the selected signal (S730). Offline learning step (S700), Receiving a real-time digital signal from the microcomputer unit (S810) and controlling the position of the mouse cursor (S820) and simultaneously detecting a click signal (S830 and S840); And displaying a mouse position control signal and a click signal output in the real time mouse control step on a window. 4. The method of claim 3, wherein the click signal selection step of the offline learning step selects an input pattern corresponding to an intentional nod and an input pattern when freely moved, and sets clicks as teacher output signals in case of intentional nods and moves freely. In the case of the computer interface method characterized in that the non-click using the input pattern vector set as the teacher output signal. 5. The computer interface method according to claim 3 or 4, wherein the click signal selection step of the offline learning step uses an input pattern vector in which each input pattern vector consists of time delay data. 6. The computer interface method according to any one of claims 3, 4 and 5, wherein the click portion selection step of the offline learning step performs offset adjustment and scaling of an input pattern vector. The method of claim 3, wherein the real-time mouse control step sets a noise range of the signal and removes the signal included in the range as noise in order to prevent the mouse cursor from shaking in response to the noise signal of the sensor. Computer interface method characterized in that using. The start position according to claim 3, wherein when a click occurs as an output of the artificial intelligence module, a click generation position is not a cursor position at the time recognized as a click, but a cursor position at which a pattern of a click starts, and a start position where a click is desired. Computer interface method, characterized in that it is recognized as being clicked on.