TW591481B - Vision-driving control system and control method thereof - Google Patents

Abstract

The vision-driving control method of the present invention comprises: providing the optical image data for the user who has vision, the optical image data having plural display regions with different flicker signals; measuring the averaged evoked response corresponding to the sensed optical image data, so as to known the display region viewed, and output the control signal corresponding to the display region for proceeding the control.

Description

[Technical field to which the invention belongs] The present invention provides a vision-driven control method and system, and more particularly, a vision-driven control method and method for measuring brain wave-evoked signals generated by a user's vision so as to know and control his gaze area and control. system. [Prior art] A currently known visual drive control system, as shown in the first figure, uses a detector located in front of the user to track eye movements, and then reverses the focus of the user to achieve the user's attention. The effect of control. Therefore, before each use, it is better to calibrate the relative position of the user and the detector for correct operation. And the more precise the vision-driven control system is, the more complicated and tedious the adjustment steps are, and the longer the required time is. During use, in order to accurately track the movement of the eyeball, it is necessary to limit the range of motion of the user's head, which is quite inconvenient to use. y restrictions or tracking, so it is still quite inconvenient. The shortcomings of financial limitations of school and head movements can not be paused during use. For this reason, some visual drive control systems choose to relatively fix the detector; the head of the camera is U 卩, forming a so-called "head cover system". In order to minimize the relative motion between the traverse and the user's head, the helmet, the helmet, must be fastened to the user's head, and the movement and rotation speed of the head must be added. At the same time, due to the above-mentioned adjustments before use, the visual drive control system also caused the user to control the pressure ball movement. The visual aid tool, due to the principle of the general visual drive control system, relies on the eye tracking, so when the user Wearing ordinary glasses or contact lenses in front of the eyeball or on the eyeball will cause blessing 591481; ::: ϋ7; ::::: 3 ...._ ': Γ Misjudgment of eyeball movement and eye focal length by the detector : As for sunglasses, it is more likely to cause the entire control system to fail. Paradoxically, users' matching will undoubtedly force some users into a dilemma. In addition, actions such as blinking, rubbing eyes, and even tears may affect the determination of eye movements by the tester, and many restrictions on use are quite inconvenient for the user. [Summary of the Invention] The wearing of vision aids is to obtain normal vision, if it is impossible to obtain normal vision abilities, it is even more impossible to use it as a visual drive control system;

It is therefore an object of the present invention to provide a method and system for visually driving control by measuring visually induced brainwave signals. Another object of the present invention is to provide a vision drive control method and system that do not need to be adjusted before use. It is still another object of the present invention to provide a vision driving control method and system that do not accumulate errors.

Still another object of the present invention is to provide a vision driving control method and system without restricting a user's head movement and wearing a vision aid. Therefore, the vision-driven control method of the present invention measures a brain wave signal when a user with visual ability feels an optical image data to output a control signal, and the control method includes the following steps: a) displaying one with a complex digital display Optical image data without zones' Each display area has a flashing signal, and the flashing signals are different from each other; b) capturing the user's brain wave signal; c) judging that the flashing signals have the same brain wave signal as the brain wave Flashing signal with maximum signal correlation; and 7

• —— ^^ · ^ Negotiable face to face ___ 恋 让 __ Negotiate d) Output the control signal corresponding to the flashing signal with the maximum correlation with the brain wave signal. The vision-driven control system using the above-mentioned vision-driven control method includes a display device that provides the optical image data and a measurement device that captures the same brain wave signals as the 5 brain waves of the user. It is characterized in that the optical image material has the display areas, each of the display areas has a flicker signal, and the flicker signals are different from each other; and the control system further includes a comparison between the brain wave signal and the flickers. A signal and a processing device that generates the control signal. [Embodiment] 10 The foregoing and other technical contents, features, and advantages of the present invention will be clearly shown in the following detailed description with reference to the second preferred embodiment of the drawings. As shown in the first figure, there are many functional regions in the human brain, including the visual cortical region n related to visual abilities. When 15 eyes receive stimuli from outside light, corresponding neurons in the visual cortical region 11 will be activated and activated, in addition to causing changes in the local ion concentration, and will be generated in the vicinity of the visual cortical region U. Changes in trace electric and magnetic fields. This change in electric and magnetic fields can be measured by an electroencephalograph via electrodes attached to the scalp surface, or by an electroencephalograph that does not need to touch the surface of the head plus 20 to detect. Because the brain wave formed by the change of the electromagnetic field has the advantage of high temporal resolution, the nerve activation signal generated in the brain can be detected in real time. The aforementioned nerve activation signal belongs to the so-called averaged evoked response. Its broad definition refers to when the brain is connected to 8 591481: sea, one ': a factory meal: the factory is stimulated by the outside world or by its own will When a certain action is to be performed, a brainwave change of a specific waveform is generated in a corresponding region of the brain. Such as the VEP (visuai evoked potential) and VEF (visuai evked field) generated when the human eye receives color or party visual stimulation. The above-mentioned brainwave signal has the characteristics of phase locked (locked) and time locked (locked). The so-called phase lock means that the initial phase of the occurrence of the brainwave signal is fixed. The so-called time lock refers to the time between the occurrence of the brain wave signal and the time point when a specific stimulus is applied, and the interval is a specific time interval. ίο 15 It can be seen that if the eye is stimulated with a predetermined blinking light, the optical signal is converted into an electrical signal by the optic nerve and transmitted to the brain, and the brain's visual cortex will generate brain waves related to the blinking signal. Signal. As shown in the second figure, 'using the above-mentioned phase locking principle, the first preferred embodiment of the visual drive control method and visual drive control system 2 of the present invention is to provide optical image data 31 to a user 10 with visual ability And measure its brainwave-evoked signal to output a corresponding control signal, so as to achieve the purpose of control or communication. The control system 2 includes-a display device 3 that provides the optical image data 31, a head of a nearby user 10 A measuring device 4 and a processing device 5 capable of receiving the information of the measuring device 4 and a controlled device 6 capable of receiving the information of the processing device 5 for the user 10 to operate. In this embodiment, The display device 3 is a liquid crystal display connected to the processing device 5 and has a liquid crystal display screen 30 displaying the optical image data frame. It must be noted here that the display device 3 is not limited to the liquid crystal display in this embodiment, whether it is direct or reflected light 9 20 591481 r: r :: a light source that stimulates the use of light signals from the eyes And-= ::: For example, the display device 3 can also be… the projector should learn the image data 31 first. There are four knifes marked with different English letters. The English alphabet

The available order is A, B, C and D. Each of the display areas 32 has a blink signal of +3, but the blink signals are different from each other. Of course, the shape and number of the optical shadow ▲ ~~ 1 mound 31 and these display areas 32 10 are not limited to this, and various changes can be made according to different needs. I will not repeat them here. In this embodiment, the ㈣ signals of each of the display areas have flicker frequencies different from each other. The English letters indicated by the display areas 32 and the flashing signals they have are shown in Table 1 below:

15 As shown in the second and third figures, in this embodiment, the measuring device 4 is a brain wave device for measuring changes in the electric field on the head using I 10. It includes a plurality of sets of measurement probes 42 applied on the epidermis of the head of the user 10 adjacent to the f-zone of the visual skin, and a signal amplifier 41 connected to the measurement probes 42. The signal amplifier 41 amplifies the obtained brain wave signal and outputs it to the processing device $. Of course, those skilled in the art can easily infer that the above brain wave device can also be replaced by a brain wave device. The processing device 5 includes a pre-processing unit 51, an analysis unit 5220, and a determination unit 53. The pre-processing unit 51 is used to receive the brain wave § # 'transmitted from the measuring device 4 and remove the non-optical image data 10 591481 dirty π -.:fenπ——κ and display H in accordance with each display. The flicker frequency analysis corresponding to the predetermined off signal of 32 calculates the correlation between each signal and the brain wave signal in the frequency domain. The frequency and other noises included in the data are helpful for subsequent calculations. The analysis early 52 is to receive the brain wave signal processed by the pre-processing unit 51, and the judgment unit 53 is used to compare the correlation degree calculated by the analysis unit 52 to determine the correlation with the measured brain wave signal The largest flash signal can then compare the maximum correlation with a preset threshold. If the maximum correlation is less than the threshold value, the pre-processing unit 51 receives the electroencephalogram signal again by the measuring device 45, and then performs correlation analysis through the analysis unit 52 for re-judgment. Otherwise, the judgment unit 53 will define the tool

The flashing signal with the above maximum correlation is the selected signal and sends the corresponding I control signal. Of course, those skilled in the art can easily understand that the aforementioned pre-processing unit 5 丨 is not absolutely necessary, and the analysis unit 52 and the judgment unit 53 can also be integrated. 15 In this embodiment, the processing device 5 has a processor 54, a storage = media 55 that stores a plurality of software and databases that can be executed on the processor 54, and-an output connected to the processing n 54 Jie φ brother. The shell library stores the aforementioned preset thresholds, the flashing signals, and a plurality of instructions corresponding to the flashing signals, respectively. These instructions and their corresponding flashing signals are shown in Table 2: Table 1 Flashing signals 3Hz 4Hz 5Hz 6Hz Corresponding commands display A display B display C display D ^ and the software includes a method for generating the optical image using the flash signals The display program of material 31, and an execution program with functions of executing the pre-processing units 51, 20 11, the analysis unit 52 and the judgment unit 53. The execution program is performed by the judgment unit 53. When the control signal is the database, The command signal of the flashing signal device 54 is transmitted to the wheel: === stroke control. The process sends the control signal. The reason: " face 56 is used to transmit electricity to the palace, and can be wired (such as wired

Benefit control, I · or ADSL line) or wireless (such as Bluetooth, 1EEE802.U agreement or mobile phone mobile communication protocol) and other types of transmission of the control 10 people 1 ^ This is not the main technical means of the present invention, and is familiar with the The technique is simple, so I won't go into details here. 15 :: the control device 6 can receive the control signal through the transmission interface 56. In this embodiment, the controlled device 6 is a monitor with _Fu Mu M: but it is not limited to this. The controlled device 6 has an effect when it accepts the control: number, so the controlled device 6 looks at the user ’s surrounding or used devices, such as _printer switches, etc., or even away from the user. The remote devices of 10 are all in the scope of application of the present invention. Therefore, the present invention will be described below with the above-mentioned vision drive control system 2

The steps of the bright vision driving control method, as shown in the fourth figure, include the following steps: s Step 200, displaying the above-mentioned optical image data M for the user 10 to watch. In this embodiment, the optical image 3m is displayed by the processing device 5 executing a display program to drive the display device 3, and the information displayed in the display areas 32 and the flicker frequency thereof are shown in Table 1. 12 591481 ..... "Factory :: two—, one :, •" belongs to the city ^^^^^^ __ away | 1 ^^^ _! 1 || ^^ 11 circle_11 Stupid! _ | Ϊ́ιήΐΙβΙΙΐ1ΐΙ__Ι ___ βΛββ®ί Step 202, then the measurement device 4 senses and obtains the brain wave signal of the user 1 ο and transmits it to the processing device 5. If the user 10 is watching the block marked with the English letter C at this time, the measured value of the brain wave signal is 5 Hz. 5 In step 204, the pre-processing unit 51 filters out the low and medium brain wave signals.

At frequencies above 2Hz and above 7Hz. The reason why the brain wave signal is filtered below 2 Hz or higher than 7 Hz is because the frequency shown in the above-mentioned optical image data 31 is only between 3 Hz and 6 Hz, but the optical image can also be disregarded. The frequency range shown in the data 31, and directly connected to the frequency of less than 0. 05 Η ζ and more than 50 Hz except for inducing brain waves. Step 206: Analyze and calculate the correlation between the brain wave signal and the flicker frequency of each of the flicker signals. In this embodiment, the analysis unit calculates the correlation between the brain wave signal with a frequency close to 5 Hz and each of the flashing signals in Table 2 above. 15 Step 208, the determination unit 53 compares the analysis unit in step 206

The correlation is obtained, and the maximum correlation is judged to be 100%. In step 210, the judging unit 53 searches for a blinking signal 'corresponding to the maximum correlation and defines it as the frequency selected by the user. In this embodiment, the flicker frequency 20 of the flicker signal having the maximum correlation with the brain wave signal is 5 Hz, and the corresponding control signal is shown in Table 2. The c ”command step 212 is displayed to compare the maximum correlation And the threshold value. In this embodiment, the threshold value is preset to 90%. Since the maximum correlation in step 208 is not less than the threshold value, proceed directly to the next step, otherwise step 13 591481 must be repeated.

202 to step 210.

^ Step 214: Output a control signal corresponding to the flash signal having the maximum correlation. It can be known from step 21G that the flashlight with the highest correlation ^ shows c, according to the corresponding command, so the control signal transmitted by the processor 54 to the output interface 56 is "display c,". Final step 216 To complete the sending of the control signal. Since the transmission interface 56 will transmit the control signal showing " c " to the controlled device 6, the controlled device 6 will be subject to the deduction of 10 by the screen 61 " Show the capital "c" instead of C "control, and then achieve the purpose of control and communication. ^ Although the display areas for selection in this example only show only part of the English language, but by the same token, Each display area can also display complete instructions such as "dial the phone", "turn on the light", "turn on the TV", etc. as a complete control interface; or-display all the phonetic symbols as writing tools at a time; even let cars or fly 15 machines The driver cooperates with a specially designed instrument panel or head-up display to achieve the "brainwave control" effect in the movie "Fire Fox". Since the time between the occurrence of the brainwave-evoked signal and the time point when a specific stimulus is applied-a specific time interval, and for the same person, the difference between the specific time interval is less than 0.1 seconds, the visual drive control method and 20 The second preferred embodiment of the visual drive control system 2 is to use the time-lock principle to similarly stimulate the eye with a predetermined blinking light, and convert the optical signal into an electrical signal via the optic nerve and conduct it to the brain. The brain's visual cortex area will generate brain wave signals related to this flashing signal after that specific time. The control system 2 required in this embodiment is substantially the same as the first preferred implementation 14 described above—2..¾, ㈣edition example, and the optical image data 31 has four display areas respectively marked with different English letters. 32. The difference is that the blinking signals of the display areas 32 have the same blinking frequency, but different timings of blinking. In this embodiment, the blinking signals all have a blinking frequency of 10 Hz, and each = the blinking signal repeats a cycle of 丨 seconds, that is, each of the blinking sequences is within a period of 1 second, From 10 groups of 01-second light and dark light signals are arranged in chronological order. The English alphabets marked in these display areas 32 and their timings are as follows:

Acupoint-η ABCD flashing signal 1010110011 1010111110 0000100001 ^ 11110100 BU Zhu · 1 "Dirty two children '〇" means dark _, and the flashing signals corresponding to these instructions and their corresponding are shown in Table 4 below: Table four hundred flashing signals 1010110011 1010111110 10000100001 1111110100 Correspond to the command display A display B display C display D 10 The processing | setting 5 is also approximately the same as the _ embodiment, but is a flickering signal with the same flicker frequency but different timing of on and off, in this example Here, the calculation method of the analysis unit 52 is to perform correlation analysis on the time domain of each of the closing signals and the wave 5 in accordance with the time sequence of each display area M <predetermined blinking_. 15 In the following, the actual & steps of the visual driving control method of the second preferred embodiment of the present invention will be described. As shown in the fifth figure, the control method includes the following step 300. The displays have the same flicker frequency, but are related to each other. Differentiated 15 591481 Extreme __ .......... ,, ..., ._, &gt; · &quot; &lt; &gt; ... :: ... &quot;. ·,. ·· &quot;. .. ν ···· νΐ.μΜ -.-. ν! »·!-^. Λ: &gt;?,.; ··· Τ · ^ »· These flashing signals before the day of death. In this embodiment, the optical image material 31 is still used to display the display areas 32 with flicker signals that are different from each other, and the information displayed on the display areas 32 and the flashing signals they have are shown in Table III. 5 Step 302 The brain wave signal of the user 10 is obtained by the measuring device 4 and transmitted to the processing device 5. If the user 10 is still watching the block marked with the non-English letter C at this time, the measured value of the brain wave signal is "00001000010000100001 ...", where "i" means bright, and "〇" means not dark. . Step 304: The unnecessary noise is filtered by the pre-processing unit 51. Step 306: Analyze and calculate the correlation between the brain wave signal and the timing of the blinking signal. Here again, the correlation between the aforementioned brain wave signal and each of the flicker signals in Table 3 is calculated by the analysis unit 52. In step 308, the determination unit 53 compares the correlation obtained in the analysis in step 306, and determines that the maximum correlation is 100%. In step 310, a blinking signal corresponding to the maximum correlation is searched by the judging unit 53 and defined as a frequency selected by the user. In this embodiment, the blinking sequence of the blinking signal that has the greatest correlation with the brain wave signal is "0000100001", and the corresponding control signal is shown in Table 4, which shows the 20 C "command. Step 312, compare the maximum correlation The degree and the threshold. If the maximum correlation is not less than the threshold, go directly to the next step, otherwise you must repeat steps 302 to 310. In this embodiment, the threshold is preset to 95%, because The maximum correlation in step 308 is not less than the threshold, so directly go to 16 591481

The next step. Step 314: Output a control signal corresponding to the blinking signal having the maximum correlation. The command corresponding to the flashing signal with the highest correlation is "display C", so the control signal sent is "display c". 5 Step 316: Send the control signal.

Since the above-mentioned blinking signals have mutually different blinking timings and the effect of binary digital coding is the same, as described above, the blinking signals formed by 10 groups of 01-second light and dark light signals arranged in time sequence have 2 ι The power of the 0th order is different from each other, that is, the optical image data 3 丨 has the ability to display the display area 32 of the 10th order of the same day 2 with different flashing signals. Of course, if the frequency of each of the blinking signals is changed, or the number of bright and dark light signals of each of the blinking signals is increased, the number of display areas 32 that can be displayed by the optical image data 31 can be increased. On the other hand, since bright and dark light signals and digital signals generally have a very good discrimination rate, the correlation between the flicker signal and the brain wave signal is very high.

The contrast effect is quite obvious, so it can not only increase the preset threshold. When the judgment unit 53 compares the degree of correlation, it can more accurately determine the blinking signal that has the highest degree of correlation with the brain wave signal and exceeds the threshold value, so as to send a corresponding control signal. 20 In summary, as the visual drive control method and system of the present invention measure the user's brain wave signal and perform analysis control, it is not necessary to perform any adjustment of the relative position of the user and the detector before operation, and Can start operation directly. The convenience of operation can effectively improve the acceptance of the visual drive control system by people with disabilities, for example. 17 591481

In addition, the user does not affect the measurement and analysis of brain wave signals regardless of the wearing of any visual aids. Therefore, the visual drive control method and system of the present invention do not avoid any visual aids. At the same time, the visual drive control method and system of the present invention only measure the 5 brainwave frequencies of the user, and there is no restriction on the movement of the 10 eyes of the user, and no error is accumulated. Therefore, it can be suspended at any time during the use and continued at any time. The use is quite user-friendly, and it can indeed achieve the purpose of the present invention. However, the above are only the two preferred embodiments of the present invention. When the scope of implementation of the present invention cannot be limited by this, that is, the simple equivalent changes made according to the scope of the patent application and the content of the invention specification And modifications should still fall within the scope of the invention patent. [Schematic description] Schematic diagram; A preferred embodiment illustrates the control system. The first picture is the position of the visual cortex of the human brain — 15 The second picture is the unbearable picture of the visual drive control system of the present invention, illustrating the control The configuration relationship of the system; the third diagram is the schematic diagram of the first preferred embodiment, the architecture of the system; 20 the fourth diagram is the steps of the motion control method of the first preferred embodiment; and the flowchart illustrates the visual drive Figures 5 and 5 are flowcharts of the visual drive control system of the present invention, and the preferred embodiment illustrates the steps of the visual drive control method. 18 591481 Mei: Ninger description (14): :: Γ [The main elements of the diagram are simply explained 1 brain 42 10 user 5 11 visual cortical area 51 2 visual drive control system 52 3 display device 53 30 display screen 54 31 Optical image data 55 32 Display area 56 4 Measuring device 6 41 Signal amplifier 61 200.202.204.206.208.210.2i2.2i4.2io. Step 300 · 302 · 304 · 306 · 308 · 310 · 312 · 314 · 316 Step measurement probe processing device Pre-processing unit analysis unit Judgment unit processor Storage media output interface Controlled device screen 19

Claims (1)

591481 ι · A vision-driven control method is to measure a brain wave signal when a user with visual ability feels an optical image data to output a control signal. The control method includes the following steps: a) displaying a display area having a plurality of numbers Optical image data, each of the display areas has a flashing signal, and the flashing signals are different from each other; b) capturing the brain wave signal of the user; c) judging that the flickering signals have a flickering signal having the maximum correlation with the electroencephalogram signal; and d) output a control signal corresponding to the flickering signal having the maximum correlation with the electroencephalogram signal. 2. The visual drive control method according to item 1 of the scope of patent application, wherein the flicker signals have flicker frequencies different from each other. 3. The vision-driven control method according to item 1 of the scope of patent application, wherein These flashing signals have the same flashing frequency and different annihilation sequences. -^ 4_ According to the visual drive control method described in item 丨 of the patent application scope, wherein step c) includes the following steps: C-1) Analyze the correlation between each flash light signal and the brain wave signal. C-2 ) Find the maximum correlation; and c-3) determine the blinking signal corresponding to the maximum correlation. 5. According to the vision-driven control method described in item 1 of the scope of the patent application, wherein a threshold value for comparison with the correlation degree is defined, and in step c) and step ^ yj-J Taj 20 Including the following steps: e) If the maximum correlation is less than the threshold, repeat step a) to step c) 〇 ·· A kind of visual drive control system is to measure a user's sense optical image data with visual ability The brain wave signal of the time to output a control signal, the control system includes a display device that provides the optical image data and a measurement device that captures the same brain wave signal as the brain wave of the user; The optical image data has a plurality of display areas, each of which has a flicker signal, and the flicker signals are different from each other; and the control system further includes-comparing the brain wave signal with the flash signals and generating the control signal Of processing equipment. 7. The visual drive control system according to item 6 of the patent claim, wherein ‘the special blinking numbers have different blinking frequencies from each other. 8. According to the description in item 6 of the patent scope, the flicker signals have the same flicker frequency and different timings of blinking 0. 9. According to the vision drive control system described in item 6 of the scope of patent application, Among them, the control signal corresponds to the flashing signal among the signals having the maximum correlation with the signal of the brain wave signal. 10. The visual drive control system according to item 6 of the patent application patent, wherein the processing device includes a database storing a plurality of instructions corresponding to the material flashing signal respectively; the processing device is corresponding to The command output of the flashing signal which has the highest correlation with the signal of the brain wave signal 21 591481 Generate the control signal.