CN110597201A - Intelligent household system controlled by AR equipment and control method thereof - Google Patents
Intelligent household system controlled by AR equipment and control method thereof Download PDFInfo
- Publication number
- CN110597201A CN110597201A CN201910836884.7A CN201910836884A CN110597201A CN 110597201 A CN110597201 A CN 110597201A CN 201910836884 A CN201910836884 A CN 201910836884A CN 110597201 A CN110597201 A CN 110597201A
- Authority
- CN
- China
- Prior art keywords
- frequency
- area
- microcontroller
- signal
- stroboscopic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 16
- 238000001514 detection method Methods 0.000 claims abstract description 57
- 238000004891 communication Methods 0.000 claims abstract description 32
- 210000004761 scalp Anatomy 0.000 claims abstract description 4
- 230000000875 corresponding effect Effects 0.000 claims description 53
- 230000000763 evoking effect Effects 0.000 claims description 35
- 230000000007 visual effect Effects 0.000 claims description 35
- 238000012545 processing Methods 0.000 claims description 28
- 210000004556 brain Anatomy 0.000 claims description 17
- 210000003128 head Anatomy 0.000 claims description 7
- 210000005252 bulbus oculi Anatomy 0.000 claims description 4
- 230000000638 stimulation Effects 0.000 claims description 3
- 210000005069 ears Anatomy 0.000 claims 1
- 240000007320 Pinus strobus Species 0.000 description 35
- 238000005516 engineering process Methods 0.000 description 3
- 238000001914 filtration Methods 0.000 description 3
- 230000002457 bidirectional effect Effects 0.000 description 2
- 230000001276 controlling effect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 210000001508 eye Anatomy 0.000 description 2
- 208000003443 Unconsciousness Diseases 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000011017 operating method Methods 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B15/00—Systems controlled by a computer
- G05B15/02—Systems controlled by a computer electric
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B19/00—Programme-control systems
- G05B19/02—Programme-control systems electric
- G05B19/418—Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control [DNC], flexible manufacturing systems [FMS], integrated manufacturing systems [IMS] or computer integrated manufacturing [CIM]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B2219/00—Program-control systems
- G05B2219/20—Pc systems
- G05B2219/26—Pc applications
- G05B2219/2642—Domotique, domestic, home control, automation, smart house
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Automation & Control Theory (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Dermatology (AREA)
- General Health & Medical Sciences (AREA)
- Neurology (AREA)
- Neurosurgery (AREA)
- Quality & Reliability (AREA)
- Manufacturing & Machinery (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
Abstract
The invention discloses an intelligent home system controlled by AR equipment and a control method thereof, wherein the intelligent home system comprises the AR equipment and a plurality of home appliance equipment, the AR equipment is in communication connection with the home appliance equipment through a short-range wireless network, the AR equipment comprises a wearing main body, a microcontroller, an AR display lens, a communication module and a brain-computer interface, the microcontroller, the AR display lens, the communication module and the brain-computer interface are all arranged on the wearing main body, and the AR display lens, the communication module and the brain-computer interface are all electrically connected with the microcontroller; the brain-computer interface comprises a detection end, and the detection end is attached to the scalp of the human body; the microcontroller is in communication connection with the short-range wireless network through the communication module; the invention aims to provide an intelligent home system controlled by AR equipment and a control method thereof, which can flexibly control household appliances and can be suitable for multi-user groups without establishing a huge database.
Description
Technical Field
The invention relates to the field of intelligent wearable equipment, in particular to an intelligent home system controlled by AR equipment and a control method thereof.
Background
With the development of the internet of things technology, the functions of smart homes become more and more abundant, the control of non-direct-contact household appliances becomes more and more important, and AI control technologies such as gesture recognition and voice control are widely developed. The steady-state visual evoked potential (SSVEP) technology is a visual frequency signal extraction technology that is originally used in medicine, and now the field of consumers also starts to show the head and corner completely.
The existing intelligent control method, no matter gesture recognition or voice control, is greatly subject to different use habits of different users, so that the recognition of action instructions or voice instructions becomes complicated. In order to be able to recognize a variety of user instructions, the intelligent algorithm must be equipped with a huge user habit database and trained to optimize its recognition capability. This promotes the time, effort and labor costs of intelligent algorithms, and the collection and construction of databases.
Disclosure of Invention
The invention aims to provide an intelligent home system controlled by AR equipment and a control method thereof, which can flexibly control household appliances and can be suitable for a multi-user group without establishing a huge database.
In order to achieve the purpose, the invention adopts the following technical scheme: an intelligent home system controlled by AR equipment comprises the AR equipment and a plurality of home appliance equipment, wherein the AR equipment is in communication connection with the home appliance equipment through a short-range wireless network, the AR equipment comprises a wearing main body, a microcontroller, an AR display lens, a communication module and a brain-computer interface, the microcontroller, the AR display lens, the communication module and the brain-computer interface are all arranged on the wearing main body, and the AR display lens, the communication module and the brain-computer interface are all electrically connected with the microcontroller; the brain-computer interface comprises a detection end, and the detection end is attached to the scalp of the human body; the microcontroller is in communication connection with the short-range wireless network through the communication module;
the AR display lens is used for generating a plurality of functional areas with different flicker frequencies to watch a user so as to enable the user to generate different electroencephalogram signals;
the detection end of the brain-computer interface is used for detecting an electroencephalogram signal of a user and sending the electroencephalogram signal to the microcontroller;
the microcontroller is used for generating a corresponding control instruction according to the electroencephalogram signal and sending the control instruction to the corresponding household appliance;
the household electrical appliance is provided with a control unit, the control unit is electrically connected with a circuit board of the corresponding household electrical appliance, and the control unit is used for receiving a control instruction and executing corresponding actions.
Preferably, the AR device further includes a camera, the camera is disposed at the front end of the wearing body, and the camera is configured to detect the household appliance currently watched by the user through image recognition and feed back the household appliance to the microcontroller; the microcontroller is used for displaying a control function menu of the household appliance fed back by the camera.
Preferably, the brain-computer interface further comprises a signal acquisition module and a signal processing module, the detection end is electrically connected with the signal acquisition module, and the signal acquisition module is electrically connected with the signal processing module; the signal processing module is electrically connected with the microcontroller;
the signal acquisition module is used for acquiring the electroencephalogram signals detected by the detection end;
the signal processing module is used for removing frequency multiplication harmonic information from the acquired electroencephalogram signal, extracting real frequency information and feeding the real frequency information back to the microcontroller;
and the microcontroller is used for receiving the frequency value fed back by the signal processing module.
Preferably, the detection end includes first detection electrode and second detection electrode, first detection electrode sets up wear the main part and paste mutually with the temple of human head, the second detection electrode sets up wear the main part and paste mutually with the rear side of human ear, first detection electrode and second detection electrode set up wear the same one side of main part.
Preferably, the AR display lens is provided with a switch display area, a functional area and a switching area, and the refresh frequency of the AR display lens is a common multiple of the strobe frequency of the switch display area, the strobe frequency of the functional area and the strobe frequency of the switching area;
the stroboscopic frequency of the switch display area, the stroboscopic frequency of the functional area and the stroboscopic frequency of the switching area are all within the range of 4Hz-75 Hz;
at least one functional area is arranged; at least one switching area is arranged; the stroboscopic frequency of the switch display area, the stroboscopic frequency of each functional area and the stroboscopic frequency of each switching area are different;
the switch display area is used for displaying the switch selection of the currently controlled household appliance, the function area is used for displaying a certain function of the currently controlled household appliance, and the switching area is used for displaying the household appliance capable of being controlled in a switching mode.
Preferably, the stroboscopic frequency of the switch display area, the stroboscopic frequency of the functional area and the stroboscopic frequency of the switching area are all in the range of 30Hz-75 Hz.
The control method of the intelligent home system controlled by the AR equipment specifically comprises the following steps:
a. a switch display area, a function area and a switching area on an AR display lens of the AR equipment flicker with different strobe frequencies;
b. a user wears the AR equipment and watches the switch display area or the functional area or the switching area of the AR display lens through human eyeballs;
c. the detection end of the brain-computer interface detects a steady-state visual evoked potential signal generated by stimulation of a human brain by the stroboscopic frequency of a corresponding gazed area;
d. the microcontroller generates a corresponding control instruction according to the detected steady-state visual evoked potential signal and sends the control instruction to corresponding household appliances;
e. and the corresponding household appliance receives the control command and executes the corresponding action.
Preferably, the following steps are further included between step c and step d:
s1, the first detection electrode and the second detection electrode are matched to detect the steady-state visual evoked potential signal and send the steady-state visual evoked potential signal to the signal acquisition module;
s2, the signal acquisition module receives the steady-state visual evoked potential signal and sends the steady-state visual evoked potential signal to a signal processing module;
s3, the signal processing module removes frequency multiplication harmonic information from the steady-state visual evoked potential signal and sends a processed frequency signal to the microcontroller;
the step d is specifically that the microcontroller generates a corresponding control instruction according to the received frequency signal and sends the control instruction to the corresponding household appliance;
and f, when the state information of the household appliance is changed, sending the state information to the AR display lens through the short-range wireless network for displaying.
Preferably, in step a, the operating modes of the stroboscopic frequency of the switch display area, the stroboscopic frequency of the functional area, and the stroboscopic frequency of the switching area include two types:
one is that the switch display area, each function area and each switching area respectively use the constant stroboflash with different stroboflash frequencies;
the other is that the switch display area, each function area and each switching area respectively use the stroboscopic frequency periodical pulse stroboscopic with different variation ranges, and the periodical pulse stroboscopic jumps from one stroboscopic frequency to another stroboscopic frequency in one period.
Preferably, in step d, the microcontroller sets a minimum fixation duration, and compares the duration of generation of the received steady-state visual evoked potential signal with the minimum fixation duration:
if the minimum gazing duration is less than the minimum gazing duration, generating no corresponding control instruction;
and if the minimum gazing duration time is greater than or equal to the minimum gazing duration time, generating a corresponding control command and sending the control command to the corresponding household appliance.
By adopting the scheme, the household appliance is controlled by the user in a watching mode without outputting a control instruction by the user, so that the operation steps are greatly simplified. And a database does not need to be established, and the control gestures or control voices of the user do not need to be collected and recognized. The method for controlling the AR equipment can simplify the control steps, does not need to collect and build a database, saves time and energy for perfecting the database, and can save cost while being convenient to control.
Drawings
The drawings are further illustrative of the invention and the content of the drawings does not constitute any limitation of the invention.
FIG. 1 is a schematic structural view of the present invention;
FIG. 2 is a flow chart of the present invention;
FIG. 3 is a block diagram of the circuit connections of the present invention;
FIG. 4 is a schematic perspective view of an AR apparatus of the present invention;
fig. 5 is a schematic diagram of a partial structure of the AR device of the present invention.
Wherein: the system comprises a frame 1, a microcontroller 2, an AR display lens 3, a switch display area 3a, a functional area 3b, a switching area 3c, a communication module 4, a first RF module 4a, a first antenna 4b, a brain-computer interface 5, a detection end 5a, a signal acquisition module 5b, a signal processing module 5c, a first detection electrode 5d, a second detection electrode 5e, a short-range wireless network 6, a control unit 7, a second RF module 7a, a second antenna 7b and an execution module 7 c.
Detailed Description
The technical scheme of the invention is further explained by the specific implementation mode in combination with the attached drawings.
Referring to fig. 1 to 5, the smart home system controlled by the AR device of the present embodiment includes an AR device and a plurality of household electrical appliances, the AR device is in communication connection with the plurality of household electrical appliances through a short-range wireless network 6, the AR device includes a wearing main body 1, a microcontroller 2, an AR display lens 3, a communication module 4 and a brain-computer interface 5, the microcontroller 2, the AR display lens 3, the communication module 4 and the brain-computer interface 5 are all disposed on the wearing main body 1, and the AR display lens 3, the communication module 4 and the brain-computer interface 5 are all electrically connected to the microcontroller 2; the brain-computer interface 5 comprises a detection end 5a, and the detection end 5a is attached to the scalp of the human body; the microcontroller 2 is in communication connection with the short-range wireless network 6 through the communication module 4.
The AR display lens 3 is used for generating a plurality of functional areas with different flicker frequencies to watch a user, so that the user can generate different electroencephalogram signals.
The detection end 5a of the brain-computer interface 5 is used for detecting the electroencephalogram signals of the user and sending the electroencephalogram signals to the microcontroller 2.
And the microcontroller 2 is used for generating a corresponding control instruction according to the electroencephalogram signal and sending the control instruction to the corresponding household appliance.
The household electrical appliance is provided with a control unit 7, the control unit 7 is electrically connected with a circuit board of the corresponding household electrical appliance, and the control unit 7 is used for receiving a control instruction and executing corresponding action.
With this structure, the short-range wireless network 6 may preferably be one of an internet of things network or a bluetooth connection network or a WiFi network or another short-range wireless network. The microcontroller 2 preferably employs an ATmega168 chip. The AR device may be AR glasses, an AR helmet, or other head-worn device. The user is through watching the different functional areas of AR display lens 3 to produce different brain electrical signal, detect this brain electrical signal and send to microcontroller 2 through brain computer interface 5, microcontroller 2 produces control signal and controls corresponding household electrical appliances execution operation through communication module 4, short range wireless network 6 and each control unit 7.
The user carries out household electrical appliance control through the mode of watching, need not user output control command, has greatly simplified the operating procedure. Meanwhile, a database does not need to be established, and the control gestures or the control voices of the user are collected and recognized. The method for controlling the AR equipment can simplify the control steps, does not need to collect and build a database, saves time and energy for perfecting the database, and can save cost while being convenient to control.
Preferably, the AR device further includes a camera, the camera is disposed at the front end of the wearing body 1, and the camera is configured to detect the household appliance currently watched by the user through image recognition and feed back the household appliance to the microcontroller 2; and the microcontroller 2 is used for displaying a control function menu of the household appliance fed back by the camera.
Adopt this kind of structure, the front end of wearing main part 1 is equipped with the camera, the camera with microcontroller 2 electricity is connected, and microcontroller 2 receives the image of camera conveying and discerns, then shows the control function menu of the tame electric installation of camera feedback on AR display lens 3. The camera can assist and identify the electric appliance watched by the user at present in an image form, and the pressure of the AR display lens 3 on displaying the menu quantity can be reduced.
Preferably, the brain-computer interface 5 further includes a signal acquisition module 5b and a signal processing module 5c, the detection terminal 5a is electrically connected with the signal acquisition module 5b, and the signal acquisition module 5b is electrically connected with the signal processing module 5 c; the signal processing module 5c is electrically connected to the microcontroller 2.
The signal acquisition module 5b is used for acquiring the electroencephalogram signals detected by the detection end 5 a.
The signal processing module 5c is used for removing frequency multiplication harmonic information from the acquired electroencephalogram signal, extracting real frequency information and feeding the real frequency information back to the microcontroller 2.
The microcontroller 2 is configured to receive the frequency value fed back by the signal processing module 5 c.
With this structure, the detection terminal 5a transmits the detected electroencephalogram signal to the signal acquisition module 5b, and the signal acquisition module 5b preferably uses TI ADS1298, and may also select a TGAM chip. After receiving the electroencephalogram signal, the signal acquisition module 5b transmits the electroencephalogram signal to the signal processing module 5c, the signal processing module 5c performs frequency doubling harmonic information elimination processing on the electroencephalogram signal, real frequency information is extracted and fed back to the microcontroller 2, and the microcontroller 2 receives the fed-back frequency value.
The frequency of the brain wave signal is 0.5Hz-40Hz, and the signal processing module 5c filters the frequencies smaller than 0.5Hz and larger than 40Hz in a high-pass filtering, low-pass filtering and power frequency interference filtering mode, so that the interference caused by other frequencies is reduced, and the acquisition result is more accurate.
Preferably, the detection end 5a includes a first detection electrode 5d and a second detection electrode 5e, the first detection electrode 5d is disposed on the body 1 and attached to the temple of the head of the human body, the second detection electrode 5e is disposed on the body 1 and attached to the rear side of the ear of the human body, and the first detection electrode 5d and the second detection electrode 5e are disposed on the same side of the body 1.
By adopting the structure, the distance between the first detection electrode 5d and the second detection electrode 5e is preferably 3-6cm, and the temple of the head of the human body and the rear side of the ear of the human body are selected as detection points, so that a larger potential difference can be formed, the signal extraction effect is enhanced, and the signal to noise ratio is improved. Of course, other positions of the head can be selected for electroencephalogram signal detection.
Preferably, the AR display lens 3 is provided with a switch display area 3a, a functional area 3b and a switching area 3c, and the refresh frequency of the AR display lens 3 is a common multiple of the strobe frequency of the switch display area 3a, the strobe frequency of the functional area 3b and the strobe frequency of the switching area 3 c.
The stroboscopic frequency of the switch display area 3a, the stroboscopic frequency of the functional area 3b and the stroboscopic frequency of the switching area 3c are all within the range of 4Hz-75 Hz.
At least one functional area 3b is arranged; the switching zone 3c is provided with at least one. The strobe frequency of the switch display area 3a, the strobe frequency of each of the functional areas 3b, and the strobe frequency of each of the switching areas 3c are different.
The switch display area 3a is used for displaying the switch selection of the currently controlled household appliance, the function area 3b is used for displaying a certain function of the currently controlled household appliance, and the switching area 3c is used for displaying the household appliance capable of being controlled in a switching mode.
With such a structure, the switch display area 3a, the function area 3b and the switching area 3c of the AR display lens 3 have a frequency conversion flashing function, which means that the AR display content can flash correspondingly at different frequencies in different working states, or regularly flash at different frequencies in a period in different working states.
The AR display lens 3, as a display device, has its own fixed screen refresh frequency, so the strobe frequency of the different display areas needs to be a divisor of the refresh frequency. For example: the refresh rate of the AR display lens 3 is 144Hz, the strobing frequency of the display area can be strobed with a frequency of 24Hz/36Hz/48 Hz.
The stroboscopic frequency of switch display area 3a, the stroboscopic frequency of functional zone 3b and the stroboscopic frequency of switch area 3c set up to different frequency values, can stimulate the human body and produce different brain electrical signal, make things convenient for brain machine interface 5 to detect and discern this brain electrical signal to send microcontroller 2, microcontroller 2 sends different control instruction according to the brain electrical signal of difference, and carry out corresponding operation through the different household electrical appliances of short-range wireless network 6 control.
The control of the household appliances is realized by distributing the limited frequency resources to different household appliance control instructions. This allocation can be freely defined by the developer, and its permutation and combination is various.
Preferably, the strobe frequency of the switch display area 3a, the strobe frequency of the functional area 3b, and the strobe frequency of the switching area 3c are all in the range of 30Hz to 75 Hz.
By adopting the structure, one of the characteristics of the human eyeball is that the stroboflash with the frequency of more than 30Hz is nearly normally bright when seen by naked eyes, the visual experience is more comfortable, and the stroboflash limit which can be distinguished by the human brain is 75Hz, so that the stroboflash frequency of each area is set within the range of 30Hz-75 Hz.
The light sources with the same frequency and flashing have different phases, so that the electroencephalogram signals can be distinguished, or the frequency and the phase are different, and richer distinction can be realized.
Preferably, the communication module 4 includes a first RF module 4a and a first antenna 4b, the first RF module 4a is electrically connected to the microcontroller 2, and the first antenna 4b is electrically connected to the first RF module 4 a.
With this configuration, the first RF module 4a preferably employs a JF24D wireless bidirectional module, modulates the control signal sent by the microcontroller into a bluetooth or wifi radio frequency signal, and transmits the signal to the first antenna 4b, and the first antenna 4b broadcasts the radio frequency signal in the form of radio magnetic waves, which are then received by the short-range wireless network 6.
Preferably, the household appliance is provided with a second RF module 7a and a second antenna 7b, the second antenna 7b is electrically connected with the second RF module 7a, and the second RF module 7a is electrically connected with the control unit 7.
With such a structure, the second RF module 7a preferably adopts a JF24D wireless bidirectional module, the second antenna 7b receives radio frequency signals in the form of wireless electromagnetic waves from the short-range wireless network 6 and transmits the radio frequency signals to the second RF module 7a, the second RF module 7a demodulates the radio frequency signals into control signals and transmits the control signals to the execution module 7c, the execution module 7c identifies the received control signals and controls corresponding household appliances to realize corresponding command control, and the execution module 7c preferably adopts RK 3399.
The control method of the intelligent home system controlled by the AR equipment specifically comprises the following steps:
a. a switch display area 3a, a function area 3b and a switching area 3c on an AR display lens 3 of the AR equipment flicker with different strobe frequencies;
b. a user wears the AR equipment and watches the switch display area 3a, the functional area 3b or the switching area 3c of the AR display lens 3 through human eyeballs;
c. the detection end 5a of the brain-computer interface 5 detects a steady-state visual evoked potential signal generated by stimulation of a human brain by the stroboscopic frequency of a corresponding gazed area;
d. the microcontroller 2 generates a corresponding control instruction according to the detected steady-state visual evoked potential signal and sends the control instruction to corresponding household appliances;
e. and the corresponding household appliance receives the control command and executes the corresponding action.
Preferably, the following steps are further included between step c and step d:
s1, the first detection electrode 5d and the second detection electrode 5e are matched to detect the steady-state visual evoked potential signal and send the steady-state visual evoked potential signal to the signal acquisition module 5 b;
s2, the signal acquisition module 5b receives the steady-state visual evoked potential signal and sends the steady-state visual evoked potential signal to the signal processing module 5 c;
s3, the signal processing module 5c eliminates frequency doubling harmonic information from the steady-state visual evoked potential signal and sends a processed frequency signal to the microcontroller 2;
and d, specifically, the microcontroller 2 generates a corresponding control instruction according to the received frequency signal and sends the control instruction to the corresponding household appliance.
And f, when the state information of the household appliance is changed, the state information is sent to the AR display lens 3 through the short-range wireless network 6 to be displayed.
By adopting the structure, the short-range wireless network 6 collects the information of the networked household appliance and displays the corresponding control instruction information on the AR display lens 3, and the user can watch the functional areas of different control instructions of the AR display lens 3, so that different steady-state visual evoked potential signals are generated due to different stroboscopic frequencies of different functional areas.
The first detection electrode 5d and the second detection electrode 5e detect the steady-state visual evoked potential signal and transmit the steady-state visual evoked potential signal to the signal acquisition module 5b, the signal acquisition module 5b receives the steady-state visual evoked potential signal and transmits the steady-state visual evoked potential signal to the signal processing module 5c to remove frequency doubling harmonic information, the signal processing module 5c transmits the processed frequency signal to the microcontroller 2, the microcontroller 2 generates a corresponding control signal according to the received frequency signal, and the communication module 4 transmits the control signal to the short-range wireless network 6.
The short-range wireless network 6 sends signals to the control unit 7 and controls the home appliance to perform corresponding functions.
When the state information of the household appliance changes, the control unit 7 generates a feedback signal and feeds the feedback signal back to the microcontroller 2 through the short-range wireless network 6 and the communication module 4, and the microcontroller 2 displays the feedback information on the AR display lens 3.
Taking the turning-on process of the television controlled by the user as an example, the television is in a standby state, the television sends the state information of the television itself to the microcontroller 2 through the short-range wireless network 6, and the microcontroller 2 displays the turning-off state information of the television and other functional information on the AR display lens 3.
The user is through watching switch display area 3a of AR display lens 3, the stroboscopic frequency of switch display area 3a stimulates human eye, makes human head produce the brain electrical signal and is steady state vision evoked potential signal, microcontroller 2 receives this steady state vision evoked potential signal and produces the control command of start-up through brain machine interface 5, communication module 4 sends the control command of start-up to control unit 7 through short distance wireless network 6, control unit 7 sends the control command of start-up to the TV set, the TV set is started up.
At this time, the state information of the television changes, the television sends the changed state information, namely the startup state information to the microcontroller 2 through the control unit 7, the short-range wireless network 6 and the communication module 4, and the microcontroller 2 displays the startup state information on the AR display lens 3 and changes the stroboscopic frequency of the switch display area 3a at the same time to wait for the next operation of the user. In order to facilitate the recognition control, the strobe frequency of the AR display lens 3 in the information state of the power-on and the strobe frequency of the AR display lens 3 in the information state of the power-off are set to different frequencies.
Preferably, in step a, the operating modes of switching the strobe frequency of the display area 3a, the strobe frequency of the functional area 3b and the strobe frequency of the switching area 3c are two types:
one is that the switch display area 3a, each function area 3b, and each switching area 3c use constant strobes of strobe frequencies different from each other, respectively;
the other is that the switch display area 3a, each function area 3b, and each switching area 3c use a strobe frequency with a range of variation different from each other, respectively, as a periodic pulse strobe that jumps from one strobe frequency to another strobe frequency in one cycle.
With this structure, for example: the first mode is to select different strobe frequencies, such as f1, f2, f3, … as strobe frequencies, 30Hz < f1, f2, f3, … <75 Hz. For example, the strobe frequency of the first function of the functional region 3b is f1, the strobe frequency of the second function is f2, and the strobe frequency of the third function is f 3.
The second mode is to select different strobe frequencies and select two of the frequencies to jump back and forth as strobe frequencies, such as f4, f5, f6, f7, …, 30Hz < f4, f5, f6, f7, … <75 Hz. For example, the strobe frequency of the functional region 3b is toggled between two frequencies with a certain time period, for example, the function four is toggled between f4 and f6, and the function five is toggled between f5 and f 7. It should be noted that the above-mentioned selection of frequencies is only an illustration and should not be interpreted in any way as limiting the scope of the invention.
By adopting the first working mode, the constant stroboscopic frequency has the advantages of simplicity, directness, low misjudgment rate and the like, but the selectable frequency is limited, and the total number of different types of control signals is limited. With the second mode of operation, the combined strobe frequency greatly enhances the richness of the signal, but requires a longer time to determine the frequency information of the signal.
Preferably, in step d, the microcontroller 2 sets a minimum fixation duration, and compares the duration of generation of the steady-state visual evoked potential signal with the minimum fixation duration:
if the minimum gazing duration is less than the minimum gazing duration, generating no corresponding control instruction;
and if the minimum gazing duration time is greater than or equal to the minimum gazing duration time, generating a corresponding control command and sending the control command to the corresponding household appliance.
By adopting the structure, the microcontroller 2 sets the minimum watching duration time, 2s can be selected as the minimum watching duration time, when in use, a user can unconsciously see other display areas, if the microcontroller 2 immediately sends out a control instruction when receiving a steady-state visual evoked potential signal, misoperation is easy to occur, so the watching duration time is set, when the continuous generation time of the steady-state visual evoked potential signal is less than the minimum watching duration time, the signal is an invalid signal, and the microcontroller 2 does not generate a corresponding control instruction; when the duration of the steady-state visual evoked potential signal is longer than the minimum watching duration, the signal is an effective signal, and the microcontroller 2 generates a corresponding control instruction and controls the corresponding household appliance to execute operation. By setting the minimum fixation duration, the interference generated by unconscious fixation can be effectively removed, and the accuracy of electroencephalogram signal identification is improved.
During the use, the user produces different brain electrical signals through watching the different functional areas of AR display lens 3, detects this brain electrical signal and sends microcontroller 2 through brain computer interface 5, and microcontroller 2 produces control signal and controls corresponding household electrical appliances execution operation through communication module 4, short distance wireless network 6 and each control unit 7.
The technical principle of the present invention is described above in connection with specific embodiments. The description is made for the purpose of illustrating the principles of the invention and should not be construed in any way as limiting the scope of the invention. Based on the explanations herein, those skilled in the art will be able to conceive of other embodiments of the present invention without inventive effort, which would fall within the scope of the present invention.
Claims (10)
1. An intelligent home system controlled by AR equipment comprises the AR equipment and a plurality of household electrical appliances, wherein the AR equipment is in communication connection with the household electrical appliances through a short-range wireless network, and is characterized in that the AR equipment comprises a wearing main body, a microcontroller, an AR display lens, a communication module and a brain-computer interface, wherein the microcontroller, the AR display lens, the communication module and the brain-computer interface are all arranged on the wearing main body, and the AR display lens, the communication module and the brain-computer interface are all electrically connected with the microcontroller; the brain-computer interface comprises a detection end, and the detection end is attached to the scalp of the human body; the microcontroller is in communication connection with the short-range wireless network through the communication module;
the AR display lens is used for generating a plurality of functional areas with different flicker frequencies to watch a user so as to enable the user to generate different electroencephalogram signals;
the detection end of the brain-computer interface is used for detecting an electroencephalogram signal of a user and sending the electroencephalogram signal to the microcontroller;
the microcontroller is used for generating a corresponding control instruction according to the electroencephalogram signal and sending the control instruction to the corresponding household appliance;
the household electrical appliance is provided with a control unit, the control unit is electrically connected with a circuit board of the corresponding household electrical appliance, and the control unit is used for receiving a control instruction and executing corresponding actions.
2. The intelligent home system controlled by the AR equipment according to claim 1, wherein the AR equipment further comprises a camera, the camera is arranged at the front end of the wearing main body, and the camera is used for detecting the household appliance currently watched by the user through image recognition and feeding back the household appliance to the microcontroller; the microcontroller is used for displaying a control function menu of the household appliance fed back by the camera.
3. The intelligent home system controlled by the AR equipment according to claim 1, wherein the brain-computer interface further comprises a signal acquisition module and a signal processing module, the detection end is electrically connected with the signal acquisition module, and the signal acquisition module is electrically connected with the signal processing module; the signal processing module is electrically connected with the microcontroller;
the signal acquisition module is used for acquiring the electroencephalogram signals detected by the detection end;
the signal processing module is used for removing frequency multiplication harmonic information from the acquired electroencephalogram signal, extracting real frequency information and feeding the real frequency information back to the microcontroller;
and the microcontroller is used for receiving the frequency value fed back by the signal processing module.
4. The intelligent home system controlled by the AR equipment according to claim 3, wherein the detection end comprises a first detection electrode and a second detection electrode, the first detection electrode is arranged on the wearing body and attached to temples of the head of the human body, the second detection electrode is arranged on the wearing body and attached to the rear side of ears of the human body, and the first detection electrode and the second detection electrode are arranged on the same side of the wearing body.
5. The intelligent home system controlled by the AR equipment according to claim 1, wherein the AR display lens is provided with a switch display area, a functional area and a switching area, and the refresh frequency of the AR display lens is a common multiple of the stroboscopic frequency of the switch display area, the stroboscopic frequency of the functional area and the stroboscopic frequency of the switching area;
the stroboscopic frequency of the switch display area, the stroboscopic frequency of the functional area and the stroboscopic frequency of the switching area are all within the range of 4Hz-75 Hz;
at least one functional area is arranged; at least one switching area is arranged; the stroboscopic frequency of the switch display area, the stroboscopic frequency of each functional area and the stroboscopic frequency of each switching area are different;
the switch display area is used for displaying the switch selection of the currently controlled household appliance, the function area is used for displaying a certain function of the currently controlled household appliance, and the switching area is used for displaying the household appliance capable of being controlled in a switching mode.
6. The intelligent home system controlled by the AR equipment according to claim 5, wherein the stroboscopic frequency of the switch display area, the stroboscopic frequency of the function area and the stroboscopic frequency of the switching area are all within a range of 30Hz-75 Hz.
7. The control method of the intelligent home system controlled by the AR equipment according to any one of claims 1 to 6, comprising the following steps:
a. a switch display area, a function area and a switching area on an AR display lens of the AR equipment flicker with different strobe frequencies;
b. a user wears the AR equipment and watches the switch display area or the functional area or the switching area of the AR display lens through human eyeballs;
c. the detection end of the brain-computer interface detects a steady-state visual evoked potential signal generated by stimulation of a human brain by the stroboscopic frequency of a corresponding gazed area;
d. the microcontroller generates a corresponding control instruction according to the detected steady-state visual evoked potential signal and sends the control instruction to corresponding household appliances;
e. and the corresponding household appliance receives the control command and executes the corresponding action.
8. The control method of the intelligent home system controlled by the AR equipment according to claim 7, further comprising the following steps between step c and step d:
s1, the first detection electrode and the second detection electrode are matched to detect the steady-state visual evoked potential signal and send the steady-state visual evoked potential signal to the signal acquisition module;
s2, the signal acquisition module receives the steady-state visual evoked potential signal and sends the steady-state visual evoked potential signal to a signal processing module;
s3, the signal processing module removes frequency multiplication harmonic information from the steady-state visual evoked potential signal and sends a processed frequency signal to the microcontroller;
the step d is specifically that the microcontroller generates a corresponding control instruction according to the received frequency signal and sends the control instruction to the corresponding household appliance;
and f, when the state information of the household appliance is changed, sending the state information to the AR display lens through the short-range wireless network for displaying.
9. The control method of the intelligent home system controlled by the AR equipment according to claim 7, wherein in the step a, the stroboscopic frequency of the switch display area, the stroboscopic frequency of the function area, and the stroboscopic frequency of the switching area have two working modes:
one is that the switch display area, each function area and each switching area respectively use the constant stroboflash with different stroboflash frequencies;
the other is that the switch display area, each function area and each switching area respectively use the stroboscopic frequency periodical pulse stroboscopic with different variation ranges, and the periodical pulse stroboscopic jumps from one stroboscopic frequency to another stroboscopic frequency in one period.
10. The method according to claim 7, wherein in the step d, the microcontroller sets a minimum gazing duration, and compares the duration generation time of the received steady-state visual evoked potential signal with the minimum gazing duration:
if the minimum gazing duration is less than the minimum gazing duration, generating no corresponding control instruction;
and if the minimum gazing duration time is greater than or equal to the minimum gazing duration time, generating a corresponding control command and sending the control command to the corresponding household appliance.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910836884.7A CN110597201A (en) | 2019-09-05 | 2019-09-05 | Intelligent household system controlled by AR equipment and control method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910836884.7A CN110597201A (en) | 2019-09-05 | 2019-09-05 | Intelligent household system controlled by AR equipment and control method thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110597201A true CN110597201A (en) | 2019-12-20 |
Family
ID=68857618
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910836884.7A Pending CN110597201A (en) | 2019-09-05 | 2019-09-05 | Intelligent household system controlled by AR equipment and control method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110597201A (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101887307A (en) * | 2010-06-03 | 2010-11-17 | 西安交通大学 | Multi-frequency time sequence combined steady-stage visual evoked potential brain-computer interface method |
| CN106339091A (en) * | 2016-08-31 | 2017-01-18 | 博睿康科技（常州）股份有限公司 | Augmented reality interaction method based on brain-computer interface wearing system |
| CN106502159A (en) * | 2016-11-21 | 2017-03-15 | 南京轩世琪源软件科技有限公司 | Household experiencing system and its method using AR |
| CN107168346A (en) * | 2017-04-28 | 2017-09-15 | 上海交通大学 | A kind of asynchronous system brain control UAS based on wearable display |
| CN107168134A (en) * | 2017-06-27 | 2017-09-15 | 福州大学 | The household electric appliance control method of brain eye combination drive |
| CN107703872A (en) * | 2017-10-31 | 2018-02-16 | 美的智慧家居科技有限公司 | Terminal control method, device and the terminal of home appliance |
| CN108614456A (en) * | 2018-05-25 | 2018-10-02 | 苏州市职业大学 | A kind of household electrical appliances safety management system using AR augmented reality technologies |
| US20190121431A1 (en) * | 2017-10-23 | 2019-04-25 | Korea University Research And Business Foundation | Method of recognizing user intention by estimating brain signals, and brain-computer interface apparatus based on head mounted display implementing the method |
| CN110134243A (en) * | 2019-05-20 | 2019-08-16 | 中国医学科学院生物医学工程研究所 | A kind of brain control mechanical arm shared control system and its method based on augmented reality |
-
2019
- 2019-09-05 CN CN201910836884.7A patent/CN110597201A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101887307A (en) * | 2010-06-03 | 2010-11-17 | 西安交通大学 | Multi-frequency time sequence combined steady-stage visual evoked potential brain-computer interface method |
| CN106339091A (en) * | 2016-08-31 | 2017-01-18 | 博睿康科技（常州）股份有限公司 | Augmented reality interaction method based on brain-computer interface wearing system |
| CN106502159A (en) * | 2016-11-21 | 2017-03-15 | 南京轩世琪源软件科技有限公司 | Household experiencing system and its method using AR |
| CN107168346A (en) * | 2017-04-28 | 2017-09-15 | 上海交通大学 | A kind of asynchronous system brain control UAS based on wearable display |
| CN107168134A (en) * | 2017-06-27 | 2017-09-15 | 福州大学 | The household electric appliance control method of brain eye combination drive |
| US20190121431A1 (en) * | 2017-10-23 | 2019-04-25 | Korea University Research And Business Foundation | Method of recognizing user intention by estimating brain signals, and brain-computer interface apparatus based on head mounted display implementing the method |
| CN107703872A (en) * | 2017-10-31 | 2018-02-16 | 美的智慧家居科技有限公司 | Terminal control method, device and the terminal of home appliance |
| CN108614456A (en) * | 2018-05-25 | 2018-10-02 | 苏州市职业大学 | A kind of household electrical appliances safety management system using AR augmented reality technologies |
| CN110134243A (en) * | 2019-05-20 | 2019-08-16 | 中国医学科学院生物医学工程研究所 | A kind of brain control mechanical arm shared control system and its method based on augmented reality |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP4287903B2 (en) | EEG interface system and activation device | |
| US20230043749A1 (en) | Brain-computer interface for user's visual focus detection | |
| CN107861651B (en) | Touch method, active pen, touch screen and touch display system | |
| CN105530307B (en) | A kind of event-prompting method and system based on more associate devices | |
| CN113110743B (en) | Parallel brain-eye fusion system and method | |
| CN109754866A (en) | Attention training method, device and computer readable storage medium | |
| CN101681201A (en) | Brain wave interface system, brain wave interface device, method and computer program | |
| CN105847538B (en) | Mobile phone and method for controlling VR (virtual reality) glasses operation based on eyeball tracking | |
| CN106681503A (en) | Display control method, terminal and display device | |
| CN115552916B (en) | Refresh rate adjusting method and electronic equipment | |
| CN104182048A (en) | Brain-computer interface based telephone system and call method thereof | |
| CN103543836A (en) | Full-automatic webpage browsing control method based on brain-computer interface | |
| CN105929937A (en) | Mobile phone music playing system based on steady-state visual evoked potential (SSVEP) | |
| CN112882567B (en) | Man-machine interaction method, man-machine interaction device and storage medium | |
| CN103976733A (en) | Multi-passage brain wave control glasses | |
| CN108897418A (en) | A kind of wearable brain-machine interface arrangement, man-machine interactive system and method | |
| CN108550354A (en) | Brightness of display screen autocontrol method and air conditioner | |
| CN106708273A (en) | Switch device based on EOG and switch key implementing method | |
| CN116360600A (en) | Space positioning system based on steady-state visual evoked potential | |
| CN103974107A (en) | Television eye movement control method and device and television | |
| CN105242778A (en) | Reminding apparatus and method for wearable device | |
| CN110597201A (en) | Intelligent household system controlled by AR equipment and control method thereof | |
| CN207394480U (en) | Intelligent lamp and the User Status system for prompting based on Intelligent lamp | |
| CN111956387B (en) | Control method of massage apparatus, and computer-readable storage medium | |
| CN109426342B (en) | Document reading method and device based on augmented reality |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191220 |
|
| RJ01 | Rejection of invention patent application after publication |