TWI557598B - Feedback learning system and method thereof based on brainwave and gesture controls - Google Patents

Description

Brain wave and gesture control feedback learning system and method thereof

The following description relates to a learning system and method thereof, and more particularly to a learning system and method for utilizing brain wave and gesture control for feedback

With the development of information technology, diversified digital teaching materials have become more and more popular, and the teaching and learning models of related innovations have emerged. Users can self-learn through digital materials open to the Internet to achieve the purpose of multi-curricular learning.

However, the above learning methods may have several disadvantages. First, when the user is in the course of physical operation, such as cooking, mechanical assembly, etc., if the digital teaching materials must be operated synchronously with the mouse and the keys, the use will be caused. It is extremely inconvenient. Second, this kind of learning method can only convey information to the user by digital teaching materials in one direction. The producers of several textbooks have no way of knowing whether the user is satisfied with the digital teaching materials or feels that they need to be modified. Third, such digital textbooks do not take into account the privacy issues of individual learning, and it is extremely easy for a third party to know what course content the user is learning.

Therefore, the inventors propose a feedback learning system and method thereof to solve the above problems.

In view of the above-mentioned problems, the brain wave and gesture control feedback learning system and method thereof are used to solve the user's synchronous operation of digital teaching materials and real The inconvenience of textbooks.

In view of the above-mentioned problems, the feedback learning system and method for brainwave and gesture control proposed by the present invention enable a producer of digital teaching materials to immediately know the learning situation of a user who views digital teaching materials through a feedback mechanism.

In view of the above-mentioned problems, the brainwave and gesture control feedback learning system and method thereof provided by the present invention provide a learning system with a privacy protection function to protect the user's personal privacy problem during learning.

Based on the above object, the present invention provides a feedback learning method for brain wave and gesture control, which is applicable to a feedback learning system, which includes a cloud server device, a gesture detection device, a slide show device, and a brain wave. The measuring device includes the following steps: respectively, defining a control message or a feedback message corresponding to the at least one gesture in the gesture detecting device. The teaching slide is played by the slide show device. The brain wave measuring device continuously detects the brain wave reading of the user who views the teaching slide, and uploads the brain wave reading to the cloud servo device. The gesture detecting device is used to detect the gesture of the user who views the teaching slide to generate a control message and a feedback message. Control the playback of the teaching slides and record feedback messages through control messages. Send feedback messages to the cloud server. The cloud server generates integrated information based on brainwave readings and feedback messages. Modify the content of the teaching slides and increase or decrease the number of pages of the teaching slides according to the unified information.

Preferably, the brainwave reading contains a blink signal.

Preferably, the feedback learning method of the present invention further includes setting, by the cloud server, a boring or uninteresting integrated information when the blink signal and the feedback message are not received within a threshold time.

Preferably, the feedback learning method of the present invention further comprises using a control message to generate a login verification code of the corresponding user. When the gesture detection device verifies that the login verification code is a legal When the user is logged in, the gesture detection device controls the playback of the teaching slide and records the feedback message according to the control message generated by the gesture.

Preferably, the feedback learning method of the present invention further comprises defining a concentration range of brainwave readings of different degrees of concentration of the user and storing them in the cloud server. When the concentration range is a low degree of concentration, the control message generated by the user is filtered and cannot be used as a reference for the unified information.

Based on the above object, the present invention further provides a feedback learning system for brain wave and gesture control, which comprises a slide show device, a brain wave measuring device, a gesture detecting device and a cloud server. A movie playback device can be used to play a teaching slide. The brainwave measuring device can be used to continuously measure and transmit brainwave readings of the user viewing the teaching slide. The gesture detecting device can be used to detect the gesture of the user who views the teaching slide to generate a control message or a feedback message, and control the playback of the teaching slide through the control message and record and transmit the feedback message, and the cloud server can perform the brain wave reading according to the brain wave reading. And feedback messages to generate integrated information.

Preferably, the brainwave reading includes a blinking signal, and when the blinking signal and the feedback message are not received within a threshold time, the cloud server generates a boring or uninteresting integrated information.

Preferably, the control message includes a login verification code corresponding to the user, and the gesture detection device is configured to verify whether the login verification code is a legitimate login user.

Preferably, the concentration range of the brain wave readings of the user's different levels of concentration is stored in advance on the cloud server. When the cloud server determines that the concentration range is a low degree of concentration, the control message generated by the user is automatically filtered. It cannot be used as a reference for integrating information.

Preferably, the control message includes play, pause, fast forward, reverse, feedback message Including good, doubtful and disagreeable, the teaching slides can modify their content and increase or decrease the number of pages based on feedback messages containing good, doubtful and disagreement.

100‧‧‧Reward learning system

10‧‧‧Slide film playback device

11‧‧‧ Teaching slides

20‧‧‧Earth wave measuring device

21‧‧‧ brainwave readings

211‧‧‧Eye signal

30‧‧‧ gesture detection device

31‧‧‧Control messages

311‧‧‧ Login verification code

32‧‧‧Feedback message

40‧‧‧Cloud Servo

41‧‧‧Consolidation of information

42‧‧‧ Concentration

43‧‧‧Locking instructions

60‧‧‧Users

61‧‧‧ gestures

S11~S18‧‧‧ Process steps

The above and other features and advantages of the present invention will become more apparent from the detailed description of the exemplary embodiments of the accompanying drawings in which: FIG. 1 is a block diagram of a feedback learning system according to an embodiment of the present invention. .

Figure 2 is a first schematic diagram of a feedback learning system in accordance with a first embodiment of the present invention.

Figure 3 is a second schematic diagram of a feedback learning system in accordance with a first embodiment of the present invention.

Figure 4 is a schematic diagram of a feedback learning system in accordance with a second embodiment of the present invention.

Figure 5 is a schematic diagram of a feedback learning system in accordance with a third embodiment of the present invention.

Figure 6 is a flow chart showing the steps of the feedback learning method according to an embodiment of the present invention.

The features, the contents and advantages of the present invention, and the advantages thereof, will be understood by the present invention. The present invention will be described in detail with reference to the accompanying drawings, The use of the present invention is not intended to be a limitation of the scope of the present invention, and the scope of the present invention is not limited by the scope and configuration of the accompanying drawings.

The advantages and features of the present invention, as well as the technical methods of the present invention, are described in more detail with reference to the exemplary embodiments and the accompanying drawings, and the present invention may be implemented in various forms and should not be construed as limited thereby. The embodiments set forth herein, and vice versa, will provide a more thorough and complete and complete disclosure of the invention. The invention will be defined only by the scope of the appended claims.

Please refer to FIG. 1, which is a block diagram of a feedback learning system according to an embodiment of the present invention. The feedback learning system 100 includes a slide show device 10, a brain wave measuring device 20, a gesture detecting device 30, and a cloud server device 40. The slide show device 10 can be a combination of a slide show playback software mounted on a host computer and a display hardware. The brain wave measuring device 20 can be a common brain wave earphone device on the market. The cloud server 40 can be a computer host, a workstation, a server, etc., and the gesture detecting device 30 can be an infrared sensor with a processor function, a motion sensor, and the like.

The video playback device 10 is configured to play a teaching slide 11 and the playback mode may be an automatic play or an input device such as a mouse, a keyboard or a microphone for playing control. The brain wave measuring device 20 is configured to continuously measure and transmit one of the brain wave readings 21 of the user 60 viewing the teaching slide 11 . It is worth mentioning that the brain wave reading 21 includes α waves, β waves, and γ waves. In addition to the delta wave, one of the user's 60 blinking signals 211 can also be included, and the transmission of the brainwave reading 21 can be transmitted to a smart phone or a computer via Bluetooth, and then transmitted through the network via the smart phone or computer. The brainwave reading 21 is to the cloud servo 40.

The gesture detecting device 30 is configured to detect a gesture 61 of the user 60 viewing the teaching slide 11 to generate a control message 31 or a feedback message 32. The user 60 can control the playback of the teaching slide 11 and record the feedback message 32 through the control message 31. In detail, the control message 31 may include a message such as play, pause, fast forward, reverse, etc., and the feedback message 32 may include messages such as "good", "question" and "disagree", and the user 60 may perform gesture detection in advance. The device 30 defines a gesture action corresponding to the control message 31 and the feedback message 32 respectively. When the gesture action 61 played by the user 60 is detected by the gesture detection device 30 and the gesture action 61 is determined to belong to a predefined gesture action, The gesture detecting device 30 can transmit related commands to the slide playing device 10 or the cloud server device 40 through the network or the electrical connection to perform different actions.

For example, when the gesture detecting device 30 determines that the gesture action 61 of the user 60 is the "pause" control message 31, the gesture detecting device 30 can send a control command corresponding to "pause" to the slide show device. 10, and the slide show device 10 can pause the content of the slide being played.

The cloud server 40 can generate a unified information 41 according to the brainwave reading 21 and the feedback message 32. The unified information 41 can include page information that the user 60 is interested in, interested in, or disagrees with the teaching slide 11. It can also be the degree of concentration and relaxation of the user 60 when viewing the teaching slide 11 , and the producer can use the integration information 41 to modify the content of the teaching slide 11 or increase or decrease the number of pages.

Please refer to FIG. 2 and FIG. 3, which are a first schematic diagram and a second schematic diagram of a feedback learning system according to a first embodiment of the present invention. Please refer to Figure 1 together. This first embodiment illustrates that when the user 60 views a teaching slide 11 for a cooking course, although the hands of the user 60 may hold a spatula or a portion of the dish or liquid, However, the user 60 is not prevented from performing the gesture before the gesture detecting device 30. As described above, the gesture detecting device 30 can define the gestures of the control message 31 "play", "fast forward", "reverse", etc. in advance. The user 60 can generate these control messages 31 in a gesture to control the playback of the teaching slide 11. In addition, the user 60 can also use the "big thumb to make a gesture", "before the food pointing" and "small to point down" to define the "good", "question" and "disagree" in advance to return the message 32, when When viewing the teaching slide 11 , the user 60 can compare the content of the teaching slide 11 with the appropriate feedback message 32 , and the gesture detecting device 30 transmits the feedback message 32 to the cloud server 40 through the network.

In addition, this login verification code 311 can also serve as a tool for protecting personal privacy when there is a specific need. The gesture detecting device 30 can include a login verification code 311 of the user 60, which can detect and verify whether the gesture action 61 made by the user 60 meets the login verification code 311 to confirm that the user 60 is a legitimate login. For example, the user 60 can set the gesture of the login verification code 311 to an "OK" gesture as a basis for verifying the identity of the user 60, when the user 60 is in the gesture. When the "OK" gesture is made within the range detectable by the detecting device 30, the gesture detecting device 30 can know the identity of the user 60 and allow the user 60 to use the feedback learning system 100.

Moreover, when the user 60 temporarily leaves the slide-playing device 10 for a period of time, the slide-playing device 10 can lock the content display of the teaching slide 11 by itself, and when the gesture detecting device 30 detects and verifies the user. When the gesture action 61 made by 60 is a correct login verification code 311, the locking of the slide-playing device 10 can be released to achieve the effect of protecting personal privacy.

Therefore, it can be known from the above that the feedback learning system 100 can not only solve the inconvenience that the user needs to operate the computer at the same time and the practical operation, but also has the function of verifying the identity of the user and protecting the privacy of the user.

Please refer to FIG. 4, which is a schematic diagram of a feedback learning system according to a second embodiment of the present invention. Please refer to Figure 1 together. When the user 60 wears the brain wave measuring device 20 to view the teaching slide 11, the measured brain wave reading 21 may include a blink signal 211, and the blink signal 211 is not received within a threshold time. And when the feedback message 32 is generated by the user 60, the cloud server 40 can generate the unified information 41 that the user 60 may have entered the sleep state, and the content that may have originated from the teaching slide 11 is inherited. Too boring or difficult to understand, at this time, the producer of the teaching slide 11 can adjust and correct the content of the slide.

Please refer to FIG. 5, which is a schematic diagram of a feedback learning system according to a third embodiment of the present invention. Please refer to Figure 1 together. The user 60 can generate an intent range 42 of brainwave readings 21 corresponding to different levels of concentration and store it in advance on the cloud server 40. As shown in the figure, the degree of concentration is divided into five levels of “highest”, “high”, “medium”, “low” and “lowest” from high to low, and the corresponding values are 100~80, 80 respectively. ~60, 60~40, 40~20 and 20~0, when the cloud servo device 40 receives the brain wave reading 21, the brain wave reading 21 can be normalized into one of 0~100 through a normalization function. An integer, and this integer is used as the intent range 42 for determining the user 60. When it is a lower degree of concentration, such as an integer of 10, it means that the user 60 may be doing other work. For example, if the mobile phone or the tablet is played, the unified information 41 generated at this time has no reference value, and the control message 31 generated by the user 60 can be set as the reference basis for the unified information 41. The normalization method described above can be implemented by different normalization functions, and since the related normalization function has been disclosed in the open literature, it will not be described herein.

Further, if the concentration range 42 belongs to a lower degree of concentration, a lock command 43 can also be returned by the cloud server 40 to lock the playback of the slide playback device 10, prompting the user 60 to raise his or her brainwave reading. The degree of concentration of 21, when the concentration range 42 is above the low patent program, an unlock command is returned by the cloud server 40 to release the lock of the slide playback device 10.

It can be seen from the above that the feedback learning system of the present invention can decide whether to add the control information to the unified information according to the degree of concentration of the user's brain wave, so as to improve the correct reference degree of the integrated information, and at the same time, the user can be promoted. Concentrate on the unpredictable effects of this feedback learning system.

Please refer to FIG. 6, which is a flow chart of the steps of the feedback learning method according to an embodiment of the present invention. The feedback learning method is applicable to a feedback learning system, and the feedback learning system comprises a cloud server device, a gesture detecting device, a slide film playing device and a brain wave measuring device. The feedback learning method comprises the following steps.

In step S11, at least one gesture action corresponding to one control message or one feedback message is defined in the gesture detection device.

Step S12 uses the slide show playback device to play a teaching slide.

Step S13 uses the brain wave measuring device to continuously detect the brain wave reading of one of the users of the teaching slide, and uploads the brain wave reading to the cloud servo device, wherein the brain wave reading may include a blink signal.

Step S14 uses the gesture detecting device to detect one of the gestures of the user viewing the teaching slide Create control messages and feedback messages.

Step S15 controls the playing of the teaching slide through the control message and records the feedback message.

Step S16 transmits a feedback message to the cloud server.

Step S17: The cloud server generates a unified information according to the brain wave reading and the feedback message, wherein when the blink signal and the feedback message are not received within a threshold time, the cloud server sets the user to be a boring, boring or user has Integrate information into sleep.

Step S18 modifies the content of the teaching slide according to the unified information and increases or decreases the number of pages of the teaching slide.

Preferably, the feedback learning method further comprises generating, by using the control message, one of the corresponding users to log in the verification code. When the gesture detecting device verifies that the login verification code generated by the user is a legitimate login user, the gesture detection device controls the teaching of the slide display and records the feedback message according to the control message generated by the gesture action.

Preferably, the feedback learning method further comprises a concentration range of brainwave readings respectively defining different degrees of concentration of the user and stored in the cloud server. When the concentration range is a low degree of concentration, the control message generated by the user is filtered and cannot be used as a reference for the unified information.

It can be seen from the above that the brain wave and gesture control feedback learning system and method thereof disclosed by the present invention can effectively solve the inconvenience of the user synchronously operating the teaching slide and the physical textbook. At the same time, through the brainwave reading and the feedback mechanism of gestures, the producer of the teaching slide can know the learning situation of the user. In addition to the above functions, the feedback learning system and the method thereof provided by the present invention can achieve the problem of protecting one of the privacy of the user while learning.

The embodiments described above are merely illustrative of the technical idea and features of the present invention, and the purpose of the present invention is to enable those skilled in the art to understand the contents of the present invention and to implement the present invention. The scope of the patents, that is, the equivalent variations or modifications made by the present invention in the spirit of the present invention, should still be included in the scope of the invention.

S11~S18‧‧‧ Process steps

Claims (8)

A feedback learning method for brain wave and gesture control is applicable to a feedback learning system, which includes a cloud server device, a gesture detecting device, a slide film playing device and a brain wave measuring device, and the feedback learning method The method includes: at least one gesture action corresponding to one control message or one feedback message is defined in the gesture detecting device; and the teaching slide is played by using the slideshow device; and the brainwave measuring device continuously detects and views the teaching One of the users of the film casts a brainwave reading and uploads the brainwave reading to the cloud server; respectively defining one of the brainwave readings of the user's different concentration levels and storing it in the cloud server; The gesture detecting device detects one of the gestures of the user viewing the teaching slide to generate the control message and the feedback message; controls the playing of the teaching slide and records the feedback message through the control message; and transmits the feedback Message to the cloud server; the cloud server reads the brainwave reading and the feedback message Health information unified whole; and the entire system based on the information content of the instruction sheet and increase or decrease the projected slide teaching of pages; Wherein, when the concentration range is a low degree of concentration, the control message generated by the user is filtered and cannot be used as a reference for the unified information. For example, in the feedback learning method of claim 1, wherein the brain wave reading includes a blinking signal. For example, in the feedback learning method of claim 2, the cloud server is set to be a boring or boring unified information when the blink signal and the feedback message are not received within a threshold time. For example, the feedback learning method of claim 1 further includes using the control message to generate a login verification code corresponding to one of the users; and when the gesture detection device verifies that the login verification code is a legitimate login user, The gesture detecting device controls the playback of the teaching slide and records the feedback message according to the control message generated by the gesture. A feedback learning system for brain wave and gesture control, comprising a slide show playing device for playing a teaching slide piece; a brain wave measuring device for continuously measuring and transmitting a user who views the teaching slide piece a brainwave reading; a gesture detecting device for detecting a gesture of the user viewing the teaching slide to generate a control message or a feedback message, and controlling the teaching slide through the control message Play and Recording and transmitting the feedback message; and a cloud server device, according to the brain wave reading and the feedback message, to generate a unified information; wherein the user concentrates on the brainwave reading of different degrees of concentration In the cloud server, when the cloud server determines that the concentration range is a low degree of concentration, the control message generated by the user is automatically filtered and cannot be used as a reference for the unified information. For example, in the feedback learning system of claim 5, the brainwave reading includes a blinking signal, and when the blinking signal and the feedback message are not received within a threshold time, the cloud server generates a boring or boring The integration of information. For example, in the feedback learning system of claim 5, the control message includes a login verification code corresponding to one of the users, and the gesture detection device is configured to verify whether the login verification code is a legitimate login user. For example, in the feedback learning system of claim 5, wherein the control message includes playing, pausing, fast forwarding, and reversing, the feedback message includes good, doubtful, and disagree, and the teaching slide film is based on good, doubtful, and disagree The feedback message modifies its content and increases or decreases its page count.