KR20140133767A - Apparatus and method for controlling display apparatus - Google Patents

Abstract

The present invention provides a method which suggests stimulation to a user using information on user environment, acquires an electroencephalogram signal of the user with regard to the suggested stimulation, and controls a display apparatus based on the acquired electroencephalogram signal; and a display apparatus. According to the method for controlling a display apparatus, and the display apparatus following an embodiment of the present invention, a user can control the display apparatus more conveniently by suggesting proper stimulation according to the information on user environment, and control the display apparatus with high accuracy according to user intention.

Description

[0001] APPARATUS AND METHOD FOR CONTROLLING DISPLAY APPARATUS [0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and an apparatus for controlling a display device, and more particularly, to a method and apparatus for controlling a display device in cooperation with a user's brain wave.

With the development of various types of display devices (e.g., TV sets, cell phones, PDAs (Personal Digital Assistants and the like)), users are often faced with situations where they have to manipulate the display device in their daily lives. In a general method for operating a device, a user inputs an input command through input means such as a button, a switch, a wheel, etc. At this time, the user uses his / her hand to operate the input means.

1 shows a TV as an example of a display device. As shown in Fig. 1, the user 105 can control the TV 10 by pressing a button included in the remote controller 20 by using his or her hand. The user 105 can input basic control commands such as channel switching, volume control, and screen brightness control to the TV 10 through the remote controller 20.

However, when the user is using both hands, for example, for tasks such as housework, eye care, driving, etc., it is very difficult to input an input command to the display device by using the hand.

In addition, as a TV including a lot of functions such as a smart TV is developed, a user can use various functions provided by a TV, for example, the Internet, a Social Networking Service (SNS) It is necessary to input the control command to the display device.

In accordance with this need, the method and apparatus for controlling the display device are becoming multifunctional and advanced.

Therefore, it is required to develop a technique that allows the user to more easily control the display device.

As a solution to this, a method of controlling a display device using a bio-signal of a user has recently been developed. Particularly, researches on techniques for analyzing the user's electroencephalography (EEG) signal from a user's bio-signals to determine a user's thoughts and feelings are actively under way.

An embodiment of the present invention provides a display device control method and apparatus in which a user can more easily control a display device.

According to an embodiment of the present invention, a stimulus suitable for a user is determined and presented according to information on a user environment, so that it operates with high accuracy according to the intention of the user, and the fatigue The present invention provides a display device control method and apparatus capable of reducing the number of display devices.

According to another aspect of the present invention, there is provided a method of controlling a display device, including: determining a stimulus to be presented to a user using information about a user environment; Presenting the stimulus to the user; Obtaining an electroencephalogram (EEG) signal of the user for the stimulus; And controlling the display device based on the electroencephalogram signal.

In the display device control method according to an embodiment of the present invention, the information on the user environment may include at least one of a distance between the display device and the user, an illuminance around the display device, and user information have.

According to another aspect of the present invention, there is provided a method of controlling a display device, the method comprising: determining a stimulus recognized by a user's body sensation; And determining the stimulation parameter.

In the method of controlling a display device according to an embodiment of the present invention, the step of determining the stimulus may include determining whether to present an auditory stimulus to the user when the distance between the display device and the user is a predetermined distance or more ; And determining a stimulation parameter that includes at least one of a frequency of the auditory stimulation, a sound pressure, a tone color, and a direction of the auditory stimulation the user feels.

In the method of controlling a display device according to an embodiment of the present invention, the stimulus includes a visual stimulus, and the step of determining the stimulus comprises: displaying on the screen of the display device the display position of the visual stimulus, Determining a stimulation parameter including at least one of a type of the visual stimulus, a color of the visual stimulus, a duration of the visual stimulus, and a repetition number of the visual stimulus.

In the method of controlling a display device according to an embodiment of the present invention, the step of presenting the visual stimulus may include: displaying at least one selection menu on a screen of the display device; And highlighting each selection menu included in the at least one selection menu in any order according to a predetermined number of times and intervals.

In the method of controlling a display device according to an embodiment of the present invention, the step of highlighting the selection menu may include displaying at least a part of the border of the selection menu with a change in brightness or changing a display position of the selection menu . ≪ / RTI >

In the method of controlling a display device according to an embodiment of the present invention, the step of controlling the display device may include selecting and executing at least one of the at least one selection menu based on the electroencephalogram signal .

In the method of controlling a display device according to an embodiment of the present invention, the step of controlling the display device includes the steps of: turning off the power of the display device when it is determined that the user is in the sleep state based on the electroencephalogram signal In the method of controlling a display device according to an embodiment of the present invention, the step of presenting the stimulus may include a step of displaying the mental state of the user using the user's electroencephalogram signal, Sensing at least one of eye motions; And displaying at least one selection menu on the screen of the display device based on at least one of the mental state and the eye action.

In a method of controlling a display device according to an embodiment of the present invention, the step of presenting a visual stimulus includes presenting a visual stimulus for at least one selected menu displayed on the display device, Wherein controlling the device comprises: using a plurality of algorithms to determine which of the at least one selection menu the user has responded to for the selection menu; And selecting and executing at least one of the at least one selection menu based on the determined result.

According to another aspect of the present invention, there is provided a display control apparatus including: a stimulus stimulus determination unit for determining a stimulus to be presented to a user using information on a user environment; A stimulus presentation unit for presenting the stimulus to the user; A signal acquisition unit for acquiring the user's electroencephalogram signal with respect to the stimulus; And a controller for controlling the display device based on the electroencephalogram signal.

In the display control device according to an embodiment of the present invention, the information on the user environment may include at least one of a distance between the display device and the user, an illuminance around the display device, and user information .

In the display control apparatus according to an embodiment of the present invention, the stimulus determination unit may determine a stimulus to be recognized by a certain body sensation of the user and determine a stimulation parameter.

In the display control apparatus according to an embodiment of the present invention, the stimulus determining unit may determine to present the auditory stimulation to the user when the distance between the display device and the user is a predetermined distance or more, The frequency, the sound pressure, the tone, and the direction of the auditory stimulation the user feels.

In the display control apparatus according to an embodiment of the present invention, the stimulus includes a visual stimulus, and the stimulus determination unit determines the display position of the visual stimulus, the brightness of the visual stimulus, The color of the visual stimulus, the duration of the visual stimulus, and the number of repetitions of the visual stimulus.

In the display control apparatus according to an embodiment of the present invention, the stimulus presentation unit displays at least one selection menu on the screen of the display device, and displays the at least one selection menu in an arbitrary order You can highlight each selection menu included in the menu.

In the display control apparatus according to an embodiment of the present invention, the stimulus suggestion section may highlight the selection menu so that the brightness of at least a part of the frame of the selection menu is changed or the display position of the selection menu is changed .

In the display control apparatus according to an embodiment of the present invention, the control section may select and execute at least one of the at least one selection menu based on the electroencephalogram signal.

In the display control apparatus according to an embodiment of the present invention, when the controller determines that the user is in a sleep state based on the electroencephalogram signal, the controller may turn off the display device.

In the display control apparatus according to an embodiment of the present invention, the stimulus presentation unit senses at least one of the mental state of the user and the eye operation of the user using the user's brain conduction signal, And at least one selection menu on the screen of the display device based on at least one of the eye action.

In the display control apparatus according to an embodiment of the present invention, the stimulus presentation unit presents a visual stimulus for at least one selection menu displayed on the display device, and the control unit displays, Determine whether the user has responded to which selection menu among at least one selection menu, and select and execute at least one of the at least one selection menu based on the determination result.

According to another aspect of the present invention, there is provided a computer-readable recording medium having recorded thereon a program for causing a computer to execute a method of controlling a display device, the method comprising the steps of: Determining a stimulus; Presenting the stimulus to the user; Obtaining an electroencephalogram signal of the user for the stimulus; And controlling the display device based on the electroencephalogram signal.

1 is a view for explaining a conventional method of controlling a display device.
2 is a conceptual diagram for explaining a display control apparatus for controlling a display apparatus according to an embodiment of the present invention.
3 is a flowchart illustrating a method of controlling a display device according to an exemplary embodiment of the present invention.
4 is a flowchart illustrating a method of generating a visual stimulus in a method of controlling a display device according to an embodiment of the present invention.
5A and 5B are views for explaining an example of a screen displayed according to a display control method according to an embodiment of the present invention.
6 is a view for explaining an example of a visual stimulus displayed according to a display control method according to an embodiment of the present invention.
7 is a block diagram illustrating a display control apparatus according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, which will be readily apparent to those skilled in the art. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

Throughout the specification, when a part is referred to as being "connected" to another part, it includes not only "directly connected" but also "electrically connected" with another part in between . Also, when an element is referred to as "comprising ", it means that it can include other elements as well, without departing from the other elements unless specifically stated otherwise.

Throughout the specification, " display device " means any device capable of displaying a screen to a user. Specifically, the display device may include a portable computer such as a notebook computer, a netbook, or a tablet PC, a portable terminal such as a smart phone or a PDA, or any electronic device capable of outputting a screen through a display unit such as a TV.

"Electroencephalogram signal" refers to the flow of electricity that occurs when a signal is transmitted between the nervous system and the nervous system. In the present specification, an electroencephalogram signal can be used in the same sense as an electroencephalogram. EEG signals are different depending on the state of mind and body, and are the most important indicators for measuring the activity of the brain.

The electroencephalogram signal is sensed through a plurality of sensors attached to the scalp, and each sensor can be a channel receiving the electroencephalogram signal.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

In controlling a display device using an electroencephalogram signal, a method of controlling a display device by grasping a user's intention using an event-related potential (ERP) occurring on a predetermined stimulus that attracts a user's interest Can be used.

For example, the P3 component, which is an event related potential detected after about 300 ms after the stimulus is received, is detected, the user's intention is determined using the detected P3 component, and based on the determined user's intention, May be used. This control scheme can perform calibration of the display control device for the user before detecting the P3 component from the electroencephalogram signal.

Calibration of the display control device means adjusting the P3 component detection method so that the intention of the user determined based on the user's actual intention and the electroencephalogram signal matches with high accuracy. In order to adjust the P3 component detection method, a method may be used in which a correlation between P3 components and corresponding stimuli for various stimuli is defined and stored in a database, and a database previously stored at the time of P3 component detection for the stimuli is used .

For example, the calibration operation of the display control device may be performed by presenting an additional stimulus other than the time before detecting the P3 component by visual stimulation from the user, The device can be calibrated.

On the other hand, when the display control device controls the display device using the electroencephalogram signal for the stimulus presented to the user, the accuracy of the display device control from the electroencephalogram signal may vary depending on the user environment.

The accuracy of the display device control means the degree to which the intention of the user matches the intention of the user determined based on the electroencephalogram signal. That is, the accuracy with which the display device is controlled according to the user's actual intention.

Therefore, in order to increase the accuracy of the display device control, the display control device needs to consider the user environment. For example, in the case of a display device such as a TV, the accuracy of the display device control may be greatly influenced by the distance between the user and the TV. For example, in order to present a visual stimulus to a user through a TV screen and to control the TV using an electroencephalogram signal for visual stimulation, an EEG signal obtained from a user, Control of the TV and control of the TV using the electroencephalogram signal obtained from the user at a distance of 1m or 5m from the TV result in a difference in the accuracy of the control.

Therefore, in order to improve the accuracy of the display device control, it is required to change the stimulus and the screen configuration presented to the user in accordance with information about the user environment such as the distance between the display device and the user.

A method and apparatus for controlling a display device according to an embodiment of the present invention includes a step of generating stimulus based on at least one of a user's peripheral environment, a user's EEG response characteristic, and a user preference, You can adjust the presented conditions.

According to the method and apparatus for controlling a display device according to an embodiment of the present invention, a stimulus to be presented to a user is determined using information on a user environment, and a display device is controlled based on an EEG signal for a determined stimulus, The accuracy of the device control can be increased.

Hereinafter, a display control apparatus 200 according to an embodiment of the present invention will be described in detail with reference to FIG. 2 is a conceptual diagram for explaining a display control apparatus for controlling a display apparatus according to an embodiment of the present invention.

2, the display control device 200 according to an exemplary embodiment of the present invention controls the display device 10 using an electroencephalography (EEG) signal acquired from the brain of the user 105 do.

The electroencephalogram signal may be acquired via a sensor attached on the head of the user 105 and transmitted wired or wirelessly to the display control apparatus 200. The display control device 200 determines the intention of the user 105 to control the display device 10 by analyzing the electroencephalogram signal obtained from the user 105 and displays the result on the display device 10 ).

The display control device 200 and the display device 10 can be connected to each other by wire or wirelessly. In addition, the display control device 200 and the display device 10 may not be physically separated from each other as shown in FIG. 2 but may be integrated.

A method of controlling a display device (10) by a display control device (200) according to an embodiment of the present invention includes determining a stimulus in consideration of information on a user environment, presenting a determined stimulus, The accuracy of the display control apparatus 200 can be improved by analyzing the user's electroencephalogram signal.

The information about the user environment may include, but is not limited to, for example, the distance between the display device and the user, the illuminance around the display device, the illuminance around the user, and the sensitivity of the user to visual stimulation.

The method of controlling the display device 10 according to an exemplary embodiment of the present invention may determine which of the five senses of the user should be presented based on information about the user environment. The stimuli presented to the user may include stimuli for at least one of visual, auditory, tactile, taste, and olfactory. For example, the display control apparatus 200 according to an exemplary embodiment of the present invention may display audible stimuli when the user is distant from the screen of the display apparatus by a predetermined distance or more, Or a method of sensing the imagery of voluntary movement.

The method of detecting the motion of the vortex is one of the Brain Computer Interface (BCI) methods for detecting the motion imagined by the user. For example, the display control device 200 can control the display device 10 to execute the leftmost menu displayed on the display device 10 based on the image of the user who wants to hold the left hand .

 In addition, the method of controlling the display device 10 according to an embodiment of the present invention may determine the stimulation parameters in consideration of information on the user environment. For example, where the stimulus includes auditory stimulation, the stimulation parameter may include at least one of a frequency of auditory stimulation, a sound pressure, a tone, and a direction of auditory stimulation the user feels. When the stimulus includes a visual stimulus, the stimulus parameter may determine the brightness of the visual stimulus, the type of visual stimulus, the color of the visual stimulus, the duration of the visual stimulus, and the number of repetitions of the visual stimulus, It does not.

Hereinafter, a method of controlling the display apparatus 10 by the display control apparatus 200 according to an embodiment of the present invention will be described in detail with reference to FIG.

3 is a flowchart illustrating a method of controlling a display device according to an exemplary embodiment of the present invention.

In step S310, the display control apparatus 200 according to an exemplary embodiment of the present invention determines stimulus to be presented to the user using information on the user environment.

The display control device 200 can determine the type of stimulus using information on the user environment. In other words, the display control device 200 can determine, as a kind of stimulus, whether a stimulus recognized by a certain body sensation should be presented from the user's five senses. For example, the display control device 200 may present visual stimuli through the display device 10, or present auditory stimuli, such as included in the display device 10 or in a separate headset or speaker, It does not.

The display control device 200 can present the visual stimulus to the user as a basic stimulus and present the auditory stimulation to the user when it is determined that the user is difficult to receive the visual stimulus using the information on the user environment. For example, if the distance between the display device and the user is greater than a predetermined distance, the display control device 200 may decide to present the auditory stimulus to the user.

Further, the display control device 200 can determine the stimulation parameter. The stimulation parameter means a value indicating a specific property of the stimulus, such as the intensity of the stimulus, the duration of the stimulus, and the number of repetitions of the stimulus.

As an example, the parameters for auditory stimulation may include at least one of the frequency of sound presented to the user as auditory stimulus, the sound pressure, the tone color, and the direction of the sound felt by the user. The display control device 200 can present a sound having a different intensity and time to the left and right sides of the user so that the user can receive a feeling that sound is transmitted from a sound source located in a specific direction.

As another example, the parameters for the visual stimulation include the display position of the visual stimulus, the brightness of the visual stimulus, the type of the visual stimulus, the color of the visual stimulus, the duration of the visual stimulation, And the number of repetition times.

At this time, the information on the user environment may include at least one of the distance between the display device 10 and the user, the illuminance around the display device 10, the illuminance around the user, and the user information.

The user information may include a degree of sensitivity to a predetermined stimulus for each user, a degree of feeling fatigue, a predetermined stimulation parameter for a specific user, and the like.

As one example of the user information, the display control device 200 can utilize the characteristics of the user's EEG signal.

The display control device 200 can determine the characteristics of the electroencephalogram signal for each user from the electroencephalogram signals obtained in the past. The display control device 200 may map and store the EEG signal characteristics determined for each user and the user in advance. For example, the display control device 200 can previously store the kind of the stimulus or the stimulus parameter that the user strongly reacted in the past as the characteristics of the user's electroencephalogram signal.

The display control device 200 can identify the user and determine the type of stimulus or determine the stimulation parameter using the mapped electroencephalogram signal characteristics for the identified user.

As another example of the user information, the display control device 200 may use stimulus parameters previously stored for each user.

The display control device 200 can receive the stimulus parameters for the user from the user. In other words, the user can minimize the fatigue caused by the stimulation felt by using the display control device 200 by setting the stimulation parameter that feels the least fatigue in advance.

The display control device 200 can previously store the stimulus parameters for each user and the corresponding user. The display control apparatus 200 can identify the user and use the stimulus parameter previously stored for the identified user as user information.

In step S320, the display control device 200 according to an embodiment of the present invention presents the stimulus determined in step S310 to the user.

The display control device 200 can generate a stimulus based on the type of the determined stimulus and the stimulus parameter, and present the generated stimulus to the user through the display device 10 or a separate device.

When using the user's electroencephalogram signal for visual stimulation, firstly, the display control device 200 can display at least one selection menu on the display device 10. [ The user can manipulate the display device 10 by selecting at least one selection menu displayed on the display device 10. [

For example, the at least one selection menu may include a menu associated with at least one TV channel. The menu related to the TV channel includes a user interface for changing a TV channel, for example, an interface for inputting a channel switching command or an input of a number corresponding to a channel, such as an arrow or a numeric pad, Lt; RTI ID = 0.0 > a < / RTI > For example, the user can switch TV channels by selecting a menu associated with at least one TV channel displayed on the TV.

Next, the display control device 200 can highlight the selection menu included in the at least one selection menu in a predetermined order according to the predetermined number of times and intervals, thereby presenting the visual stimulation to the user. The number and interval of highlighting the selection menu may be predetermined by the user's input, or may be stored in advance according to the type or number of selection menus to be displayed. The predetermined order in which each selection menu is highlighted may be a pre-stored order or a randomly determined order.

For example, in order to highlight a selection menu, the display control apparatus 200 may be configured to change the brightness of at least a part of the border of the selection menu, change the display position of the selection menu, or change the shape or color of the selection menu , And can display a selection menu on the display device 10. [

With regard to the method of presenting the visual stimulus, it will be described later in more detail with reference to FIG.

When using the user's electroencephalogram signal for auditory stimulation, the display control device 200 can generate sound corresponding to at least one selection menu on the display device 10. [ The user can select at least one selection menu and manipulate the display device 10 by responding to sounds corresponding to at least one selection menu presented by the display device 10. [ The sound corresponding to each selection menu may be a sound having different auditory stimulation parameters. For example, when the selection menu is a menu for selecting a TV channel, the sound corresponding to each selection menu may include a sound broadcast through a TV channel corresponding to each selection menu.

In step S330, the display control apparatus 200 according to an embodiment of the present invention acquires the user's electroencephalogram signal for the stimulus presented in step S320.

The display control device 200 can acquire an electroencephalogram signal sensed by the user's scalp. The display control device 200 may include a plurality of sensing modules attached to a scalp of a user to obtain an electroencephalogram signal.

The electroencephalogram signal may be obtained through a plurality of channels corresponding to each of a plurality of sensing modules attached to a scalp of a user. For example, the acquired electroencephalogram signal may include a P3 component that is an event related potential that is detected about 300 ms after the visual stimulus is presented.

The display control apparatus 200 can acquire the user's electroencephalogram signal with respect to the visual stimulus presented for the specific selection menu when the user concentrates on the specific selection menu with the intention of selecting.

For example, when the display control device 200 displays at least one selection menu and a visual stimulus for each selection menu through the display device 10, the user can select at least one selection menu . The display control apparatus 200 can acquire an electroencephalogram signal received from one or more channels in synchronization with the presented visual stimulus.

In step S340, the display control device 200 according to an embodiment of the present invention controls the display device 10 based on the electroencephalogram signal obtained in step S330.

In order to control the display device 10, the display controller 200 may perform a process of pre-processing the obtained electrocardiographic signal and determining a visual stimulus in which the user has reacted from the preprocessed electrocardiographic signal.

For example, the display control device 200 can pre-process the acquired electrocardiogram signal as follows. First, the display controller 200 may apply various filtering to the obtained electroencephalogram signal to remove various noise that may be included in the electroencephalogram signal.

For example, band pass filtering may be performed on the acquired EEG signal to leave only a signal within a certain critical frequency, for example, 0.5 to 20 Hz.

In addition, as another example, the display control apparatus 200 may perform a common average reference derivation, Laplacian filtering, Independent Component Analysis, and the like on the obtained electrocardiogram signal, A similar filtering scheme can be applied.

Next, the display control apparatus 200 can display the stimulated stimulus from a point of time before the stimulus is presented, for example, about 100 ms before the stimulus is presented, by matching the calibrated corrected electroencephalogram signal from which the noise is removed and the point at which the stimulus is presented to the user The brain wave signal of 300 ms is obtained as the region of interest (ROI) data.

The display control device 200 compares the region of interest electroencephalogram data according to the presented stimulus when the region of interest electroencephalogram data is obtained for each stimulus. Based on the comparison result, the display control device 200 determines which stimulus the user has responded most strongly to.

The user's strongest response to a stimulus can be determined by analyzing the event-related potential for the stimulus. Particularly, in analyzing the event related potentials related to the stimuli perceived by the user's body senses, the most strongly stimulated stimulus is generated based on the amplitude magnitude or the latency of P3, N2, It can be judged.

The display control device 200 can utilize the area of interest electroencephalogram data for a single stimulus to determine which stimulus the user has responded most strongly to. In order to increase the accuracy, it is also possible to utilize the area-of-interest electroencephalogram data for the same kind of repeated stimulation by repeatedly presenting the same kind of stimulus.

Meanwhile, the display controller 200 may further perform an operation of calibrating the region-of-interest EEG data by utilizing the characteristics of the user's EEG signal accumulated for each user.

The user-specific EEG signal characteristics may include at least one of a channel at which an event-related potential is detected when the user responds to the stimulus, an amplitude of the detected event-related potential, and an amplitude generation time.

The display control device 200 may be configured to display the stimulus on the basis of the characteristics of the user's EEG signal with respect to the stimulus when the user responds to the target stimulus (for example, a stimulus for the selected menu to be selected) It is possible to learn the electroencephalogram signal at the time of the reaction. Accordingly, the display control device 200 can determine whether the user has responded to the stimulus by analyzing the newly obtained EEG signal using the previously learned EEG characteristic of each user.

The display control device 200 can select and execute at least one of the at least one selection menu displayed on the display device 10 based on the user's electroencephalogram signal.

The display control device 200 can determine and analyze the electroencephalogram signal using a plurality of algorithms to determine which of the at least one selection menu displayed on the display device 10 has responded to which selection menu. An algorithm for analyzing a user's electroencephalogram signal may be a linear discriminant analysis (LDA), a support vector machine (SVM), a random forest, or a similar algorithm.

The display control device 200 can select at least one of the at least one selection menu displayed on the display device 10 based on the result determined using a plurality of algorithms.

The display control apparatus 200 can determine a selection menu responded by the user to at least one selection menu displayed on the display device 10 using each algorithm included in the plurality of algorithms. The display control apparatus 200 can finally determine the selection menu to be selected by the user by synthesizing the results determined by a plurality of algorithms.

For example, the display control apparatus 200 can finally determine the most-selected selection menu as a selection menu for the user to select according to a result determined by a plurality of algorithms. Alternatively, the display control apparatus 200 may finally determine, as a selection menu to be selected by the user, based on the average value of the results determined by the plurality of algorithms.

The display control device 200 can control the display device 10 to execute the finally selected selection menu. For example, the display control apparatus 200 may display a submenu corresponding to a selected menu, execute an application corresponding to the selected menu, play a video / movie corresponding to the selected menu, It is possible to switch the channel to the TV channel corresponding to the TV channel.

Also, the display control device 200 can turn off the display device 10 when it is determined that the user is in the sleep state based on the user's electroencephalogram signal.

Whether or not the user is in the sleep state can be judged by detecting the sleeping brain wave based on the user's electroencephalogram signal, detecting a state in which the concentration of the user is continuously deteriorated, or detecting the state in which the user closes his eyes . Further, it can be judged in consideration of the detection result in a comprehensive manner. The state in which the concentration of the user is continuously decreased can be determined by comparing the concentration determined based on the user's electroencephalogram signal with a specific reference value. The state in which the user closes his or her eyes can be detected based on the electroencephalogram signal and can also be detected through a separate motion sensor or the like that detects the eye movement of the user.

 For example, when the concentration of the user is less than or equal to a predetermined reference value, the display control device 200 provides the user with feedback on the concentration of the user, determines that the user is in the sleep state, The power can be turned off. Alternatively, if it is detected that the user is closing the eyes for a predetermined time or more based on the user's EEG signal, the display control device 200 determines that the user is in the sleep state and can turn off the display device 10 have.

4 is a flowchart illustrating a method of generating a visual stimulus in a method of controlling a display device according to an embodiment of the present invention.

The display control device 200, as shown in Fig. 3, presents the stimulus determined at step S310 to the user at step S320. At this time, the display control device 200 can present the visual stimulus to the user as a stimulus for acquiring the electroencephalogram signal. Hereinafter, a method of presenting a visual stimulus to at least one selection menu to the user so that at least one selection menu can be selected and executed based on the user's electroencephalogram signal will be described with reference to FIG.

In step S422, the display control apparatus 200 according to an embodiment of the present invention may receive a trigger signal for displaying a selection menu.

According to one embodiment, the trigger signal for displaying the selection menu may include a signal generated based on the eye movement sensed using the user's brain conduction signal.

The display control apparatus 200 may determine that a trigger signal for controlling the display of the selection menu is received when the detected eye movement corresponds to a predetermined eye movement using the user's brain conduction signal.

For example, the display control device 200 displays a selection menu when the eye movement sensed by the user using the EEG signal is a continuous blinking operation of the eyes or an eye closing operation for a specific time .

According to another embodiment, the trigger signal for displaying the selection menu may comprise a signal generated based on the mental state of the user sensed using the user's electroencephalogram signal.

The mental state of the user detected by using the user's electroencephalogram signal can be expressed as the degree of user's concentration, feeling of boredom, and feeling of tiredness. Therefore, the display control device 200 can compare the predetermined value indicating the mental state of the user with a predetermined reference value, and judge that the trigger signal for controlling the display of the selection menu is received.

For example, the display control device 200 senses a mental state of a user who is viewing a general screen displayed by the display device 10, and when the user's concentration is below a predetermined reference value, And display on the display device 10 whether or not to switch to a screen including a selection menu.

Alternatively, the display control device 200 may control the display device 10 to automatically switch the screen currently displayed to a screen including a selection menu when the concentration of the user is less than or equal to a specific reference value.

According to yet another embodiment, the trigger signal for displaying the selection menu may include a signal generated based on the user's mental state and eye motion sensed using the user's electroencephalogram signal. The display control apparatus 200 can determine that the trigger signal for controlling the display of the selection menu is received when the concentration of the user is less than or equal to a certain reference value and the sensed eye movement corresponds to a predetermined eye movement .

According to yet another embodiment, the trigger signal for displaying the selection menu may include a signal generated based on at least one of a user's gesture, a user's voice, and a user's eye action. When at least one of the detected user gesture, the user's voice, and the user's eye operation corresponds to a predetermined condition, the display control device 200 can determine that the trigger signal for controlling the display of the selection menu is received have.

In step S424, the display control apparatus 200 according to an embodiment of the present invention can display a selection menu when a trigger signal for displaying a selection menu is received.

When the trigger signal for displaying the selection menu is received, the display control device 200 can control the display device 10 to switch the displayed basic screen to a screen including the selection menu.

The selection menu may be displayed on the display device 10 in the form of video, moving picture, text, or the like.

5A and 5B show an example of a screen including a selection menu displayed on the display device 10 by the display control device 200. Fig.

5A and 5B, when the trigger signal for displaying the selection menu is received, the basic screen 510 displayed on the display device 10 can be switched to the partial screens 525 and 535 have.

For example, when the display device 10 is reproducing a moving image as a basic screen, a trigger signal for displaying a selection menu is received, so that the moving image being reproduced can be switched to a partial screen. The display control device 200 can control the display device 10 so that the moving image is continuously reproduced even when the display screen is switched to the partial screen.

5A, when a trigger signal for displaying a selection menu is received, a screen 520 including selection menus 521, 522, 523, and 524 may be displayed on the display device 10 .

For example, the selection menus 521, 522, 523, and 524 may include a thumbnail for an image or a video, a thumbnail for a TV channel, an application that can be displayed through the display device 10 An icon for executing the program, a graphical user interface (GUI) for setting the functions of the display device 10 or the display control device 200, and the like.

When the display device 10 is a TV, the display control device 200 can select a channel broadcasting the most suitable TV program based on the mental state of the user. In this case, the display control device 200 can display a thumbnail for the selected channel on the display device 10 as a selection menu.

Alternatively, the display control device 200 analyzes the past TV viewing record of the user, selects channels on which the user watches TV programs with a high frequency of watching TV, and displays a thumbnail of the selected channel on the display device 10 Can be displayed.

5B, when the trigger signal for displaying the selection menu is received, the basic screen 510 is switched to the partial screen 535, and the selection menus 531, 532, 533 , ≪ / RTI > 534) may be displayed.

 The display control apparatus 200 can display a plurality of submenus 531.1, 531.2, 531.3 and 531.4 on the display device 10 as a single selection menu 531 by grouping them.

For example, the display control apparatus 200 can classify a plurality of TV channels according to a category, and display a selection menu on the display device 10 from which a category can be selected. When the category is selected by the user, a selection menu for selecting a plurality of TV channels included in the selected category can be displayed on the display device 10. [

In step S426, the display control apparatus 200 according to an embodiment of the present invention may receive a trigger signal for presenting a visual stimulus.

For example, a trigger signal for presenting a visual stimulus may be generated based on a user's mental state using a signal generated based on a sensed eye motion using the user's electroencephalogram signal or a user's mental state signal Signal.

As another example, a trigger signal for presenting a visual stimulus may include a signal generated based on at least one of a user's gesture, a user's voice, and a user's eye action. The display control device 200 can determine that a trigger signal for presenting the visual stimulus is received when at least one of the detected user's gesture, the user's voice, and the user's eye action corresponds to a predetermined condition .

In step S428, the display control device 200 according to an embodiment of the present invention displays at least one selection menu displayed on the display device 10 as a highlight can do.

The user can control the display device 10 to return to the basic screen by performing a predetermined eye operation when there is no selection menu to be selected among the displayed selection menus. The display control device 200 controls the display device 10 to switch the screen including the selection menu to the basic screen if the detected eye movement is associated with a predetermined eye movement using the user's electroencephalogram signal can do.

Also, the display control device 200 may control the display device 10 to switch the screen including the selection menu to the basic screen when the user does not react to the presented visual stimulation for a predetermined time.

6 is a view for explaining an example of a visual stimulus displayed according to a display control method according to an embodiment of the present invention.

The display control apparatus 200 according to an embodiment of the present invention can present a visual stimulus to a user by highlighting a selection menu.

For example, the display control apparatus 200 may display a visual stimulus by using a visual effect such as a part of the screen on which the selection menu is displayed is blinking or a part of the screen is vibrated, Can be minimized.

6A, the display control device 200 displays a visual stimulus to the user by displaying a selection menu 610 such that the brightness of a portion 620 of the border 620 of the selection menu 610 is changed can do.

6 (b), the display control apparatus 200 displays the selection menu 610 so that the brightness of the border 630 of the selection menu 610 is changed, thereby providing a visual stimulation to the user Can be presented.

6 (c), the display control apparatus 200 may be configured so that the display position of the selection menu 610 is changed, for example, the selection menu 610 vibrates or rotates A visual stimulus can be presented to the user by displaying the selection menu 610 so that the visual effect appears.

The display control device 200 according to the embodiment of the present invention is configured to generate the visual stimulus based on the changeable visual stimulation parameter, thereby minimizing the user's fatigue by visual stimulation.

As an example, prior to the presentation of the visual stimulus, the display control apparatus 200 may display the display position of the visual stimulus, the color of the visual stimulus, the magnitude of the visual stimulus, the type of the visual stimulus, The user's fatigue due to the visual stimulation can be reduced.

As another example, the display control device 200 may determine the display position of the visual stimulus, the color of the visual stimulus, the size of the visual stimulus, the type of the visual stimulus, the duration of the visual stimulation, Thus, the user's fatigue due to the visual stimulation can be reduced.

For example, the display control apparatus 200 can determine the degree to which the user is sensitive to visual stimulation, based on the distance between the display apparatus 10 and the user. The display control apparatus 200 can lower the intensity of the visual stimulus as the user reacts sensitively to the visual stimulus.

The visual stimuli presented through the display device 10 can be presented in a predetermined order according to a predetermined number of times and duration for each of the at least one selection menu displayed on the display device 10. [ Or the visual stimulus for each of the at least one selection menu displayed on the display device 10 may be presented at random. Accordingly, the visual stimuli presented through the display device 10 can be presented in an order that the user can not predict in advance.

7 is a block diagram illustrating a display control apparatus according to an embodiment of the present invention.

Each component of the display control apparatus 200 according to an embodiment of the present invention is configured to perform each step of the display apparatus control method shown in FIG. Therefore, it is understood that the contents described above with respect to the display device control method shown in FIG. 3 also apply to the display control device 200 of FIG. 7, even if omitted from the following description.

The display control apparatus 200 according to an embodiment of the present invention controls the display apparatus 10 by utilizing an electroencephalogram signal obtained from the brain of the user 105. The display control apparatus 200 according to an embodiment of the present invention can be implemented in various forms. For example, the display control device 200 may be a device connected to the display device 10 wirelessly, or may be a device included in the display device 10.

7, the display control apparatus 200 according to an exemplary embodiment of the present invention includes a stimulus determination unit 210, a stimulus presentation unit 220, a signal acquisition unit 230, and a control unit 240 .

The stimulus determination unit 210 determines a stimulus to be presented to the user using information on the user environment. Information about the user environment may be input by a user, obtained through various sensors, or received from an external device. The information on the user environment may include at least one of the distance between the display device 10 and the user, the illuminance around the display device 10, the illuminance around the user, and the user information.

The stimulus presentation unit 220 generates the stimulus determined by the stimulus determination unit 210 and presents it to the user. The stimulus presentation unit 220 can present a stimulus recognized by at least one sensation of the user's five senses to the user.

As an example, the stimulus suggestion unit 220 can present a visual stimulus based on the electroencephalogram signal of the user 105 obtained by the signal acquisition unit 230. [ That is, the stimulus presentation unit 220 can receive a signal generated based on at least one of a mental state of the user and the eye action sensed by using the electroencephalogram signal as a trigger signal for controlling to display a selection menu. When the trigger signal is received, the stimulus presentation unit 220 may display a selection menu and present a visual stimulus highlighting at least one selection menu.

As another example, the stimulation presentation unit 220 may present an auditory stimulation based on the user's 105 brain wave signal acquired at the signal acquisition unit 230. [ That is, the stimulus presentation unit 220 controls the display device 10 to generate a sound for the selection menu based on at least one of the mental state and the eye action of the user sensed using the electroencephalogram signal Can be received as a trigger signal. When the trigger signal is received, the stimulus presentation unit 220 may generate a sound for the selection menu and present an auditory stimulus having different stimulation parameters corresponding to at least one selection menu.

The signal acquisition unit 230 acquires the user's electroencephalogram signal with respect to the visual stimulus. The signal acquisition unit 230 may acquire an electroencephalogram signal sensed by the user's scalp. The signal acquisition unit 230 may receive an electroencephalogram signal from a plurality of sensing modules attached to the scalp of the user 105. In addition, the signal acquisition unit 230 may include a plurality of sensing modules attached to the scalp of the user 105 to acquire an electroencephalogram signal. The control unit 240 controls the display device 10 based on the electroencephalogram signal.

The controller 240 controls the overall operation of the display controller 200 and includes a stimulus determiner 210, a stimulus presenting unit 220, The control unit 230 can be controlled.

The control unit 240 may control the display device 10 to display at least one selection menu based on the electroencephalogram signal. The control unit 240 can control the stimulation presentation unit 220 to generate a visual stimulus and to cause the display device 10 to present the generated visual stimulus to the user. In presenting the visual stimulus, the control unit 240 may control the display device 10 to highlight at least one selection menu in an arbitrary order according to the predetermined number of times and intervals.

In addition, the control unit 240 can determine which of the at least one selection menu displayed on the display device 10 the user 105 has responded to. In determining the selection menu in response to the user 105, the control unit 240 may synthesize the results derived from various algorithms and use the synthesized results.

The control unit 240 controls the display device 10 to perform an operation corresponding to the selection menu selected by the user 105 (for example, a sub-selection menu display, application execution, movie reproduction, channel switching, can do.

As described above, according to the method and apparatus for controlling a display device according to an embodiment of the present invention, a user can more easily control the display device, and can control the display device with high accuracy according to the intention of the user , It is possible to reduce the fatigue of the user due to stimulation for detecting the electroencephalogram signal.

One embodiment of the present invention may also be embodied in the form of a recording medium including instructions executable by a computer, such as program modules, being executed by a computer. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, the computer-readable medium may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically includes any information delivery media, including computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transport mechanism.

It will be understood by those skilled in the art that the foregoing description of the present invention is for illustrative purposes only and that those of ordinary skill in the art can readily understand that various changes and modifications may be made without departing from the spirit or essential characteristics of the present invention. will be. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive. For example, each component described as a single entity may be distributed and implemented, and components described as being distributed may also be implemented in a combined form.

The scope of the present invention is defined by the appended claims rather than the detailed description and all changes or modifications derived from the meaning and scope of the claims and their equivalents are to be construed as being included within the scope of the present invention do.

Claims (23)

Determining a stimulus to be presented to the user using information about the user environment;
Presenting the stimulus to the user;
Obtaining an electroencephalogram (EEG) signal of the user for the stimulus; And
And controlling the display device based on the electroencephalogram signal. The method according to claim 1,
The information on the user environment may include,
A distance between the display device and the user, an illuminance around the display device, and user information. The method according to claim 1,
The step of determining the stimulus comprises:
Determining which stimulus is recognized by the user's body sensation; And
And determining the stimulation parameter. The method according to claim 1,
The step of determining the stimulus comprises:
Determining to present the auditory stimulus to the user if the distance between the display device and the user is greater than a predetermined distance; And
Determining a stimulation parameter that includes at least one of a frequency of the auditory stimulation, a sound pressure, a tone color, and a direction of the auditory stimulation the user feels. The method according to claim 1,
Wherein the stimulus comprises a visual stimulus,
The step of determining the stimulus comprises:
A display position of the visual stimulus, a brightness of the visual stimulus, a shape of the visual stimulus, a color of the visual stimulus, a duration of the visual stimulus, and a repetition number of the visual stimulus And determining a stimulation parameter to be applied to the display device. The method according to claim 1,
Wherein the step of presenting the stimulus comprises:
Displaying at least one selection menu on a screen of the display device; And
And highlighting each selection menu included in the at least one selection menu in an arbitrary order according to a predetermined number of times and intervals. The method according to claim 6,
Wherein highlighting the selection menu comprises:
And displaying a change in brightness of at least a part of a border of the selection menu or a display position of the selection menu to be changed. The method according to claim 6,
The step of controlling the display device includes:
And selecting and executing at least one of the at least one selection menu based on the electroencephalogram signal. The method according to claim 1,
The step of controlling the display device includes:
And turning off the power of the display device when it is determined that the user is in a sleep state based on the electroencephalogram signal. The method according to claim 1,
Wherein the step of presenting the stimulus comprises:
Detecting at least one of mental state of the user and eye operation of the user using the user's electroencephalogram signal; And
And displaying at least one selection menu on a screen of the display device based on at least one of the mental state and the eye action. The method according to claim 1,
Wherein the step of presenting the stimulus comprises:
And presenting a visual stimulus for at least one selection menu displayed on the display device,
The step of controlling the display device includes:
Determining, using a plurality of algorithms, which of the at least one selection menu the user has responded to for a selection menu; And
And selecting and executing at least one of the at least one selection menu based on the determined result. A stimulus stimulus decision unit for determining a stimulus to be presented to a user using information about the user environment;
A stimulus presentation unit for presenting the stimulus to the user;
A signal acquisition unit for acquiring the user's electroencephalogram signal with respect to the stimulus; And
And a control unit for controlling the display device based on the electroencephalogram signal. 13. The method of claim 12,
The information on the user environment may include,
A distance between the display device and the user, an illuminance around the display device, and user information. 13. The method of claim 12,
The stimulus-
Wherein the stimulation parameter determining unit determines whether a stimulus recognized by the user's body sensation is to be presented, and determines a stimulation parameter. 13. The method of claim 12,
The stimulus-
Determining at least one of a frequency of the auditory stimulation, a sound pressure, a tone, and a direction of auditory stimulation felt by the user when the distance between the display device and the user is a predetermined distance or more, And determines a stimulation parameter to be included. 13. The method of claim 12,
Wherein the stimulus comprises a visual stimulus,
The stimulus-
A display position of the visual stimulus, a brightness of the visual stimulus, a shape of the visual stimulus, a color of the visual stimulus, a duration of the visual stimulus, and a repetition number of the visual stimulus Wherein the stimulation parameter determining unit determines the stimulation parameter. 13. The method of claim 12,
The stimulus presentation unit,
Displays at least one selection menu on a screen of the display device and highlights each selection menu included in the at least one selection menu in an arbitrary order according to a predetermined number of times and intervals. 18. The method of claim 17,
The stimulus presentation unit,
And highlighting the selection menu so that the brightness of at least a part of the border of the selection menu is changed or the display position of the selection menu is changed. 18. The method of claim 17,
Wherein,
And selects and executes at least one of the at least one selection menu based on the electroencephalogram signal. 13. The method of claim 12,
Wherein,
And turns off the display device when it is determined that the user is in a sleep state based on the electroencephalogram signal. 13. The method of claim 12,
The stimulus presentation unit,
Detecting at least one of the mental state of the user and the eye action of the user by using the user's electroencephalogram signal and displaying at least one selection on the screen of the display device based on at least one of the mental state and the eye action And displays a menu. 13. The method of claim 12,
The stimulus presentation unit,
Presenting a visual stimulus for at least one selected menu displayed on the display device,
Wherein,
Wherein the plurality of selection menus are used to determine whether the user has responded to which selection menu among the at least one selection menu and to select and execute at least one of the at least one selection menu based on the determination result . A computer-readable recording medium recording a program for causing a computer to execute a method of controlling a display device,
The method comprises:
Determining a stimulus to be presented to the user using information about the user environment;
Presenting the stimulus to the user;
Obtaining an electroencephalogram signal of the user for the stimulus; And
And controlling the display device based on the electroencephalogram signal.

Images