JP6553492B2 - Object control method and control apparatus using electroencephalogram - Google Patents

Description

  The present invention relates to an object control method capable of controlling an object such as a computer by using an electroencephalogram, and more specifically, the computer etc. from the change in frequency before and after performing a predetermined operation image The present invention relates to an object control method that can be controlled according to an image.

  In recent years, methods of controlling a computer or the like using human brain waves have been studied. For example, Patent Document 1 below proposes a control device that can extract a brain wave from a sensor attached to the head and control a computer. This controller measures the brain waves at a certain time after the start of the image when the subject periodically images the movement of a predetermined body part of the subject, and also measures the power spectrum of the β wave band (frequency 13 to 30 Hz) from the brain waves. Thus, an object such as a brain computer interface can be controlled depending on whether the power spectrum exceeds a threshold value determined based on the power spectrum at the frequency of the brain wave before the image. If such a system is used, for example, a subject can control a computer etc. only by imaging a predetermined exercise | movement in a brain, A hand etc. may perform operation assistance of the test subject. It is expected to be possible.

JP 2013-128642 A

  However, in general, such changes in frequency are easily affected by individual differences such as the environment, brain function state and mental state, and only by looking at changes in a predetermined frequency, an image is accurately created in the brain. It may not be possible to determine whether or not there is. Specifically, the electroencephalogram before and after the motion image when exercising in an environment that is very susceptible to external stimuli, situations where the brain function is weak, or individual differences such as psychological state It may not be possible to judge exactly whether the image is created or not just by looking at the change state of.

  Therefore, the present invention has been made focusing on the above-mentioned problems, and an object of the present invention is to provide a method of controlling an object by using an electroencephalogram which is less susceptible to the environment, brain function state, individual difference and the like.

  That is, in order to solve the above problems, the present invention relates to the steps of extracting an electroencephalogram from an electroencephalogram detection sensor attached to one of the head corresponding to the right brain or the left brain, and having the subject perform a predetermined operation. The step of storing the change state of the extracted electroencephalogram, the step of extracting the electroencephalogram from the electroencephalogram detection sensor attached to the head on the opposite side to the attached side, and the electroencephalogram detection sensor on the opposite head Controlling the object so as to operate the body corresponding to the brain on which the brain wave detection sensor is attached, when the change state of the extracted brain wave matches the stored change state; It is intended to provide.

  In this way, the waveform change of the electroencephalogram acquired on one side of the brain is compared with the waveform change acquired on the other side of the brain to determine the presence or absence of an action image, and the corresponding brain on the other side is supported. Since the object is controlled as if the body part is moving, the object can be controlled without being subject to fluctuations such as the environment, the brain function state, and individual differences.

  Further, in such an invention, the determination is made by the combination pattern of the change state of each frequency band.

  In this way, by extracting the temporal change of the θ wave band, α wave band, and β wave band when the operation image is performed, compared to the case of extracting the frequency change of each band alone, It becomes possible to determine whether the operation image is being performed more accurately.

  According to the present invention, the waveform change of the electroencephalogram acquired on one brain side is compared with the waveform change acquired on the opposite brain to determine the presence or absence of an operation image, and the corresponding brain is supported Since the object is controlled as if the body part is moving, the object can be controlled without being subject to fluctuations such as the environment, the brain function state, and individual differences.

1 is an overall schematic diagram of an object control apparatus showing an embodiment of the present invention. Functional block diagram in the same form The figure which shows the change state of the power spectrum in each frequency band in the same form Normalized power spectrum before and after the operation image in the same form The figure which shows the allowable value of a change state and a coincidence state in the same form Flow chart in the same form

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

  The object control device 1 according to this embodiment is an object such as the tablet terminal 8 as if the body having the functional disorder is moved with respect to the subject having the functional disorder in either the left or right body. As shown in FIG. 1 and FIG. 2, an electroencephalogram detection unit 2 mounted on the head corresponding to the right brain or the left brain (preferably the healthy brain), as shown in FIGS. The frequency analysis unit 3 and the change state extraction unit 4 that extract the change state of the electroencephalogram detected by the electroencephalogram detection unit 2 when the operation is performed, and the storage unit 5 that stores these change states together with the test subject's operation The electroencephalogram is detected from the electroencephalogram detection unit 2 attached to the head on the opposite side to the attached side, and the change state of the electroencephalogram detected from the electroencephalogram detection unit 2 on the opposite head is stored Consistent with the change state If that one in which the brain wave detecting unit 2 has to a control unit 7 for controlling the screen of the tablet device 8 as by operating body corresponding to the brain that is mounted. And, by controlling the tablet terminal 8 on the basis of the change state of the same subject's electroencephalogram in this way, it has a functional disorder without being influenced by individual differences such as environmental and psychological factors. It is intended to rehabilitate the side half of the body. Hereinafter, the object control apparatus 1 in this Embodiment is demonstrated in detail.

  First, the electroencephalogram detection unit 2 is attached to a human head so as to detect a brain signal. The electroencephalogram detection sensor 21 attached to the head or the head attached to the head. A slight potential difference is detected from the brain wave detection sensor 21 of the set, and a brain wave signal which is a temporal change of this potential difference is detected. At this time, since the current flowing to the muscle of the face and the like are included as noise together in the detected electroencephalogram signal, the noise is removed using a filter, and the electroencephalogram signal is amplified. Then, the frequency analysis unit 3 performs frequency analysis on the amplified signal so that it can be extracted in time series how much power spectrum is included in which frequency band.

  This frequency analysis unit 3 (see FIG. 2) mainly includes how much power spectrum of the frequency of the θ wave band (4 to 7 Hz), the α wave band (8 to 13 Hz), and the β wave band (13 to 30 Hz). Analyzes whether or not

  When this frequency analysis is performed, as a first method, the power spectrum of each frequency band can be extracted by short-time Fourier transform (STFT). When performing this short-time Fourier transformation, first, while shifting the analysis frames little by little while overlapping analysis frames, perform short-time Fourier transformation within each analysis frame to calculate discrete frequency bins in the time and frequency domain To do. And since it is not possible to analyze the temporal change of the power spectrum of the frequency as it is, the power spectrum in the obtained frequency bin is linked in the time axis direction so that the temporal change of the power spectrum can be calculated. . However, when calculating the temporal change of the power spectrum by this short-time Fourier transform, a time lag will inevitably occur, and since the width of the analysis frame is fixed, the time resolution is improved even if the frequency resolution is increased. There is a problem of not. For this reason, preferably, wavelet transformation is used to calculate the temporal change of the power spectrum. If such a wavelet transform is used, there is an advantage that the time change of the power spectrum can be calculated instantaneously as compared with the case where the short-time Fourier transform is performed.

Next, the change state in each frequency band is extracted by the change state extraction unit 4 from the power spectrum thus analyzed. The change state extraction unit 4 first measures brain waves when the subject is not imaging anything, and the power spectra of the θ wave band, the α wave band, and the β wave band at that time are θ ₀ , α ₀ , respectively. It memorize | stores in the memory | storage part 5 as (beta) ₀ (refer FIG. 4). Next, an electroencephalogram when the subject performs an operation such as opening the hand is measured. At this time, for example, the subject is asked to actually operate based on a predetermined cue signal by pressing a button or the like, and an electroencephalogram signal at a predetermined time before the cue signal is extracted and a power spectrum is stored. In general, when the subject is given a motion image, the power spectrum of the θ wave band tends to decrease as shown in FIG. 3, and the power spectrum of the α wave band further decreases significantly, conversely, β It is known that the power spectrum of the wave band tends to increase. Since these increase / decrease values vary depending on individual differences such as environment, psychological factors, brain function status, etc., if the subjects are different, these increase / decrease widths also vary greatly. Therefore, it is possible to first extract the power spectrum in each frequency band at normal times from one brain, and also extract the power spectrum in each frequency band at the time of performing the operation image, and extract the pattern of each change state. Do. In addition to extracting the power spectrum, a moving image when the subject actually exercises is taken.

  Thus, as shown in FIG. 4, the pattern of the change state of the powers spectrum stored in the storage unit 5 is stored as a “value” of increase or decrease in each frequency band, or “rate (%)” of the increase or decrease As a method for storing as When memorizing the increase / decrease value at this time, it is possible that the waveform on one brain side is relatively weaker than the waveform on the other side due to brain dysfunction. Incidentally, even if the waveform output of one of the brains is reduced as a whole, the power spectrum is made common and can be compared.

  The match state determination unit 6 determines whether the pattern of the change state of the power spectrum extracted from the brain on the opposite side matches the pattern of the change state of the power spectrum stored in the storage unit 5 or not. . When judging the coincidence state, the subject is similarly asked to attach the electroencephalogram detection unit 2 at a position corresponding to the opposite brain, and the power spectrum of each frequency band in normal times and the power of each frequency band at the present time Extract the spectrum. Then, the change state of each frequency band is calculated from each value, and whether the pattern of the change state matches the pattern of the change states (Δθ, Δα, Δβ) stored in the storage unit 5 or not to decide. At this time, when a plurality of operation states (for example, an operation to grasp a hand, an operation to shake a hand, etc.) and patterns of change states of the power spectrum are stored in storage unit 5, the power spectrum in each operation state It is determined whether or not the pattern matches the change state pattern. In addition, since there is a possibility that the pattern of the change state on the right brain side and the pattern of the change state on the left brain side do not completely match, as shown in FIG. If so, it is determined that they match. With regard to the tolerance value, one tolerance value may be set for all frequency bands, or a tolerance value may be individually set for each frequency band. In this way, if an allowable value is set for each frequency band individually, it becomes possible to determine a matching state by dividing a frequency band having a large change and a frequency band having a small change individually.

  Next, when such change state patterns match, the control unit 7 controls the object as if using the hand on the half body side corresponding to the brain. As an example of control of such an object, here, as shown in FIG. 1, an example of controlling the screen so as to open the hand projected on the camera of the tablet terminal 8 is given, and the hand is opened by paralysis Assume that a subject who cannot perform rehabilitation by showing the screen as if the hand is open only with an electroencephalogram signal. When this screen is controlled, an electroencephalogram is detected from the electroencephalogram detection unit 2 attached to the brain corresponding to the hand, and the change pattern of the detected state of the power spectrum of the electroencephalogram is stored in the storage unit 5. When the patterns of the states (Δθ, Δα, Δβ) coincide with each other within the allowable range, the moving image at the time when the operation is actually performed is read out from the storage unit 5 and the moving image reversed horizontally is displayed on the screen. Alternatively, when a paralyzed hand is positioned behind the tablet terminal 8, the position of the hand is detected by the camera of the tablet terminal 8, and a moving image that is horizontally reversed according to the position of the hand is displayed on the screen. At this time, it is preferable to display on the screen an image actually shot by the camera for backgrounds other than the hand, and for the hand, to flip the moving image previously shot and to display it.

  Next, the case of controlling the tablet terminal 8 which is an object using such an object control device 1 will be described with reference to a flowchart shown in FIG.

  First, in order to register the relationship between the change state and movement of the subject's brain wave in advance, the brain wave signal is detected by attaching the brain wave detection sensor 21 of the brain wave detector 2 to either the left or right side of the head (step S1). ). When detecting this electroencephalogram signal, noise is removed from the weak potential change detected by the electroencephalogram detection sensor 21, and the signal from which the noise has been removed is amplified to obtain an electroencephalogram signal.

  Then, in a state where the electroencephalogram signal is detected, the subject is allowed to perform a predetermined operation such as opening a hand based on a predetermined signal. When this signal is given, the signal is output to the computer by the operator pressing a button (not shown) or the like (step S2).

  Further, when the subject performs an action in this way, here, as shown in FIG. 1, a hand is put on the back side of the tablet terminal 8, and the action is acquired as a video by the camera of the tablet terminal 8. (Step S3).

  Next, when a cue signal is output in a state where an electroencephalogram signal is being detected, an electroencephalogram signal from a predetermined time before is extracted from the cue signal, and a time change of the power spectrum of each frequency band is extracted by wavelet transform. (Step S4).

Then, the power spectrum of each frequency band in a normal state where no image is given before the signal is given is stored in the storage unit 5 as θ ₀ , α ₀ , β ₀ (step S5). At this time, the values of θ ₀ , α ₀ , β _0, etc. may be power spectra at a certain time or may be average values of power spectra within a predetermined time width.

Next, the power spectrum of each frequency band at the time of performing the “operation image” slightly before giving a signal is extracted, and the value is output to the change state extraction unit 4 to change the power spectrum of each frequency band. It extracts (Step S6). In addition, since the time when the "operation image" is performed can not be judged visually, the change of the power spectrum in the time series is extracted, and the power where the change is the largest within a predetermined time width from the cue signal The time of the spectrum is estimated as the time when the “motion image” is performed, and the change state is extracted. When extracting this change state, the time difference T between the time when the θ ₀ , α ₀ , and β ₀ are extracted and the time when the “operation image” is performed is extracted, and the “value” or “ The ratio etc. are also stored.

  Then, the storage unit 5 stores the pattern of the change state of the power spectrum of each frequency band and the moving image. When storing the change state, if there is a difference between the healthy states of the right and left brains, the change state is normalized and stored in the storage unit 5.

  Next, an electroencephalogram detection sensor 21 of the electroencephalogram detection unit 2 is attached to the head on the opposite side to the head side to detect an electroencephalogram signal (step T1), and time change of power spectrum of each frequency band is extracted by frequency analysis. (Step T2). Then, it is determined whether or not the pattern of the change state of the power spectrum of each frequency band matches the pattern of the change state (Δθ, Δα, Δβ) stored in the storage unit 5 (step T3). When determining the coincidence of the change state patterns, the “value” and “ratio” of the change state of the power spectrum within the time difference T of the power spectrum when the change state is extracted first are extracted. Then, when the pattern of the change state matches within the range of the allowable value, the moving image stored in the storage unit 5 is read out, horizontally reversed, and displayed as a moving image on the screen of the tablet terminal 8 (step T4) ).

  As described above, according to the present embodiment, the electroencephalogram detection unit 2 attached to one of the head corresponding to the right brain or the left brain and the electroencephalogram detection unit 2 when the subject performs a predetermined operation An electroencephalogram is detected from the storage unit 5 for storing the change state of the electroencephalogram and the electroencephalogram detection unit 2 mounted on the head on the opposite side to the attached side, and the electroencephalogram detection unit 2 on the opposite head When the change state of the detected electroencephalogram matches the stored change state, the tablet terminal 8 is controlled so as to operate the body corresponding to the brain on which the electroencephalogram detection unit 2 is attached. Since the control unit 7 is provided, it is possible to determine the presence or absence of the operation image by comparing the waveform change of the electroencephalogram acquired on one brain side with the waveform change acquired on the opposite brain, In response to changes in the environment, brain function status, and individual differences Kuku and it is possible to control the tablet terminal 8.

  In addition, the temporal change in the power spectrum of the θ-wave band, α-wave band, and β-wave band is extracted as a combination pattern of the change state of each frequency band when the operation image is performed. As compared with the case of extracting the change in frequency, it is possible to determine whether or not the operation image is performed more accurately.

  In addition, this invention is not limited to the said embodiment, It can implement in a various aspect.

  For example, in the present embodiment, the change state of three bands of the θ wave band, the α wave band, and the β wave band is extracted as the frequency band, but other frequency bands such as the δ wave band are added thereto. Alternatively, the change state may be extracted using any combination of two frequency bands.

  In the above-described embodiment, the “value” or “ratio” of the change in the power spectrum is described as an example of the change state, but other than this, a differential value of the power spectrum in time series may be used.

  Furthermore, in the above embodiment, the object is controlled so as to cause the tablet terminal 8 to display a moving image that opens the hand, but in addition to this, it outputs a predetermined signal to a computer or another device to operate You may make it control to do. As such an object, for example, a wide variety of objects can be considered, such as switches of lighting and electronic devices such as televisions, and outputs of sounds, lights, images, and the like.

DESCRIPTION OF SYMBOLS 1 ... Object control apparatus 2 ... EEG detection part (21 EEG detection sensor)
DESCRIPTION OF SYMBOLS 3 ... Frequency analysis part 4 ... Change state extraction part 5 ... Memory | storage part 6 ... Coincidence state judgment part 7 ... Control part 8 ... Tablet terminal (target object)

Claims (4)

Extracting an electroencephalogram from an electroencephalogram detection sensor mounted on one of the head corresponding to the right brain or the left brain;
Storing the extracted electroencephalogram change state when the subject performs a predetermined action;
Extracting an electroencephalogram from an electroencephalogram detection sensor attached to the head opposite to the attached side;
When the change state of the electroencephalogram extracted from the electroencephalogram detection sensor of the head on the opposite side matches the stored change state, the body corresponding to the brain on which the electroencephalogram detection sensor is attached is operated Control the object as
An object control method comprising: The object control method according to claim 1, wherein the change state of the detected electroencephalogram is a change state of a combination pattern of each frequency band. An electroencephalogram detector attached to one of the heads corresponding to the right or left brain,
A storage unit for storing a change state of the electroencephalogram detected by the electroencephalogram detection unit when the subject performs a predetermined action;
The electroencephalogram is detected from the electroencephalogram detection unit attached to the head on the opposite side to the attached side, and the change state of the electroencephalogram detected from the electroencephalogram detection unit on the opposite head is the stored change state A control unit that controls the object so as to operate the body corresponding to the brain on which the electroencephalogram detection unit is worn,
An object control device comprising: The object control apparatus according to claim 3, wherein the change state of the detected electroencephalogram is a change state of a combination pattern of each frequency band.

Images