JP5150942B2 - Activity assistance system - Google Patents

Description

  The present invention relates to a system for assisting user activities with a device.

In order to assist the movement of users (passengers) such as electric wheelchairs and automobiles whose physical motor functions have deteriorated, an object that the user is interested in is estimated based on the line of sight, and the object is electrically driven in the vicinity of the object. A technique for moving a wheelchair has been proposed (see, for example, Patent Document 1). This presupposes that the line of sight of the user (human) is closely related to the user's intention and attention.
JP 2005-006897 A paragraphs 0002-0007

  However, the relationship between the user's line of sight and the intention and attention is not constant, and the user sometimes pays attention to an object in a direction different from the line of sight. For this reason, according to the technique, the user may be guided to a place different from the intended place, and the auxiliary content may become inappropriate in view of the user's intention.

  Therefore, an object of the present invention is to provide a system that can more appropriately assist a user's activity in view of the user's intention and attention.

The activity assist system of the present invention for solving the problem is a system that assists a user's activity with an assist device, and is a brain activity variation that is a variation amount of a physical quantity induced by a brain activity according to a user's visual perception. A first measurement unit that measures the distribution of the amount in the head and measures the attention area in the visual field of the user based on the distribution of the brain activity variation , and the time of the spatial distribution of the user's brain activity variation A second measurement unit that measures a change pattern and measures the degree of attention of the user based on a temporal change pattern of the spatial distribution of the brain activity variation amount, and the first measurement unit is in the visual field, that is, in retinal coordinates Define a first function representing the relationship between the distribution of the user's brain activity variation measured in a situation where the user's attention is directed to each of a plurality of regions and the attention region in the visual field, and Based on the measured distribution of the brain activity fluctuation amount of the user, the attention area in the visual field of the user is measured according to the first function defined above, and the second measurement unit Defining a second function representing a relationship between a temporal change pattern of the spatial distribution of the brain activity fluctuation amount of the user measured in a situation where the attention area is changed and the attention degree of the user according to the change situation; Based on the temporal change pattern of the spatial distribution of the brain activity fluctuation amount of the user newly measured above, the attention level of the user is measured according to the second function defined above, and the first measurement unit And a control unit that controls the operation of the auxiliary device according to the degree of attention measured by the second measurement unit in addition to the attention region in the visual field of the user measured by Toss .

According to the activity assistance system of the present invention, the attention area in the visual field of the user is measured based on the distribution of the brain activity fluctuation amount in the user's head. Since the attention area in the user's visual field is closely related to the brain activity according to the user's visual perception, and thus the brain activity fluctuation amount, the attention area in the user's visual field can be accurately measured. Furthermore, the attention level of the user is measured based on the change pattern of the brain activity fluctuation amount of the user. Since the user's attention level is closely related to the brain activity corresponding to the user's visual perception, and hence the brain activity fluctuation amount, the user's attention level can be accurately measured. Furthermore, by following the first function defined to represent the relationship between the user's brain activity variation distribution measured earlier and the attention area in the visual field, the measurement accuracy of the attention area in the user's visual field is improved. ing. Further, the user's attention level in each attention area is measured by following the second function defined to represent the relationship between the change pattern of the user's brain activity variation and the attention level in each attention area. The accuracy is improved. Therefore, by controlling the operation of the auxiliary device according to the attention level in addition to the measured attention area in the visual field, it is possible to appropriately assist the user's activities in view of the user's true intention and attention. . The “physical quantity” includes an electric field, a magnetic field, absorbance of near infrared light, and the like. The “activity” includes not only the physical activity of the user but also the mental activity.

Furthermore, activities auxiliary systems of the present invention, the first measuring unit further measures the gaze of the user, and, based on the distribution and line of sight of the measured the brain activity variation of the user, of the user world coordinates or It is characterized by measuring an attention area in body coordinates.

According to the activity assistance system of the present invention, the attention area of the user is measured by comprehensively considering the brain activity variation distribution and the line of sight in the user's head. Thereby, the measurement accuracy of the user's attention area can be improved. Then, by controlling the operation of the auxiliary device according to the measured attention area and the attention level, the user's activity can be appropriately assisted in view of the user's true intention and attention. aid system, wherein the first measurement unit measures the EEG intensity as brain activity variation, concentration or characteristic values of the specific ingredients of a specific component in the brain blood.

  According to the activity assistance system of the present invention, the attention area in the visual field is measured based on the distribution of the user's brain wave intensity and the like, and the operation of the auxiliary device is performed according to the measured attention area in the visual field and the user's line of sight. Be controlled. Thus, the user's activities can be appropriately assisted in view of the intention and attention reflected in the distribution of the user's brain wave intensity and the like. The brain waves may include those in various frequency bands such as θ waves (4 to 8 Hz), α waves (8 to 12 Hz), β waves (12 to 40 Hz), and the like.

Furthermore, activities auxiliary systems of the present invention, the control unit, the fluctuation amount of the attention region in the field of view of the user measured by the first measurement unit to another whether it is less than a predetermined value for a predetermined time in response, and controlling to distinguish the operation of the auxiliary device.

  According to the activity assisting system of the present invention, for example, the operation of the assisting device is controlled separately in a situation where the attention area in the user's visual field frequently changes and a situation where the attention area does not. Thereby, the operation of the apparatus is appropriately controlled in view of the difference in the situation regarding the user's intention and attention (mental activity).

Also, activities auxiliary systems of the present invention, the second measurement unit, the reaction rate of the user is measured with respect to a change in the attention area in the field of view of the user to evaluate the alertness of the user based on the reaction rate It is characterized by that.

  According to the activity assistance system of the present invention, the second function is defined based on the degree of attention of the user evaluated based on the user's response speed to the change of the attention area in the visual field.

Furthermore, activities auxiliary systems of the present invention, the second measurement unit, measures the attention degree of the user on the basis of the intensity of brain waves of the user, to change the pattern of concentrations or the specific Ingredient characteristic values of the specific component in the brain blood It is characterized by doing.

  According to the activity assistance system of the present invention, the attention level is measured based on a change pattern such as the brain wave intensity of the user, and the operation of the auxiliary device is controlled according to the measured attention level. Accordingly, the user's activity can be appropriately assisted in view of the intention and attention reflected in the change pattern such as the user's brain wave intensity.

Also, activities auxiliary systems of the present invention, the control section, the attention degree of the user measured by the second measuring unit according to another whether it is less than a predetermined value for a predetermined time, the auxiliary device It is characterized by controlling the operation separately.

  According to the activity assistance system of the present invention, for example, the operation of the apparatus is controlled separately in a situation where the user is paying attention to a specific object and a situation where the user is not. Thereby, the operation of the apparatus is appropriately controlled in view of the difference in the situation regarding the user's attention (mental activity).

Furthermore, activities auxiliary systems of the present invention, the control unit is a device for assisting in the movement of the user by moving by riding the user, the movement of the user by applying a force on the body of the user and controlling the physical or operation of a device for assisting in mental activities of the user by providing information to the device, or the user to assist.

  According to the activity assistance system of the present invention, activities such as movement and exercise of the user can be assisted according to the attention area within the visual field of the user.

  An embodiment of the activity assistance system of the present invention will be described with reference to the drawings.

  FIG. 1 is a structural example diagram of the activity assistance system of the present invention, and FIGS. 2 to 5 are functional example diagrams of the activity assistance system of the present invention.

  An activity assistance system 10 shown in FIG. 1 is configured by a computer mounted on an electric wheelchair (device for assisting a user's activity) 20, and includes a first measurement unit 11, a second measurement unit 12, and the like. The control unit 13 is provided.

  The first measurement unit 11 includes a first sub measurement unit 111, a second sub measurement unit 112, and a first function definition unit 114.

  The first sub-measurement unit 111 measures the user's line of sight based on the imaging data of the CCD camera 101 installed facing the user's face. The height and angle of the CCD camera 101 can be adjusted by an appropriate adjustment mechanism. The second sub-measurement unit 112 measures the distribution of α-wave intensity (brain activity fluctuation amount) with the plurality of electrodes 102 arranged in the occipital region corresponding to the visual cortex of the user. The first function definition unit 114 predefines the first function and stores it in the memory (storage unit) based on the α wave intensity distribution that is a reference in the user's occipital region measured by the second sub-measurement unit 112. Then, the first measuring unit 11 includes the first function defined by the first function defining unit 114 based on the line of sight measured by the first sub measuring unit 111 and the α wave intensity distribution measured by the second sub measuring unit 112. The user's attention area is measured according to the function.

  The second measurement unit 12 includes a sub measurement unit 122 and a second function definition unit 124.

  The sub-measurement unit 122 measures the change pattern of the intensity of the α wave with the plurality of electrodes 102 arranged on the back of the head corresponding to the visual cortex of the user. The second function definition unit 124 predefines the second function based on the reference α-wave intensity change pattern measured by the sub-measurement unit 112 and stores the second function in the memory. Then, the second measurement unit 12 measures the degree of attention of the user according to the second function defined by the second function definition unit 124 based on the α wave intensity distribution measured by the sub measurement unit 122.

  The control unit 13 controls the operation of the electric wheelchair 20 according to the user's line of sight measured by the first processing unit 11, the attention area in the visual field, and the degree of caution measured by the second processing unit 12.

  The function of the activity assistance system having the above-described configuration will be described with reference to FIGS.

  First, the second sub-measurement unit 112 measures the reference α-wave intensity distribution (FIG. 2 / S012). When measuring the reference α wave intensity distribution, among the 25 signs arranged in 5 rows and 5 rows as shown in FIG. Fix your line of sight. In addition, an arbitrary one of the signs (a hatched sign) is used as a caution sign, and the color is differentiated from the colors of other signs, thereby causing the user to pay attention to the sign. For example, each label is provided with a light emitting element such as an LED, and the brightness of the light emitting element provided on the attention sign is made higher than the brightness of the light emitting elements provided on other attention signs, so that the brightness of the attention sign is increased. Differentiated from the brightness of other signs.

  As a method for measuring the intensity of the α wave, a method disclosed in Japanese Patent Application Laid-Open No. 2003-325467 may be employed. Since a certain amount of time is required until the change of the user's attention is reflected in the intensity of the α wave, the intensity of the α wave is measured after taking into account the time difference. In addition, the measurement of the α wave intensity distribution is statistically processed, such as averaging the measurement results for each label multiple times in order to suppress variation, and this statistical result is used as the reference α wave intensity distribution. May be measured. Further, these signs may be displayed on a front wall surface or the like by a projector (not shown) installed in the electric wheelchair 20. In addition, the arrangement and number of the signs may be appropriately changed, and the signs may be arranged on any form of surface such as a flat surface, a smooth curved surface, and an uneven surface.

It has recently been clarified that the α wave intensity distribution measured in the visual cortex area corresponding to the attention area in the visual field is reduced as compared with the distribution measured in other visual cortex areas. For example, The Journal of Ne u roscience, 2000, according to Vol.20 RC63 1 of 6, FIG. 5 (a) ~ (d) as shown in a α-wave intensity distribution is measured. The circles shown in FIGS. 5A to 5D represent the user's head, the upper side of the circle represents the forehead, and the lower side of the circle represents the back of the head. When the sign on the upper left is a caution sign, as shown in FIG. 5 (a), the contour of the α wave intensity gradually increases from the front to the back of the head, and is slightly to the left of the center of the back of the head. It represents a steep mountain with the top at the top. When the sign on the upper right side is a caution sign, as shown in FIG. 5 (b), the contour line of the α wave intensity gradually increases from the front to the back and slightly to the right of the center of the back. This represents a steep mountain with the peak at the top. Further, when the sign on the lower left side is used as a caution sign, as shown in FIG. 5 (c), the contour line of the α wave intensity gradually increases from the front to the back of the head and to the left from the center of the back of the head. Represents a gentle mountain with a certain peak at the top. When the sign on the lower right side is a caution sign, as shown in FIG. 5 (d), the contour of the α wave intensity gradually increases from the front to the back and slightly to the right of the center of the back. Represents a gentle mountain with the part at the top.

  In addition, it has been confirmed that the α wave intensity is reduced in the visual cortex region where attention is paid in the study by the inventors of the present invention by measuring the magnetic field of the brain (see Cognitive Brain Research in press).

  Next, the first measurement unit 11 measures the position of the caution sign as a caution area in the field of view when the α-wave intensity distribution serving as a reference is measured for each caution sign (FIG. 2 / S014).

  Then, the first function definition unit 114 defines a first function representing the relationship between the reference α-wave intensity distribution and the attention area in the field of view (FIG. 2 / S016). The defined first function is stored in a storage unit such as a memory.

  Further, the sub-measurement unit 122 measures a reference α-wave intensity change pattern (FIG. 2 / S022). The change pattern of the reference α wave intensity is based on the appearance or disappearance time point of the caution sign in the same situation (see FIG. 4) in which the reference α wave intensity distribution is measured by the second sub-measurement unit 112. As measured. For example, the change pattern of the α wave intensity over a predetermined time from the time when the color of a caution sign is differentiated from the color of other signs when all the signs have the same color, the α wave that is the reference for this caution sign It is measured as an intensity change pattern. Note that the change pattern measurement start time is shifted in consideration of the time delay until the change in the user's attention is reflected in the α wave intensity as described above.

  The change pattern of α wave intensity includes a change pattern of α wave intensity at one location on the head, a change pattern of an average value of α wave intensity at a plurality of locations on the head, and contour lines representing predetermined values (FIGS. 5A to 5 ( d) Measured as a change pattern of the area surrounded by (2), a change pattern of the length of contour lines representing a predetermined value, and the like.

  Further, the second measuring unit 12 measures the degree of attention to the attention sign when the change pattern of the reference α wave intensity is measured (FIG. 2 / S024). Specifically, the reaction time from when the color of the caution sign is differentiated from the color of other signs until the user notices the appearance of the caution sign and presses a button (not shown) is measured. The shorter it is, the higher the degree of attention to the attention sign (attention area in the field of view) is measured (evaluation).

  Then, the second function definition unit 124 represents a second function for representing the relationship between the reference α-wave intensity change pattern and the attention level when the reference α-wave intensity change pattern is measured. Each label is defined (FIG. 2 / S026). The defined second function is stored in a storage unit such as a memory.

  A specific definition example of the first and second functions will be described. When the number of measurement sites is small, the simplest case is, for example, when electrodes are placed at the O1 (left) and O2 (right) positions of the International 10-20 method, the signals obtained from the two channels are compared, and α A threshold function that assumes attention toward the lower band power is the “first function”. The threshold is determined by the reference α wave intensity distribution. The non-linear function that associates the power difference or ratio between channels with the attention level to the attention area corresponds to the “second function”.

Next, a case where an artificial neural network is used will be described. Prepare neural networks (for example, three-layer neural network with input terminals for the number of electrodes) as many as the number of electrodes arranged in each area of the electroencephalogram measurement area (back of the head), and repeatedly pay attention to each area in the field of view. The brain wave intensity distribution is input to each neural network corresponding to each region and learned. Each learned neural network corresponds to a “first function”. Suppose that the brain wave intensity distribution at the time of measurement was input to each learned neural network, and attention was directed to the area corresponding to the neural network that showed high output.
Similarly, a learned neural network having the change rate or change pattern of the electroencephalogram intensity when the attention level is measured as an input and the attention level as an output corresponds to the “second function”.

  Further, as shown in the inventors' research (NeuroImage 23 (2004) 806-826), a method using a hierarchical Bayesian method is also conceivable. Experiments to measure attention to each region in the field of view are performed in advance by fMRI, and the attention place / attention level and the fMRI data of the visual cortex activation state are matched to obtain the hierarchical prior distribution P0 of the hierarchical Bayesian model. A prior distribution P indicating the correspondence between the distribution of current sources of brain activity and the distribution state of the electroencephalograms observed in a plurality of channels in that case is prepared in advance by means such as computer simulation. With respect to the measured electroencephalogram data, the current source and intensity are obtained using the distribution P with P0 as the constraint condition. The part for obtaining the current source corresponds to the “first function”, and the part for obtaining the intensity corresponds to the “second function”.

  As described above, after the first and second functions are defined, the operation of the electric wheelchair 20 is controlled according to the procedure described below.

  That is, in an arbitrary situation, the first sub-measurement unit 111 measures the user's line of sight (FIG. 3 / S111). As a method for measuring the user's line of sight, a method disclosed in Japanese Patent No. 319076, Japanese Patent Laid-Open No. 7-289519, or the like can be employed. The second sub-measurement unit 112 measures the α wave intensity distribution in the user's head (FIG. 3 / S112). Furthermore, the first measurement unit 11 uses the first function definition unit 114 based on the user's line of sight measured by the first sub measurement unit 111 and the user's α wave intensity distribution measured by the second sub measurement unit 112. The attention area in the visual field of the user is measured according to the first function defined in advance (FIG. 3 / S114).

  The sub-measurement unit 122 measures the change pattern of the α wave intensity of the user (S122 in FIG. 3). Further, the second measurement unit 12 uses the second function definition unit 124 in advance based on the α wave intensity change pattern measured by the sub-measurement unit 122 and the attention area in the field of view measured by the first measurement unit 11. The degree of attention to the attention area in the user's visual field is measured according to the defined second function (FIG. 3 / S124).

  Then, the control unit 13 controls the operation of the electric wheelchair 20 based on the user's line of sight measured by the first measurement unit 11, the attention area in the visual field, and the attention level measured by the second measurement unit 12 ( FIG. 3 / S130). For example, the moving direction of the electric wheelchair 20 is controlled so that the electric wheelchair 20 moves toward the attention area within the visual field. Further, the moving speed of the electric wheelchair 20 is controlled according to the level of attention.

  According to the activity assistance system 10 of the present invention, the attention area in the world coordinates or body coordinates of the user is measured based on the distribution of the α wave intensity (brain activity fluctuation amount) and the line of sight in the user's head (FIG. 3). / S114). Since the user's attention area is closely related not only to the line of sight but also to the α wave intensity that reflects the visual perception, the user's attention area can be accurately measured. Further, the attention level of the user is measured based on the change pattern of the α wave intensity (S124 in FIG. 3).

  Therefore, the movement direction and speed of the electric wheelchair (auxiliary device) 20 are controlled in accordance with the measured attention area and degree of attention, so that the user's movement is appropriately assisted in view of the user's true intention and attention. Can be done. For example, when a plurality of works of art are displayed in a direction in which a user on the electric wheelchair 20 faces an eye in a museum and the user's attention is directed to the work in the field of view, the electric wheelchair 20 Move towards the work. Moreover, the higher the user's attention level, the higher the speed at which the electric wheelchair 20 approaches the work. Thereby, the user can approach the work which became interested in the some work which jumped into the visual field, and can enjoy the appreciation according to the preference.

  In the above embodiment, the α wave intensity is measured as the brain activity fluctuation amount. However, as another embodiment, the intensity of the brain wave having a frequency band different from the α wave, such as a β wave and the θ wave, is measured as the brain activity fluctuation amount. May be. Further, the oxygen saturation and relaxation time in the blood of the brain may be measured as the amount of brain activity fluctuation by the BOLD (Blood Oxygen Level Dependent) method using fMRI (Functional Magnetic Resonance Imaging). Further, even if NIRS (Near-InfraRed Spectroscopy) is used to measure the oxyhemoglobin concentration, deoxyhemoglobin concentration in brain blood, and the total hemoglobin concentration that is the sum of these, as the brain activity fluctuation amount Good.

  In the above-described embodiment, the user's movement is assisted by the electric wheelchair 20 according to the user's attention area and degree of attention. However, as another embodiment, the user's movement may be assisted by another moving body such as an automobile.

Further, for example, JP-by walking assist device as described in 2004-3295 2 0 JP, by the action of the body the force of the user in response to the attention area and attention of the user, the walking motion assist May be.

  Furthermore, information may be provided to the user through an image display device, an audio device, or the like according to the user's attention area and attention level. As a result, for example, in a museum, the guide information of works that the user's attention is directed to among the works of art included in the user's field of view is displayed. Therefore, mental activities such as understanding and thinking about the user's art appreciation Is assisted.

  In addition, the control unit 13 determines whether the amount of change in the attention area in the user's visual field measured by the first measurement unit 11 is less than a predetermined value over a predetermined time, depending on whether or not the electric wheelchair (auxiliary device) 20. These operations may be controlled separately. Thereby, for example, the operation of the electric wheelchair 20 is controlled by distinguishing between a situation where the attention area in the user's visual field frequently fluctuates and a situation where the attention area does not. Thereby, the movement direction and speed of the electric wheelchair 20 are appropriately controlled in view of the difference in the situation regarding the user's intention and attention (mental activity).

  Further, the control unit 13 controls the operation of the electric wheelchair (auxiliary device) 20 depending on whether or not the user's attention level measured by the second measurement unit 12 is less than a predetermined value over a predetermined time. May be. Thereby, for example, the moving direction and speed of the electric wheelchair 20 are controlled separately in a situation in which the user is paying attention to a specific target and a situation in which the user is not. Thereby, the operation of the electric wheelchair 20 is appropriately controlled in view of the difference in the situation regarding the user's attention (mental activity).

Configuration example diagram of activity support system of the present invention Functional illustration of the activity assistance system of the present invention Functional illustration of the activity assistance system of the present invention Functional illustration of the activity assistance system of the present invention Functional illustration of the activity assistance system of the present invention

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Activity assistance system, 11 ... 1st measurement part, 111 ... 1st sub measurement part, 112 ... 2nd sub measurement part, 114 ... 1st function definition part, 12 ... 2nd measurement part, 122 ... Sub measurement part, 124 ... 2nd function definition part, 13 ... Control part, 20 ... Electric wheelchair (auxiliary device)

Claims (8)

A system for assisting a user's activity with an auxiliary device,
The distribution of the brain activity fluctuation amount, which is a fluctuation amount of the physical quantity induced by the brain activity according to the user's visual perception, is measured in the head, and based on the distribution of the brain activity fluctuation amount, A first measuring unit for measuring the attention area of
Time change pattern of the spatial distribution of brain activity variation of the user is measured, and, based on the temporal change pattern of the spatial distribution of the brain activity change amount, a second measuring unit for measuring the attention degree of the user,
Relationship between the distribution of the brain activity fluctuation amount of the user measured by the first measurement unit in a state where the user's attention is directed to each of the plurality of regions in the visual field, that is, the retinal coordinates, and the attention region in the visual field Measure the attention area in the visual field of the user according to the first function defined above based on the distribution of the fluctuation amount of the brain activity of the user newly measured. And
The second measurement unit includes a temporal change pattern of the spatial distribution of the brain activity fluctuation amount of the user measured in a situation where the attention area of the user is changed, and the attention degree of the user according to the change situation. A second function representing a relationship is defined, and based on the temporal change pattern of the spatial distribution of the brain activity fluctuation amount of the user newly measured on the second function, the user's attention is determined according to the second function defined above. Measure the degree,
A control unit that controls the operation of the auxiliary device according to the degree of attention measured by the second measurement unit in addition to the attention region in the visual field of the user measured by the first measurement unit. An activity assistance system characterized by The activity assistance system according to claim 1,
The first measurement unit further measures the user's line of sight, and measures the attention area in the user's world coordinates or body coordinates based on the measured distribution and line of sight of the brain activity variation of the user. Feature activity support system. The activity assistance system according to claim 1,
Activities auxiliary systems, characterized in that the first measurement unit measures the strength of the brain waves, the concentration or characteristic values of the specific Ingredients of a specific component in the brain blood as the brain activity variation. The activity assistance system according to claim 1,
The controller controls the operation of the auxiliary device according to whether or not the amount of change in the attention area in the user's visual field measured by the first measurement unit is less than a predetermined value over a predetermined time. Activity assistance system characterized by being controlled separately. The activity assistance system according to claim 1 ,
The activity assisting system, wherein the second measuring unit measures a response speed of the user with respect to a change in an attention area in the visual field of the user, and evaluates the attention level of the user based on the response speed. The activity assistance system according to claim 1,
Activity assist the second measuring unit, and measuring the attention degree of the user on the basis of the intensity of brain waves of the user, to change the pattern of concentrations or the specific Ingredient characteristic values of the specific component in the brain blood system. The activity assistance system according to claim 1,
The control unit distinguishes and controls the operation of the auxiliary device according to whether or not the user's attention level measured by the second measurement unit is less than a predetermined value over a predetermined time. Activity assistance system to do. The activity assistance system according to claim 1,
The control unit is a device that assists the movement of the user by boarding and moving the user, a device that assists the user's movement by applying a force to the body of the user, or information to the user. An activity assisting system characterized by controlling the operation of a device that assists the user's physical or mental activity by providing.

Images