JP2009020656A - State detection system, interface system, health appliance, game system, state detection method and state detection program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To estimate the state of a body of a person seated on a device based on a plurality of sensor values. <P>SOLUTION: On the surface of a balance ball 100, the balance ball 100 is detachably installed with: acceleration sensors 200 each detecting an inclination θ of the balance ball 100 changing according to the movement of the person seated on the balance ball 100; and pressure sensors 210 and strain sensors 220 detecting positions of the gravity centers Ga, Gb applied to the balance ball 100 changing according to the movement of the person seated on the balance ball 100. A behavior and a posture of the body of the person seated on the balance ball 100 are estimated based on inclinations θ<SB>x</SB>, θ<SB>y</SB>of two axes obtained from sensor values of the acceleration sensors 200, the gravity center Ga obtained from sensor values of the pressure sensors 210, and the gravity center Gb and additional information Gc obtained from sensor values of the strain sensors 220. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a state detection system, an interface system, a health device, a game system, a state detection method, and a state detection program for estimating the state of a person or a device using a sensor attached to a predetermined device.

  Conventionally, techniques for estimating the state of a person or a body of a device using a sensor attached to a predetermined device have been proposed (see, for example, Non-Patent Documents 1 and 4 and Patent Documents 2 and 3). In Non-Patent Document 1, there is a motion capture type system that calculates the movement of a person's whole body from a change in pressure distribution on the sole of a foot that occurs when a person moves on a sheet on which a number of pressure sensors are arranged, and uses it for animation. Proposed. Patent Document 2 discloses a technique for performing a game by attaching an acceleration sensor to an exercise device or a wearing tool on a human body, detecting an acceleration component, and transmitting a control signal to a game machine according to the detected value. . Patent Document 3 proposes a technique for sensing a movement of an arm or a wrist using a game controller that is operated by grasping with a hand and performing a game in conjunction with the movement. Non-Patent Document 4 proposes a collaborative game in which each person sits on two balance balls, maintains a sitting posture and cooperates with each other to enjoy a pacman game.

“FootSee: an Interactive Animation System”, KangKang Yin and Dinesh K. et al. Pai, ACM SIGGRAPH / Eurographics Symposium on Computer Animation July 26-27, 2003 San Diego, California JP 2006-34436 A JP 2007-83024 A “Collabola” [Search June 4, 2007], Internet <URL: http: // www. collabolla. com />

  However, Non-Patent Document 1 is an interactive animation system in which a large number of pressure sensors are spread on a sensor pad, a pressure distribution on the sole of a human foot located on the pad is detected, and a person's movement is estimated based on the result. is there. Therefore, postures that can be estimated by this system are limited to standing postures, and it is very difficult to appropriately estimate the state of the body of a person sitting on an arbitrary device. Also, with this technology, the position and number of sensors cannot be changed.

  Further, in Patent Document 2, it is almost impossible to appropriately estimate the state of the upper body and lower body of a person sitting on an arbitrary device only by the acceleration and speed detected by the acceleration sensor according to the operation of the exercise device. Is possible.

  Further, in Patent Document 3, the game progresses in conjunction with the movement of the arm or wrist, and therefore the state of the upper body and lower body of a person sitting on an arbitrary device is appropriately estimated from input information regarding the movement of the arm and wrist. It is not possible.

  Further, in Non-Patent Document 4, a sensor is arranged on the floor surface instead of the surface of the balance ball, and the arranged sensor functions only as a switch for confirming the vertical and horizontal directions. Therefore, the state of the upper body and lower body of a person sitting on an arbitrary device cannot be estimated appropriately from these input information.

  In order to solve the above-described problem, according to an aspect of the present invention, a first sensor that is detachably attached to a device and detects rotation of the device that changes with movement of a person sitting on the device; A second sensor that is detachably attached to the device and detects a force acting on the device that changes with the movement of a person sitting on the device; and a rotation of the device detected by the first sensor And a processing device that estimates at least one of the state of the person sitting on the device and the state of the device based on the force acting on the device detected by the second sensor, and a state detection system comprising: Provided.

  According to this, by attaching the first sensor and the second sensor to the device in a detachable manner, the state of the body of the person sitting on the device and the state of the device based on the sensor value detected using them can be determined. Can be estimated.

  As a specific estimation method, a plurality of acceleration sensors (as an example of a first sensor) are detachably provided in a plane horizontal to the ground plane on which the device is placed, and the processing device includes X, Y, The state of the lower body of a person sitting on the device may be estimated based on the rotation of two or three axes of the device detected by the plurality of acceleration sensors in the Z axis.

  At this time, the processing apparatus has an initial state in which the position of the waist of the person sitting on the device is located almost directly above the device, and the initial position of the waist of the person sitting on the device. The state of the lower body of the person may be estimated from the deviation by specifying the deviation based on the detected 2-axis or 3-axis rotation of the device.

  As another specific estimation method, a plurality of pressure sensors (as an example of a second sensor) are detachably provided on the seating surface or bottom surface of the device, and the processing device is detected by the plurality of pressure sensors. The center of gravity acting on the device may be obtained from the pressure distribution applied to the device, and the upper body state of the person may be estimated based on the obtained center of gravity.

  Alternatively, a plurality of strain sensors (as an example of a second sensor) are detachably provided on the seating surface of the device, and the processing apparatus acts on the device from the strain state of the device detected by the plurality of strain sensors. The center of gravity of the person may be obtained, and the state of the upper body of the person may be estimated based on the obtained center of gravity.

  According to this, rotation of a device having two or more axes is obtained from the acceleration sensor, and thereby the state of the lower body of a person is mainly estimated. Further, the center of gravity acting on the device is obtained from the pressure distribution and the degree of distortion of the device, whereby the state of the upper body of the person is mainly estimated. In this way, by using at least two or more types of sensors in combination, it is possible to accurately estimate the posture of the whole person sitting on the device by simultaneously detecting the movement of the center of gravity and the change in the rotation of the device. be able to.

  Various sensors are detachably attached to the device. Therefore, if it is a device that can maintain a posture in which a person is sitting to some extent, it is possible to estimate the state of a person's body using the device only by attaching these sensors.

  For example, using an interface system with such a configuration as an interface, it is mounted on any device such as a balance ball, chair or cradle that allows a person to maintain a sitting posture, and is performed in conjunction with the estimated human body condition. A game system for performing a game while sitting on the device and moving the body can be provided.

  The processing apparatus may determine that the device has moved in a state of the device when the rotation of the device obtained using the acceleration sensor is smaller than a given first threshold value.

  The processing apparatus is configured such that the rotation of the device obtained using the acceleration sensor is larger than a given first threshold value, and the center of gravity of the device obtained using the pressure sensor is given a given second threshold value. If smaller, the state of the device may be determined that an impact has been applied to the device.

  The processor has a rotation of the device determined using the acceleration sensor greater than a given first threshold and a center of gravity of the device determined using the second pressure sensor is a given second If it is equal to or greater than the threshold value, it may be determined that the device has bounced the state of the device.

  According to this, by combining the pressure sensor and the acceleration sensor, it is possible to detect specific operations such as device movement, device bounce, and device impact.

  The state detection system can be used as an interface system that uses a signal indicating the state of the person sitting on the device or the device detected by the system as an input signal when executing an arbitrary application. For example, when creating a game program using the state detection system as an interface, not only information on the posture of the whole body of a person sitting on the device, but also specific operation information of the device such as movement of the device, device bounce, and impact on the device Can also be used to create game programs. As a result, it is possible to create an application that operates in conjunction with both the movement of the device and its own movement by sitting on the device and moving the body.

  The processing apparatus is configured such that the human body state estimated based on the determined rotation of the device and the center of gravity of the determined device is one or more predetermined human body states. It may be determined which of the above applies.

  According to this, not only can the state of the whole body of the person sitting on the device be estimated to a certain degree, but the estimated state of the person's body is compared with the state of the reference body 1 or 2, and the body A rough movement of the user can be recognized by determining which of the representative examples of the posture is close. For example, instead of acquiring the rotation of the upper body in increments of 1 degree, it is possible to estimate a rough posture of the user who only acquires whether it is leaning forward or leaning backward. As a result, noise is reduced and estimation that is more appropriate for the user's intuition is possible. As a result, this system can be used not only as an input means for an application that requires fine movement, such as a game, but also as an input means for an application that requires rough movement.

  For example, by attaching an interface system having such a configuration to an arbitrary device as an interface, it is possible to develop a health device that sits on the device and trains its body. In this health appliance, it is preferable to determine whether the motion and posture are appropriate for a general movement of a person. Thus, if the posture is appropriate, it is possible to progress to the next step, obtain a score, or to show a privilege that exists for the user, train the body with a sense of play, and solve the lack of exercise.

  The processing device obtains a deviation of a human figure sitting on the device from a predetermined standard figure, and takes into account the individual figure of the person sitting on the device based on the deviation from the standard figure. Then, the state of the person's body may be estimated.

  In order to use the system comfortably, the system is adjusted to a state corresponding to the user's physique by inputting and measuring the user's physique and the movement range of the device before use (calibration). For example, when using it for diet or exercise, this system is adjusted so that it doesn't react unless it moves greatly. Thereby, a big physical exercise | movement can be promoted and the higher diet effect can be acquired. On the other hand, for example, when the user wants to use the system for a long time, the system may be adjusted so as to respond with a small operation in order to reduce the load on the user. By such adjustment, it is possible to provide an interface that prompts the user to exercise the whole body on a daily basis.

  In order to solve the above problems, according to another aspect of the present invention, the rotation of the device changes with the movement of a person sitting on the device using a first sensor detachably attached to the device. And detecting a force acting on the device that changes with the movement of a person sitting on the device using a second sensor detachably attached to the device, and detecting the force by the first sensor. A method for estimating the state of a person sitting on the device based on the rotation of the device and the force acting on the device detected by the second sensor is provided.

  Moreover, in order to solve the said subject, according to the other aspect of this invention, the said device which changes with the motion of the person sitting in the said device using the 1st sensor detachably attached to the device A process of detecting the rotation of the device, a process of detecting a force acting on the device that changes with the movement of a person sitting on the device using a second sensor detachably attached to the device, Causing the computer to execute a process of estimating a state of a person sitting on the device based on the rotation of the device detected by the first sensor and the force acting on the device detected by the second sensor. A state detection program is provided.

  According to the present invention, the state of the body of a person sitting on a device can be estimated using a plurality of sensors.

  An interface system according to an embodiment of the present invention will be described below with reference to the accompanying drawings. Note that, in the following description and the accompanying drawings, the same reference numerals are given to components having the same configuration and function, and redundant description is omitted.

(Whole interface system configuration)
First, the overall configuration of the interface system according to the present embodiment will be described with reference to FIG. The interface system 10 includes a balance ball 100, various sensors (acceleration sensor 200, pressure sensor 210, and strain sensor 220) attached to the balance ball 100, a microcomputer 300 attached to the balance ball 100, and a receiver 400. And a personal computer 500. The interface system 10 may include a display (monitor) 550.

  The balance ball 100 is an example of a device in which a plurality of sensors and a board on which a microcomputer 300 for control is mounted are mounted on an object (ball) that can change the posture while the user U maintains a sitting posture. is there. The receiver 400 receives a radio signal emitted from the microcomputer 300 on the device side and transmits the signal to the computer 500. A computer 500 (corresponding to a processing apparatus) processes a signal sent from a device via the receiver 400. The personal computer 500 is an example of a state detection device (processing device). However, the state detection device is not limited to the personal computer 500 and may be any electronic device as long as signal processing of data described later can be performed.

  A plurality of acceleration sensors 200 are detachably mounted on the surface of the balance ball 100 and in a plane horizontal to the ground plane on which the balance ball 100 is placed. As the acceleration sensor 200, one that can detect rotation of two or three axes among the X, Y, and Z axes is used. In the present embodiment, the rotation of the two axes (X axis and Y axis) of the balance ball 100 is detected. The biaxial rotation of the balance ball 100 is mainly used to estimate the state of the lower body of the user U and to specify the movement and impact of the balance ball 100. The acceleration sensor 200 is detachably attached to the device, and is an example of a first sensor that detects rotation of the device that changes with the movement of a person sitting on the device. Another example of the first sensor is a gyro sensor.

  A plurality of pressure sensors 210 are detachably attached to the seating surface of the balance ball 100. The pressure sensor 210 detects a plurality of pressures (force and pressure distribution acting on the balance ball 100) applied to the seat surface, and the center of gravity acting on the balance ball 100 is calculated from the detected pressure distribution. The center of gravity acting on the balance ball 100 is mainly used for estimating the state of the upper body of the user U and distinguishing the impact and bounce of the balance ball 100. The pressure sensor 210 is detachably attached to the device, and is an example of a second sensor that detects a force acting on the device that changes as a person sitting on the device moves.

  A plurality of strain sensors (bending sensors) 220 are detachably provided on the bearing surface of the balance ball 100. The strain sensor 220 is rod-shaped and detects its deformation (bending), and the center of gravity acting on the balance ball 100 is calculated from the detected strain state of the balance ball 100. The center of gravity acting on the balance ball 100 is mainly used for estimating the state of the upper body of the user U. The strain sensor 220 is another example of the second sensor.

  When obtaining the center of gravity acting on the device, the strain sensor 220 is effective when the device is easily deformed, and the pressure sensor 210 is effective when the device is difficult to deform. In the case of the balance ball 100, either the pressure sensor 210 or the strain sensor 220 can be used. However, since there are various types of balance balls 100 from hard to soft, in this embodiment, both sensors are used to act on the device in order to prevent deterioration of sensing accuracy due to the hardness of the balance ball 100. Find the center of gravity.

(Sensor arrangement)
2A to 2C show an example of the arrangement of various sensors from three angles. 2A is a view of the balance ball 100 viewed from the Z direction (directly above), FIG. 2B is a view of the balance ball 100 viewed from the Y direction (side surface), and FIG. It is the figure which looked at the balance ball 100 from the front.

  A total of three acceleration sensors 200 are mounted at approximately the center of the front surface and positions shifted from the front surface by 60 degrees in the horizontal direction. The acceleration sensor 200 is disposed at a position where biaxial or triaxial acceleration can be measured.

  The pressure sensor 210 and the strain sensor 220 are provided centering on the position of the user's buttocks, and are also provided in part where the foot is in contact. The pressure sensor 210 may be mounted on the floor surface, but in order to increase accuracy, it is better to be mounted on the seat surface. The strain sensor 220 is mounted around a position where the balance ball 100 is most easily deformed by weight shift.

  Various sensors are provided symmetrically with respect to the XZ plane that passes through the center of the balance ball 100 and cuts the balance ball 100 in the vertical direction. By combining a plurality of sensors in this way, different types of information can be obtained, and rough movements of the body can be acquired even with a small number of sensors by combining the obtained information. In addition, since all the various sensors can be attached and detached, the number of sensors, the arrangement position of the sensors, and the type of the sensors can be flexibly changed.

  3, the computer 500 includes a ROM 505, an HDD 510, a RAM 515, a CPU 520, a bus 525, an internal interface (internal I / F) 530, and an external interface (external I / F) 535. have. For example, the ROM 505 records basic programs, programs that are activated in the event of an abnormality, and the like, and the RAM 515 stores various programs and data (for example, a user model table described later) necessary for estimating the posture and behavior of the user from the sensor values. Is accumulated. The bus 525 is a path for exchanging information between the devices of the ROM 505, HDD 510, RAM 515, CPU 520, internal interface 530, and external interface 535. Various sensor values received by the receiver 400 are transmitted via the bus 525. Stored in the RAM 515. The internal interface 530 inputs data from the keyboard 540 and the mouse 545 and displays necessary information on a monitor (display) 550.

  The CPU 520 estimates the user's posture and behavior by executing a program using the data. Since the internal configuration of the microcomputer 300 is basically the same as that of the computer 500, description thereof is omitted.

  Next, the operation of the interface system 10 will be described in the order of processing of the microcomputer 300 (signal processing in the device) and processing of the computer 500 (posture estimation processing).

(Signal processing in the device)
The signal processing in the balance ball 100 will be described with reference to FIG. When the process is started (S400) and the user U changes his / her posture while sitting on the balance ball 100 and operates the balance ball 100 (S405), the movement of the balance ball 100 is sensed by various sensors. That is, the plurality of strain sensors 220 measure the deformation of the balance ball 100 (S410), the plurality of pressure sensors 210 measure the pressure at a plurality of points provided with the pressure sensor 210, and the plurality of acceleration sensors 200 The acceleration of the balance ball 100 applied to each point where the acceleration sensor 200 is provided is measured (S420).

  The microcomputer 300 aggregates the various sensor values measured in S410 to S420 for each sensor, converts them into a transmission format to the computer 500 (S425), transmits the signal (S430), and then ends this process. The process ends (S435). This process is repeatedly executed every predetermined time (for example, several seconds) while the user U is keeping the sitting posture with the balance ball 100.

(Attitude estimation principle)
Before explaining the processing of the computer 500 (posture estimation processing), the human posture estimation principle will be described with reference to FIG. 5 showing the relationship between the user posture, device tilt (rotation), and the center of gravity of the user U. To do. The balance ball 100 is based on the sitting posture shown in FIG. In the initial state shown in FIG. 5A, the position of the waist W is on an axis Zo passing through the center O of the balance ball 100. At this time, the balance ball 100 does not rotate in any of the X-axis, Y-axis, and Z-axis directions, and the tilt (rotation) R of the device is zero. In this state, the center of gravity G acting on the balance ball 100 is also on the axis Zo. In FIG. 5A, for the sake of convenience, the inclination R and the center of gravity G of the device are shifted from each other, but in reality, both are on the axis Zo and overlap in the initial state.

  In FIG. 5B, the upper body of the user U is tilted forward with respect to the state of FIG. 5A, and the lower body is not moving. This state is linked to the change in the inclination R and the center of gravity G of the device. That is, in the state of FIG. 5B, the position of the waist W is on an axis Zo in the Z direction passing through the center O of the balance ball 100. Therefore, as in the case of FIG. 5A, the balance ball 100 does not rotate at all from the initial state, and the inclination R of the device is zero. Thus, when the inclination R of the device is 0, it can be estimated that the lower body has not moved from the initial state. On the other hand, the center of gravity G in the case of FIG. 5B has moved largely forward with respect to the axis Zo. Thereby, it can be estimated that the upper body of the user U is inclined forward by a change amount (deviation) from the position of the center of gravity G with respect to the initial state.

  In FIG. 5C, the waist W of the user U moves backward with respect to the state of FIG. 5B, and the upper body is also warped backward. In this case, the position of the waist W is rotated by θ with respect to the axis Zo. Therefore, it can be estimated from the inclination R of the device that the lower half of the user U has shifted to the back of the balance ball 100. Further, the center of gravity G in the case of FIG. 5C is moved backward with respect to the axis Zo. Thereby, it can be estimated that the upper body of the user U is inclined backward by the amount of change in the position of the center of gravity G with respect to the initial state.

  Furthermore, when only the upper body is tilted forward from the posture of FIG. 5C as shown in FIG. 5D, the position of the waist W remains rotated by θ with respect to the axis Zo, and only the position of the center of gravity G is present. Move forward with respect to axis Zo. Thereby, the lower body of the user U is the same as the state of FIG. 5C, and it is estimated that only the upper body of the user U has moved and tilted forward.

  The position of the waist W of the user U always exists in the zenith portion of the device in the initial state, and the user U tries to operate the device by moving the whole body while keeping this as much as possible. On the assumption of this state, the state of the lower body of the user U can be estimated by estimating the inclination of the device obtained from the acceleration sensor 200 as the movement of the position of the waist W. On the other hand, the state of the upper body of the user U can be estimated from the pressure distribution at a plurality of points obtained from the pressure sensor 210. The state of the whole body of the user U can be estimated by combining these.

(Calculation method of device tilt and center of gravity)
In the interface system 10 according to the present embodiment, the tilt (rotation θ) of the device is calculated as follows. 6A and 6B show an initial state of the balance ball 100. FIG. 6A is a view of the acceleration sensor 200 attached to the surface of the balance ball 100 as seen from the X-axis direction (front), and FIG. 6B is a view of the acceleration sensor 200 as seen from the Y-axis direction (side). It is a figure. The acceleration in the direction perpendicular to the contact surface between the balance ball 100 and the acceleration sensor 200 is a _v , and the acceleration in the horizontal direction is a _u . The gravitational acceleration g works in the vertical direction with respect to the ground plane.

The balance ball 100 from this state, is rotated by theta _x about the X-axis as shown in FIG. 6 (C), is rotated by theta _y about the Y-axis as shown in FIG. 6 (D). The relationship between the angles θ _x and θ _y and the accelerations a _u and a _v at this time is expressed by the following equations (1) and (2).
θ _x = α sin (a _u / g) (1)
_{θ y = αsin (a v /} g) ··· (2)
α is a predetermined eigenvalue.

ここで、ｒ_ｋはバランスボール１００の中心点Ｏからの距離、θ_ｋは中心点Ｏに対する角度、ｆ_ｋは力を示す。 Next, calculation of the center of gravity using the pressure sensor 210 will be described. The actual calculation of the center of gravity is sufficient if it uses a rough change in the center of gravity rather than the exact center of gravity of the entire body. The position (x, y) of the center of gravity G is calculated using the following equations (3) and (4) based on the pressure distribution detected by the pressure sensors 210 provided at n locations.

Here, r _k is a distance from the center point O of the balance ball 100, θ _k is an angle with respect to the center point O, and f _k is a force.

  When the balance ball 100 is difficult to deform, a method of calculating the position of the center of gravity G using the pressure sensor 210 is effective. However, when the balance ball 100 is easily deformed, a method using the strain sensor 220 is effective.

  As shown in FIG. 2A, the strain sensors 220 are arranged in a pair on the left and right and in a pair on the front and back, and the center of gravity on the left and right is determined from two values detected from the pair of strain sensors 220. The center of gravity before and after is obtained separately. The centroid resulting from the addition is mapped onto a two-dimensional plane. For example, as shown in FIG. 7A, when the user U puts weight on the left foot side, the strain (bending) sensor 200a located near the right leg of the user U and the position near the left leg of the user U are located. The strain (bending) sensor 200b to bend bends according to the strain of the balance ball 100 as shown in FIG.

Weights are set for each sensor pair in the X-axis direction (left and right) and Y-axis direction (front and rear), and the center of gravity is determined separately for X and Y. The weight is determined by the position of the sensor and the ease of reaction. N sensor pairs having weights in the X-axis direction, m sensor pairs having weights in the Y-axis direction, X value of a certain sensor pair as X _k , Y value as Y _k , and X-axis direction Assuming that the sensor weight is A _k and the sensor weight in the Y-axis direction is B _k , the center of gravity acting on the balance ball 100 uses equations (5) and (6) based on the strain (bending) of the plurality of strain sensors 200. Is required.

  In order to obtain the center of gravity more accurately, based on the direct force measurement at a plurality of points using the pressure sensor 210 and the indirect force measurement by deformation on a plurality of surfaces using the strain sensor 220, etc. Comprehensive calculations are made using a combination of information from multiple sensors depending on the type and material of the device used.

  Specifically, the center of gravity G uses the additional information Gc obtained from the force applied to a specific part in addition to the center of gravity Ga measured using the pressure sensor 210 and the center of gravity Gb measured using the strain sensor 220. Calculated. The center of gravity is determined mainly by the pressure distribution of the buttocks for both the pressure sensor 210 and the strain sensor 220. In addition, the weight supported by the foot, the weight supported by the balance ball 100, and the position of the center of gravity can be estimated from the ratio and total amount of pressure applied to the left and right thighs, and this additional information Gc is added to the calculation of the center of gravity as an auxiliary. To do.

Specifically, for example, the weight of the center of gravity Ga measured using the pressure sensor 210 is 1, the weight of the center of gravity Gb measured using the strain sensor 220 is 0.5, and the weight of the additional information is 0.1. In this case, the center of gravity G can be appropriately obtained from each sensor value using the following equation (7).
G = (1 × Ga + 0.5 × Gb + 0.1 × Gc) × 1 / 1.6 (7)
However, the above weighting is only an example.

(Attitude estimation processing)
Next, a user posture estimation process executed by the computer 500 (CPU 520) will be described with reference to the process flow of FIG.

  When the posture estimation process is started from step S800 of FIG. 8, the CPU 520 periodically receives various sensor values from the receiver 400, and the sensor values of the acceleration sensor 200, that is, a plurality of values measured by the balance ball 100 are measured. Based on the acceleration of the point, the inclination (biaxial rotation) of the balance ball 100 is calculated (S805). Next, the CPU 520 specifies the waist position of the user U from the calculated inclination of the balance ball 100 (S810), and mainly estimates the lower body state of the user U from the waist position of the user U.

  Next, the CPU 520 calculates the center of gravity Ga acting on the balance ball 100 based on the sensor value of the pressure sensor 210, that is, the measured force (pressure distribution) at a plurality of points (S815), and the sensor value of the strain sensor 220, That is, the center of gravity Gb is calculated based on the measured deformation of the balance ball 100 (S820), and additional information Gc related to the center of gravity is obtained (S825). By substituting the calculated centroids Ga and Gb and the additional information Gc into the equation (6), a more accurate centroid G is specified (S830). From the identified position of the user's waist and the position of the center of gravity, the model most closely matched among the models stored in advance in the user model table is estimated as the state of the body of the user U (S835), and this process is terminated (S840). ).

  According to this, the body is balanced so that the pelvis is brought in the vicinity of the zenith portion of the balance ball 100 without attaching sensors to the person or operating the controller by hand. The state of the user's body on the balance ball 100 can be estimated using two types of sensors.

(Move, bounce, shock treatment)
In the system 10, in addition to the estimation of the posture, the movement of the balance ball 100 on the two-dimensional plane, the impact on the balance ball 100, and the bounce can be specified. These processes will be described with reference to FIG. 9 showing a process flow for specifying the movement, bounce, and impact on the balance ball 100 of the balance ball 100.

When the movement, bounce, and impact identification processing is started from step S900 in FIG. 9, the CPU 520 first detects the balance ball 100 detected by the acceleration sensor 200 among the various sensor values periodically received from the receiver 400. Based on the acceleration at a plurality of points, it is determined whether or not a sudden acceleration change has occurred (S905). Specifically, it is determined whether or not there is a sudden change in acceleration depending on whether or not the total amount f of acceleration measured during a predetermined time t ₁ from the n acceleration sensors 200 on the balance ball 100 exceeds a threshold value F. The In the case of n> 1, the CPU 520 determines a position where there is a sudden change in acceleration from the difference between the total acceleration amounts f _k (1 ≦ k ≦ n).

  If it is determined in step S905 that there is no sudden change in acceleration, the CPU 520 determines that the balance ball 100 has moved, and executes a movement process (S910). The actual movement process depends on the application to which the system 10 is applied. For example, in the case of an adventure game in which the balance ball 100 is moved and the virtual space on the display is adventured, the movement distance of the balance ball 100 is calculated, and the virtual space on the display is adventured accordingly. May be.

When it is determined in step S905 that there has been a sudden change in acceleration, the CPU 520 determines whether there has been a sudden change in pressure (S915). Specifically, there was a sudden change in pressure depending on whether the total amount p of the measured pressure for a predetermined time t ₂ of m of the pressure sensor 210 on the balance ball 100 has exceeded the threshold value P is determined The m> For 1, CPU 520 determines the position where there is a rapid change in pressure from the difference between the total _p h of the respective pressure (1 ≦ h ≦ m).

  If it is determined in step S905 that there has been a sudden change in acceleration, and if it is determined in step S915 that there has been a sudden change in pressure, the CPU 520 determines that the balance ball 100 has bounced and executes a bounce process. (S920). The actual bouncing process depends on the application to which the system 10 is applied. For example, in the case of the adventure game, the bouncing state of the balance ball 100 may be processed so as to jump over the obstacle in the virtual space on the display. .

  If it is determined in step S905 that there has been a sudden change in acceleration, and if it is determined in step S915 that there has not been a sudden change in pressure, the CPU 520 applies an impact to the balance ball 100, such as by striking the balance ball 100. The impact process is executed (S925). For example, in the case of the adventure game, processing may be performed to warp to another space on the display according to the number of times of impact on the balance ball 100. The CPU 520 ends the process (S930) after completing any one of the movement process (S910), the bounce process (S920), and the impact process (S925). This process is executed every predetermined time.

  Both the thresholds F and P used in the determinations in steps S905 and S915 are determined by calibration described later, and the numerical values can be changed thereafter. Actually, it is determined that there is an abrupt change when both conditions of exceeding the absolute threshold value and how many times the absolute value has been increased compared to the previous value are cleared. However, since the absolute threshold value is a force (weight) applied to the device, it greatly varies depending on the weight of the user. As for the condition of how many times it has become, for example, an average value is calculated from the measurement records of the past 5 to 10 times (the number of times is arbitrary), and it is determined how many times the current measurement value has been increased. The standard is about twice the pressure and about 1.5 times the acceleration, but is not limited thereto.

(Modification)
The interface system 10 according to the embodiment described above can have the following various functions.

(Correction process)
For example, as shown in the processing flow of FIG. 10, after the above-described user posture estimation processing (FIG. 8) and movement, bounce, and impact processing (FIG. 9) are executed by the computer 500 of this system (S1005), The correction process may be performed (S1010), the corrected result may be output (S1015), and the process may be terminated (S1025).

  For example, in the correction process, the computer 500 applies the state of the human body estimated based on the values detected by the various sensors to any one of one or more predetermined reference body states. It may be determined.

  According to the present system 10, it is possible to estimate the state of the whole body of a person sitting on the balance ball 100 to some degree. On the other hand, the human body state estimated by executing the above correction process is compared with the body state serving as the reference 1 or 2, and it is determined which one of the representative examples of the body posture is closer. By doing so, for example, it is possible to recognize a rough movement of the user that does not acquire the inclination of the upper body in increments of 1 degree but only acquires whether it is inclined forward or inclined backward. As a result, noise is reduced and estimation that is more appropriate for the user's intuition is possible. As a result, the system 10 can be used not only for applications such as games that require fine movements but also for applications that require input information for rough movements.

(Command)
Further, for example, as shown in the processing flow of FIG. 10, after the above-described user posture estimation processing (FIG. 8) and movement, bounce, and impact processing (FIG. 9) are executed by the computer 500 of this system (S1005). ), Command processing is performed (S1020), the result is output (S1015), and this processing may be terminated (S1025).

  Commanding means, for example, moving the waist quickly to the left or right, moving in a circle, tilting the upper body after playing, double-clicking the mouse on the balance ball, Arbitrary assignment to input on the computer, such as starting or turning off the computer. As described above, the system 10 is used as a health promotion system for encouraging the user U to perform physical exercise by registering and using the estimated user posture, estimated device behavior, and a series of these operations as commands. Or the system 10 can be used as information input means.

(Calibration)
In order to use the system 10 comfortably, as shown in the processing flow of FIG. 11, the user U's physique and movement range are input and measured before the system 10 is used, You may adjust so that the system 10 may be utilized.

  Specifically, when there is a request to use the system 10, the process is started only once before the system is used (S1100), and the CPU 520 uses the balance ball 100 placed in the initial state. The body weight is calculated based on the sensor value from the pressure sensor 210 (S1105), the physique is calculated based on the sensor value from the strain sensor 220 using the balance ball 100 placed in the initial state (S1110), and the balance ball 100 is The range of motion of the balance ball 100 is calculated using the sensor value from the acceleration sensor 200 by moving back and forth, and left and right (S1115), and a table (user model table) suitable for the user is created from the model library (S1120). This process ends (S1125).

  For example, when the system 10 is used for the purpose of dieting or exercising, the system 10 is adjusted so that it does not react unless the user moves greatly. Thereby, a big exercise | movement can be encouraged. In addition, since the system 10 is expected to be used for a long time, when it is desired to reduce the load during use, the system 10 may be adjusted so that it reacts with a small operation.

(Model library)
Before using the system 10, the developer side actually uses the system 10 in advance using a computer according to the processing flow shown in FIG. 12 to determine the force distribution, device deformation, tilt, and impact. The posture and behavior of the user U are estimated from such values. (S1205). At the same time, the motion capture of the actual user posture and behavior is performed (S1210), and the comparison between the actually measured value (S1215) obtained thereby and the calculated value obtained in S1205 is repeated (S1220). The developer side inputs data representing the physique such as the height and weight of the user together with the data obtained by comparison (S1225), and stores the posture, behavior, user height, weight, sensor value, etc. of the user in the database. (S1230) and registered as a model library (S1235).

  As described above, the system 10 is a user interface system and requires real-time performance, and therefore, it is necessary to perform calculation at high speed while reducing the processing load. In order to satisfy this requirement, a model library is created in advance, and by using the data stored in the model library, a table for immediately identifying the posture and behavior of the user U from the sensor values according to the user's physique Is created during calibration. Then, by using this table (user model table) for estimating the posture of the user, the processing load is reduced and high-speed calculation is realized.

  As described above, the interface system 10 according to the present embodiment is a state in which the state of the person sitting on the balance ball and the state of the balance ball itself are detected from various sensors attached to the balance ball and detection signals from the sensors. Functions as a detection system. Furthermore, the interface system 10 according to the present embodiment can function as an interface for various applications by using a signal indicating the detected human state or device state as an input signal for various applications.

  For example, the present system can be used as a user interface of various software that models the posture and behavior of the user U using various sensors and estimates the posture and behavior of the whole body. Since the basic software is made into a library, a developer can freely create an application using part or all of the user's attitude and behavior by applying this system.

  For example, as shown in FIG. 13A, the user U balances while maintaining a sitting posture on the balance ball, or bounces on the balance ball 100 or moves his body greatly as shown in FIG. 13B. By doing this, it is possible to create an application in which the game progresses. This game may be executed in cooperation with another person in cooperation with another person.

  Further, for example, the user U bounces on the balance ball 100 as shown in FIG. 14 (A), rotates as shown in FIG. 14 (B), or variously as shown in FIG. 14 (C). By taking a simple pose and drawing a picture or character on the display, the user U can be promoted to exercise like a game.

  In addition, the present system 10 can be used for a computer pointing device, a game controller, a mouse, a joystick, and the like. For example, if you move the center of gravity backward on the balance ball 100, the cursor will move up, if you move the center of gravity forward, the cursor will move down, and if you strike the balance ball 100 twice, a link will be clicked In addition, the balance ball 100 can be used as one of input means to the computer.

  Furthermore, the system 10 can be applied to art works, musical instruments, motion capture, and the like. For example, if a posture on the balance ball 100 can be maintained, a sound linked to the posture may be generated. In this way, the user's movement and posture can be changed to sound or an image.

  In the above embodiment, the operations of the respective units are related to each other, and can be replaced as a series of operations in consideration of the relationship between each other. And by replacing in this way, the embodiment of the operation of the state detection system can be expressed as an embodiment of the state detection method. The operation of the state detection system can also be realized by using a computer to execute a state detection program described so that the computer can understand the functions of the state detection system.

  As mentioned above, although preferred embodiment of this invention was described referring an accompanying drawing, it cannot be overemphasized that this invention is not limited to the example which concerns. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Understood.

  For example, the state detection apparatus according to the present invention can be applied as an entertainment (game) or a health appliance by being mounted on various devices as a user interface. As a device, a chair or a cradle can be used in addition to a balance ball. When mounting on a device, the type of sensor, the number of sensors, and the position of the sensor can be arbitrarily changed, thereby improving processing accuracy and reducing costs.

1 is an overall configuration diagram of an interface system according to an embodiment of the present invention. It is the figure which showed the example of arrangement | positioning of a sensor from several angles. It is an internal block diagram of a computer. It is the figure which showed the signal processing flow on a device. It is a figure for demonstrating the relationship between a user's attitude | position, the inclination of a device, and a gravity center. It is a figure for demonstrating the relationship between the sensor value of an acceleration sensor, and the inclination of a device. It is a figure for demonstrating the relationship between a user's attitude | position and a deformation | transformation of a device. It is the figure which showed the processing flow which estimates a user's attitude | position from the sensor value of various sensors. It is the figure which showed the processing flow which specifies the movement of a device, a bounce, and an impact from the sensor value of various sensors. It is the figure which showed the processing flow at the time of adding correction and command after estimating a user's attitude | position and behavior. It is the figure which showed the processing flow of calibration. It is the figure which showed the model library creation flow. It is the figure which showed the example of application of the interface system concerning this embodiment. It is the figure which showed the other application example of the interface system concerning this embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Interface system 100 Balance ball 200 Acceleration sensor 210 Pressure sensor 220 Strain sensor 300 Microcomputer 400 Receiver 500 Computer 520 CPU

Claims (15)

A first sensor that is removably attached to the device and detects rotation of the device that varies with the movement of a person sitting on the device;
A second sensor that is removably attached to the device and detects a force acting on the device that changes with the movement of a person sitting on the device;
Estimating at least one of the state of the person sitting on the device and the state of the device based on the rotation of the device detected by the first sensor and the force acting on the device detected by the second sensor A state detecting system. The first sensor is an acceleration sensor that is detachably provided in a plane parallel to a ground plane on which the device is placed,
The processing apparatus estimates a state of a lower body of a person sitting on the device based on rotation of two or three axes of the device detected by the plurality of acceleration sensors among the X, Y, and Z axes. The state detection system described in.   The processing apparatus sets a deviation from a position of an initial state of a waist of a person sitting on the device, with an initial state where a waist position of the person sitting on the device is positioned almost directly above the device. The state detection system according to claim 2, wherein the state of the lower body of the person is estimated from the deviation by specifying the detected device based on two-axis or three-axis rotation. The second sensor is a plurality of pressure sensors detachably provided on a seating surface or a bottom surface of the device,
The said processing apparatus calculates | requires the gravity center which acts on the said device from the pressure distribution added to the said device detected by these pressure sensors, and estimates the state of the said upper body based on the calculated | required gravity center. The state detection system described in any one of. The second sensor is a strain sensor provided in a detachable manner on the seating surface of the device,
5. The processing apparatus according to claim 1, wherein the processing device obtains a center of gravity acting on the device from the strain states of the device detected by the plurality of strain sensors, and estimates the state of the upper body of the person based on the obtained center of gravity. The state detection system described in any one.   The state detection system according to claim 1, wherein the device is a balance ball or a chair that allows a person to maintain a sitting posture with the device.   The said processing apparatus determines with the said device having moved the state of the said device, when rotation of the device calculated | required using the said acceleration sensor is smaller than a predetermined 1st threshold value. The state detection system described in.   The processing apparatus is configured such that the rotation of the device obtained using the acceleration sensor is larger than a given first threshold value, and the center of gravity of the device obtained using the pressure sensor is given a given second threshold value. The state detection system according to claim 4, wherein if it is smaller, the state of the device is determined that an impact has been applied to the device.   The processor has a rotation of the device determined using the acceleration sensor greater than a given first threshold and a center of gravity of the device determined using the second pressure sensor is a given second The state detection system according to claim 4, wherein if the threshold value is equal to or greater than a threshold value, the device state is determined to have bounced. The processor is
The deviation of the figure of a person sitting on the device from a predetermined standard figure is obtained, and the figure of the individual person sitting on the device is added to the device based on the deviation from the standard figure. The state detection system according to claim 1, wherein at least one of a state of a sitting person and a state of the device is estimated.   A signal indicating the state of a person sitting on the device or the state of the device detected by the state detection system according to any one of claims 1 to 11 is used as an input signal when executing an arbitrary application. Interface system.   A health appliance for sitting on the device and training its body by attaching the interface system according to claim 11 to an arbitrary device.   A game program that is executed in conjunction with at least one of a human body state estimated by the processing apparatus and the device state by mounting the interface system according to claim 11 on an arbitrary device. A game system for performing a game while sitting on the device and moving its body. Detecting rotation of the device that varies with the movement of a person sitting on the device using a first sensor removably attached to the device;
Detecting a force acting on the device that changes with the movement of a person sitting on the device using a second sensor detachably attached to the device;
A state detection method for estimating a state of a person sitting on the device based on a rotation of the device detected by the first sensor and a force acting on the device detected by the second sensor. Detecting rotation of the device that changes with the movement of a person sitting on the device using a first sensor detachably attached to the device;
A process of detecting a force acting on the device that changes with the movement of a person sitting on the device using a second sensor detachably attached to the device;
A process for estimating the state of the body of a person sitting on the device based on the rotation of the device detected by the first sensor and the force acting on the device detected by the second sensor. Condition detection program

Images