JP2015204091A - vector data output system and video display system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reflect the actual operation of a human body on the control of a remote operation robot or the like or the operation of a character on a video in a computer game or the like and an effect on a video given by the operation by detecting the operation of the human body in accordance with a change in the size of a power accompanied by the operation, and converting it into vector data.SOLUTION: A vector data output system includes: a myoelectric signal detection device capable of measuring the surface myoelectric potential of one or more specific muscles of a human body, and outputting it as a myoelectric signal; a motion capture device capable of detecting the motion of the arbitrary specific part of the human body, and performing the projective transformation of the three-dimensional coordinates of each part of the human body into the coordinates of a projective coordinate system by perspective projection, and outputting a position change as coordinate displacement data; and a control device capable of inputting the myoelectric signal and the coordinate displacement data. The control device calculates and outputs vector data defining a vector volume with a change in the myoelectric signal of the specific muscle of the human body as a change in the size of a power and with the direction of the coordinate displacement of the specific part of the human body as a direction of the power in the projective coordinate system.

Description

  According to the present invention, the motion of the human body and the magnitude of the force are vectorized from the myoelectric signal obtained by measuring the surface myoelectric potential of the specific muscle of the human body and the coordinate displacement of the specific part of the human body detected by the motion capture device. The present invention relates to a vector data output system capable of calculating and outputting as data, and a video display system capable of outputting vector data from the vector data output system as an arbitrary figure or the like on a video of a computer game or the like.

  Motion capture technology is known as a technology for detecting an actual motion of a human body and reflecting this in the motion of a character such as a person on a video, and has been introduced to video production in movies and game software. By introducing motion capture, it is possible to reproduce more natural movements of people, etc., compared to video produced only by computer graphics, and it is also effective in improving the efficiency of the video production process.

  Motion capture technology includes optical, mechanical, magnetic, etc., but optical production is generally used for video production, and humans wearing special suits with markers perform, A general method is to optically detect the movement of the marker accompanying the movement of the human body with a tracker and record the displacement of the coordinates as digital data.

In recent years, by combining an RGB camera and a depth sensor as in the product described in Non-Patent Document 1 (URL http://www.xbox.com/ja-JP/kinect) A motion capture device has also been developed that detects the movement of the skeleton while measuring the distance to the main joints of the human body and reflects it in real time on the motion of the character on the video of the computer game. If such a device is used as a game input device, it is not necessary to wear the above-described suit with a marker, and a degree of freedom and a sense of reality that are higher than those of a conventional game controller in which a player operates buttons and sticks can be realized. For this reason, it is conceivable that computer games that use the motion capture device as a game controller will become popular in the future.
Microsoft's game device "Kinect" (registered trademark)

By the way, the motion capture device detects the coordinates of a specific part of the human body in the three-dimensional coordinate system, performs projective transformation on the coordinates of the projected coordinate system by perspective projection, and outputs the displacement as coordinate displacement data. is there. Therefore, the magnitude of the force accompanying the displacement, specifically, the muscle force used by the human body for the displacement cannot be directly detected. Therefore, the strength of striking power such as punching and kicking in so-called fighting games can only be expressed with the speed or acceleration of fist or leg displacement as a parameter, and the muscle strength actually applied by the player to the fist or leg is reflected in the character. It cannot be made, and there is a limit to the realism of expression.

  In many games, an effect is introduced in which a character emits directional energy (so-called “beam” or the like) toward a target as a “killer technique”. In this case, the “destructive power” may be set so that the character charges energy (so-called “reservoir”) by his / her own muscular load prior to launch, and varies according to the amount of energy that can be charged. However, since motion capture cannot detect the muscular strength load inside the human body, it is impossible to express such a “reservoir”, and there is a limit to the realism of the expression.

As a technique for detecting such a human body, in particular, the state of muscle load, a method of measuring the surface myoelectric potential generated by a muscle during voluntary movement through an electrode placed on the skin surface in the vicinity of the muscle to be detected is known. It is popular for medical and sports science purposes. In recent years, in the field of so-called wearable robots as walking support devices for elderly and handicapped persons, an electromyographic sensor detects the movement from the surface myoelectric potential of a predetermined muscle of the wearer and controls an actuator such as a motor. A method has been put into practical use, and as shown in Patent Document 1, a technique applied to the operation of a robot or a character on a game image has been proposed.
JP 2009-226127 A

  The present invention detects an actual movement of the human body together with a change in the magnitude of the force accompanying the movement and converts it into vector data that defines a vector quantity consisting of the direction, starting point, and magnitude of the force, It is an object of the present invention to make it possible to reflect the above-mentioned control, or the action of a character on an image such as a computer game and the effect on the image that the operation brings.

  In order to solve the above-described problem, the vector data output system according to claim 1 is a myoelectric signal detection device capable of measuring a surface myoelectric potential of one or a plurality of specific muscles of a human body and outputting it as a myoelectric signal. And a motion capture device capable of detecting the movement of any specific part of the human body and outputting the position change of the three-dimensional coordinates of each part of the human body as coordinate displacement data, and inputting the myoelectric signal and the coordinate displacement data The control device comprises a change in the magnitude of the force of the electromyogram signal of the specific muscle of the human body and the direction of the coordinate displacement of the specific part of the human body in the three-dimensional coordinate system. It is possible to calculate and output vector data that defines a vector quantity with the direction of force as.

  The motion of a specific part of the actual human body can be measured by the coordinate displacement of the specific part in the three-dimensional coordinate system by motion capture. On the other hand, by detecting a change in the voluntary muscle contraction strength of the muscle that operates the specific part, it is possible to measure a change in the muscle force load related to the operation. By combining these, the movement of a specific part of the human body is a dynamic change in which the voluntary muscle contraction strength changes, the force magnitude changes, the coordinate displacement start point becomes the force start point, and the coordinate displacement direction becomes the force direction. It can be expressed as a vector quantity.

  The vector data defining the dynamic vector quantity output by the vector data output system according to the present invention can be used as control data for a remote control robot, for example. Specifically, the specific part of the operator's body is associated with the specific part of the remote control robot, and vector data defining coordinate displacement in the former three-dimensional coordinate system is transmitted to the control device on the remote control robot side. Then, the control device converts it into coordinate displacement in a three-dimensional coordinate system in which the latter is present, calculates operation data such as an actuator necessary for displacement of a specific part of the remote control robot, and activates the actuator. At this time, by controlling the change in voluntary muscle contraction strength of the predetermined muscle of the operator in accordance with the change in the output of the actuator or the like (motor torque, etc.), the magnitude of the force that operates the specific part of the remote operation robot Can be reflected. As a result, the remote control robot, for example, not only operates the arm by reflecting the movement of the operator's arm, but also reflects the magnitude of the force that the operator puts in the arm muscles in the movement of the arm. Therefore, more delicate control of the remote control robot becomes possible.

  Next, a vector data output system according to a second aspect of the present invention includes a myoelectric signal detection device capable of measuring a surface myoelectric potential of one or a plurality of specific muscles of a human body and outputting it as a myoelectric signal, and an arbitrary human body A motion capture device capable of detecting a movement of a specific part of the human body and projecting the three-dimensional coordinates of each part of the human body to a coordinate in a projected coordinate system by perspective projection, and outputting the position change as coordinate displacement data; And a control device capable of inputting the coordinate displacement data, wherein the control device changes a myoelectric signal of a specific muscle of the human body in a change in magnitude of the force in the projective coordinate system, the human body The vector data defining the vector quantity having the direction of the coordinate displacement of the specific part as the force direction can be calculated and output.

  As described above, in order to reflect the motion of the human body detected by the motion capture device in the motion of a character or the like on a game image that is a two-dimensional coordinate system, the coordinates of a specific part of the human body in the three-dimensional coordinate system are detected, This is projectively transformed into coordinate displacement data in the projection coordinate system by perspective projection. As a vector data defining a vector quantity defining a change in the magnitude of the force as a change in the myoelectric signal of the specific muscle of the human body and a direction of the coordinate displacement of the specific part of the human body as a force direction, a game machine or the like Output to the control device.

  With such a configuration, for example, when a character makes a punch attack against a target (such as a battle character), the coordinate displacement of the player's fist is displayed as a motion of the character's fist, and the position where the fist hits the target is displayed. You can define a vector quantity where the coordinates are the starting point of the force, the direction of the coordinate displacement of the fist is the direction of the force, and the voluntary muscle contraction strength value of the arm muscle at the time of the hit is the magnitude of the force (ie punch power) . This improves the realism of fighting games by calculating the effectiveness of punching based on the hit position and punching direction, in other words, calculating the damage given to the target and displaying it as a video or numerical value on the game video. be able to. In addition, the muscle strength of the legs that shoot soccer, baseball bats, tennis rackets, and the strength of the arms holding the golf club can also be measured as voluntary muscle contraction strength values. It can also be applied to these ball games.

  Next, a vector data output system according to a third aspect of the present invention includes a myoelectric signal detection device capable of measuring a surface myoelectric potential of any one or a plurality of muscles of a human body and outputting voluntary muscle contraction strength as a myoelectric signal. Motion capture capable of detecting the movement of any specific part of the human body and the reference part and projecting and transforming the three-dimensional coordinates of the specific part into the coordinates of the projected coordinate system by perspective projection and outputting the position change as coordinate displacement data And a control device capable of inputting the myoelectric signal and the coordinate displacement data. The control device continues the state where the voluntary muscle contraction intensity value is equal to or greater than a predetermined first threshold for a predetermined time or longer. After that, when the value of the voluntary muscle contraction strength first falls below the first threshold, the coordinates of the specific part is the starting point, the direction from the coordinates of the reference part to the starting point is the direction of the force, Serial first threshold value and calculates the vector data defining a vector quantity and magnitude of the force, characterized in that a possible output.

  As described above, when applied to the launch of directional energy as a setting on the game, for example, the motion (gesture) of a specific body part of the player is detected as a coordinate displacement and displayed as a video, It is possible to detect a change in the value of voluntary muscle contraction strength of a predetermined muscle of the player and to display a moving image of the emission of directional energy on the game image using the change pattern as a trigger.

Specifically, the aim of directional energy emission is determined by the positional relationship of the coordinates of two specific body parts of the player (for example, two points of the elbow and wrist) at the end of the gesture, and the voluntary muscles of the arm muscles after the gesture starts A state in which the state in which the value of the contraction intensity exceeds a predetermined threshold continues for a predetermined time and then falls below the threshold, that is, the moment when the muscle is relaxed is set as a trigger for firing. With such a configuration, it is possible to display a moving image in which directional energy is emitted when the character in the game image points the wrist toward the target while applying force to the muscles of the arm and relaxes the force.

Next, a vector data output system according to claim 4 is a myoelectric signal detection device capable of measuring a surface myoelectric potential of any one or a plurality of muscles of a human body and outputting voluntary muscle contraction strength as a myoelectric signal. Motion capture capable of detecting the movement of any specific part of the human body and the reference part and projecting and transforming the three-dimensional coordinates of the specific part into the coordinates of the projected coordinate system by perspective projection and outputting the position change as coordinate displacement data And a control device capable of inputting the myoelectric signal and the coordinate displacement data. The control device continues the state where the voluntary muscle contraction intensity value is equal to or greater than a predetermined first threshold for a predetermined time or longer. After that, when the value of the voluntary muscle contraction strength first falls below the first threshold value and the coordinates of the specific part are displaced with respect to the coordinates of the reference part, the specification after the displacement is performed. It is possible to calculate and output vector data defining a vector quantity in which the coordinates of the position are the starting point, the direction from the coordinates of the reference portion toward the starting point is the direction of the force, and the first threshold is the magnitude of the force. And

When expressing the launch of directional energy as a game setting, a specific gesture of the character is often used as a trigger for the launch. In this configuration, when the state in which the muscle contraction strength reaches the predetermined first threshold at any time continues for a predetermined time or longer, preparation for directional energy emission is completed (that is, “energy filling state”), and then the character is specified. Directive energy is emitted at the moment when the gesture is performed. Such a gesture may be an action of literally pulling a handgun trigger that the index finger of one hand of the player moves to pull the trigger, or an action of swinging the two arms raised above the head toward the target. . The aiming direction at that time is determined by the direction of the extension line of the line connecting the elbow and the wrist, for example, with the player's elbow as the reference part and the wrist as the specific part, and the starting point of the vector amount is set near the wrist. For example, vector data can be obtained that can depict a moving image in which directional energy is emitted from the character's hand in the direction in which the character is pointing with the second arm.

Next, a vector data output system according to claim 5 is a myoelectric signal detecting device capable of measuring the surface myoelectric potential of any one or a plurality of muscles of a human body and outputting voluntary muscle contraction strength as a myoelectric signal. Motion that can detect the motion of any two specific parts of the human body and project the three-dimensional coordinates of the specific parts to the coordinates of the projected coordinate system by perspective projection, and then output the position change as coordinate displacement data A capture device; and a control device capable of inputting the myoelectric signal and the coordinate displacement data. The control device has a value of the voluntary muscle contraction strength equal to or greater than a predetermined first threshold value and the two locations. After the state where the value of the distance of the specific part is equal to or less than the predetermined second threshold continues for a predetermined time or longer, the value of the voluntary muscle contraction strength first decreases below the first threshold, and In the middle of a specific site Vector data defining a vector quantity having the coordinates of the intermediate point after the displacement as the starting point, the direction of the displacement as the direction of the force, and the first threshold as the magnitude of the force. It is possible to calculate and output.

When expressing the launch of directional energy as a game setting, a more complex gesture of the character may be used as a trigger for filling and launching energy. For example, when a character fills energy by gradually approaching both hands while putting power into the whole body, and expresses a “skill” in which directional energy is emitted at the moment when both hands are projected toward the target. . In this configuration, for example, the distance and the middle point are detected using two locations on both wrists, the voluntary muscle contraction strength of one arm or both arms is detected, and this exceeds a predetermined first threshold value. When the state where the neck distance is equal to or smaller than the predetermined second threshold value continues for a certain period of time, the charging of energy is completed. Then, by triggering the player to release the force while projecting both wrists toward the target (ie, to “release” the energy), the middle point of the wrists is set as the starting point, and the coordinate displacement direction of the middle point is changed. Vector data capable of describing a moving image in which directional energy is emitted as the aiming direction is obtained.

Next, a vector data output system according to a sixth aspect is the vector data output system according to the third to sixth aspects, wherein the magnitude of the force of the vector amount is the value of the voluntary muscle contraction strength. Calculating vector data defined as an electromyographic integral value obtained by integrating the voluntary muscle contraction intensity value according to the elapsed time from the time when the threshold value is equal to or greater than 1 to the time when it first falls below the first threshold value. The output is made possible.

In the vector data output system according to any one of claims 3 to 6, since the magnitude of the force in the output vector quantity is determined by the first threshold value, the magnitude of the emitted directional energy (representation on the game) (Corresponding to “destructive power”) is always fixed. However, as described above, from the viewpoint of the fun of the game, before the launch, the character charges the energy by his own muscular load (so-called “accumulation”), and the directional energy “destroy” according to the amount of the charge. It is preferable that the expression “force” can be changed.

In this configuration, the myoelectric integration value obtained by integrating the voluntary muscle contraction strength value according to the elapsed time when the voluntary muscle contraction strength value of the specific muscle exceeds the threshold during a predetermined gesture in the vector quantity. Can be set as the magnitude of force. Therefore, for example, the magnitude of the directional energy, that is, the “destructive power” can be changed according to the time when the player keeps putting the force on the arm muscles. It becomes possible.

Next, a video display system according to a seventh aspect includes the vector data output system according to any one of the second to sixth aspects and a video output apparatus, and the video output apparatus includes the vector data output system. The vector quantity defined by the output vector data can be output in an arbitrary display form on the projected coordinate system image.

  The video output device is any video monitor device including a computer game machine equipped with a processing device and software capable of processing vector data input from the vector data output system and displaying it as a two-dimensional video in the projected coordinate system. The display form of the vector data is not only the character's movement, but also a figure such as a beam shape, a wave shape, a spherical shape, a bullet shape, etc. according to the magnitude of the vector amount of force that directs the emitted directional energy toward the target (target) It may be displayed as an animation, or may be displayed as an animation in which the hit target receives damage without displaying the directivity energy itself. Moreover, it is good also as a structure displayed on the indicator etc. on an image | video as the state of the directional energy amount which emits the magnitude | size of the force of a vector amount. Also, in fighting computer game machines, the amount of vector power is not the launch of directional energy, but the power of skills such as punches and kicks that a character delivers, and the damage to the target hit by the skill is animated Or an indicator or the like.

  Finally, the video display system according to claim 8 is the video display system according to claim 7, wherein the video output device is a game machine or a computer equipped with fighting or shooting game software. It is characterized by that.

  According to the configuration of the present invention, in addition to the motion of the human body detected by the motion capture device, the vector amount including the magnitude of the force during the motion of the human body is determined from the change in the value of the surface myoelectric potential of a specific muscle of the human body. Since it can be output as vector data, for example, the magnitude of force can be added as a parameter to the control information of the remote control robot to enable more delicate control. In addition, when applied to a fighting or shooting computer game, it is possible to reflect the magnitude of the power accompanying the player's motion in the motion of the character on the video and the effect brought about by that motion. Can be made more realistic and diverse, and the sense of reality can be enhanced.

  In the following, an embodiment assuming the case where the present invention is implemented as an input device for operating a fighting computer game will be described with reference to the drawings. Note that a human body that is a target for detection of myoelectric potential and coordinate displacement of a specific part is hereinafter referred to as a “player”.

  FIG. 1 is a configuration diagram showing a configuration in which an output device is connected to a vector data output system according to the present invention. The vector data output system includes a myoelectric signal detection device 1, a motion capture device 2, and a control device 3. In this configuration diagram, the myoelectric signal detection device 1 is assumed to transmit data wirelessly via a receiving device 13. Are connected to the control device 3. On the other hand, the motion capture device 2 is connected to the control device 3 by wire. The control device 3 is connected to an external output device 4. The output device 4 is a computer game machine.

  The myoelectric signal detection device 1 measures a voluntary muscle contraction intensity from a myoelectric sensor 10 composed of an electrode for detecting a surface myoelectric potential attached to the skin in the vicinity of an arbitrary muscle of the player and the surface myoelectric potential detected by the myoelectric sensor 10. The myoelectric conversion unit 11 that converts the signal into a myoelectric signal and the transmission unit 12 that wirelessly transmits the myoelectric signal data to the receiving device 13. The control device 3 includes an input / output unit 30, a control unit 31, a storage unit 32, and a memory 33. The input / output unit 30 is an interface that inputs myoelectric signal data received by the receiving device 13 and coordinate displacement data from the motion capture device 2 and outputs vector data from the control unit 31 to the output device 4. The control unit 31 is a processor that processes myoelectric signal data and coordinate displacement data based on an algorithm to control generation and output of vector data. The storage unit 32 stores various specification setting data and the like for generating and outputting vector data, and the memory 33 is used for temporarily storing various data used in the vector data generation process.

  FIG. 2 is a schematic diagram showing the motion capture device 2 and a state in which the myoelectric signal detection device 1 is attached to the player. The motion capture device 2 is, for example, a depth sensor or IR emitter that is an RGB image recognition camera and an infrared camera for depth (depth) measurement, such as “Kinect (registered trademark)” commercially available from Microsoft Corporation. A device including an optical sensor (not shown) composed of (infrared laser irradiation unit) and having a processor that operates dedicated software can be used, but an optical sensor having another configuration having the same function may be used. An optical sensor may be connected to an external control device having dedicated software or a processor.

  In the case of the above-mentioned “Kinect (registered trademark)”, the motion capture device 2 is an infrared ray that reflects a state in which invisible light having a predetermined pattern (specifically, a dot distribution pattern) is projected onto the player's body with an infrared laser. Image data and distance data obtained by photographing with a camera, calculating the distance from the camera to the surface of each part of the body using predetermined parameters, and recognizing the image photographed with the RGB image recognition camera with dedicated software The process of converting to the coordinates of the projective coordinate system is performed continuously for each frame of the video. On the other hand, the myoelectric signal detection device 1 is worn with a belt or the like at a position where a myoelectric potential of a specific muscle of the player's arm can be detected as shown in FIG. Transmission from the transmission unit 12 to the reception device 13 is performed wirelessly.

(First embodiment)
FIG. 3 is an algorithm showing a processing procedure of myoelectric signal data and coordinate displacement data in the control device 3 according to the first embodiment of the present invention. In the first embodiment, for example, when a player emits directional energy toward a character in a game image toward a target, the aim is aimed from the direction of the elbow to the wrist, and a specific muscle of the arm (for example, biceps or biceps) The case is assumed that the release (relaxation) of the force placed in the arm (radius) is used as a trigger.

In this embodiment, myoelectric potential calibration for measuring the maximum voluntary muscle contraction value MVC is performed in advance for specific muscles of individual players prior to the calculation of vector data. Since the character on the computer game is a virtual existence, the voluntary muscle contraction strength% as a relative value to the maximum voluntary muscle contraction value MVC for each individual player, rather than reflecting the absolute value of the physical strength of the actual player, that is, the muscle strength on the game. This is because it is appropriate to measure MVC as a myoelectric signal.

Specifically, the myoelectric potential calibration measures the myoelectric potential value with a force applied as much as possible for a predetermined time after the player measures the resting myoelectric potential value of the arm muscles before starting the game. The maximum voluntary muscle contraction value MVC is set by excluding abnormal values through a low-pass filter. After the game starts, the myoelectric signal detection device 1 continuously detects the myoelectric potential value of the muscle of the arm, measures the voluntary muscle contraction intensity% MVC, which is a ratio to the MVC, and outputs it as a myoelectric signal (in the figure). "ME" or later). The voluntary muscle contraction strength% MVC measured at an arbitrary time point is calculated by calculating the myoelectric potential value EMG and the resting time with respect to the difference between the maximum voluntary muscle contraction value MVC and the resting myoelectric potential value EMGrelax. It is calculated as the ratio of the difference between the myoelectric potential values EMGrelax.

  On the other hand, the motion capture device 2 can always detect the entire skeleton of the player and continuously measure the coordinate displacement by converting the three-dimensional coordinates of all the main joints into a projective coordinate system. The coordinate P1 of the elbow and the coordinate P2 of the elbow joint are set as specific parts, and the positional relationship of the relative coordinates is measured as a parameter for determining the aiming direction described later (after “MC” in the figure).

  After the game is started, the myoelectric signal detection device 1 continuously measures the voluntary muscle contraction strength% MVC of the player, and when this exceeds a predetermined threshold value T1, that is, the player deliberately exceeds a certain value in the arm muscle. When power is started, the time counter C in the control device 3 is started. Then, when the voluntary muscle contraction strength% MVC continues for a predetermined time T2 or more and the voluntary muscle contraction strength% MVC falls below the threshold T1, the control device 3 determines that the elbow joint at that time Vector data defining a vector Rdir having the direction from the coordinate P2 to the coordinate P1 of the wrist joint as the vector direction and the wrist joint coordinate P1 as the vector start point is output. Thereafter, the time counter C is reset and returns to the measurement of voluntary muscle contraction strength% MVC again.

  By transmitting the vector data output by the above process to the game machine, the character that moves on the game image reflecting the movement of the whole body of the player charges the energy of the arm muscles while charging the wrist. It can be configured to display an animation of emitting directional energy from the wrist at the moment when the energy is released toward the target.

  FIG. 4 is an algorithm for explaining the second embodiment. In the algorithm of FIG. 4 and subsequent figures, the processing steps before “ME” and “MC” are the same, and thus the description is omitted. In the first embodiment, the magnitude of the vector force defined by the vector data is constant regardless of the duration of the state in which the voluntary muscle contraction strength% MVC is equal to or greater than the threshold value T1, but in the present embodiment, the magnitude of the force depends on the duration. The magnitude of the vector force can be changed.

After the time counter C starts when the voluntary muscle contraction strength% MVC is equal to or greater than the threshold value T1, in the present embodiment, the control device 3 sets the voluntary muscle contraction strength% MVC to the start time t1 of the counter C as shown in Equation 2. The myoelectric integration value S obtained by integration over the elapsed time from the first to the time point t2 when it first falls below the threshold value T1 is calculated. That is, as time elapses in a state where the muscles of the arm are put in force, the charged energy increases cumulatively.

  When the voluntary muscle contraction intensity% MVC continues for a predetermined time T2 or more and the voluntary muscle contraction intensity% MVC decreases below the threshold T1, the control device 3 determines the coordinates of the elbow joint at that time. Vector data defining a vector Rdir having the direction from P2 to the wrist joint coordinate P1 as the vector direction, the wrist joint coordinate P1 as the vector start point, and the myoelectric integral value S as the magnitude of the force is output. Thereafter, the time counter C is reset, and returning to the measurement of voluntary muscle contraction strength% MVC is the same as in the first embodiment.

  As described above, the expression that the character on the game image increases the destructive power of the directional energy that is emitted from the wrist toward the target as the time for charging the energy of the arm muscles becomes longer. It becomes possible.

  FIG. 5 is an algorithm for explaining the third embodiment. In the present embodiment, as in the second embodiment, the magnitude of the force of the vector Rdir is made variable by the myoelectric integration value S obtained by integrating the voluntary muscle contraction strength% MVC with the elapsed time, and vector data is output. Instead of relaxing the muscles, the trigger is replaced by re-tensioning the relaxed muscles.

  After the game starts, the voluntary muscle contraction strength% MVC is equal to or greater than the threshold value T1, and the time counter C starts and the myoelectric integral value S increases. If the voluntary muscle contraction strength% MVC decreases below the threshold value T1 after the state continues for a predetermined time T2 or more, the myoelectric integration value S accumulated up to that point is stored in the memory 33. Thereafter, when the voluntary muscle contraction strength% MVC again becomes the threshold value T1, the direction from the elbow joint coordinate P2 to the wrist joint coordinate P1 at that time is stored as the force direction, and the wrist joint coordinate P1 is stored as the force start point. The vector Rdir having the force of the myoelectric integration value S as the magnitude of the force is output as vector data. Thereafter, the time counter C is reset, and the process returns to the measurement of the voluntary muscle contraction strength% MVC.

  With this algorithm structure, after the character on the game video puts power into the arm muscles and completes the energy charge, aiming at the wrist toward the target and putting power into the arm muscles again as a trigger An animation of launching directional energy can be displayed. In the present embodiment, since the once charged energy is stored in the memory 33, the character can maintain a state in which directional energy can be emitted at an arbitrary timing even after time has elapsed after the completion of charging.

  FIG. 6 is an algorithm for explaining the fourth embodiment. In this embodiment, as a trigger for emitting directional energy, for example, the displacement of the coordinate P1 of the wrist joint with reference to the coordinate P2 of the elbow joint is used.

  After the game starts, the time counter C starts when the voluntary muscle contraction strength% MVC is equal to or greater than the threshold value T1, and until the myoelectric integration value S increases, the state is the same as in the second embodiment. When the voluntary muscle contraction strength% MVC decreases below the threshold value T1 after continuing for the time T2 or more, and the wrist joint coordinate P1 is displaced with respect to the elbow joint coordinate P2, the wrist joint from the displaced elbow joint coordinate P2 is changed. The vector Rdir is the vector Rdir in which the direction of the coordinate P1 is the direction of the force, the coordinate P1 of the wrist joint is the starting point of the force, and the electromyographic integration value S when the voluntary muscle contraction strength% MVC falls below the threshold T1. Thereafter, the time counter C is reset, and the process returns to the measurement of voluntary muscle contraction strength% MVC again.

  With this algorithm structure, the character on the game video puts energy into the arm muscles and charges the energy for a predetermined time or more, swings the wrist toward the target at any time, and relaxes the arm muscles at the same time. Only when a gesture of releasing energy is performed, an animation that directional energy is emitted can be displayed.

  Finally, FIG. 7 shows an algorithm for explaining the fifth embodiment. In this embodiment, as a condition for charging directional energy and a trigger for launching, gestures by two parts of the body where the distance changes, such as a change in the distance between the coordinates P3 of the right wrist joint and the coordinates P4 of the left wrist joint, The displacement of those coordinates is added. Specifically, the motion capture device 2 is configured to detect the coordinates P3 and P4 of both wrists of the player as specific parts and continuously measure the distance D of both coordinates and the coordinate M of the intermediate point.

After the game starts, the voluntary muscle contraction strength% MVC is equal to or greater than the threshold value T1, and the distance D is equal to or less than the predetermined threshold value T3, the time counter C starts, and the myoelectric integration value S increases cumulatively with time. . After a predetermined time T2, when the muscle contraction strength% MVC decreases below the threshold value T1 and the coordinate M of the intermediate point is displaced, the displacement direction of the coordinate M is changed to the direction of force, and after the displacement. A vector Rdir is output as vector data with the myoelectric integration value S when the muscle contraction intensity% MVC falls below the threshold value T1, and the time counter C is reset. Then, it returns to the measurement of voluntary muscle contraction strength% MVC again.

With this algorithm configuration, the character on the game video is charged with energy over a predetermined time while putting both wrists (or both palms) close to each other while putting power on the arm muscles, and at the desired timing both wrists (or both wrists). It is possible to display an animation in which directional energy is emitted only when the gesture of releasing the energy by releasing the energy by releasing the palm muscle toward the target is performed. In this embodiment, the launch point and aim of the directional energy are determined by the positional relationship and displacement of two specific parts of the body whose distance and relative position change three-dimensionally, such as both wrists. The character becomes more difficult to hit, and while charging with both wrists (or both palms) close to each other, the character will be unprotected and unable to take defense, thus improving the fun of game production. be able to.

In the present embodiment, the two myoelectric signal detection devices 1 are respectively mounted at locations where the surface myoelectric potential of the arm muscles of the player's arms can be detected, and both or one of the myoelectric signal detection devices 1 detects. Although it can be configured to measure the relationship between the surface myoelectric potential and the threshold value T1, it may be worn only on one arm as in the other embodiments.

  The specific configuration of the vector data output system according to the present invention has been described above. However, the present invention is not limited to the above embodiment, and can be improved or changed within the scope of the technical idea of the present invention. They are within the technical scope of the present invention.

  For example, the muscles to be detected for the surface myoelectric potential by the myoelectric signal detection device 1 are not limited to the biceps brachii and brachiofibular muscles, and other muscles may be appropriately detected. The specific part of the human body for detecting the coordinate displacement is not limited to the wrist or elbow, and may be another part.

  The present invention can be applied to a game controller for operating a character on a game image in a computer game, or a control controller such as a remote control robot.

Configuration diagram of the present invention Schematic diagram showing the motion capture device 2 and a state in which the myoelectric signal detection device 1 is attached to the player Algorithm for explaining the first embodiment Algorithm for explaining the second embodiment Algorithm for explaining the third embodiment Algorithm for explaining the fourth embodiment Algorithm for explaining the fifth embodiment

DESCRIPTION OF SYMBOLS 1 Myoelectric input apparatus 10 Myoelectric sensor 11 Myoelectric conversion part 12 Transmission part 13 Reception apparatus 2 Motion capture apparatus 3 Control apparatus 30 Input / output part 31 Control part 32 Memory | storage part 33 Memory 4 Output device H Player (human body)
P1 specific part (wrist)
P2 reference part (elbow)
P3, P4 specific part (wrist)
D Distance between both wrists M Middle point between both wrists

Claims (8)

  A myoelectric signal detection device capable of measuring the surface myoelectric potential of one or more specific muscles of the human body and outputting it as a myoelectric signal, and detecting the movement of any specific part of the human body to detect 3 of each part of the human body It consists of a motion capture device capable of outputting a change in the position of a dimensional coordinate as coordinate displacement data, and a control device capable of inputting the myoelectric signal and the coordinate displacement data, the control device in the three-dimensional coordinate system, It is possible to calculate and output vector data defining a vector quantity in which the change in the magnitude of the force is a change in the myoelectric signal of a specific muscle of the human body, and the direction of the coordinate displacement of the specific part of the human body is the direction of the force. A vector data output system characterized by that.   A myoelectric signal detection device capable of measuring the surface myoelectric potential of one or more specific muscles of the human body and outputting it as a myoelectric signal, and detecting the movement of any specific part of the human body to detect 3 of each part of the human body It consists of a motion capture device capable of projecting a dimensional coordinate to a coordinate in a projected coordinate system by perspective projection and outputting the position change as coordinate displacement data, and a control device capable of inputting the myoelectric signal and the coordinate displacement data. In the projective coordinate system, the control device is a vector in which a change in the magnitude of the force is a change in a myoelectric signal of a specific muscle of the human body, and a direction of a coordinate displacement of the specific part of the human body is a direction of the force. A vector data output system characterized in that vector data defining a quantity can be calculated and output.   A myoelectric signal detection device capable of measuring the surface myoelectric potential of any one or more muscles in the human body and outputting the voluntary muscle contraction intensity as a myoelectric signal, and detecting the movement of any specific part of the human body and the reference part A motion capture device capable of projecting and transforming the three-dimensional coordinates of the specific part into coordinates in a projected coordinate system by perspective projection, and outputting the position change as coordinate displacement data, and inputting the myoelectric signal and the coordinate displacement data The control device is configured such that after the state where the voluntary muscle contraction strength value is equal to or greater than a predetermined first threshold continues for a predetermined time or longer, the voluntary muscle contraction strength value is first set to the first value. When the value falls below the threshold value, a vector amount is defined that defines the coordinates of the specific part as the starting point, the direction from the coordinates of the reference part toward the starting point as the direction of force, and the first threshold value as the magnitude of force. Wherein the torque data calculated by the set to be output, the vector data output system.   A myoelectric signal detection device capable of measuring the surface myoelectric potential of any one or more muscles in the human body and outputting the voluntary muscle contraction intensity as a myoelectric signal, and detecting the movement of any specific part of the human body and the reference part A motion capture device capable of projecting and transforming the three-dimensional coordinates of the specific part into coordinates in a projected coordinate system by perspective projection, and outputting the position change as coordinate displacement data, and inputting the myoelectric signal and the coordinate displacement data The control device is configured such that after the state where the voluntary muscle contraction strength value is equal to or greater than a predetermined first threshold continues for a predetermined time or longer, the voluntary muscle contraction strength value is first set to the first value. And when the coordinates of the specific part are displaced with respect to the coordinates of the reference part, the coordinate of the specific part after the displacement is the starting point, and the direction from the coordinates of the reference part to the starting point The power of direction, and characterized in that a possible calculates and outputs vector data defining a vector quantity and magnitude of the force to the first threshold value, the vector data output system.   A myoelectric signal detection device capable of measuring the surface myoelectric potential of any one or more muscles in the human body and outputting the voluntary muscle contraction intensity as a myoelectric signal, and detecting the motion of two specific parts of the human body Then, after projecting and transforming the three-dimensional coordinates of the specific part into the coordinates of the projected coordinate system by perspective projection, the motion capture device capable of outputting the position change as coordinate displacement data, the myoelectric signal and the coordinate displacement data are input. The control device is configured such that the voluntary muscle contraction strength value is equal to or greater than a predetermined first threshold value, and the distance value between the two specific parts is equal to or less than a predetermined second threshold value. After the state has been maintained for a predetermined time or longer, the voluntary muscle contraction strength value first decreases below the first threshold value, and the coordinates of the intermediate point between the two specific parts are displaced in an arbitrary direction. Before the displacement Vector data output characterized in that vector data defining a vector quantity having the intermediate point as the starting point, the direction of displacement as the direction of force and the first threshold as the magnitude of force can be calculated and output. system.   7. The vector data output system according to claim 4, wherein the magnitude of the force of the vector quantity is a first threshold value from the time when the voluntary muscle contraction strength value becomes equal to or greater than the first threshold value. A vector data output system capable of calculating and outputting vector data defined as an electromyographic integral value obtained by integrating the voluntary muscle contraction strength value according to an elapsed time until a time point when it decreases to less than .   A vector data output system according to any one of claims 2 to 6 and a video output device, wherein the video output device outputs a vector quantity defined by the vector data output from the vector data output system. An image display system, characterized in that it can be output in an arbitrary display form on a projected coordinate system image.   The video display system according to claim 7, wherein the video output device is a game machine or a computer equipped with fighting or shooting game software.