WO2016079828A1

WO2016079828A1 - User interface system for hit operation, operation signal analyzing method, and program

Info

Publication number: WO2016079828A1
Application number: PCT/JP2014/080665
Authority: WO
Inventors: 加藤　敬郎
Original assignee: ニャフーン・ゲームス・ピーティーイー・エルティーディー; 加藤　敬郎
Priority date: 2014-11-19
Filing date: 2014-11-19
Publication date: 2016-05-26
Also published as: JP6064111B2; JPWO2016079828A1

Abstract

[Problem] To enable intuitive operation and use thereof in inexpensive and general purpose devices. [Solution] A controller comprises a gyro sensor 164 and a main body that can be held and operated by a user with both hands. A hit detection unit 211 is provided with a hit detection method of using a series of angular velocity values for the controller, which are detected continuously by the gyro sensor 164, to detect hit operation characteristics relating to rotational movement of the controller which could be caused when the user holding the controller with both hands pushes either of the left and right hands forward to perform a hit operation. Using the hit detection method achieves a function of determining when a user has inputted a hit operation.

Description

User interface system, operation signal analysis method and program for batting operation

The present invention relates to a user interface system for batting operation that specifies the timing of batting in a sports game such as boxing, tennis, squash, table tennis, etc., and a method and program for analyzing information such as operation signals input from the controller About.

Conventionally, as a form of a video game, devices and systems for executing game software on a general-purpose computer such as a personal computer in addition to a home-use game machine and an arcade game for business have been provided. Also, with respect to the game software providing method, a so-called online game service that provides a game program via a communication network such as the Internet, in accordance with the recent development of communication infrastructure, from a form via a conventional recording medium such as a CD-ROM. Is also spreading.

As one of the above-mentioned video games, there is a sport action game such as a boxing game, for example. In this sport action game, various actions related to the action of the player object such as the timing at which the player object punches, the punch type, and the punching power. Enter the condition. The input of these conditions is performed by an input interface such as a controller of a game machine or a touch panel through a graphical user interface (GUI: Graphical User Interface) displayed on the screen (see, for example, Patent Document 1). .

The technique disclosed in Patent Document 1 is to advance a boxing game by moving an object in the screen in accordance with the position of the finger touching the touch screen and the pressure of the finger. However, since the game device disclosed in Patent Document 1 detects the punch timing by tapping or swiping the touch screen, it is far from the actual sports operation. As a result, there was a problem that intuitive operability was lost. The same applies to a game device that is operated by a user pressing a button on the controller.

In order to solve such problems, in recent years, a controller that has an acceleration sensor that detects acceleration in a predetermined direction and inputs various conditions according to the acceleration detected by this acceleration sensor has been developed. Corresponding video game machines and game software have also been developed (see, for example, Patent Document 2).

The technique disclosed in Patent Document 2 detects three-axis accelerations in the front-rear (Y-axis), left-right (X-axis), and vertical (Z-axis) directions, respectively, in a glove unit that functions as a controller of a game device. An acceleration sensor is provided, and the type of punch fed out by the user is identified by analyzing the output waveform output from the acceleration sensor. In a game apparatus using such a glove unit as a controller, the operation of the controller operator can be traced by the player object in the screen, and intuitive operability can be experienced.

JP 2013-58136 A Special table 2006-320706

However, since the technique disclosed in Patent Document 2 is configured to operate the user's left and right hands with separate controllers, two controllers having acceleration sensors are required until the game is introduced. There is a problem that the cost is expensive. In addition, the technique disclosed in Patent Document 2 uses a glove-type input unit as a controller, and is designed to be used only for a boxing game. Its design and functions are specialized. There was a problem that it could not be used and its versatility was low.

Therefore, the present invention has been made in view of the above points. For example, in a game where a user needs to input a batting operation, such as a boxing game, intuitive operability is possible, and it is inexpensive and versatile. It is an object of the present invention to provide a user interface system, method, and program that can be used in a certain device.

The present invention includes a user interface system as described above, that is, a controller and an arithmetic device that processes information input from the controller, and an operation in which the arithmetic processing device or software executed by the arithmetic processing device is input from the controller. A user interface system having a hit detection unit that is a means for determining that a user has input a hit operation based on information such as a signal, and in order to solve the above problem, the controller includes an angular velocity A sensor and a main body that can be operated with both hands, and the hit detection unit uses a series of values of the angular velocity of the controller that is continuously detected by the angular velocity sensor. A batting operation in which the controller is held with both hands and either the left or right hand is pushed forward. A hit detection method for detecting a feature of the hit operation related to the rotational motion of the controller that can occur when the hit is performed, and a function for determining that the user has input a hit operation by using the hit detection method It is characterized by realizing. *

Further, another invention is an operation signal analysis method in a user interface system having a batting detection step of determining that there has been a batting operation by a user based on information such as an operation signal input from a controller,
(1) The controller has an angular velocity sensor and a main body that the user can operate with both hands,
(2) When the hit detection step performs a hit operation in which the user holds the controller with both hands and pushes out either the left or right hand using a series of values of the controller's angular speed continuously detected by the angular speed sensor. A batting detection method for detecting the characteristics of the batting operation related to the rotational movement of the controller that can occur in the
(3) By using a hit detection method, a function for determining that a user has input a hit operation is realized.

In the present invention, the user holds the single controller main body with both hands, and inputs a striking operation by pushing one hand forward as if the user himself performed a striking motion. According to the present invention as described above, since the angular velocity sensor is provided in the controller, and the movement of the user's hand having the controller is detected by the angular velocity sensor, for example, a game in which a left and right fist is hit such as a boxing game The user can operate the object intuitively, and the user can obtain a better game experience.

In the above invention, the hit detection unit can have a function of determining whether the hitting operation is performed by the user's right hand or the left hand by using the angular velocity of the controller detected by the angular velocity sensor. Become. In this case, since the left and right of the batting operation can be distinguished using the direction of the angular velocity detected by the angular velocity sensor, it is sufficient to have a single control device, for example, widely used general-purpose devices such as smartphones and tablet terminals. A reliable operation device can be used, and the overall cost can be reduced.

In the above invention, as one of the hit detection methods, the angular velocity value around the predetermined axis that can be regarded as the natural rotation axis of the controller when the hit operation is performed, or the unit time of the angular velocity value, which is detected by the angular velocity sensor Per minute, or a minute time predicted by using a series of angular velocity values (about 0.01 to 0.05 seconds) ahead angular velocity values, or values obtained by combining these values A method of comparing with a threshold value can be used. Here, the threshold value may be a predetermined value, but may be changed depending on the game situation, or may be changed according to user settings or a user environment.

In the above invention, a function for causing the object to be operated to perform a preliminary motion related to the striking motion in conjunction with a rotation angle around a predetermined axis of the controller calculated based on information input from the controller, or a preliminary motion for the object It is preferable to have a function of transmitting a control signal for executing the above. In this case, before the batting operation is performed by the user, the rotation based on the preliminary operation is detected, and the preliminary operation (e.g., swinging up the arm before punching) is performed on the object on the screen in advance. By doing so, the operational feeling of the game can be improved.

That is, in the conventional technique, since the object is started with the punch preliminary operation after simply detecting the hitting operation, there is a delay between the timing intended by the user and the timing at which the object actually punches. In the present invention, it is natural that the user pulls the hand holding the controller in the opposite direction to the batting operation before the batting operation, and the object is punched in conjunction with the rotation angle of the game device detected by the angular velocity sensor. By performing a preliminary motion (twist up and swinging up the arm), the object can start a preliminary motion before the batting operation is detected, and the object actually punches after the batting operation is detected. Can be shortened. Thereby, the delay of the timing which an object punches can be shortened, and a better game experience can be given to the user.

In the above invention, the hit detection unit detects the hitting operation using the angular velocity detected by the angular velocity sensor or the change amount per unit time of the angular velocity, but the change amount per unit time of the detected angular velocity or angular velocity is Since it is difficult to determine whether the operation is a hitting operation or a preliminary operation, it is preferable to use an acceleration sensor in addition to the angular velocity sensor. That is, in the above invention, the controller further includes an acceleration sensor, and the impact detection unit uses the acceleration detected by the acceleration sensor, so that the controller is pushed forward when the controller makes a rotational motion. It is determined whether or not it is pulled back, and if it is determined that the controller is pulled back, the rotation motion of the controller is assumed to be due to the reaction after the preliminary operation or the batting operation, and the detection of the batting operation is suppressed.

In carrying out the above invention, if it is possible to add a condition that the batting operation cannot be performed until the preliminary operation is completed according to the requirements of the game application, the preliminary operation is completed without using the acceleration sensor. Until then, it can be determined that the rotation of the control device is due to the preliminary operation. However, in a boxing game, such a condition cannot be added because there is a batting operation like a jab without a preliminary operation.

In addition, by using the acceleration sensor, it is possible to distinguish the batting operation and the reaction after the batting operation to some extent, but in order to more reliably distinguish the batting operation unit, the rotation of the controller by the user's batting operation is performed. By detecting the moment when the movement stops and predicting that a recoil will occur immediately after that, limiting the detection of the characteristics of the batting operation by the batting detection method, the recoil after the batting operation by the user is erroneously detected. It is preferable to prevent this.

On the other hand, by limiting the detection of the hitting operation in this way, there arises a problem that it becomes difficult to input continuous hitting operations such as a combination punch. Therefore, in the above invention, in order to accurately detect the input of the continuous hitting operation by the user, the hitting detection unit includes at least two hitting detection methods, one of which detects the characteristics of the moment when the hitting operation is started. By using a method and detecting the feature during the hitting operation, the other hitting operation can be detected. Here, one is a hit detection method 1 and the other is a hit detection method 2.

In this case, the batting detection method 1 detects that the batting operation has been input with emphasis on the temporal change of angular velocity, that is, angular acceleration. By using the angular acceleration, the hit detection method 1 can detect the moment when the user starts to rotate the control device for the hit operation, and the response speed is improved. However, the angular acceleration increases not only at the moment when the control device starts to rotate, but also at the moment when the rotation stops, the moment when the control device returns to the original direction after stopping the rotation, Since it becomes large even at the moment of stopping, it is necessary to prevent the hitting operation from being erroneously detected at these moments. The angular acceleration at the moment when the rotation stops can be distinguished by whether the angular velocity and the direction are the same. On the other hand, the angular acceleration due to the reaction that stopped rotating detects the moment when the rotation stops and temporarily increases the threshold value based on the angular acceleration at that time, so that the angular acceleration caused by the reaction and the batting operation Differentiate angular acceleration due to.

On the other hand, the hit detection method 2 detects the input of the hit operation with an emphasis on the angular velocity. Since a certain amount of time is required until the magnitude of the angular velocity reaches a certain level or more after the user starts rotating the control device for inputting the batting operation, the batting detection method 2 starts the batting operation like the batting detection method 1. It cannot be detected instantaneously. However, the value of the angular velocity remains large for a relatively long time when the user performs a hitting operation. Therefore, the hit detection method 2 ignores the input of the hit operation detected within a very short period from the moment when the rotational motion of the controller by the hit operation stops, and erroneously detects the reaction after the hit operation as the hit operation. Even if this is prevented, a continuous hitting operation such as a combination punch can be detected.

In the above invention, the controller further includes an input unit arranged in the controller body, and the hit detection unit has a function of changing the type or strength of the hit according to the presence or type of the input signal to the input unit. Is preferred. Although the input of the striking operation by the angular velocity sensor is intuitive, unlike the touch, swipe, or controller button input, it is not suitable for inputting the type of striking operation separately. However, if a detection means different from the angular velocity detection is prepared in the controller and is used, it is possible to achieve both an intuitive batting operation input by the angular velocity sensor and an operation for changing the striking type or strength.

Further, these user interface system and operation signal analysis method can be realized by executing the game program of the present invention described in a predetermined language on a computer. That is, an operation signal analysis program in a user interface system having a batting detection step for determining that a batting operation is input by a user based on information such as an operation signal input from a controller,
(1) The controller has an angular velocity sensor and a main body that the user can operate with both hands,
(2) When the hit detection step performs a hit operation in which the user holds the controller with both hands and pushes out either the left or right hand using a series of values of the controller's angular speed continuously detected by the angular speed sensor. A batting detection method for detecting the characteristics of the batting operation related to the rotational movement of the controller that can occur in the
(3) By using a hit detection method, a function for determining that a user has input a hit operation is realized.

According to such an invention, by using the operation signal analysis program, the above-described user interface system and operation signal analysis method can be realized in various information processing apparatuses without requiring a special processing device. . That is, by installing this operation signal analysis program on a general-purpose computer or IC chip and executing it on the CPU, a user interface system having the above-described functions can be easily constructed. This program can be distributed through a communication line, for example, and can be transferred as a package application that operates on a stand-alone computer. Such an operation signal analysis program can be recorded on a computer-readable recording medium, and the above-described system and method can be implemented using a general-purpose computer or a dedicated computer. The program can be easily stored, transported and installed.

As described above, according to the present invention, for example, a batting operation that specifies the timing of batting is input, such as a boxing game, and the batting operation by the user is recognized based on the angular velocity sensor in a game that advances the game. A user interface system capable of intuitive operation can be realized with an inexpensive and versatile device.

It is a conceptual diagram which shows the whole structure of the game device which concerns on embodiment. It is a block diagram which shows the internal structure of the game device which concerns on embodiment. It is a block diagram which shows the virtual internal structure in CPU which concerns on embodiment. (A) And (b) is explanatory drawing which shows operation | movement of the controller which concerns on embodiment, and the display screen of a game. It is the graph which showed the change of the rotation angle around the Y-axis of the controller according to user operation concerning an embodiment, and shows the change at the time of performing batting operation once with one hand. It is the graph which showed the change of the rotation angle around the Y-axis of the controller according to user operation concerning an embodiment, and shows the change at the time of performing batting operation alternately with both hands. It is the graph which showed the change of the rotation angle around the Y-axis of the controller according to user operation concerning an embodiment, and shows the change at the time of performing striking operation once with one hand. It is the graph which showed the change of the rotation angle around the Y-axis of the controller according to user operation concerning an embodiment, and shows the change at the time of performing batting operation twice with one hand. It is explanatory drawing which shows the hit | damage selection area | region on the screen which concerns on embodiment. It is a flowchart figure which shows the outline of the operation signal analysis process which concerns on embodiment. It is a flowchart figure which shows the subroutine of the peak detection process which concerns on embodiment. It is a flowchart figure which shows the subroutine of the impact detection process which concerns on embodiment. It is a flowchart figure which shows the subroutine of the impact detection process which concerns on embodiment. It is explanatory drawing which shows the recording medium which recorded the program which concerns on embodiment, and the information processing apparatus which can read the said recording medium.

(Basic configuration of game device)
Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. FIG. 1 is a perspective view showing a game apparatus 1 constituting a user interface system for inputting a batting operation by a user according to the present embodiment. In the present embodiment, a case where boxing game software is executed on the game apparatus 1 is illustrated. Further, in the present embodiment, the case where the present invention is applied to boxing game software will be described as an example, but the present invention is not limited to this, for example, sports action games such as tennis, squash, table tennis, etc. The present invention can also be applied to game software for executing game software for executing a game by inputting a batting operation for designating the timing of batting on an object such as a player object placed in a virtual space. In addition, it can be used to control a driving device such as a robot toy in the real space, and further, the object to be operated does not necessarily exist in the real space or the virtual space. The present invention can also be applied to software that performs a simulation that a virtual object performs a hitting operation and outputs only the calculation result. For example, music game software that makes a sound as if a virtual player object struck a drum by a user's hitting operation.

The game apparatus 1 according to the present embodiment is an arithmetic processing apparatus including a CPU, and can be realized by a portable information processing terminal or a dedicated apparatus specialized in function, and can be implemented by a mobile computer or a PDA (Personal Digital Assistance) and mobile phone. In the present embodiment, the game apparatus 1 has a communication function and can access a server through a communication network, and the game program according to the present invention can be downloaded through the server. As a communication method of the game apparatus 1, for example, 4G method, LTE method, 3G method, FDMA method, TDMA method, CDMA method, W-CDMA, PHS (Personal Handyphone System) method, Wi-Fi (registered trademark) And a wireless LAN system such as Bluetooth (registered trademark).

As shown in FIG. 1, the game apparatus 1 has a rectangular control device main body 100, and the main body 100 is provided with a liquid crystal display 165 that is a display device. The liquid crystal display (LCD) 165 includes: The state of the object is displayed in the screen, and the game is progressed by operating the displayed object. In addition, the game apparatus 1 includes a CPU 200 in the control apparatus main body 100, and also has a function as an arithmetic processing apparatus.

In this embodiment, the game apparatus 1 itself has a function as a controller. Specifically, the control device main body 100 of the game apparatus 1 includes a gyro sensor (angular velocity sensor) 164 therein, and can detect the angular velocity of the game apparatus 1 in the three-axis directions. Here, the angular velocity is the speed of rotation when the object makes a rotational motion, which is represented by a rotation angle per unit time. The user holds the game apparatus main body horizontally with both hands, and inputs a batting operation by pushing one hand forward as if punching. In the present embodiment, in addition to the angular velocity detected by the gyro sensor 164, the angular acceleration and rotation angle acting on the control device main body 100 are calculated from the angular velocity continuously detected, and the object is operated according to those values. .

Further, the control device main body 100 of the game apparatus 1 includes an acceleration sensor 169 inside, and can detect acceleration applied to the game apparatus 1 from the outside. In the present embodiment, the detected acceleration is used to determine whether the rotational motion of the game apparatus 1 by the user operation is a preliminary operation or a batting operation.

In the present invention, it is not necessary that the display device, the arithmetic processing device, and the controller are integrated as in the present embodiment. Rather, it is preferable that the controller and the display device are separate, but this embodiment The form shows that the present invention can be implemented by widely used devices such as smartphones and tablet terminals.

In the present embodiment, as shown in FIG. 1, the longitudinal direction of the game apparatus 1 is the X axis, and the direction from the left hand to the right hand when the user holds the game apparatus 1 with both hands is the positive direction of the X axis. To do. In addition, the user holds the game apparatus 1 with both hands and performs a striking operation in which one of the left and right hands is pushed forward, in parallel with the natural rotation axis, perpendicular to the X axis and on the surface of the LCD 165 of the game apparatus 1 The parallel vertical direction is the Y axis, and the direction from the bottom to the top is the positive direction of the Y axis. A direction perpendicular to the X axis and the Y axis and perpendicular to the surface of the LCD 165 is defined as a Z axis, and a direction from the game apparatus 1 toward the user is defined as a positive direction of the Z axis. Note that the above-described coordinates (X axis, Y axis, Z axis) for describing the orientation and angular velocity of the game apparatus 1 are used only for explaining the contents of the present embodiment. It may be different from the coordinate system it has.

The game apparatus 1 includes a touch panel 300 as an input device for an operation signal for operating the object. The touch panel 300 is an input device that inputs an operation signal by a touch operation using a user's fingertip, pen, or the like, and an LCD 165 that is a display unit that displays a graphic and a touch sensor 168 that receives an operation signal by the touch operation are superimposed. Configured. For this reason, the position where the touch operation is detected by the touch sensor 168 can be specified in correspondence with the coordinate position of the graphic displayed on the LCD 165.

(Internal structure of game device 1)
A specific internal configuration of the game apparatus 1 having the touch panel 300 and the gyro sensor 164 capable of display / operation described above will be described in detail below. FIG. 2 is a block diagram showing an internal configuration of the game apparatus 1 according to the present embodiment. The “module” used in the following description is a functional unit for achieving a predetermined operation, which is configured by hardware such as a device or equipment, software having the function, or a combination thereof. Indicates.

As shown in FIG. 2, the game apparatus 1 includes a duplexer 102 connected to the antenna 101, and a reception system module and a transmission system module connected to the duplexer 102. The reception system module includes a low noise amplifier 110, a mixer 111, an IF amplifier 112, an orthogonal mixer 113, an A / D converter 114, a demodulator 115, a channel decoder 116, an audio decoder 117, a D / A converter 118, and an amplifier 119 with a switch. And a speaker 120. On the other hand, the transmission system module includes a microphone 140, an amplifier 139, an A / D converter 138, an audio encoder 137, a channel encoder 136, a modulator 135, a D / A converter 134, an orthogonal mixer 133, an IF amplifier 132, a mixer 131, and A power amplifier 130 is provided.

The game apparatus 1 also includes a synthesizer 103, a time base 150, a CPU 200, a RAM 152, a ROM 153, and an EEPROM 151 as control system modules, and a gyro sensor 164, an LCD 165, operation buttons 166, an LED 167, and a touch sensor 168 as user interface system modules. And a vibrator 174, and further, as a power system module, a power system battery 171, a power source 172, and an A / D converter 173 are provided. The touch panel 300 described above is configured by superimposing an LCD 165 that displays a graphic and a touch sensor 168 that receives an operation signal corresponding to the coordinate position of the graphic displayed on the LCD 165.

The antenna 101 transmits and receives signals to and from a base station (not shown) via a radio wave line. The duplexer 102 is a circuit that switches between input and output of a signal to be transmitted and received. The duplexer 102 inputs a signal received by the antenna 101 to the low noise amplifier 110 and outputs a signal output from the power amplifier 130 to the antenna 101.

In the reception system module, the low noise amplifier 110 amplifies the signal input from the duplexer 102 and outputs the amplified signal to the mixer 111. The mixer 111 receives the output of the low noise amplifier 110, separates it by a specific frequency, and outputs it as an intermediate frequency signal. The IF amplifier 112 amplifies the intermediate frequency signal output from the mixer 111. The orthogonal mixer 113 receives the output of the IF amplifier 112 and performs orthogonal demodulation. The A / D converter 114 digitizes the output of the orthogonal mixer 113. The demodulator 115 demodulates the output of the A / D converter 114. Channel decoder 116 performs error correction on the output of demodulator 115. The error-corrected signal includes a control message and voice data. The control message is sent to the CPU 200 and the voice data is sent to the voice decoder 117.

The signal input from the channel decoder 116 to the audio decoder 117 is decoded into audio data and delivered to the D / A converter 118. The D / A converter 118 converts the output of the audio decoder 117 into an analog signal. The switch-equipped amplifier 119 is switched at an appropriate timing based on a control signal from the CPU 200 and amplifies the output of the D / A converter 118 when the switch is ON. The speaker 120 amplifies the output of the switch-equipped amplifier 119.

On the other hand, in the transmission system module, the microphone 140 receives an audio signal from the user and outputs the audio signal as an analog signal. The amplifier 139 amplifies the analog signal output from the microphone 140. The A / D converter 138 converts the output of the amplifier 139 into a digital signal. The audio encoder 137 encodes and compresses the output of the A / D converter 138 and outputs it as audio data. The channel encoder 136 combines the control message from the CPU 200 and the audio data from the audio encoder 137 and adds an error correction code.

The modulator 135 modulates the output of the channel encoder 136. The D / A converter 134 converts the output of the modulator 135 into an analog signal. The orthogonal mixer 133 converts the output of the D / A converter 134 into an IF frequency signal (intermediate frequency signal). The IF amplifier 132 amplifies the output of the quadrature mixer 133. The mixer 131 increases the frequency of the signal output from the IF amplifier 132. The power amplifier 130 amplifies the output of the mixer 131. The synthesizer 103 synchronizes the mixer 111, the quadrature mixer 113, the mixer 131, and the quadrature mixer 133 during communication. The time base 150 supplies a clock signal to each unit.

In the user interface system module, the gyro sensor 164 is a sensor that detects the magnitude and direction of the angular velocity. In this embodiment, the gyro sensor 164 detects angular velocities in the triaxial directions (X, Y, and Z axial directions). However, the triaxial angular velocities are not necessarily required in the control of the hitting operation in the present invention. Alternatively, only the angular velocity around the Y axis may be detected.

It should be noted that by detecting the angular velocity of the three axes, it is possible to control operations other than striking by the gyro sensor 164. For example, in a boxing game, the angular velocity around the Y axis is used to detect the input of the batting operation by the user, but the X axis detected by the gyro sensor and the input of the operation to avoid hitting the opponent are detected. The angular velocity around the Z axis is used.

An operation signal corresponding to the angular velocity detected by the gyro sensor 164 is output to the CPU 200 of the game apparatus 1 main body. In this embodiment, other sensors 154 such as an acceleration sensor 169 that detects acceleration applied to the game apparatus 1 and a magnetic sensor that detects geomagnetism and identifies the absolute orientation of the game apparatus 1 are provided. It may be.

The LCD 165 is a liquid crystal display that displays a game screen, GUI, user input characters, and the like. The touch panel 300 can display a graphic such as characters, graphics, and moving images via the LCD 165 to form a GUI, and obtains an operation signal through the touch sensor 168 on the touch panel 300.

The LED 167 is for transmitting a message to the user by turning on and off. The touch sensor 168 detects that the user's finger has touched the surface of the touch panel, and acquires an operation signal based on a change in pressure or capacitance on the surface of the touch panel 300. The vibrator 174 is a device that notifies an incoming call, and vibrates when the incoming call is received. The power system battery 171 supplies power to the power source 172 and the A / D converter 173. The power source 172 is a power source for the game apparatus 1. The A / D converter 173 supplies a signal to the CPU 200.

The CPU 200 is an arithmetic processing unit that controls each of the above-described units, and sequentially executes program commands stored in the RAM 152 and the ROM 153 to perform various functions. The CPU 200 may include a dedicated arithmetic processing unit GPU specialized for displaying graphics on the LCD 165.

The RAM 152 is used as a working memory of the CPU 200 and temporarily stores programs executed by the CPU 200 and calculation results by the programs. A program for the CPU 200 is recorded in the ROM 153, and program execution instructions are sequentially output in response to a request from the CPU 200. In the EEPROM 151, user data such as abbreviated dial, ID unique to the machine, and the like are recorded.

In this embodiment, the CPU 200 executes a predetermined game program (software). With this software, the CPU 200 functions as an arithmetic processing device constructed virtually by various modules.

(Virtual internal configuration of CPU 200)
Next, the internal structure of the arithmetic processing device that is virtually constructed by software executed on the CPU 200 will be described. FIG. 3 is a block diagram showing an internal configuration virtually constructed on the CPU 200 according to the present embodiment.

In the CPU 200, an operation signal acquisition unit 201, an application execution unit 202, a display information generation unit 204, and an angular velocity detection unit 205 are virtually constructed by executing a predetermined game program (software). .

The operation signal acquisition unit 201 is a module that receives an operation signal input from an operation device 206 such as an operation button 166 or a touch sensor 168 arranged in the control device body, and outputs the operation signal to the application execution unit 202 as an operation command. For example, an input from the operation button 166 is used as part of a menu operation (such as returning to the previous menu) in the application execution unit 202. In addition, the operation signal acquisition unit 201 includes an operation position detection unit 201a. The operation position detection unit 201a is a module that detects the coordinate position of the operation signal input to the touch panel 300, which is one of the input units arranged in the control device main body, and the operation position detection unit 201a detects the operation signal detected by the touch sensor 168. The input coordinate position is transmitted to the application execution unit 202 as a coordinate position corresponding to the coordinate system of the graphic displayed on the LCD 165. Thereby, the application execution unit 202 can execute selection of a menu displayed on the LCD 165 and selection of a batting type or a defense type during the game.

The angular velocity detection unit 205 operates at a predetermined time interval (for example, 60 Hz) by a timer interruption, and receives an operation signal such as an angular velocity in each direction (X axis, Y axis, Z axis) from the gyro sensor 164 in the control device main body 100. This is a reading module. Furthermore, the angular velocity detection unit 205 integrates the angular velocity continuously detected at a predetermined time interval, and tracks the direction (rotation angle) of the game apparatus 1. For tracking the orientation of the game apparatus 1, in addition to the angular velocity continuously detected at a predetermined time interval, data detected by other sensors 154 such as gravitational acceleration and magnetic sensor detected by the acceleration sensor 169 are used. You may use together.

Further, the angular velocity detection unit 205 calculates the acceleration applied to the game apparatus 1 using the acceleration sensor 169 in the control apparatus main body 100. The acceleration sensor 169 also detects gravitational acceleration, but it is assumed that the acceleration calculated by the angular velocity detection unit 205 is obtained by removing the gravitational acceleration. Further, the acceleration is expressed as an apparent acceleration (actually measured acceleration) applied to an internal spring when viewed in the stationary system of the game apparatus 1, or the game apparatus 1 itself when the game apparatus 1 is observed from the outside. Although the direction of the acceleration is reversed depending on whether it is expressed as the acceleration applied to the device, in the following description, the acceleration calculated by the angular velocity detection unit 205 is the acceleration applied to the game device 1 itself when the game device 1 is observed from the outside. It shall be expressed as

The angular velocity, direction, and acceleration of the game apparatus 1 obtained by the angular velocity detection unit 205 are held in a storage area in the RAM 152. Then, the application execution unit 202 may operate at a timing different from the timer interrupt of the angular velocity detection unit 205, and by accessing a storage area in the RAM 152, the latest apparatus main body obtained by the angular velocity detection unit 205 100 angular velocities, orientations, accelerations, etc. are acquired.

Here, in order to describe the present invention in detail, the notation method of the angular velocity obtained by the angular velocity detection unit 205 and held in the storage area in the RAM 152 and the orientation of the game apparatus 1 is defined. First, the angular velocity vector amount read by the angular velocity detection unit 205 at a certain time t is a vector ω (t), the X-axis component value of the angular velocity (angular velocity around the X axis) is ω _x (t), and the angular velocity Y The value of the axis component (angular velocity around the Y axis) is expressed as ω _y (t), and the value of the Z-axis component of the angular velocity (angular velocity around the Z axis) is expressed as ω _z (t). The value of the angular velocity around each axis is positive in the counterclockwise direction with respect to that axis. Here, since the angular velocity detection unit 205 operates at a predetermined time interval by a timer interruption, the time t takes a discrete value, but even when other values are used at the time t, the vector ω _Let (t), ω _x (t), ω _y (t), and ω _z (t) represent the values of the angular velocities detected most recently before time t. Here, for convenience of explanation, the angular velocity is expressed as a function of time t for the sake of the following description. However, in carrying out the present invention, it is not necessary to obtain the value of the angular velocity retroactively, and the angular velocity detection unit 205 It is sufficient that only the latest value obtained at the time of the operation is held in the storage area in the RAM 152.

Next, the orientation of the game apparatus 1 at a certain time t is expressed by a rotation matrix A (t). Further, only the rotation angle around the Y axis of the game apparatus 1 is necessary for controlling the batting action in the present invention, and the rotation angle around the Y axis of the game apparatus 1 is expressed as θ _y (t). Similarly to the angular velocity, the rotation angle θ _y (t) about the Y axis is positive in the counterclockwise direction with respect to the Y axis. Here, the time t also takes a discrete value, but it is expanded to take a continuous value in the same way as the angular velocity. Here, for convenience, it is expressed as a function of time t. However, when implementing the present invention, it is not necessary to go back to the past and acquire the rotation matrix A (t) and the rotation angle θ _y (t).

The rotation angle θ _y (t) around the Y axis of the game apparatus 1 is obtained by integrating the angular velocity ω _y (t) at the time t even when the gyro sensor 164 can detect only the angular velocity ω _y (t) around the Y axis. It can be calculated by Although this value may deviate from the actual value when rotation about another axis is added, it is not necessary to detect the rotation angle with high accuracy in implementing the present invention, and automatic calibration described later is implemented. This will allow you to control the object without any problems.

On the other hand, when the rotation matrix A (t) representing the orientation of the game apparatus 1 can be used, the rotation angle θ _y (t) around the Y axis is more accurately calculated from the rotation matrix A (t). However, in three-dimensional rotation, there are various methods for defining the rotation angle θ _y (t) around the Y axis, and the calculation method differs depending on the definition. Although the present invention can be implemented using any calculation method, in the present embodiment, the rotation angle θ _y (t) about the Y axis is calculated as follows as an example.

First, the vector z is set as a unit vector in the Z-axis direction as follows.
z = (0,0,1) (Formula 1)
Then, a vector obtained by rotating the vector z with the rotation matrix A (t) is defined as a vector z ′ (t), and the vector z ′ (t) is obtained as follows.
z '(t) = A (t) z (Formula 2)
If the X component of the vector z ′ (t) obtained by Equations 1 and 2 is z ′ _x (t) and the Z component is z ′ _z (t), the following relationship is established.
tanθ _y (t) = z ′ _x (t) / z ′ _z (t) (Formula 3)
Here, tan .theta _y (t) is the tangent of the angle of rotation θ _y (t) (tangent), can calculate the rotation angle theta _y (t) about the Y axis from Equation 3 by using the inverse function of the tangent function . However, if z ' _x (t) and z' _z (t) are both zero or very small, the calculation error will increase, so use the unit vector in the X-axis direction instead of the vector z in Equation 1. The same calculation shall be performed.

The display information generation unit 204 is a module that generates information for object display on the LCD 165 and outputs the information to the LCD 165 in accordance with control by the application execution unit 202. Specifically, the display information generation unit 204 includes a 3D configuration unit 204a, a 2D configuration unit 204b, and a GUI configuration unit 204c, and arranges a virtual camera that defines a visual field range in a three-dimensional space. The object imaged by the virtual camera is displayed on the LCD 165 as a two-dimensional plane.

The 3D configuration unit 204a is a module that virtually configures a three-dimensional space and controls the coordinate positions of objects and cameras existing on the three-dimensional coordinates in the three-dimensional space. The 2D configuration unit 204b is a module that two-dimensionally displays the three-dimensional space on the shooting screen, acquires the data of the three-dimensional space configured by the 3D configuration unit 204a, and generates a two-dimensional image corresponding to the field of view of the shooting screen. Is processed to display the three-dimensional space two-dimensionally on the photographing screen.

Specifically, a virtual camera that defines a visual field range is arranged in a three-dimensional space, and an object captured by the camera is determined based on the positional relationship between the object calculated by the 3D configuration unit 204a and the virtual camera. By forming the photographing screen as a two-dimensional plane, the display of the object state on the LCD 165 is changed. In the present embodiment, the shooting screen is a main screen that captures an image of the player and the opponent's object and part of the ring through the player object.

The GUI composition unit 204c is a module for displaying a graphic user interface (GUI) such as information about a game and a menu on the shooting screen generated by the 2D composition unit 204b. The GUI is a graphic that visually conveys an operation method for an object and information on the object in an easy-to-understand manner. For example, in the case of a boxing game, the GUI includes power gauges 304 and 305 of each character as shown in FIG. .

The application execution unit 202 generates an object based on the input signals from the angular velocity detection unit 205 and the operation signal acquisition unit 201 and arranges the object in the virtual space, and controls or deletes the object in the virtual space. For example, in a boxing game, the player or opponent object in the virtual space performs operations such as hitting, defending, and avoiding, and controlling preliminary operations before hitting. And make it work.

In addition, the application execution unit 202 manages the progress of the game in accordance with the boxing rules (scoring process during the game, such as the class of the selected object, the ranking, the number of rounds according to the ranking, etc.) Based on the ability parameter of the opponent object, the type of punch, and the like, the hit condition is calculated, and the damage to the object in the virtual space is calculated.

The application execution unit 202 includes a motion control unit 210 that controls the entire motion of the object, and a batting detection unit 211 that detects a batting operation by the user. The motion control unit 210 is a module that controls the entire motion of the object, and causes the object to perform a striking motion in response to a batting operation input by the user detected by the striking detection unit 211. Further, the motion control unit 210 has a function of executing a preliminary motion related to the striking motion according to the value of the rotation angle θ _y (t) obtained by the angular velocity detecting unit 205 in addition to the striking motion. Further, the motion control unit 210 has a function of causing the object to perform a motion to avoid hitting the opponent using the rotation matrix A (t) and the angular velocity vector ω (t) obtained by the angular velocity detection unit 205.

The hit detection unit 211 is either the value of the Y-axis component ω _y (t) of the angular velocity obtained by the angular velocity detection unit 205, or the amount of change per unit time of the value of ω _y (t), or these A striking detection method is provided for detecting a feature of the rotational motion of the controller that may be caused by the input of the striking operation by the user by comparing the value of the angular velocity ahead of the minute time predicted by using both with a threshold value. The hit detection unit uses this hit detection method to detect that the user has input a hitting operation. At this time, whether the hit with the right hand or the left hand can be determined by the sign of ω _y (t). If it is positive, it will be hit with the right hand. The batting detection unit 211 determines whether the rotational motion of the controller is due to the user's batting operation or the preliminary operation according to the direction of the acceleration of the game apparatus 1 calculated by the angular velocity detection unit 205, and the preliminary operation If it is determined that it is due to, the input of the batting operation is not detected. If it is determined that the hitting operation is performed, the motion control unit 210 starts the hitting operation of the object, and the preliminary motion of the object is suppressed while the hitting operation is being executed.

Further, in the present embodiment, the hit detection unit 211 detects the moment when the rotational motion of the controller due to the hit operation of the user stops, and predicts that the recoil will occur immediately thereafter, and detects the hit operation by the hit detection method. I try to limit it. By doing so, it is possible to prevent erroneous detection of the angular velocity and the change pattern of the angular velocity due to the reaction when the batting operation is stopped as the batting operation.

Further, in the present embodiment, the hit detection unit 211 uses two hit detection methods, one is a method for detecting the characteristics of the moment when the hit operation is started, and the other is a method in which the hit operation is performed. Use a method to detect the features inside. In detail, the first hit detection method detects the moment when the input of the hit operation is started by using a method of comparing the amount of change per unit time of ω _y (t), that is, the angular acceleration, with a threshold value. . In this case, in order to prevent erroneous detection of angular acceleration due to reaction when the batting operation is stopped, the batting operation is performed because the value of ω _y (t) is set to zero, or the sign is reversed across the zero. The moment when the rotation of the controller stops is detected, and the threshold value is temporarily increased based on the angular acceleration at that time. On the other hand, the second hit detection method cannot detect the start of the hitting operation instantaneously by detecting hitting with emphasis on the value of ω _y (t), but compared with the case where angular acceleration is used. Therefore, it is easy to distinguish the recoil when the batting operation is stopped. That is, the second batting detection method also detects the moment when the rotational motion of the controller by the batting operation stops, and ignores the batting operation detected again only for a very short time from the moment when the rotational motion of the controller stops. Although the recoil when the batting operation is stopped is prevented from being erroneously detected as the batting operation, the detection of the continuous batting operation such as a combination punch is not hindered.

As shown in FIG. 4 (a), when the right hand holding the game apparatus 1 is pulled forward (opposite to the operation direction of the batting operation) by the control by the operation control unit 210 and the hit detection unit 211, the Y axis is set. On the other hand, when a clockwise rotation is detected, a preliminary movement is performed to twist the body of the object to the right, and then an operation to push the right hand forward (the direction of movement of the batting movement) is accepted. Control is performed so as to perform the striking motion of paying out. Further, when the left hand holding the game apparatus 1 is pulled forward (opposite to the direction of motion of the batting motion), a preliminary motion of twisting the object body to the left is performed, and then the left hand is strongly moved forward (motion direction of the batting motion). When an operation to push out is accepted, control is performed so as to perform a striking motion for delivering a left hand punch. Hereinafter, the control of the preliminary operation in the operation control unit 210 and the detection process of the batting operation in the batting detection unit 211 will be described.

(1) Preliminary Operation Control First, preliminary operation control in the operation control unit 210 will be described. In the present embodiment, the motion control unit 210 controls the preliminary motion of the object based on the rotation angle θ _y (t) about the Y axis calculated by the angular velocity detection unit 205. Here, description will be made separately on (a) calculation of the body rotation (torsion) size φ, (b) object rotation control, and (c) automatic calibration.

(A) Calculation of object body rotation (torsion) size φ First, the motion control unit 210 calculates the object body rotation (torsion) size φ at a certain time t. Specifically, the time when the game starts and t _0, the rotation angle theta ₀ about the Y axis at the time t ₀ is determined as follows.
θ ₀ = θ _y (t ₀ ) (Formula 4)
Then, the magnitude φ of the rotation (torsion) of the body of the object at a certain time t is obtained as follows.
φ = S × (θ _y (t) −θ ₀ ) (Formula 5)
Here, S in Equation 5 represents a predetermined scale parameter, which is for enabling the object to perform a sufficiently large action without greatly rotating the game apparatus 1. Further, the rotation angle of the body of the object has an upper limit Φ, and the magnitude φ of the rotation (twisting) is truncated to the range [−Φ, Φ]. As the magnitude φ of rotation (twist) approaches the upper limit, the object swings up the arm as well as the body and prepares for punching. Further, the rotation angle calculated by Expression 5 can be used as a target value without being used as it is. For example, if control is performed so that the amount of change in the rotation angle of the body of the object per unit time does not exceed a certain value, the object tries to rotate the body at an unrealistic speed. Can be prevented.

(B) Object Rotation Control Next, a method for controlling the rotation of the body in accordance with the magnitude φ of the rotation (twisting) will be described. First, the posture of the object is controlled by N parameters, and a set of these parameters is an N-dimensional vector a. In this case, in order for the object to rotate the body according to the value of the rotation (torsion) magnitude φ, the value of the parameter set a (φ) corresponding to the rotation (torsion) magnitude φ is determined. There is a need to. Here, if the form of the function of the vector a (φ) is simple, it can be calculated by a program, but in most cases, the vector a (φ) is a complicated function and is manually set by a human hand. Need to be set.

Therefore, in this embodiment, M representative values of the rotation (torsion) magnitude φ are selected, and these are (φ ₁ , φ ₂ ,..., Φ _M ). These values are set so that φ ₁ = −Φ, φ _M = Φ, φ ₁ <φ ₂ <... <φ _M. Then, when the magnitude of rotation (torsion) φ becomes the above selected value, a set of values that the control parameter vector a (φ) should take is set in advance. Here, a set of values that the parameter vector a should take is a ₁ = a (φ ₁ ), a ₂ = a (φ ₂ ),..., A _M = a (φ _M ). After that, given a rotation (torsion) magnitude φ which is an arbitrary value, the value of the vector a (φ) is calculated by interpolating the above a ₁ , a ₂ , ..., a _M . As an interpolation method in this embodiment, linear interpolation is used,
If φ <φ1,
a (φ) = a ₁ (Formula 6)
If φ _i ≦ φ <φ _{i + 1} ,
a (φ) = s × a _i + (1−s) × a _{i + 1} ,
s = (φ _{i + 1} −φ) / (φ _{i + 1} −φ _i ) (Expression 7)
If φ _M ≦ φ,
a (φ) = a _M (Formula 8)
As a result, the value of the vector a (φ) is calculated. In addition to the linear interpolation, other interpolation methods such as higher-order spline interpolation may be used as the interpolation method.

(C) Automatic Calibration Next, automatic calibration will be described. In the present embodiment, the rotation angle θ _y (t) around the Y axis of the game apparatus 1 is used for the movement of the body of the object, but this value is obtained when the user changes the direction. Will also change. For example, if a user is playing a game on a train or car, the user's direction may change without noticing, and the objects in the game may rotate their body in an unexpected direction. . Further, since the value of the rotation angle θ _y (t) around the Y axis includes an error when the angular velocity vector ω (t) is integrated, there is a possibility that it may become so large that it cannot be ignored.

However, the change amount of the rotation angle θ _y (t) about the Y axis due to these effects is smaller than the amount of change when the user controls the body of an object intended, Y at time t ₀ of the above Correction can be made by sequentially changing the value of θ ₀ representing the rotation angle around the axis.

Specifically, the current time is t _j, and the value of θ ₀ representing the rotation angle around the Y axis at time t ₀ is updated as in the following equation.
θ ₀ → θ ₀ + s × (θ _y (t _j ) −θ ₀ ) (Equation 9)
Here, the parameter s in Equation 9 represents the degree of calibration. If the value of s is 0, the calibration effect is completely lost. If the value of s is 1, θ ₀ is instantaneously set to the current direction. Will be calibrated. The value of s is preferably proportional to the calibration time interval so that the degree of calibration does not depend on the frequency of calibration. That is, when the time at the previous calibration is t _j−1 , the value of s can be obtained by the following equation.
s = C × (t _j −t _j−1 ) (Equation 10)
Here, C may be a constant, and C represents the speed of calibration (the degree of calibration per unit time). The appropriate value of C can be variously changed depending on the application, but it is preferable to set a value of about 1 to 5 seconds- ¹ . Further, it is assumed that calibration is performed at a sufficiently high frequency (at least 10 Hz or more), and the value of s must always be a value sufficiently smaller than 1.

In the present embodiment, the value of C is reduced when the user is moving the game apparatus 1 violently, and the value of C is increased when the user is hardly moving. For example, the value of C is obtained as follows.
C = C ′ × exp (− | ω (t) | ² / Ω ² ) (Equation 11)
Here, exp represents an exponential function, and Ω is a predetermined constant, which determines how much the angular velocity increases and the value of s decreases. C ′ is a constant and should be set to a value of about 1 to 5 seconds− ¹ .

(2) Detection of batting operation Next, a batting detection method in which the batting detection unit 211 detects the feature of the rotational motion of the controller that can be caused by the batting operation by the user will be described. In this embodiment, the hit detection unit 211 uses two hit detection methods, one is (1) the hit detection method 1 for detecting the characteristics of the hit operation using angular acceleration, and the other is (2) the angular velocity and the angle. The impact detection method 2 detects the characteristics of the impact operation using both acceleration and acceleration. However, the implementation of the present invention does not necessarily require these two methods, and either one can be used according to the requirements of the game application, or a modified method can be used. Note that since both detection methods may cause erroneous detection, a method for improving false detection described later is used in combination. Here, erroneous detection means that a punch is detected at a timing not intended by the user. Hereinafter, each detection method and the false detection improvement method will be described.

(A) Impact detection method 1
The hit detection method 1 for detecting the timing at which the hitting operation is started using the angular acceleration is the change amount per unit time of ω _y (t) that is the Y component of the angular velocity, that is, the angular acceleration around the Y axis. A hit is detected when the threshold is exceeded.

Here, when the time when the Y component ω _y (t) of the current angular velocity is detected is t _j and the time when the previous angular velocity is detected is t _j−1 , the angular acceleration around the Y axis at time t _j is α _y (t _j ) is obtained as follows.
α _y (t _j ) = {ω _y (t _j ) −ω _y (t _j−1 )} / (t _j −t _j−1 ) (Equation 12)
When the magnitude of the angular acceleration α _y (t _j ) thus obtained exceeds the threshold value Tα, it is detected that a hitting operation has been input. The merit of this method is that the value of the angular acceleration α _y (t _j ) increases at the moment when the user pushes out the game apparatus 1 to input the batting operation, so that the batting can be detected without delay with respect to the user's operation. is there. Further, the user only needs to apply power for a moment, and can input a batting operation without moving the game apparatus 1 greatly.

On the other hand, the value of the angular acceleration α _y (t _j ) increases because it includes not only the moment when the game apparatus 1 starts to rotate suddenly but also the moment when the game apparatus 1 that has rotated suddenly stops. In this detection method, a hitting operation may be erroneously detected at such a moment. A method for improving this erroneous detection will be described later.

(B) Impact detection method 2
Next, the hit detection method 2 using both the angular velocity and the angular acceleration will be described. The detection method here uses the angular acceleration α _y (t) to predict the value of the angular velocity ω _y (t + Δt) after a minute time Δt from the current time t, and compares the magnitude with the threshold value Tω. It is a method. Here, to be described as ω _y (t + Δt) of the predicted value _{ω 'y (t + Δt)} , predicted as follows.
ω ′ _y (t + Δt) = ω _y (t) + α _y (t) Δt (Expression 13)
Then, when the magnitude of ω ′ _y (t + Δt) predicted by Expression 13 exceeds the threshold value Tω, it is detected that a batting operation has been input. The merit of this method is that the value of the angular velocity ω _y (t) is increased only when the game apparatus 1 is rotating fast, so that false detection is reduced compared to the hit detection method 1. .

However, on the other hand, since it takes time until the value of the angular velocity ω _y (t) increases after the user pushes the game apparatus 1 to input the batting operation, in this method, the user starts to input the batting operation. Then, there is a demerit that a delay occurs until it is detected. Δt in Equation 13 is intended to reduce this disadvantage. However, if the value of Δt is too large, the influence of angular acceleration increases and false detection increases, so the value of Δt is set according to the requirements of the game application. It needs to be adjusted. If priority is given to delay reduction, it is better to set the value of Δt to about 0.05 seconds, and if you want to reduce false detections, set the value of Δt to about 0.01 seconds. good. In the present embodiment, a setting for reducing erroneous detection is used in order to detect an input of a continuous striking operation such as a combination punch separately from a recoil when the striking operation is stopped.

(C) Measure for Improving False Detection Next, a measure for improving the false detection of the hitting operation that occurs in each hit detection method will be described. FIG. 5 is a graph showing a change in the rotation angle θ _y (t) around the Y axis when the batting motion is input once with the right hand, and FIG. 6 is a diagram in which the batting motion is sequentially input in the order of the right hand and the left hand. It is the graph which showed the change of rotation angle (theta) _y (t) around the Y-axis at the time of doing. FIG. 7 is a graph showing a change in the rotation angle θ _y (t) around the Y axis when the hitting action is strongly input once with the right hand, and FIG. 8 is the hit action twice with only the right hand. It is the graph which showed the change of rotation angle (theta) _y (t) around the Y-axis at the time of inputting continuously.

5 to 8, the horizontal axis represents time t and the vertical axis represents the rotation angle θ _y (t) around the Y axis. It should be noted that the graphs in each figure are not smooth but are stepped due to different timings of operation between the angular velocity detection unit 205 and the hit detection unit 211, and can be obtained directly from the gyro sensor. It does not mean that the value of angular velocity is zero or the value of angular acceleration is large.

(A) About the case where the batting operation is performed once with one hand First, the case where the batting operation is performed once with one hand will be described. As shown in FIG. 5, the operation here starts from the time point A of the graph by starting to pull the right hand toward the front and enters the preliminary punch operation, and at the time point D, the right hand is strongly pushed out to input the punch. Yes. And the hand pushed out at the time of F is stopped, and the hand is pulled back by the reaction at the time of G. This graph shows a typical pattern that can be seen when a punch is operated. In particular, the action of pulling back the hand at time G is a reflex movement of the muscle, not what the user intended. This is a reaction that is sure to be seen. There is a possibility that erroneous detection occurs at each time point A to H. By adjusting the threshold value Tα in the impact detection method 1, the threshold value Tω in the impact detection method 2, and Δt in Equation 13, the false detection can be reduced to some extent, but some drastic measures are required. Since there are certain points in time, a description will be given of erroneous detection that can occur at these points in time and improvement processing for the detection.

-About time A At the time A when the preliminary operation is started, the angular acceleration α _y (t) when the user pulls the hand holding the game apparatus 1 may be erroneously detected. Due to this erroneous detection, it is determined that a batting operation has been input with a hand opposite to the hand intended by the user, so that the user's operational feeling is greatly impaired. This erroneous detection occurs in the hit detection method 1 described above, but even if the hit detection method 2 hit detection method 2 is used, it may occur when the value of Δt in Expression 13 is set large. The following improvement method is used for erroneous detection at this point.

(Improvement processing A-1)
The improvement process here distinguishes the preliminary action and the striking action by using the acceleration sensor 169 included in the control device main body 100. That is, when the Z component of the acceleration detected by the acceleration sensor 169 is positive, the game apparatus 1 is pulled back, so the reason why the angular acceleration α _y (t) has increased is the preliminary operation. Therefore, it is determined not to detect the striking operation. However, when the user pulls one hand forward, it tends to push the other hand slightly at the same time, and may detect a small reverse acceleration. Therefore, instead of checking whether the acceleration Z component is positive, it is checked whether the acceleration Z component is larger than the threshold value −Ta. In this embodiment, the size of Ta is about 0.1 G (1/10 of the gravitational acceleration).

-About time point C At time point C, the angular acceleration α _y (t) increases due to stopping the operation of pulling back the game apparatus 1 to the front by a preliminary operation. appear. In this case, since it is determined that the user hits the right hand as intended, there is no major problem. However, since the time point C is not the timing intended by the user, if the user only imitates hitting with a feint, False detection. When using the hit detection method 2, if the value of Δt in Equation 13 is set too large, a false detection may occur at time C. If the value of Δt is about 0.05 seconds, The possibility is very low.

(Improvement process C-1)
The improvement process C-1 is a process for monitoring not only the angular acceleration but also the direction of the angular velocity. That is, even if the magnitude of the angular acceleration α _y (t) exceeds the threshold value Tα, the angular velocity ω _y (t) around the Y axis and the angular acceleration α _y (t) are different from each other or around the Y axis. When the magnitude of the angular velocity ω _y (t) is not sufficient (when it is smaller than a certain threshold value T′ω), the game apparatus 1 in which the reason for the increase in the angular acceleration α _y (t) is rotating is It is determined that the operation has stopped, and the hitting operation is not detected. Note that the threshold value T′ω is different from the threshold value Tω in the impact detection method 2, and the value of the threshold value T′ω is set to a value considerably smaller than the threshold value Tω.

-About point D Since point D is the time when the user performed the batting operation, it is correct to detect the batting here, but if the user only imitates batting with a feint, At this point, a hit may be detected. When applying a feint, the user should be careful not to cause a reaction, but the following improvement process can increase the probability that the feint will succeed.

(Improvement process D-1)
The improvement process D-1 is a process for detecting the moment when the rotation of the game apparatus 1 stops and predicting that the game apparatus 1 will move immediately before increasing the threshold temporarily. Specifically, first, the moment when the rotation stops is defined as the time when the value of the angular velocity ω _y (t) becomes zero, or the time when the positive and negative signs are reversed across zero. This time the rotational angle around Y axis of the game apparatus θ _y (t) is the time that will peak, is referred to as a peak time t _p. Then, the angular acceleration α _y (t _p ) at the peak time t _p is used to prevent the hit from being detected while the following conditional expression is satisfied.
| α _y (t) | ≦ | α _y (t _p ) | · exp (−λ (t−t _p )) (Equation 14)
That is, the hit is not detected unless the angular acceleration α _y (t) becomes larger than the value obtained by exponentially decaying the angular acceleration α _y (t _p ) at the peak time t _p . Since the value calculated on the right side of Expression 14 is attenuated immediately and becomes smaller than the threshold value Tα, the period during which this process is applied is limited. In this embodiment, a value of about 20 seconds ⁻¹ is used as the value of λ.

-About the time G At the time G, a false detection occurs due to the angular acceleration α _y (t) that increases when the right hand that pushed out the game apparatus 1 is suddenly stopped and the hand is pulled back. Although the erroneous detection here occurs in the impact detection method 1, even when the impact detection method 2 is used, it may occur when the value of Δt in Expression 13 is set large. The erroneous detection here can be prevented by applying the improvement process A-1 and the improvement process D-1. However, these improvement processes sometimes fail to detect a hit when the hit must be detected, that is, when the user performs a continuous hitting operation such as a combination punch.

・ About time point H As in time point G, the time point H is erroneously detected by the hit detection method 2 because the magnitude of the angular velocity ω _y (t) increases when the hand is pulled back by a reaction that suddenly stops the game apparatus 1. Detection occurs. At time G, applying the improvement process A-1 and the improvement process D-1 sometimes fails to detect the left and right continuous hitting operations such as a combination punch. Therefore, an attempt is made to distinguish and detect the reaction and the combination punch at time H. Therefore, first, the difference between the case where the combination punch is input and the case where the hand is pulled back by the reaction is analyzed, and then the improvement processing for preventing erroneous detection is described.

(B) When a combination punch is input First, a case where a batting operation is input twice in succession will be described. FIG. 6 is a graph showing a change in the rotation angle θ _y (t) about the Y axis when the batting operation is continuously input in the order of the right hand and the left hand. This figure shows that at the time point I, the right hand is pushed forward and the first batting operation is inputted, and at the time point J, the right hand stops and at the same time the left hand is pushed out and the second batting operation is inputted.

In this case, the first right-hand hitting operation can be detected without any problem, but the second left-hand hitting detection becomes a problem. This is because the hitting operation may fail to be detected at time J due to the effects of the improvement process A-1 and the improvement process D-1. Therefore, the angular velocity ω _y (t), which remains large for a while from time J to time M, is detected using the impact detection method 2, and this is detected as the angular velocity due to the reaction at time H in FIG. It must be distinguished.

Here, FIG. 7 is a graph when a striking operation is strongly input with the right hand in order to emphasize the erroneous detection at the time point H in FIG. 5, and FIG. 8 is a striking operation with the right hand twice in succession. It is a graph of time. In FIG. 7, the game apparatus 1 is pushed forward vigorously at time N, and the rotation angle θ _y (t) around the Y axis changes greatly from reaction time O to time P. For this reason, even if the threshold value Tω in the hit detection method 2 is increased, erroneous detection occurs in the section between the O time point and the P time point. However, as can be seen in FIG. 7, the interval T1 between the time O and the time P from the start of the reaction until the angular velocity becomes small is characterized by a very short time (about 0.05 seconds). In the embodiment, the following improvement processing is applied using this feature.

(Improvement processing H-1)
The improvement process H-1 is to prevent the hit detection method 2 from detecting the hit operation in the hit detection method 2 for a very short period from the time when the rotation angle θ _y (t) around the Y axis reaches the peak. It is. Taking FIG. 7 as an example, the striking motion is not detected during the minute period ε from time O when the rotation angle θ _y (t) around the Y axis reaches the peak. This minute period ε is set to about 0.055 seconds, which is slightly longer than T1. If this is set too long, a combination punch cannot be detected.

In the combination punch shown in FIG. 6, the angular acceleration α _y (t) is still small at the time point K and the angular velocity ω _y (t) remains large for a while. However, in the single punch shown in FIG. At the time P, the angular acceleration α _y (t) in the reverse direction has already been applied, and deceleration has started. Therefore, by predicting the angular acceleration ω ′ _y (t + Δt) slightly ahead using the angular acceleration α _y (t) in Equation 13, the difference between the two can be better distinguished.

If the angular velocity detection unit 205 is operating at a frequency of about 60 Hz, the angular velocity is detected at intervals of about 0.017 seconds, because this is longer than the difference between the section T1 and the minute period ε. Depending on the timing at which the detection process is performed, there is a possibility that an erroneous detection occurs at the H time or the P time, or that the combination punch detection at the K time fails. Therefore, in order to increase the accuracy of the improvement process H-1, it is necessary to more accurately know the time when the rotation angle θ _y (t) around the Y axis reaches the peak. For example, in FIG. 7, the peak is detected at time O, but it can be estimated that it is a time before that that actually falls on the peak. Therefore, the time at which the peak is detected and t _j, and the time of detecting the angular velocity before that one and t _j-1, as follows, the angular velocity of the time t _j and time t _j-1 at each time point by internally dividing by the size ratio of, estimating the time t _p that of us to peak.
t _p = (| ω _y (t _j ) | t _j-1 + | ω _y (t _j-1 ) | t _j ) / (| ω _y (t _j-1 ) | + | ω _y (t _j ) |) ...... (Formula 15)
Further, the time when the angular velocity is detected for the first time after elapse of a minute period ε from the time t _p when the peak is reached is assumed to be t _ε, and at the time t _ε , the angular velocity value is estimated by the following equation instead of the equation 13. To do.
ω ′ _y (t _p + ε + Δt) = ω _y (t _ε ) + α _y (t _ε ) (Δt + t _p + ε−t _ε ) (Equation 16)
At time t _epsilon, by detecting the batting operation compared to a threshold Tω resulting omega _'y a (t _p + ε + Δt) in equation 16, not so much affected by the timing of the angular velocity detecting unit 205 is operated In addition, the improvement process H-1 can be applied.

(C) When the batting operation is performed twice with the same hand Next, the case where the batting operation is performed twice with the same hand as shown in FIG. 8 will be described. In FIG. 8, the first batting operation is input at the time point Q, and the right hand is pulled back by reaction from the time point R to the time point S. From time S to time T, the second preliminary operation is performed with the reaction force, and the second batting operation is input at time T.

FIG. 8 and FIG. 6 are very similar and seem to be difficult to distinguish, but in FIG. 8, the time between the point R and the point S where the peak is detected is short, and at the point S, deceleration starts. In contrast, the preliminary operation is gradually started and the time point T is reached. In FIG. 6, since the steep slope continues from immediately after the time point J to immediately before the time point M, a large angular velocity is maintained. By applying the processing H-1, these differences can be determined, and erroneous detection at the point S can be prevented.

Further, in the combination punch of FIG. 6, the game device 1 is rotated by pushing out the opposite hand while pulling back the hand put out by the input of the first batting operation. Since the angular velocity is higher than when pulling back for preliminary operation, the threshold value Tω of the hit detection method 2 is set to a slightly larger value so that erroneous detection at the time S can be prevented more reliably. become. By setting the threshold value Tω of the hit detection method 2 to a slightly larger value, it becomes difficult to detect the input of the normal hit operation, but there is no problem because the input of the normal hit operation can be detected by the hit detection method 1. .

(4) Other motion control The motion control unit 210 also has a function of controlling the motion of an object different from the batting motion and the preliminary motion. For example, in a boxing game, the gyro sensor 164 is placed on the player object to the opponent. It can be used for the control that makes the operation to avoid hitting.

As the avoiding operation, as shown in FIG. 4B, when the game apparatus 1 is tilted forward, the object is crouched to avoid punching, and when the game apparatus 1 is tilted backward, the object When the game apparatus 1 is tilted to the right and the game apparatus 1 is tilted to the right, the object body is tilted to the right to avoid the punch, and when the game apparatus 1 is tilted to the left, the object body is It is controlled to tilt to the left to avoid punching.

For such an operation, the motion control unit 210 uses the rotation matrix of the game apparatus 1 obtained by the angular velocity detection unit 205 for A (t). Specifically, first, the vector y is set as a unit vector in the Y-axis direction as in the following equation.
y = (0,1,0) (Equation 17)
Then, the rotation matrix of the game apparatus 1 at the start of the game is A _0, and its inverse matrix is A ₀ ⁻¹ . Then, a vector y ′ (t) obtained by rotating the unit vector y in the Y-axis direction by A (t) A ₀ ⁻¹ is obtained as follows.
y ′ (t) = A (t) A ₀ ⁻¹ y (Equation 18)
Since the vector y ′ (t) calculated by Expression 18 represents how much the Y axis of the game apparatus 1 is tilted from the start of the game, the X component of the vector y ′ (t) is detected as the left / right tilt. And the Z component of the vector y ′ (t) can be used to detect the forward / backward inclination.

The slope information obtained in this way can be used as an analog value for object control as it is. For example, in a tennis game, the X component and the Z component of the vector y ′ (t) can be used as they are for the front / rear / left / right movement amount of the object. Also in the boxing game, the angle at which the object tilts (swayes) the body may be controlled according to this value.

For example, in a boxing game, such processing is not used, and the punch avoidance action may be taken when the X component or the Z component of the vector y ′ (t) exceeds a certain threshold value. . In this case, since it takes time until the inclination actually exceeds the threshold after the user operates the game apparatus 1 in an attempt to take an avoidance action, it is necessary to detect an input of a punch avoidance operation by the user. There is a slight delay.

Therefore, in this embodiment, the vector y ′ (t + Δt) after a minute time Δt is predicted using the angular velocity vector ω (t) and the angular acceleration vector α (t) obtained from the change amount per unit time. To eliminate the delay. Therefore, first, how much the game apparatus 1 rotates from the current direction after a minute time Δt is predicted as the rotation vector Δr as follows.
Δr = ω (t) Δt + 0.5 × α (t) Δt ² (Equation 19)
Then, the rotation vector Δr at the minute time Δt seconds predicted by Expression 19 is decomposed into the rotation magnitude Δθ and the rotation axis vector a as follows.
Δθ = | Δr |, (Equation 20)
a = Δr / Δθ (Formula 21)
Using the rotation angle Δθ and the rotation axis vector a thus obtained, the unit vector y in the Y-axis direction of Equation 17 is rotated as a vector y ″, and the vector y ″ is set as follows: Ask.
y ″ = (1−cosΔθ) (a · y) a + (cosΔθ) y + (sinΔθ) (a × y) (Equation 22)
Here, (a · y) represents the inner product of the vector a and the vector y, and (a × y) represents the outer product of the vector a and the vector y. Then, by rotating the vector y ″ obtained by the equation 22 instead of the vector y of the equation 18 by the rotation matrix A (t) A ₀ ⁻¹ , the vector y ′ (t + Δt after the minute time Δt is as follows. ).
y ′ (t + Δt) = A (t) A ₀ ⁻¹ y ″ (Expression 23)
By using y ′ (t + Δt) obtained by Equation 23 instead of y ′ (t), the delay in detecting the input of the punch avoidance operation can be eliminated. In the present embodiment, the minute time Δt is set to 0.05 seconds.

As described above, since the user can concentrate on moving the hand holding the game apparatus 1 by controlling the object with the gyro sensor 164 for operations other than the striking operation and the preliminary operation, other operations such as a button are used. You will be able to operate the object as you want without being distracted by the operation.

(5) Action Type Selection Function Further, the hit detection unit 211 has a function of changing the hit type and the defense type in accordance with the presence or absence or type of an input signal to the input unit such as the touch panel 300. For example, in the case of a boxing game, the hit type may be jab, straight, hook, body blow, upper cut, and the defense type may be a head card or a body guard.

The hitting by the hit detection unit 211 is selected according to an operation signal to the touch panel 300 from the operation position detection unit 201a. Specifically, as shown in FIG. 9, the screen has

selection areas

306 and 307 for selecting the type of batting and defense action for the right hand and the left hand, respectively. Here, the selection area 306 on the right side of the screen is assigned types for the right hand hitting and defense, and the selection area 307 on the left side of the screen is assigned the type for hitting and defending the left hand. In each of the

ranges

306 and 307, there are three areas, A

area

306a and 307a,

B area

306b and 307b, and

C area

306c and 307c, so that the action of hitting and defense differs depending on each area. Is set.

Here, in the case of a boxing game, a straight can be assigned to the

A areas

306a and 307a, a body blow can be assigned to the

B areas

306b and 307b, and an upper cut can be assigned to the

C areas

306c and 307c. That is, when inputting the batting operation, if the finger of the hand on the side where the user's batting operation is input touches any of these areas, the batting operation becomes the batting type assigned thereto. In addition, when no area is touched, the hitting operation of the hook can be performed to divide the four types of hitting. Further, if the preliminary operation is small when the batting operation is input, a jab may be hit instead of a hook or a straight.

Also, if there is no batting operation, the above area can be assigned to the defensive action. For example, while the

A region

306a, 307a is being pressed, the head is operated to be guarded, and while the

B region

306b, 307b or the

C region

306c, 307c is being pressed, the body is guarded. Make it work.

Note that the positions, shapes, and ranges of the

areas

306 and 307 shown in FIG. 9 are not limited to this, and can be variously changed according to the type of game. In this embodiment, the touch panel 300 is used to change the hit type and the defense type. However, a button may be used in a controller that does not have a touch screen.

(Operation signal analysis method)
In the user interface system of the present invention as described above, an operation signal analysis method for determining that the user has performed a batting operation based on information such as an operation signal input from the controller will be described below. . FIG. 10 is a flowchart showing an outline of the operation signal analysis method according to this embodiment, and FIG. 11 is a flowchart showing a subroutine of peak detection processing according to this embodiment. FIG. 12 is a flowchart showing a subroutine of the hit detection process 1 according to the embodiment, and FIG. 13 is a flowchart showing a subroutine of the hit detection process 2 according to the embodiment.

In the present embodiment, the hit detection method 1 uses the hit detection method 1 and the hit detection processing 2 subroutine uses the hit detection method 2. The hit detection method 1 uses the improvement processing A-1 and the improvement processing C. -1 and improvement processing D-1, and the hit detection method 2 uses improvement processing A-1 and improvement processing H-1.

Further, in the hit detection method 2, the hit operation is not detected for a moment, but may be continuously detected for a while. Furthermore, since two detection methods are used in combination, an input of one hitting operation may be detected twice. Therefore, in order to prevent double punch detection, processing is performed using the punch variable and peak detection processing. The peak detection process is also used for the improvement process D-1 and the improvement process H-1. These processes will be described later.

In addition, the values of the parameters used in the present embodiment are set as follows. First, the threshold value Tα of the impact detection method 1 is 3500 degrees / second ² , the threshold value Tω of the impact detection method 2 is 360 degrees / second, and the threshold value T′ω of the improvement process C-1 is 90 degrees Set to / sec. Further, a minute time Δt of the striking detection method 2 to 0.01 seconds, the improved process A-1 of Ta to 0.1 G (the size of G is gravitational acceleration), the improvement process D-1 lambda 20 seconds ^{- First} , the minute period ε of the improvement process H-1 is set to 0.055 seconds, and the constant T ″ ω in the peak detection process is set to 180 degrees / second. The constant T ″ ω is a constant used in the peak detection process for detecting the moment when the rotation of the game apparatus 1 stops. The constant T ″ ω is the angular velocity around the Y axis of the game apparatus 1 between the peaks. If the maximum value ω _max does not exceed T ″ ω, the improvement process H-1 is not started.

(1) Overview of Operation Signal Analysis Method An overview of the operation signal analysis method in the present embodiment will be described.
First, in the first step, the hit detection unit 211 sets initial values for each variable necessary for the hit detection process (S101). Specifically, the start time is set as a variable t _prev and the angular velocity around the Y axis of the game apparatus 1 at the start time is set as a variable ω _prev (ω _prev = ω _y (t _prev )). Further, the peak elapsed time t _peek , which is the elapsed time since the peak was detected, is set to 0 (t _peek = 0). Further, a variable ω _max representing the maximum value of the angular velocity around the Y axis of the game apparatus 1 between the peaks and a variable α _peek representing the angular acceleration around the Y axis at the time of the most recent peak are set to 0. Initialize (ω _max = 0, α _peek = 0). At the start time, it is set that neither the left or right batting operation is detected (punch = NONE), and the command queue is initialized.

When the variable initialization is completed, the application execution unit 202 generates an object to be arranged in the virtual space and sets an initial state (S102), and enters a main loop process. When the main loop is entered, a virtual space and object display information generation step (S103) is first performed. In this process, the display information generation unit 204 generates information for displaying the virtual space and the object on the LCD 165, synchronizes with the LCD 165, and waits for the previous display information update to be completed on the LCD 165. The newly generated display information is transferred to the LCD 165. Here, in order to synchronize with the LCD 165, the main loop is repeatedly executed at a frequency of the refresh rate of the LCD 165 (for example, about 60 Hz).

Next, the hit detection unit 211 reads information related to the motion of the game apparatus 1 calculated by the angular velocity detection unit 205, and sets the read value as a variable necessary for the hit detection process (S104). Specifically, the current time is set to t _cur and the angular velocity ω _y (t _cur ) around the Y axis of the game apparatus 1 is set to the variable ω _cur . Further, the Z component of acceleration applied to the game apparatus 1 is set to the variable a _z .

In the next step (S105), an elapsed time dt from the time t _prev at which the previous angular velocity was read to the current time t _cur is calculated (dt = t _cur −t _prev ), and the obtained elapsed time dt is calculated. Utilizing this, the angular acceleration α _cur around the Y axis is calculated (α _cur = (ω _cur −ω _prev ) / dt).

Furthermore, in step S105, the value of the peak elapsed time t _peek is updated using the calculated elapsed time dt. Specifically, the peak elapsed time t _peek is updated with a value (t _peek = t _peek + dt) obtained by adding the t _peek before the update and the elapsed time dt.
Further, the angular acceleration α _peek around the Y axis detected at the peak time is attenuated as in the right side of Expression 14 (α _peek = α _peek · exp (−λdt)).

Thereafter, the hit detection unit 211 performs a series of hit detection steps (S106 to S108). After each of these processes is executed, the current time and angular velocity are stored in the variable t _prev and the variable ω _prev for the next main loop process, and the value of ω _max is updated (S109). . Here, the value of ω _max is updated with the larger value by comparing the value of ω _max before the update with the current magnitude of the angular velocity around the Y axis | ω _cur |. Since the value of ω _max is reset to 0 every time a peak is detected in the peak detection process (S106), the Y-axis of the game apparatus 1 detected since the previous peak was detected until now. It represents the maximum value of the angular velocity around. When this step ends, the process moves to the operation control unit 210, and the operation control step (S110) is executed. In the motion control step, the punch command generated by the hit detection unit 211 is received from the command queue, and if the object is in a state where the hit operation can be started, the punch command is taken out from the command queue and the object starts the hit operation. .

In the motion control step, the motion control unit 210 responds to the value of the rotation angle θ (t _cur ) around the Y axis of the game apparatus 1 calculated by the angular velocity detection unit 205 if the object is not in the hitting motion state. To cause the object to perform a preliminary motion related to the striking motion. Further, in the action control step, the action of avoiding hitting the object is also controlled according to the degree of inclination of the game apparatus 1.

When the operation control step ends, the application execution unit 202 confirms the progress of the game (S111). If the game has not ended yet ("NO" in S111), the first step of the main loop (S103) If the game is over ("YES" in S111), the process is terminated.

(2) Peak Detection Process Next, the peak detection process in S106 will be described (FIG. 11). First, the hit detection unit 211 determines whether or not the rotation of the game apparatus 1 has stopped and the rotation angle around the Y axis of the game apparatus 1 has reached a peak from the previously detected angular velocity and the angular velocity detected this time ( S201). Specifically, the peak is obtained when the multiplication value of the angular velocity ω _prev around the Y axis detected when the operation control unit 210 was executed last time and the angular velocity ω _cur around the Y axis at the current time is positive or negative. It is determined whether or not it is detected (ω _cur × ω _prev ≦ 0).

When the multiplication value of the angular velocity ω _prev detected last time and the angular velocity ω _cur detected this time is positive (“NO” in S201), it is determined that no peak is detected, and the process is terminated. On the other hand, when the multiplication value is 0 or less (“YES” in S201), it is determined that the peak has been detected because the positive / negative is reversed between the angular velocity ω _prev detected last time and the angular velocity ω _cur detected this time. The maximum value ω _max of the magnitude of the angular velocity around the Y axis detected between the previous peak and the current peak is confirmed (S202).

If the value of ω _max is larger than the constant T ″ ω (“YES” in S202), there is a possibility that the game apparatus 1 has rotated more rapidly as the reaction becomes larger, so the elapsed time from the peak detection time After t _peek and the angular acceleration α _peek around the Y axis at the peak time are set (S203), the process proceeds to step S204. Thereby, application of the improvement process H-1 and the improvement process D-1 is started. On the other hand, when the value of ω _max is equal to or smaller than the constant T ″ ω, the game apparatus 1 is not rotating as the reaction becomes larger, so the elapsed time t _peek from the peak detection time and the Y axis around the peak time The process proceeds to the next step S204 without setting the angular acceleration α _peek .

In the process of setting the elapsed time t _peek in step S203, the peak arrival estimated time t _p according to Equation 15 is used. In other words, even if the peak was detected at the current time, the peak was actually exceeded a little before the current time, and that time was estimated and the elapsed time from the estimated time to the current time _Is set to t _peek .

In step S204, it is determined whether or not the game apparatus 1 has been rotated to the left (counterclockwise) immediately before the rotation is stopped (S204). Here, when the previously detected angular velocity ω _prev is greater than 0 (“YES” in S204), it is determined that the rotation is counterclockwise (counterclockwise) from the previous peak detection. The fact that the game apparatus 1 is rotating to the left (counterclockwise) means that the punching operation is performed with the right hand.

Therefore, the hit detection unit 211 determines whether a hit operation with the right hand is detected (punch == RIGHT) (S206). If a hitting operation with the right hand is detected (“YES” in S206), the variable punch is reset (punch = NONE) assuming that the hitting operation has ended (S207). When the hit operation with the right hand is not detected (“NO” in S206), the process proceeds to step S208 without resetting the variable punch.

On the other hand, when the angular velocity ω _prev detected last time is 0 or less (“NO” in S204), it is determined that the rotation is clockwise (clockwise) from the previous peak detection. The fact that the game apparatus 1 is rotating to the right (clockwise) means that the left hand is performing the punching operation.

Therefore, the hit detection unit 211 determines whether a hitting operation with the left hand is detected (punch == LEFT) (S205). If a hitting operation with the left hand is detected (“YES” in S205), the variable punch is reset (punch = NONE) assuming that the hitting operation has ended (S207). On the other hand, when the left hand hitting operation is not detected (“NO” in S205), the process proceeds to step S208 without resetting the variable punch.

In step S208, since the peak is detected, the variable ω _max representing the maximum value of the angular velocity around the Y axis detected between the peaks is reset, and the process is terminated.

(3) Impact detection process 1
Next, the hit detection process 1 in S107 will be described (FIG. 12). In this process, the hit detection method 1 is combined with the improvement process A-1, the improvement process C-1, and the improvement process D-1 to detect a hitting action.

First, as shown in FIG. 12, the hit detection unit 211 adds the condition of the improvement process A-1 to the hit detection method 1, and determines whether or not the detected angular acceleration is due to a preliminary operation. Specifically, it is determined whether the Z component a _{z of the} acceleration acquired in step S104 is larger than a threshold value −Ta (S301). Here, when the acceleration Z component a _z is larger than the threshold value −Ta (“YES” in S301), it is determined that the detected angular acceleration is due to the preliminary operation, and the punch detection processing 1 is terminated. . On the other hand, when the Z component a _{z of} acceleration is equal to or less than the threshold −Ta (“NO” in S301), it is determined that the detected angular acceleration is due to the batting operation.

Next, the hit detection unit 211 adds the conditions of the improvement process C-1 and the improvement process D-1, and the angular velocity ω _cur around the Y axis of the game apparatus 1 calculated in S104 and the game apparatus calculated in S105. The angular acceleration α _cur around the Y axis is the same as the angular acceleration α _cur around the Y axis, and the angular velocity ω _cur around the Y axis exceeds a certain threshold value T′ω, and the angular acceleration α _cur is equal to the threshold value Tα. Then, it is determined whether the angular acceleration α _peek at the peak time attenuated with time in step S105 is _exceeded (S302 and S303).

Here, first, the angular acceleration α _cur and the angular velocity ω _cur are positive with each other, and the angular acceleration α _cur exceeds the threshold value Tα and the angular acceleration α _peek at the peak time, or the angular velocity ω _cur is the threshold value. It is determined whether or not T′ω is exceeded (S302). If all the conditions are satisfied (“YES” in S302), it is determined that a batting operation has been input, and the process proceeds to step S304.

On the other hand, when the condition of S302 is not satisfied (“NO” in S302), the angular acceleration α _cur and the angular velocity ω _cur are negative with each other, and the angular acceleration α _cur is a threshold value −Tα and a peak time. It is determined whether or not the angular acceleration α _{peek at is} smaller than the angular velocity ω _cur is smaller than the threshold −T′ω (S303). If all the conditions are satisfied (“YES” in S303), it is determined that a batting operation has been input, and the process proceeds to step S306. On the other hand, when the condition of S303 is not satisfied (“NO” in S303), it is determined that the hit operation is not performed, and the process is terminated.

In step S304 and step S306, a process is performed in which the same batting operation is not detected twice by the batting detection process 1 and the batting detection process 2. Specifically, in step S304, the punch variable is checked to determine whether a hit operation with the right hand has not been detected yet (punch! = RIGHT). Here, if the hit operation with the right hand has not yet been detected (punch! = RIGHT) (“YES” in S304), it is determined to hit with the right hand, and the “punch with right hand” command is entered in the command queue in step S305. , And after setting the punch variable to RIGHT, the hit detection method 1 is terminated. On the other hand, if the condition in step S304 is not satisfied (“NO” in S304), the batting detection method 1 is terminated as it is.

In step S306, the same batting operation is performed so as not to be detected twice by the batting detection process 1 and the batting detection process 2. Specifically, the punch variable is examined to determine whether a left hand hitting motion has not yet been detected (punch! = LEFT) (S306). If the left hand hitting operation has not yet been detected (punch! = LIGHT) (“YES” in S306), it is determined to punch with the left hand, and the “punch with left hand” command is added to the command queue in step S307. At the same time, the hit detection method 1 is terminated after setting the punch variable to LEFT. On the other hand, if the condition in step S306 is not met (“NO” in S306), the batting detection method 1 is terminated as it is.

In addition, in step S305 and S307, when adding a punch command to the command queue, a hitting type is selected according to the presence or absence or type of an input signal to the touch panel 300 arranged in the control device main body 100. A corresponding punch command is added to the command queue.

(4) Impact detection process 2
Next, the hit detection method 2 in S108 will be described (FIG. 13). In this process, the hit detection method 2 is combined with the improvement process A-1 and the improvement process H-1 to detect a punch. First, the hit detection unit 211 applies the improvement process H-1 and the improvement process A-1 in step S401. Specifically, it is confirmed whether the elapsed time t _peek after the peak is detected in the peak detection process exceeds a predetermined minute period ε and whether the acceleration Z component a _z is equal to or less than the threshold −Ta. Then, it is determined whether or not to perform the hit detection process (S401). If any of the conditions is not satisfied (“NO” in S401), the hit detection process is terminated so as not to erroneously detect the reaction.

Next, the hit detection unit 211 calculates the predicted value ω ′ of the angular velocity after the minute time Δt. At this time, the hit detection unit 211 determines whether the hit detection process is the first after the previous peak is detected (S402). If the difference between the peak elapsed time t _peek and the minute period ε is larger than the elapsed time dt calculated in step S105 (“NO” in S402), the predicted value ω ′ _y (t _cur + Δt) is calculated using Equation 13. Then, the variable ω ′ is set (S403). Otherwise (“YES” in S402), since the first hit detection process after the peak detection is performed, the predicted value ω ′ _y (t _peek + ε + Δt) is calculated using Expression 16 and set to the variable ω ′. (S404).

Then, it is determined whether or not the calculated value of ω ′ is larger than the threshold value Tω and a hitting action with the right hand has not been detected yet (punch! = RIGHT) (S405). If the condition of step S405 is satisfied (“YES” in S405), it is determined that punching is performed with the right hand, the “punch with right hand” command is added to the command queue, and the punch variable is set to RIGHT ( S406), the hit detection process 2 is terminated. If any of the conditions in step S405 is not satisfied (“NO” in S405), the calculated value of ω ′ is smaller than the threshold value −Tω, and the left hand hitting motion has not been detected yet ( It is determined whether punch! = LEFT) (S407).

If the condition of step S407 is satisfied (“YES” in S407), it is determined to punch with the left hand, and the “punch with left hand” command is added to the command queue,
After setting the punch variable to LEFT (S408), the hit detection method 2 is terminated. If any of the conditions in step S407 is not satisfied (“NO” in S407), the process ends with no hit detection.

In addition, in step S406 and S408, when adding a punch command to the command queue, the hit type is selected according to the presence or absence or type of an input signal to the touch panel 300 arranged in the control device main body 100. A corresponding punch command is added to the command queue.

(Operation signal analysis program)
The above-described user interface system and operation signal analysis method according to the present embodiment can be realized by executing an operation signal analysis program described in a predetermined language on the computers 71 to 74. That is, as shown in FIG. 14, this program is a personal digital assistant (PDA) integrated with a mobile phone / communication function 71, a personal computer 72 used on the client side, and arranged on the network to receive data on the client side. It is possible to easily construct a user interface system having the above-described functions by installing it on a server device 73 that provides a function or a function, a dedicated device 74 such as a game device, or an IC chip 86 and executing it on the CPU. And an operation signal analysis method can be executed.

That is, by installing the operation signal analysis program on the game apparatus 1 having a main body that can be operated by the user with both hands, the game apparatus 1 is input from the gyro sensor 164, the acceleration sensor 169, and the like. Based on information such as an operation signal, it is possible to execute a batting detection step for determining that a batting operation has been performed by the user, and the batting detection step is continuously detected by the gyro sensor 164. Using the series of values, a batting that detects a characteristic of the batting operation related to the rotational movement of the game apparatus 1 that may occur when the user performs the batting operation that holds the game apparatus 1 with both hands and pushes the left or right hand forward. Including the detection method, by using the hit detection method, it is determined that the user has input the hit operation It is possible to realize the function.

Further, the hit detection step uses the angular velocity of the game apparatus 1 detected by the gyro sensor 164 to realize a function of determining whether the hit operation is performed by the user's right hand or the left hand. it can. In this embodiment, as one of the hit detection methods, the angular velocity value around the Y axis detected by the gyro sensor 164, the amount of change in angular velocity value per unit time, or a series of angular velocity values are used. A method is used in which a predicted value of the angular velocity ahead of a minute time or a value obtained by combining these values is compared with a threshold value.

In addition, the game apparatus 1 further includes an acceleration sensor 169, and the hit detection step uses the acceleration detected by the acceleration sensor 169 so that the game apparatus 1 is pushed forward when the game apparatus 1 rotates. If it is determined whether the game apparatus 1 is pulled back, it is assumed that the rotational motion of the game apparatus 1 is due to the preliminary operation or the reaction after the batting operation, and the batting operation is detected. The function that suppresses is realized.

Further, the hit detection step detects the moment when the rotational motion of the game apparatus 1 is stopped by the hit operation of the user, predicts that the reaction will occur immediately thereafter, and limits the detection of the hit operation feature by the hit detection method. This realizes a function to prevent erroneous detection of the reaction after the hitting operation by the user.

In order to accurately detect the input of a continuous hitting operation by the user, the hitting detection step includes two hitting detection methods, one of which is a method of detecting the characteristics of the moment when the hitting operation is started, and the other is By adopting a method for detecting a feature during a hitting operation, it is possible to detect a continuous hitting operation. The operation signal analysis program further includes an operation control step, and strikes an object to be operated in conjunction with a rotation angle around a predetermined axis of the game apparatus 1 calculated based on information input from the game apparatus 1. A function to execute a preliminary operation related to the operation is also realized. The game apparatus 1 further includes an input step arranged on the main body of the game apparatus 1, and realizes a function of changing the type or strength of the hit in the hit detection step according to the presence or absence or type of the input signal to the input step. is doing.

And this program can be distributed through a communication line, for example, and can be transferred as a package application that operates on a stand-alone computer. Such a program can be recorded on recording media 81 to 85 readable by a personal computer. Specifically, as shown in FIG. 14, there are various magnetic recording media such as a flexible disk 83 and a cassette tape 82, an optical disk 81 such as a CD-ROM and a DVD-ROM, a USB memory 85, a memory card 84, and the like. Can be recorded on any recording medium.

(Action / Effect)
According to this embodiment, the game apparatus 1 includes the gyro sensor 164, and the gyro sensor 164 detects the movement of the user's hand holding the game apparatus 1, and according to the movement of the user's hand, Since the left and right batting operations are recognized, the object can be operated intuitively even in a game in which the batting is performed with the left and right fists, such as a boxing game, and the user can obtain a better game experience. Furthermore, in this embodiment, since the left and right of the batting operation are distinguished using the direction of the angular velocity detected by the gyro sensor 164, a single control device is sufficient, for example, widespread use such as smartphones and tablet terminals. The versatile operation device can be used, and the overall cost can be reduced.

Further, in the present embodiment, the motion control unit 210 causes the operation target object to execute a preliminary motion related to the batting operation in conjunction with the rotation angle around the predetermined axis of the game apparatus 1, so that the reaction to the batting operation is performed. The speed can be increased and the operational feeling of the game can be improved.

That is, in this embodiment, when the user inputs a batting operation, the action of pulling the hand holding the game apparatus 1 slightly forward is natural, and the action is linked to the preliminary action of punching by the object. Can make it natural to prepare to punch before the user actually enters the batting operation. As a result, the time from when the batting operation is input until the object actually punches is shortened, and the user can obtain a better game experience.

In such a method of controlling the object by detecting the movement of the user's hand, there is a possibility that erroneous detection such as detecting a hit at a timing not intended by the user may occur. Therefore, it is possible to prevent such erroneous detection.

First, in the present embodiment, the hit detection unit 211 determines whether the detection of the hit operation is actually due to the user's hit operation or the preliminary operation with the opposite hand according to the detected direction of acceleration. Thus, it is possible to prevent erroneous detection of the striking operation at the timing of the preliminary operation. If the condition that the batting operation cannot be performed until the preliminary operation is completed can be added according to the requirements of the game application, the rotation of the control device until the preliminary operation is completed without using the acceleration sensor 169. Can be determined to be due to the preliminary operation. However, in the boxing game, there is a batting operation without a preliminary operation such as jab, so it is preferable to use the acceleration sensor 169.

Further, the hit detection unit 211 detects the moment when the rotational motion of the game apparatus 1 is stopped by the hit operation of the user, and restricts the detection of the hit operation by predicting that a recoil will occur immediately thereafter. It prevents the recoil when the operation is stopped from being erroneously detected as a batting operation.

On the other hand, by limiting the detection of the hitting operation in this way, there is a problem that detection of a continuous hitting operation such as a combination punch is hindered. By using the batting detection method, one of them is a method for detecting the feature at the moment when the batting operation is started, and the other is a method for detecting the feature during the batting operation. A continuous hitting operation such as a combination punch is detected separately from a recoil when the hitting operation is stopped.

Furthermore, in the present embodiment, the hit type (strength) such as jab, straight, and hook and the defense type such as guard are selected according to the touch position of the touch panel 300 which is a detection means different from the angular velocity detection of the gyro sensor 164. Therefore, it is supplemented with the selection of the type of hit and the input of the defensive action, which are difficult to realize by the method of inputting the hit operation by the movement of the user's hand.

[Example of change]
The above description of the embodiment is an example of the present invention. For this reason, the present invention is not limited to the above-described embodiment, and various modifications and configurations can be combined depending on the design and the like as long as the technical idea according to the present invention is not deviated. Of course. Although modifications are shown below, the present invention is not limited to these modifications. In particular, the hit detection method 1 and the hit detection method 2 described above, and a method in which various improvement methods are combined with these hit detection methods, use a series of angular velocity values input from the game apparatus 1 having the gyro sensor 164. When the user performs a batting operation in which the user holds the game apparatus 1 with both hands and pushes out one of the left and right hands, any characteristic regarding the rotational movement of the game apparatus 1 is detected, and the user inputs the batting operation. Any other method can be used as long as it can be used to realize the function of determining the occurrence.

(1) Regarding the hit detection method In the embodiment described above, the hit detection method 1 using the angular acceleration and the hit detection method 2 using both the angular velocity and the angular acceleration are used in the detection of the hit operation. For example, the hitting operation by the user may be detected using another method. Hereinafter, other hit detection methods will be described.

(A) Impact detection method 3
This hit detection method 3 detects that a hit operation is input by the user using only the angular velocity. Specifically, the hit detection method 3 is to detect a punch when the angular velocity ω _y (t) of rotation about the Y axis exceeds a certain threshold value Tω. By using this method, if the threshold value Tω is increased to some extent, it becomes easier to implement because there are fewer false detections. However, until the user actually starts the batting operation until the batting operation is detected. Another disadvantage is that it takes time.

(B) Impact detection method 4
As the hit detection method 4, the hit detection method 2 described above is extended, and the angular velocity after a minute time Δt is predicted using data for the past M times. Specifically, the hit detection method 2 described above can be said to be a method of predicting the angular velocity after a minute time Δt by approximating the time change of the angular velocity ω _y (t) with a straight line using data for the past two times. In the hit detection method 4, the accuracy is further improved by using the data for the past M times and predicting the change in angular velocity using an Nth-order curve instead of a straight line. Here, the Nth order curve can be obtained by the method of least squares, but it is necessary that N + 1 <M in order to prevent overfitting, and the value is preferably about N = 2, M = 4 or 5.

Further, in this hit detection method 4, the Nth order curve obtained by the least square method is developed around the current time so that the influence of the higher order terms is reduced when the predicted value after the minute time Δt is obtained. .
That is, it is assumed that the angular velocity ω ′ _y (t + Δt) after the elapsed time Δt has the following format.
ω ′ _y (t + Δt) = a ₀ + a ₁ × Δt +... + a _N × Δt ^N (Equation 24)
Then, the values of parameters a ₀ , a ₁ , ..., a _N are determined by the least square method, and a punch is detected when the magnitude of the obtained value of ω ' _y (t + Δt) exceeds the threshold value Tω I will decide.

In this case, minimum 2 and curves obtained in the multiplication to calculate the value of the square sum of the error values for the past M times, it is assumed that this value is compared with the allowable square error E ² predetermined. If the calculated sum of squares of the error is larger than the allowable square error E ² , it is not suitable for approximating the change in angular velocity around the current time with an Nth-order curve. and small, again, repeat the least square method is repeated until the sum of squares of errors is less than the allowable square error E ^2. When N eventually becomes 1, the punch is detected by switching to the hit detection method 2. The value of the allowable error E is preferably set as a ratio (about 5% to 10%) with respect to the threshold value Tω.

(2) Error Detection Improvement Process at Other Timings Next, other error detection improvement processes in the hit detection method 1 and the hit detection method 2 described above, and the hit detection method 3 and the hit detection method 4 described in the modification example The false detection improvement process will be described.

(A) About the case where the batting motion is performed once with one hand. At time A, at the time A when the preliminary motion is started, the angular acceleration α _y (t) when the user pulls the hand holding the game device 1 is calculated. Incorrect detection may occur. Due to this erroneous detection, the user strikes with a hand opposite to the hand intended by the user. In particular, this erroneous detection may occur when any one of the hit detection method 1, the hit detection method 2, and the hit detection method 4 is used. As an improvement method in this case, the following improvement method is used.

(Improvement A-2)
In the improvement process A-2, the threshold value Tα is increased. Specifically, when the hit detection method 1 is used, erroneous detection is prevented by increasing the threshold value Tα. In this case, the threshold value Tα is set to a size that can detect the batting operation to be detected at the time point D. However, the user is excited while playing the game, and the hand movement of the game apparatus 1 tends to become intense, so if the improvement process A-1 using acceleration can be used, that person Is good.

(Improvement A-3)
In the improvement process A-3, the threshold value Tω is increased. Specifically, when any one of the hit detection methods 2 to 4 is used, erroneous detection is prevented by increasing the threshold value Tω. Also in this case, the threshold value Tω is set to a size that can detect the striking motion to be detected at the time point D. In this case, it is better if the improvement process A-1 can be used.

(Improvement A-4)
As another improvement process when the impact detection method 2 or 4 is used, the influence of the angular acceleration α _y (t) can be reduced by reducing the value of the minute time Δt. However, when the value of the minute time Δt is decreased, the detection reaction rate is decreased.

(Improvement A-5)
Further, as another improvement process, it is possible to perform a process so as not to detect the angular acceleration α _y (t) when the preliminary operation is not completed. That is, the magnitude of the rotation angle θ _y (t) around the Y axis is larger than a certain threshold value T _θ , the angular velocity ω _y (t) around the Y axis, and the rotation angle θ _y ( The striking motion is detected only when the sign differs from t). This method is effective for a tennis game or the like that does not necessarily require a preliminary operation, but cannot be applied to a boxing game with a hit without a preliminary action.

(Improvement A-6)
In the improvement process A-6, when the preliminary operation is not completed, the threshold value Tα is set large. By this process, similarly to the improvement process A-2, it is possible to prevent erroneous detection and to detect the batting action at the time point D by limiting to the preliminary operation. Moreover, although it can be applied to a boxing game or the like, it becomes difficult to input a hit like a jab without a preliminary operation.

-About time B At the time of B, the magnitude | size of preliminary operation becomes the maximum, and when using any of the impact detection methods 2 thru | or 4, a false detection may generate | occur | produce.

(Improvement process B-1)
In the improvement process B-1, the value of the threshold value Tω is set large. However, if this value is increased too much, the batting operation at the time point E cannot be detected, and the game apparatus 1 must be moved greatly in order to input the batting operation. It is assumed that the batting operation at the time point E can be detected.

(Improvement process B-2)
The improvement process B-2 is a process that does not detect the input of the batting operation in the state where the preliminary operation is not completed, like the improvement process A-5.

(Improvement B-3)
In the improvement process B-3, the threshold value Tω is increased only when the preliminary operation is not completed as in the improvement process A-6. Thereby, similarly to the improvement process B-1, it is possible to detect the striking operation at the time point E by preventing erroneous detection and limiting it to the preliminary operation.

-About time point C Since the preliminary operation has stopped at time point C, the angular acceleration α _y (t) increases, and even when the impact detection methods 2 and 4 are used, erroneous detection may occur at time point C. .

(Improvement process C-2)
The reason why erroneous detection occurs at the time point C when using the impact detection method 2 or 4 is that the value of the minute time Δt is too large. Therefore, the improvement process C-2 prevents erroneous detection at time C by reducing the value of the minute time Δt.

-Regarding the F time point At the F time point, the angular acceleration α _y (t) when the batting operation is stopped is detected as in the erroneous detection at the C time point. In particular, the hitting operation may be determined to be a hand opposite to the hand intended by the user. Here, the improvement process C-1 or C-2 is used as an improvement process for erroneous detection.

-About the time point G At the time point G, the angular acceleration α _y (t) generated when the hand is pulled back by the reaction when the right hand pushing out the game apparatus 1 is suddenly stopped is detected.

(Improvement G-1)
The improvement process G-1 is to prevent the punch from being detected for a certain time once the hitting operation is detected. Specifically, after the hitting operation is detected at time D, the hitting operation is not detected during a predetermined time interval δ. In the boxing game of the present embodiment, the object hitting operation is extremely fast, and the time interval δ cannot be set large so that the user can continuously input the hitting operation. The time interval δ is preferably set to a length of about 0.2 seconds.
Depending on the requirements of the game application, once a batting operation is input, it may not be necessary to perform the batting operation for a while. In that case, it is possible to increase the value of the time interval δ. For example, in the case of a tennis game, once the racket is shaken, it is not necessary to shake the racket for the next one second or so, so that the time interval δ can be made as large as one second.

-About the H time point At the H time point, not only the hit detection method 2 but also the hit detection methods 3 and 4 cause false detection. However, the hit detection method 3 is a special case in which the minute time Δt in the hit detection method 2 is set to zero, and the hit detection method 4 is a method in which the hit detection method 2 is further expanded. It can be applied to the hit detection method.

(Improvement processing H-2)
As another improvement process, even if a hit operation is detected by the hit detection method 2 after the rotation angle θ _y (t) around the Y axis reaches a peak, the rotation angle around the Y axis within a predetermined time. Until θ _y (t) changes sufficiently large, it is not processed as a batting operation. For example, even if a hitting operation is detected at time K in FIG. 6, detection is suspended until time L when the value of the rotation angle θ _y (t) around the Y axis crosses a line with θ _y = 0 within a predetermined time. Perform the process. This improvement process can also be used for the impact detection methods 3 and 4. In this improvement process, an erroneous detection is performed in S of FIG. 8 when the batting operation is performed twice with one hand, so that it is difficult to apply to the above-described embodiment, but it is sufficiently practical depending on the requirements of the game application. It can be a possible improvement.

(3) Change of threshold value according to user environment Various threshold values have been used in the hit detection method and improvement processing described so far. These have predetermined values, and the threshold values are changed according to the situation in the improvement process D-1 and the improvement processes A-6 and B-3. However, these threshold values may be changed according to the user's environment. That is, by changing the threshold value according to the user's muscular strength, the weight of the controller, etc., the user can make the hitting operation more easily successful. These changes may be made directly according to user settings, or the user may perform several hitting operations and measure an appropriate threshold value.

(4) If the hit detection method and improvement processing that have been discussed so far are generalized to the hit detection identification function and machine learning that integrate the above hit detection method and improvement processing into one, input of the hit operation by the user It can be identified by analyzing the change pattern (waveform) of the angular velocity ω _y (t) around the Y axis and the Z component of the acceleration of the game apparatus 1. For example, if a pattern analysis is performed on the 10 most recently detected angular velocity values and acceleration Z component values using a neural network or the like, an identification function using the neural network can be used as the hit detection method. However, this method requires a multi-layered network and takes time to learn, so it is not practical. Therefore, it is necessary to extract some features instead of using the most recently detected data for 10 times as it is. Considering the hit detection methods 1 and 2 and the improvement processes A-1, C-1, D-1, H-1, and the like, the features are the angular velocity ω _y (t) and the angular acceleration at the current time t. α _y (t), the Z component a _z of the acceleration of the game apparatus 1, the angular acceleration α _y (t _p ) at the time t _p when the rotation angle around the Y axis estimated by Equation 15 reaches its peak, and the time t angular acceleration α _y (t _p) the applied value in the elapsed time t _peek and time t _p from _{_{_{p (t peek · α y (}}} t p)), or if there is such. The value obtained by multiplying t _peek by α _y (t _p ) is used as a feature because the calculation of the attenuation of α _y (t _p ) by the right side of Equation 14 is performed using α _y (t _p ) · exp (−λt _peek ) α _y (t _p ) −λt _peek · α _y (t _p ) (Equation 25)
This is because of approximation. In our experience, a better discriminant function can be created by adding the Z component of the acceleration detected before the rotation to the feature.

In addition to the features described here, information specific to the game application may be added. For example, in the case of the embodiment of the boxing game described above, if the hitting type is selected by the touch screen, it can be predicted that there is an input of the hitting operation immediately thereafter, so that the currently selected hitting type and the previous time An elapsed time after the selection of the hit type is considered as a feature. However, since it is difficult to machine-learn the discriminant function, it is not good to increase the features unnecessarily.

The features as described above are input to the discriminant function, but the output is two Boolean values, that is, whether there has been a right hand hitting operation and whether there has been a left hand hitting operation.
Here, the discriminant function may be programmed based on the discussion so far, but it can also be estimated by machine learning using a neural network or the like. In addition, the machine learning may be able to change the threshold value according to the user environment as described above, or to absorb differences between game machines that are running game applications or games for each user. . For example, the threshold value Tα in the hit detection method 1, the threshold value Tω in the hit detection method 2, and the minute period ε of the improvement process H-1 are changed depending on the user's muscle strength, the weight and size of the game device, etc. Can detect the batting operation more accurately, and depending on the position where the acceleration sensor is arranged, one of the features is the Z component a _z of acceleration and the other feature is the angular acceleration α _y (t). It can be expected that the same process as the improvement process A-1 can be performed more accurately by changing the weights of.

In order to perform machine learning for each user's environment, it is necessary for the user to perform a batting operation determined at a specified timing. To do this, in the game application, a mini game, which is different from the original game play, is required. It is good to have you play. For example, in a game in which the user performs a batting operation according to the rhythm, the user instructs the user with the marker that flows on the screen, the timing of the batting, the type of batting, and the left / right distinction, and the user sets the line where the marker is determined. The hitting operation determined in accordance with the color and shape of the marker and the location of the marker is performed at the timing of reaching.

A machine learning teacher signal can be obtained by the mini-game as described above. However, it is necessary to devise when learning the discriminant function using this teacher signal. For example, when considering the improvement process H-1, it is difficult to distinguish between the reaction after the batting operation and the input of the combination punch operation in a very short period after the peak is detected. The hit must be detected at a timing slightly later than the timing at which it was performed. Also, the user cannot input the striking operation at the specified timing exactly like a machine. Therefore, the amount of change per unit time of the angular acceleration α _y (t) around the Y axis of the game apparatus 1 is not in the vicinity of the instructed timing, but is assumed that the batting operation is performed at the timing instructed by the mini game. Is the time when the input of the batting operation is started, and from that time until the time when the direction of the angular acceleration α _y (t) is reversed, that is, the time when the magnitude of the angular velocity falls to the maximum value It is assumed that the interval is a batting operation period, and the teacher signal detects the batting operation during the batting operation period. When learning is performed based on the teacher signal obtained in this way, data other than the batting operation period is weighted, and the batting operation does not necessarily need to be detected in all data of the batting operation period. Like that.

This kind of machine learning has the advantage of absorbing the difference in the environment for each user and making it possible to detect the batting operation more accurately, but it is necessary to prepare a mini game to obtain a teacher signal, Even if a teacher signal is obtained, there is a problem that it takes time to learn the neural network. In our experience, we were able to obtain a discriminant function with sufficient performance using a neural network with only one intermediate layer of 5 units, but about 4600 teacher signals (63 punches) were used for learning. It was necessary to learn about 5,000 batches.

In the above-described embodiment, the case where a widespread device such as a smartphone or a tablet terminal in which a display device, an arithmetic processing device, and a controller are integrated is described as an example. The device, the arithmetic processing device, and the controller are not limited to be integrated, and the controller, the arithmetic processing device, and the display device may be provided separately. In this case, for example, a smartphone or the like is used as a controller, a general-purpose computer or a game device that can be wirelessly connected to the smartphone (such as Bluetooth (registered trademark)) or a computing device as a computing device, and a television device that can be connected to the computing device as a display device. Can also be used. In this case, an object displayed on a large screen can be operated.

Furthermore, for example, a toy such as a robot capable of connecting a smartphone or the like to a controller and a wireless communication (such as Bluetooth (registered trademark)) with the smartphone can be used as an object to be operated. In this case, the object is a driving device such as a robot toy, and the motion control unit 210 transmits a control signal to the driving device.

In the present invention, the object does not necessarily need to exist in the real space or the virtual space. That is, the present invention can also be applied to an application in which the arithmetic processing unit performs a simulation that a virtual object has performed a hitting operation based on an input from the controller and outputs only the result. . For example, when a user inputs a batting operation, a virtual object not displayed on the screen performs a batting operation, and another object displayed on the screen is destroyed. A game application can be considered. As another example, a game application in which the user performs a batting operation in accordance with the rhythm and changes the display on the screen or outputs a sound effect according to the result is conceivable. In this case, the virtual object is like a performer with a drum drumstick.

DESCRIPTION OF SYMBOLS 1 ... Game device 71 ... Portable terminal 71-74 ... Computer 72 ... Personal computer 73 ... Server apparatus 74 ... Dedicated device 81 ... Optical disk 81-85 ... Recording medium 82 ... Cassette tape 83 ... Flexible disk 84 ... Memory card 85 ... USB Memory 86 ... IC chip 100 ... Control device main body 100 ... Main body 101 ... Antenna 102 ... Duplexer 103 ... Synthesizer 110 ... Low noise amplifier 111 ... Mixer 112 ... IF amplifier 113 ... Orthogonal mixer 114 ... Converter 115 ... Demodulator 116 ... Channel decoder 117 ... Audio decoder 118 ... Converter 119 ... Amplifier 120 ... Speaker 130 ... Power amplifier 131 ... Mixer 132 ... IF amplifier 133 ... Orthogonal mixer 134 ... Converter 135 ... Modulator 136 ... Catcher channel encoder 137 ... audio encoder 138 ... Converter 139 ... amplifier 140 ... microphone 150 ... time base 151 ... EEPROM
152 ... RAM
152, EEPROM 151 ... RAM
153 ... ROM
164 ... Gyro sensor 165 ... Display 165 ... Liquid crystal display 165 ... LCD
166 ... Operation buttons 167 ... LED
168 ... Touch sensor 169 ... Acceleration sensor 171 ... Power supply system battery 172 ... Power supply 173 ... Converter 174 ... Vibrator 200 ... CPU
DESCRIPTION OF SYMBOLS 201 ... Operation signal acquisition part 201a ... Operation position detection part 202 ... Application execution part 204 ... Display information generation part 204a ... 3D structure part 204b ... 2D structure part 204c ... GUI structure part 205 ... Angular velocity detection part 206 ... Operation device 210 ... Operation | movement Control unit 211 ... Impact detection unit 300 ...

Touch panel

304, 305 ...

Power gauge

306, 307 ...

Selection area

306a, 307a ... A

area

306b, 307b ...

B area

306c, 307c ... C area

Claims

A controller and an arithmetic processing unit for processing information input from the controller;
A hit detection unit that is a means for determining that a user has input a hitting operation based on information such as an operation signal input from the controller by the arithmetic processing unit or software executed by the arithmetic processing unit. A user interface system comprising:
The controller has an angular velocity sensor and a main body that a user can operate with both hands,
Using the series of values of the angular velocity of the controller, which is continuously detected by the angular velocity sensor, the batting detection unit performs a batting operation in which the user holds the controller with both hands and pushes either the left or right hand forward. A hit detection method for detecting characteristics of the hit operation related to the rotational movement of the controller that may occur at the time,
A user interface system that realizes a function of determining that a user has input a batting operation by using the batting detection method.
The hit detection unit has a function of determining whether the hitting operation is performed by a user's right hand or a left hand by using an angular velocity of the controller detected by the angular velocity sensor. The user interface system according to claim 1.
As one of the hit detection methods,
A value of an angular velocity around a predetermined axis that can be regarded as a natural rotation axis of the controller when the striking operation is performed detected by the angular velocity sensor, or a change amount per unit time of a value of the angular velocity, or a series of The value of the angular velocity predicted by using the angular velocity value, or the value obtained by combining these values, can be changed according to the predetermined or situation, or changed according to the user settings and the user environment 3. User interface system according to claim 1 or 2, characterized in that it uses a method for comparing with a threshold.
The controller further includes an acceleration sensor inside,
The hit detection unit uses the acceleration detected by the acceleration sensor to determine whether the controller is pushed forward or pulled back when the controller rotates. 4. The controller according to claim 1, wherein the controller has a function of suppressing detection of a striking operation on the assumption that the rotational motion of the controller is caused by a pre-operation or a reaction after the striking operation when it is determined that the striking operation has been pulled back. A user interface system according to any of the above.
By detecting the moment when the rotational motion of the controller due to the user's batting operation stops and predicting that a recoil will occur immediately thereafter, the detection of the characteristics of the batting operation by the batting detection method is limited, and the batting operation performed by the user 5. The user interface system according to claim 1, further comprising: a function that prevents a subsequent reaction from being erroneously detected.
In order to accurately detect an input of a continuous hitting operation by a user, the hitting detection unit includes at least two hitting detection methods, one of which is a method of detecting a feature at the moment when the hitting operation is started. 6. The user interface system according to claim 4, wherein a method for detecting a feature during a hitting operation is used.
A function of causing the object to be operated to perform a preliminary motion related to a striking motion in conjunction with a rotation angle around a predetermined axis of the controller calculated based on information input from the controller, or 7. The user interface system according to claim 1, wherein the user interface system has a function of transmitting a control signal for executing a preliminary operation.
The controller further comprises an input unit arranged in the controller body,
The user interface according to any one of claims 1 to 7, wherein the hit detection unit has a function of changing the type or strength of the hit in accordance with the presence or absence or type of an input signal to the input unit. system.
An operation signal analysis method in a user interface system having a batting detection step for determining that a batting operation is input from a user based on information such as an operation signal input from a controller,
The controller has an angular velocity sensor and a main body that a user can operate with both hands,
In the hit detection step, using a series of values of the angular velocity of the controller continuously detected by the angular velocity sensor, a hit operation is performed in which the user holds the controller with both hands and pushes the left or right hand forward. A batting detection method for detecting characteristics of the batting operation relating to the rotational movement of the controller that may occur in the event of
An operation signal analysis method characterized by realizing a function of determining that a user has input a batting operation by using the batting detection method.
The hit detection step realizes a function of determining whether the hitting operation is performed by a user's right hand or a left hand by using an angular velocity of the controller detected by the angular velocity sensor. The operation signal analysis method according to claim 9.
As one of the hit detection methods,
A value of an angular velocity around a predetermined axis that can be regarded as a natural rotation axis of the controller when the striking operation is performed detected by the angular velocity sensor, or a change amount per unit time of a value of the angular velocity, or a series of The value of the angular velocity predicted by using the angular velocity value, or the value obtained by combining these values, can be changed according to the predetermined or situation, or changed according to the user settings and the user environment 11. The operation signal analyzing method according to claim 9, wherein a method for comparing with a threshold value is possible.
The controller further includes an acceleration sensor inside,
The hit detection step uses the acceleration detected by the acceleration sensor,
When the controller performs a rotational motion, it is determined whether the controller is pushed forward or pulled back. If it is determined that the controller is pulled back, the rotational motion of the controller is a preliminary operation. The operation signal analysis method according to claim 9, further comprising a function of suppressing detection of a batting operation as a result of a reaction after the batting operation.
The hit detection step detects the moment when the rotational motion of the controller due to the hit operation of the user stops, and predicts that a recoil will occur immediately thereafter, thereby limiting the detection of the hit operation feature by the hit detection method. The operation signal analysis method according to claim 9, further comprising: a function for preventing erroneous detection of a reaction after a hitting operation by the user.
In order to accurately detect the input of continuous hitting operations by the user, the hit detection step uses at least two hit detection methods, one is a method for detecting the characteristics of the moment when the hit operation is started, and 14. The operation signal analysis method according to claim 12 or 13, wherein a method for detecting a feature during a hitting operation is used.
The operation signal analyzing method causing the object to be operated to perform a preliminary motion related to a striking motion in conjunction with a rotation angle around a predetermined axis of the controller calculated based on information input from the controller; The operation signal analysis method according to claim 9, further comprising a step of transmitting a control signal for executing a preliminary operation.
The controller further comprises an input unit arranged in the controller body,
The operation according to any one of claims 9 to 15, wherein the hit detection step realizes a function of changing a hit type or strength according to presence or absence or type of an input signal to the input unit. Signal analysis method.
An operation signal analysis program in a user interface system having a batting detection step for determining that a batting operation is input from a user based on information such as an operation signal input from a controller,
The controller has an angular velocity sensor and a main body that a user can operate with both hands,
In the hit detection step, using a series of values of the angular velocity of the controller continuously detected by the angular velocity sensor, a hit operation is performed in which the user holds the controller with both hands and pushes the left or right hand forward. A batting detection method for detecting characteristics of the batting operation relating to the rotational movement of the controller that may occur in the event of
An operation signal analysis program that realizes a function of determining that a user has input a batting operation by using the batting detection method.
The hit detection step realizes a function of determining whether the hitting operation is performed by a user's right hand or a left hand by using an angular velocity of the controller detected by the angular velocity sensor. The operation signal analysis program according to claim 17.
As one of the hit detection methods,
A value of an angular velocity around a predetermined axis that can be regarded as a natural rotation axis of the controller when the striking operation is performed detected by the angular velocity sensor, or a change amount per unit time of a value of the angular velocity, or a series of The value of the angular velocity predicted by using the angular velocity value, or the value obtained by combining these values, can be changed according to the predetermined or situation, or changed according to the user settings and the user environment The operation signal analysis program according to claim 17 or 18, wherein a method for comparing with a threshold value is possible.
The controller further includes an acceleration sensor inside,
The hit detection step uses the acceleration detected by the acceleration sensor,
When the controller performs a rotational motion, it is determined whether the controller is pushed forward or pulled back. If it is determined that the controller is pulled back, the rotational motion of the controller is a preliminary operation. The operation signal analysis program according to any one of claims 17 to 19, wherein a function for suppressing detection of a hitting operation is realized as a result of a reaction after the hitting operation.
The hit detection step detects the moment when the rotational motion of the controller due to the hit operation of the user stops, and predicts that a recoil will occur immediately thereafter, thereby limiting the detection of the hit operation feature by the hit detection method. 21. The operation signal analysis program according to claim 17, wherein the operation signal analysis program realizes a function to prevent erroneous detection of a reaction after a hitting operation by the user.
In order to accurately detect the input of continuous hitting operations by the user, the hit detection step uses at least two hit detection methods, one is a method for detecting the characteristics of the moment when the hit operation is started, and The operation signal analysis program according to claim 20 or 21, wherein a method for detecting a feature during a single hitting operation is used.
The operation signal analysis program causing the object to be operated to perform a preliminary motion related to a striking motion in conjunction with a rotation angle around a predetermined axis of the controller calculated based on information input from the controller; The operation signal analysis program according to any one of claims 17 to 22, further comprising a step of transmitting a control signal for executing a preliminary operation.
The controller further comprises an input unit arranged in the controller body,
The operation according to any one of claims 17 to 23, wherein the hit detection step realizes a function of changing a hit type or strength according to presence or absence or type of an input signal to the input unit. Signal analysis program.