JP6031801B2 - Performance device, method and program - Google Patents

Description

  The present invention relates to a performance device, a method, and a program.

2. Description of the Related Art Conventionally, there has been proposed a performance device that generates a sound corresponding to a performance operation when the performance operation of the performer is detected. For example, a performance device that produces percussion instrument sounds with only a stick-shaped member is known. In this performance device, a sensor is provided on the stick-shaped member, and the player holds the member by hand. By shaking, the sensor detects the performance and produces a percussion instrument sound.
According to such a virtual performance device, the musical sound of the musical instrument can be generated without the need for an actual musical instrument, so that the performer can enjoy the performance without being restricted by the performance place or performance space. be able to.

  As a virtual performance device as described above, for example, in Patent Document 1, an acceleration sensor is provided on a stick-shaped member, and an output (acceleration value) obtained from the acceleration sensor by shaking this member reaches a predetermined threshold value. Then, a performance device configured to generate a musical tone is disclosed.

JP 2007-256736 A

  However, in Patent Document 1, there is no description of determining the sound volume when sounding, and if the invention of Patent Document 1 is applied as it is, the sound volume must be determined according to the magnitude of acceleration. However, the shot strength is determined according to the acceleration in the deceleration direction, but the speed is obtained by integrating the acceleration, so if an error occurs in the acceleration, the error is integrated, and the stick keeps moving. As a result, the accuracy of the speed is remarkably deteriorated, and it becomes impossible to distinguish between deceleration and acceleration.

  For this reason, there is a restriction to place a virtual sound source only on a virtual horizontal plane, and since the direction opposite to the direction of gravity is the deceleration direction, a method has been proposed for determining the amount of sound produced by the stick from the acceleration component in that direction. Yes. However, this method has a problem that it cannot be applied when virtual sound sources are arranged vertically in a virtual space.

  The present invention has been made in view of such a situation, and even when a virtual sound source is arranged vertically in a virtual space, an appropriate volume that reflects the strength of the swing motion is reflected. An object is to provide a performance device that can be controlled.

In order to achieve the above object, a performance device according to one aspect of the present invention includes:
A holding member that can be held by the user;
An acceleration sensor that detects acceleration generated in the directions of the three axes perpendicular to each other of the holding member;
An angular velocity sensor that detects an angular velocity that occurs when rotating about each of the three orthogonal axes;
First acquisition means for acquiring a maximum value of the angular velocity of the holding member during a performance operation based on an angular velocity generated corresponding to the swing-down operation of the holding member detected by the angular velocity sensor;
Second acquisition means for acquiring a maximum acceleration value of the holding member based on an acceleration generated in response to a swing-down operation of the holding member detected by the acceleration sensor;
Third acquisition means for acquiring a maximum value of the speed of the holding member based on acceleration generated in response to the swing-down operation of the holding member detected by the acceleration sensor;
Based on the maximum value of the angular velocity acquired by the first acquisition unit, the maximum value of the acceleration acquired by the second acquisition unit, and the maximum value of the velocity acquired by the third acquisition unit. Volume control means for controlling the volume of the musical sound to be generated,
It is characterized by that.

  According to the present invention, even when a virtual sound source is arranged vertically in the virtual space, appropriate volume control is possible reflecting the strength of the swing motion.

It is a figure which shows the outline | summary of one Embodiment of the performance apparatus of this invention. It is a block diagram which shows the structure of the stick part of the said performance apparatus. It is a block diagram which shows the structure of the sound generation part of the said performance apparatus. It is a figure explaining the detail about the stick part of the said performance apparatus. It is a figure explaining the world coordinate system which concerns on one Embodiment of the performance apparatus of this invention. It is a flowchart which shows the flow of the process which CPU of the said stick part performs. It is a flowchart which shows the flow of an attitude | position sensor and an initialization process of an attitude | position. It is a flowchart which shows the flow of an attitude | position estimation process. It is a flowchart which shows the flow of a position estimation process. It is a flowchart which shows the flow of a front-end | tip movement prediction process. It is a figure which shows the relationship between the stick tip rotating shaft of the said stick part, and a swing direction. It is a flowchart which shows the flow of the sound generation timing detection process in a 1st Example. It is a flowchart which shows the flow of the sound generation timing detection process in a 2nd Example. It is a figure which shows the transition of the operation state of the said stick part. It is a flowchart which shows the flow of a pronunciation | sound_production amount detection process. It is a flowchart which shows the flow of a note event production | generation process. It is a flowchart which shows the flow of the sound search process in a modification. It is a flowchart which shows the flow of a sound generation process.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

Embodiments of the present invention will be described below with reference to the accompanying drawings.
FIG. 1 is a diagram showing an outline of a performance device 1 according to an embodiment of the present invention. As shown in FIG. 1, the performance device 1 of the present invention includes a stick unit 10 and a sound generation unit 30.
The stick part 10 is a stick-like member extending in the longitudinal direction. The performer performs a performance operation by holding up one end (base side) of the stick unit 10 and performing a swing-up and swing-down operation around the wrist or the like. In order to detect such a player's performance, a posture sensor 12 is provided at the other end (tip side) of the stick unit 10 (see FIG. 2).

The sound generator 30 generates a musical sound.
In such a performance device 1, the sound generator 30 generates a predetermined musical sound in response to a performance operation using the stick unit 10, that is, a swing-down operation of the stick unit 10. Thereby, a performance by a performer is performed.

[Configuration of the performance device 1]
First, with reference to FIG.2 and FIG.3, each component of the performance apparatus 1 of this embodiment, specifically, the structure of the stick part 10 and the sound generation part 30, is demonstrated.

[Configuration of Stick Unit 10]
FIG. 2 is a block diagram showing the configuration of the stick unit 10. As shown in FIG. 2, the stick unit 10 includes a CPU (Central Processing Unit) 11, an attitude sensor 12, an I / F (interface) device 13, a ROM (Read Only Memory) 14, a RAM (Random Access Memory) 15, and an input. The unit 16 includes a wireless communication device 17 and an LED 18.
The posture sensor 12 includes a triaxial magnetic sensor 121 that measures the geomagnetic direction, a triaxial acceleration sensor 122 that measures acceleration applied to the stick, and a triaxial angular velocity sensor 123 that measures the rotational movement of the stick. The Each sensor can measure in the axial direction of three axes x, y, and z.

Referring to FIG. 4, in this embodiment, an axis that coincides with the longitudinal axis of the stick portion 10 is defined as a y-axis, parallel to a substrate (not shown) on which the attitude sensor 12 is disposed, and the y-axis. The axis orthogonal to the x axis is the x axis, and the axis orthogonal to the x axis and the y axis is the z axis. The magnetic sensor 121 can acquire the geomagnetic value of each component of the x-axis, y-axis, and z-axis of the geomagnetic vector. Further, the acceleration sensor 122 can acquire acceleration values of components of the x-axis, y-axis, and z-axis. Further, the angular velocity sensor 123 can acquire values (ωx, ωy, ωz) of angular velocity components centered on the respective axes.
Here, the performer holds one end (base side) of the stick unit 10 and performs a swing-up and swing-down operation around the wrist or the like, thereby causing a rotational motion or a parallel motion to the stick unit 10. Let These movements are detected by the attitude sensor 12 and sent to the CPU 11.

  Returning to FIG. 2, the CPU 11 reads the acceleration value, angular velocity value, geomagnetic direction and intensity detected by the posture sensor 12, and detects the posture of the stick and the timing of sound generation according to these values. Thereby, the CPU 11 generates a note-on event corresponding to the sound generation, and transmits the note-on event to the sound generation unit 30 via the I / F device 13 and the wireless communication device 17. Here, the note-on event is command information for instructing the tone generator 371 (see FIG. 3) to generate a musical tone, and in this embodiment, in addition to the tone color and pitch information of the musical tone to be generated. In addition, it includes information such as the sound volume.

  In this way, the CPU 11 transmits a note-on event to the sound generation unit 30 based on each sensor value of the posture sensor 12 included in the stick unit 10, so that the sound generation unit 30 is based on a performance operation using the stick unit 10. Can produce a predetermined tone.

  The ROM 14 stores processing programs for various processes executed by the CPU 11, that is, processing programs such as data acquisition by the attitude sensor 12, note-on event generation processing, and note-on event transmission processing as described above. The RAM 15 stores each sensor value of the attitude sensor 12 and a value acquired or generated in the processing. The I / F 13 outputs data to the wireless communication device 17 in accordance with an instruction from the CPU 11.

  The input unit 16 has various switches (not shown) and receives various instruction inputs from the performer.

  The wireless communication device 17 performs predetermined wireless communication with the sound generation unit 30. The predetermined wireless communication may be performed by an arbitrary method. In the present embodiment, the wireless communication device 17 performs wireless communication with the sound generation unit 30 by infrared communication. The LED 18 emits light and goes off according to control from the CPU 11. The LED 18 is used for indicating a processing state of the stick such as initialization of the stick posture.

[Configuration of Sound Generation Unit 30]

  Next, the configuration of the sound generation unit 30 will be described with reference to FIG. FIG. 3 is a block diagram illustrating a configuration of the sound generation unit 30. As shown in FIG. 3, the sound generation unit 30 includes a CPU 31, an I / F (interface) 32, a ROM 33, a RAM 34, a display unit 35, an input unit 36, and a sound system 37. The I / F 32 of the sound generation unit 30 receives data from the stick unit 10 (for example, a note-on event that is control data for generating a musical sound), stores the data in the RAM 34, and notifies the CPU 31 of the data reception. In the present embodiment, the stick unit 10 is provided with a wireless communication device 17, and the sound generation unit 30 is provided with an infrared communication device 38. The I / F 32 receives data from the stick unit 10 by receiving the wireless signal generated by the wireless communication device 17 of the stick unit 10 by the infrared communication device 38.

  The CPU 31 controls the performance device 1 as a whole, in particular, controls the sound generator 30, detects operations of key switches (not shown) constituting the input unit 36, and notes from the stick unit 10 received via the I / F 32. Various processing such as sound generation based on on-event is executed.

  The ROM 33 controls the performance device 1 as a whole, in particular, the sound generation unit 30, the detection of operation of a key switch (not shown) constituting the input unit 36, and a musical tone based on a note-on event received via the I / F 32. Stores various processing programs such as pronunciation. The ROM 33 stores waveform data of various timbres, for example, wind instruments such as flutes, saxophones, and trumpets, keyboard instruments such as pianos, stringed instruments such as guitars, bass drums, hi-hats, snares, cymbals, and toms. Includes waveform data area to store.

  The RAM 34 stores programs read from the ROM 33, data and parameters generated in the course of processing. The data generated in the process includes the operation state of the switch of the input unit 36, the note-on event received via the I / F 32, the state of the player's physical movement (forward flag and displacement), and the like.

  The display unit 35 is composed of, for example, a liquid crystal display device, and displays the selected tone color and volume, the state of the player's body movement (displacement from the reference position), and the like as an image. The input unit 36 has various switches (not shown) and accepts input of various information from the performer.

  The sound system 37 includes a sound source unit 371, an audio circuit 372, and a speaker 373. The sound source unit 371 reads out waveform data from the waveform data area of the ROM 33 in accordance with an instruction from the CPU 31, and generates and outputs musical sound data. The audio circuit 372 converts the musical sound data output from the sound source unit 371 into an analog signal, amplifies the converted analog signal, and outputs the amplified analog signal to the speaker 373. As a result, a musical tone is output from the speaker 373.

[Description of coordinate system]
Next, the posture of the stick and the world coordinate system in this embodiment will be described with reference to FIG. Here, the world coordinate means a coordinate system centered on the performer. As shown in FIG. 5, the right horizontal direction as viewed from the performer is defined as the X axis, the front horizontal direction is defined as the Y axis, and the zenith direction is defined as the Z axis. This is different from the x-axis, y-axis, and z-axis, which are the three axes of the stick unit 10 described in FIG. In the present embodiment, the three axes of the stick unit 10 are referred to as a local coordinate system and expressed in lowercase letters, whereas in the world coordinate system, the three axes are expressed in uppercase letters. When the performer swings the stick unit 10, the relationship between the local coordinates and the world coordinates changes with time. The orientation of the local coordinate system of the stick unit 10 when viewed from the world coordinate system is referred to as the posture of the stick.

  When the stick unit 10 moves, the origin of the local coordinate system as viewed from the origin of the world coordinate system also moves. The origin of the local coordinate system viewed from the world coordinate system is called the position of the stick unit 10.

[Processing of the performance device 1]

  Next, processing of the performance device 1 according to this embodiment will be described. First, an outline of the processing executed in the stick unit 10 will be described with reference to FIG.

  FIG. 6 is a flowchart showing a flow of processing executed by the CPU 11 of the stick unit 10. The CPU 11 reads the program code from the ROM 14 and executes it.

  First, the CPU 11 executes initialization processing including initialization of various sensors included in the attitude sensor 12 and clearing of data in the RAM 15 (step S11). When the initialization process is completed, the CPU 11 executes offset and gain adjustments that vary depending on device variations and temperatures of various sensors included in the attitude sensor 12 (step S12). Next, the CPU 11 reads the switch state of the input unit 36 and stores it in the RAM 15 (step S14). The state of the switch stored in the RAM 15 is used for assisting position detection, which will be described later.

  Next, when the user swings the stick unit 10, the CPU 11 executes a process of generating a sound according to the swing (steps S16 to S24). First, the CPU 11 stores the acceleration value, angular velocity value, and geomagnetic direction and intensity of the attitude sensor 12 in the RAM 15 (step S16).

  The CPU 11 executes the estimation of the posture of the stick unit 10 according to the read value (step S17), and then executes the position estimation of the parallel movement of the stick unit 10 (step S18). Further, the CPU 11 performs motion prediction of the tip of the stick unit 10 (step S19), and executes sound generation timing detection using the predicted motion (step S20). Further, the CPU 11 detects the sound generation amount (step S21). If it is the sound generation timing in step S20 (the operation state is “SHOT” to be described later) (if YES is determined in step S22), the CPU 11 determines the tone and tone of the musical tone according to the posture and position of the stick unit 10. The note-on event is generated by determining the height (step S23), and the generated note-on event is transmitted to the sound generator 30 (step S24).

[Attitude sensor and initialization of attitude]
With reference to FIG. 7, the posture sensor and the posture initialization process in step S12 of FIG. 6 will be described in detail.

FIG. 7 is a flowchart showing the flow of the posture sensor and posture initialization processing.
First, the CPU 11 performs display so that the user has the tip of the stick unit 10 pointed forward and stops (step S71). As an example of display, the CPU 11 turns on the LED 18 in green.

  Next, the CPU 11 reads data of each axis of the angular velocity sensor 123. The angular velocity sensor 123 usually varies in offset value when the angular velocity is zero due to device variations and temperature variations. The CPU 11 stores the value of the angular velocity sensor 123 as an offset value in the RAM 15 (step S72). In subsequent steps, when the value of the angular velocity sensor 123 is read, the CPU 11 subtracts this offset value from the read value so that the angular velocity when the stick unit 10 is stationary is adjusted to zero.

  Next, the CPU 11 reads the data of each axis of the acceleration sensor 122 and determines whether or not a change in the direction of gravity is detected compared with the previous acceleration (step S74). Specifically, when the stick unit 10 is in a stationary state, the value of the acceleration sensor 122 indicates only each axis component in the gravitational direction. Therefore, the CPU 11 determines that the stick does not change when the value of each axis component does not change. It can be detected that it is stationary. In the present embodiment, the CPU 11 detects that the stick is stationary when there is no change for about 1 second. If the CPU 11 detects that the stick is stationary, the CPU 11 completes registration of the offset value of the angular velocity and moves the process to step S75. On the other hand, if the CPU 11 detects that the stick has been operated, the process proceeds to step S72 and the offset value is registered again.

  Subsequently, the CPU 11 initializes the posture (step S75). Specifically, the CPU 11 has the direction in which the tip of the stick unit 10 faces in the state where the user points the tip of the stick unit 10 in the forward direction, which is the front of the world coordinate system (Y-axis direction). Initialize the relationship between the world coordinate system and the local coordinate system. In addition, the CPU 11 stores the angular velocity, acceleration, and geomagnetism axis components at the time of initialization in the RAM 15.

  Subsequently, the CPU 11 performs display for causing the user to move the stick unit 10 in various directions (step S76). For example, the CPU 11 blinks the LED 18 in red.

  Further, the CPU 11 obtains correction data for offset and gain of the magnetic sensor 121 (step S77). Specifically, the CPU 11 measures the geomagnetism value when the user points the stick unit 10 in various directions by the magnetic sensor 121 according to the process of step S76, and corrects it using the measured geomagnetism value. Ask for data. As an example of how to obtain the offset and gain values, the CPU 11 calculates the median value from the maximum value and the minimum value of each axis of the magnetic sensor 121, and sets the calculated median value as the offset value of each axis.

  Further, the CPU 11 calculates the gain correction value so that the values from the center of each axis to the maximum value are the same, and stores the calculated offset value and gain correction value in the RAM 15. In the subsequent steps, when the CPU 11 reads the value of the magnetic sensor 121, it subtracts the offset value from the read value and further multiplies the correction gain to correct the geomagnetism value to be a detection value.

  Subsequently, the CPU 11 determines whether or not the detected geomagnetism value is a value detected on a general ground (step S78). When the detected value of the geomagnetism is not a value detected on the general ground, the CPU 11 returns the process to step S76 and obtains the offset value and gain correction data of the magnetic sensor 121 again. When the detected geomagnetism value is a value detected on the general ground, the CPU 11 shifts the processing to step S79 and sets the yaw correction mode to the geomagnetic mode.

  Further, if the detected geomagnetism value is not a value detected on the general ground even if the process of step S78 is executed a predetermined number of times (for example, five times), the CPU 11 shifts the process to step S80. And set the yaw correction mode to the shot mode. When the process of step S79 or step S80 ends, the CPU 11 shifts the process to step S81 and displays whether the yaw correction mode is the geomagnetic mode or the shot mode. For example, the CPU 11 lights the LED 18 in red in the shot mode, and lights the LED 18 in green in the geomagnetic mode. Note that the order of the sensor value correction processing of the angular velocity sensor 123 and the magnetic sensor 121 may be switched.

[Attitude estimation]
With reference to FIG. 8, the posture estimation process in step S17 of FIG. 6 will be described in detail.

FIG. 8 is a flowchart showing the flow of posture estimation processing.
X-axis of the local coordinate system as viewed from the world coordinate system, y-axis, a basis vector _e x and _z-axis, e y, as _{e z,} the matrix of each basis vectors arranged in columns _{T L → W = (e x} e _{y ez} ) is defined, the following relational expression (1) exists between the point p = (xyz) ^{T in the} local coordinate system and the point P = (XYZ) ^T in the world coordinate system corresponding to the point p. ), (2) holds (the operation symbol T means a transposed matrix).
_{P = T L → W p (} 1) p = T L → W T P (2)

Thus, _{TL → W} indicates the relationship between the two coordinate systems, and thus the current stick posture state can be expressed using the relationship between the two coordinate systems. In addition, as a method of expressing the posture, an angle such as pitch, yaw, or roll may be used.

By the way, a tertiary angular velocity vector having each axis component of the local coordinate system can be acquired from the value of the angular velocity sensor 123 read at the current time t. The rotation angle vector θt = (θ _xt θ _yt θ _zt ) ^T can be obtained by multiplying the value of each axial component of this cubic angular velocity vector by δT, which is the sampling time of the angular velocity sensor 123.

The posture of the stick can be obtained from this rotation angle vector. Specifically, the posture T _{L → Wt at} time t is a combination of θ _xt rotation about the x axis, θ _yt rotation about the y axis, and θ _zt rotation about the z axis. By using the matrix R _all (θ _t ) and the previous posture T _{L → Wt−1,} it can be estimated that T _{L → Wt} = R _all (θ _t ) · T _{L → Wt−1} . As described above, the CPU 11 updates the previous posture based on the value of the current angular velocity sensor 123, and estimates the current posture (step S101).

  However, errors are accumulated in the posture obtained in this way. In addition, once the posture is wrong, the accuracy of the posture estimation thereafter is significantly deteriorated. For example, consider that the stick unit 10 is rotated straight from the front horizontal toward the right horizontal. If there is an error in the rotation in the vertical direction instead of in the horizontal direction, the trajectory of the tip of the stick will rotate while moving up diagonally. After that, a new rotation will be added to this posture, so the original posture will be accelerated. Deviate. Therefore, by performing gravity estimation, posture estimation is corrected and posture accuracy is improved.

  The acceleration vector detected by the acceleration sensor 122 is composed of components of each axis in the local coordinate system. This acceleration vector is represented by a world coordinate system vector using the posture information obtained in step S101.

  If no external force is applied, the acceleration sensor 122 detects only gravity. Therefore, when the acceleration vector in the local coordinate system detected by the acceleration sensor 122 is converted into the world coordinate system, the acceleration sensor 122 always indicates the same direction. However, since an external force such as a centrifugal force is applied during the operation of the stick unit 10, it does not always face the same direction. However, since the user swings the stick unit 10 in a stopped state, the position of the stick unit 10 does not move. In addition, the speed becomes 0 when the stick unit 10 is swung up or down. That is, the integration of the external force obtained by removing the gravity component from the acceleration vector becomes zero every time. Therefore, the vector obtained by integrating the acceleration vector of the world coordinate system generally indicates the direction of gravity. This property is used to estimate the direction of gravity in the world coordinate system.

Specifically, the CPU 11 sets the acceleration value of each axis component acquired by the acceleration sensor 122 read at the current time t as _L A _T = (A _xt A _yt A _zt ) ^T , which is a local coordinate system. Based on the equation (3), conversion to the world coordinate system is calculated (step S102).
_W A _T = T _{L → Wt} · _L A _T (3)

Next, the CPU 11 calculates an average value from the current time to a predetermined time before each component for the value of the acceleration vector converted into the world coordinate system to obtain an average acceleration vector. The predetermined time is experimentally determined in a swing state or the like. The CPU 11 estimates this average acceleration vector as the gravity direction vector _W e _gt (step S103).
Specifically, the following formula (formula (4)) is defined, and accumulation is performed by recursive addition. K _g is a damping coefficient, and is experimentally determined in a swing state or the like.
Gw _t = k _g · Gw _t−1 + _W A _t (4)
From this Gw _t, the gravity direction vector _W e _gt of the world coordinate system is estimated as _W e _gt = Gw _t / | Gw _t |.

Next, the CPU 11 corrects the posture parameter T _{L → Wt} so that the gravity direction vector _W e _gt becomes the negative Z-axis direction vector (0 0 −1) ^T (step S104). Specifically, the CPU 11 rotates the angle formed by these two vectors around an axis orthogonal to the two vectors of the gravity direction vector _W e _gt and the negative direction vector (0 0 −1) ^T. By performing the operation, the posture parameter T _{L → Wt} is corrected. Thereby, since the gravity direction is corrected, the accuracy of posture estimation is improved.

  Since the posture correction described above is only in the direction of gravity, the error accumulation of the yaw angle around the Z axis remains. In the present embodiment, two methods for correcting the yaw angle are proposed: a switch operation by the user and a correction using geomagnetism.

  First, correction by a switch operation by a user will be described. In this correction, the CPU 11 corrects the yaw angle when the user periodically presses the input unit 16 (for example, a switch) included in the stick unit 10 at a timing when the stick unit 10 faces forward. That's it. The CPU 11 determines whether or not the yaw direction forced switch is turned on (step S109). In this process, the CPU 11 determines whether or not a signal indicating that the yaw direction forced switch has been turned on has been received from the input unit 16.

If the determination in step S109 is YES, the CPU 11 obtains a rotation matrix that matches the direction vector of the world coordinate system at the tip of the stick unit 10 with the direction vector of the Y axis of the world coordinate system, and uses this rotation matrix. Then, the posture parameter is corrected (step S110). Thereby, the yaw angle at the tip of the stick unit 10 is set to 0 degree.
This rotation matrix is, for example, a length of an angle that forms two vectors, and considers a rotation angle vector that faces a vector direction orthogonal to the two vectors. The composite matrix transformation can be obtained by the same method as the method of obtaining the rotational angle vector and the composite rotational matrix for the posture correction of the angular velocity sensor 123.

  Next, correction using geomagnetism will be described.

When the determination in step S109 is NO, the CPU 11 determines whether or not the yaw direction correction mode is a correction mode based on the geomagnetic mode (step S105). If the correction mode is not the geomagnetic mode, the CPU 11 ends the posture estimation process.
When the correction mode is the geomagnetic mode, the CPU 11 sets the value of the geomagnetic vector obtained from the magnetic sensor 121 read at the current time t as _L M _T = (M _xt M _yt M _zt ) ^T , which is a local coordinate. Since it is a system, it is converted to the world coordinate system based on the equation (5) (step S106).
_W M _T = T _{L → Wt} · _L M _T (5)

Next, the CPU 11 executes, for example, an IIR (Infinite Impulse Response) filter to perform smoothing. The direction obtained by smoothing is estimated as the current geomagnetic direction _W e _mt (step S107).
Specifically, smoothing is performed by executing the following IIR filter operation (formula (6)). k _m is determined experimentally in such a swing state in filter coefficients.
_{Mw t = (1-k m} ) · Mw t-1 + k m · W A t (6)
From this Mw _t, the current geomagnetic direction vector _W e _mt of the world coordinate system is obtained.
_{W e mt = Mw t / |} Mw t | and to estimate.

Subsequently, the CPU 11 determines the geomagnetic direction _W e _m0 in the world coordinate system in the initial state.
Are stored in the RAM 15, and the current geomagnetic direction _W e _mt and the initial geomagnetic direction _W e _m0 in the world coordinate system are recorded.
A rotation matrix is obtained so that the direction of is oriented in the same direction on the XY plane of the world coordinate system, and the posture parameter T _{L → Wt} is corrected using this rotation matrix (step S108). Thereby, since the correction of the geomagnetic direction is performed, the accuracy of posture estimation is improved.
Specifically, the rotation matrix is a vector in which the Z component of each vector is set to 0. From the two vector directions, two vectors having the length of an angle that forms two vectors at the center, as in gravity correction, are used. Considering a vector oriented in the vector direction orthogonal to, it can be expressed by a composite matrix transformation of the rotation of each component value.

[Position estimation]
With reference to FIG. 9, the position estimation process in step S18 of FIG. 6 will be described in detail.

FIG. 9 is a flowchart showing the flow of the position estimation process.
When estimating the posture, the gravity direction _W e _gt of the world coordinates was estimated in step S103 of FIG. CPU11 obtains a vector excluding the component of the gravity direction _W e _gt from the acceleration vector in the world coordinate system to define this vector and acceleration _W F _T due to an external force (step S201). Furthermore, CPU 11 obtains the moving velocity _W v _t by integrating the acceleration _W F _T due to an external force, calculating the moving amount _W D _t of the horizontal plane and further integrates the moving velocity _W v _t (step S202).

  In addition, it is desirable to use incomplete integration because integration is affected by error accumulation in complete integration. The damping coefficient (time constant) is adjusted by experiment. The value obtained in this way is used in a later note event generation process (see FIG. 16).

[Tip motion prediction]
With reference to FIG.10 and FIG.11, the tip movement prediction process of step S19 of FIG. 6 is demonstrated in detail.

  It is desirable to control the sound intensity at the speed of the stick tip. There is a method for obtaining the speed from an acceleration sensor, but there is a problem in accuracy. In addition, since the swing of the stick swings while rotating the elbow or the like around the elbow or the shoulder only by the angular velocity, the angular velocity and the velocity cannot be said to be in a proportional relationship. Therefore, in this embodiment, the speed is predicted by the angular velocity and angular acceleration of the tip.

FIG. 10 is a flowchart showing the flow of the tip motion prediction process.
As an example of how to obtain it, first, the CPU 11 uses the angular velocity obtained by removing the component around the y-axis from the angular velocity vector as the observed value ωV _t = (ω _xt 0 ω _zt ) ^T of the stick tip angular velocity, and uses that value as the Kalman filter or the like. The angular velocity ωs and the angular acceleration δωs are used for estimation (step S210).

For example, each angular velocity component is obtained by adding an angular acceleration component to the previous angular velocity component, and the angular acceleration component can be obtained using a state model in which drive noise is added to the same angular acceleration component as the previous time. CPU11 using the angular velocity of the stick tip thus obtained to estimate the most recent average vector direction of the angular velocity as a stick tip speed rotary shaft e _r (step S211).

  In this embodiment, the Kalman filter is used. However, the angular velocity and the angular acceleration may be obtained by smoothing the sensor value and its difference using an IIR filter.

Stick tip rotation axis direction perpendicular to the direction y-axis of the stick (cross product direction, see FIG. 11) is defined as the swing direction e _s. The tangential direction of the rotational movement of the tip is in this direction. CPU11 is the component _L As _t swing direction of the acceleration vector _L A _t, and calculating the inner product of L _{A t} and _{e s} (step S212).

[Sounding timing detection processing]
With reference to FIG. 12 or FIG. 13, the detailed description of the sound generation timing detection process of step S20 of FIG. 6 is given.

[Sound generation timing detection processing (first embodiment)]
FIG. 12 is a flowchart showing the flow of sound generation timing detection processing in the first embodiment.
First, the CPU 11 branches according to the operation state (step S302). There are four operating states: IDLE, ACTION, SHOT, and DONE. The CPU 11 shifts the process to step S303 in the case of IDLE, shifts the process to step S320 in the case of ACTION, shifts the process to step S330 in the case of SHOT, and shifts the process to step S330 in the case of DONE. Then, the process proceeds to step S340.

  Here, the transition of the standard operation state is shown in FIG. In FIG. 14, the horizontal axis represents the time axis, and the vertical axis represents the magnitude of the angular velocity (the length of the vector) in FIG. The sign has little meaning, but for the sake of convenience, the direction in which the stick unit 10 is started to swing is positive. The vertical axis indicates the magnitude of the angular acceleration (the length of the vector) in FIG. The sign is positive in the same direction as the positive direction of angular velocity. FIG. C shows the transition of the operating state.

The state where the stick is stopped is IDLE, and the operation state is initialized to IDLE in the initial state. The operation state is IDLE at time t0 in the initial state. At this time, if the angular acceleration of the stick unit 10 is not positive, the CPU 11 ends the sound generation timing detection process (step S303). Even if the angular acceleration of the stick unit 10 is positive, if the angular velocity of the stick unit 10 is “threshold 1” or less, the CPU 11 ends the sound generation timing detection process (step S304). When the angular velocity of the stick unit 10 exceeds the value of “threshold 1”, the CPU 11 shifts the processing to step S305 (step S304). FIG. 14 shows that “threshold value 1” was exceeded at time t2. In step S305, the CPU 11 changes the operation state to ACTION (step S305), initializes the maximum angular velocity to 0 (step S306), and ends the sound generation timing detection process.
When the operation state becomes ACTION, the stick keeps accelerating for a while. Since the angular acceleration is positive during this period, the CPU 11 determines YES in step S320 and compares the current angular velocity with the maximum angular velocity held ( In step S321), when the angular velocity is larger than the maximum angular velocity, the maximum angular velocity is updated (step S322), and "threshold value 2" is updated (step S323). When the angular velocity is smaller than the maximum angular velocity, the CPU 11 ends the sound generation timing detection process.

  “Threshold 2” is determined by the value of the maximum angular velocity. For example, “threshold 2” = (maximum angular velocity) × C, where C is experimentally determined by a value of 0 to 1. Usually about 0.1 is desirable. This “threshold 2” assumes a prior angular velocity corresponding to a delay time from when a shot is detected until sound is generated. This is because a radio transmission delay and a later-described sound generation process are delayed, so that the sound is sounded in advance from the original sounding timing in order to guarantee the delay. In general, it is assumed that the shot angular velocity curve is proportional to the intensity of the sound, and the threshold is determined from the maximum value of the angular velocity.

  When the stick unit 10 changes to the deceleration state (t3), the angular acceleration of the stick unit 10 becomes 0 or less, so the CPU 11 determines NO in step S320. In step S324, the CPU 11 compares whether or not the absolute value of the current angular velocity is smaller than “threshold value 2”. While the stick has not been sufficiently decelerated, the CPU 11 determines NO in step S324, ends the sound generation timing detection process, and waits for the shot timing.

  When the stick is sufficiently decelerated and becomes smaller than “threshold 2”, the CPU 11 determines that it is close to the shot timing, determines YES in step S324 (timing of t4), and sets the operation state to SHOT (step S325). The shot passage time is initialized to 0 (step S326). Thereafter, the CPU 11 ends the sound generation timing detection process. This SHOT state informs the sound generation timing, and is used to determine the sound generation timing in step S22 of FIG.

  If the operation state is SHOT, the CPU 11 updates the operation state to DONE in step S330, and ends the sound generation timing detection process.

  When the operation state is DONE, the CPU 11 determines whether or not the angular velocity is “threshold 3” or less (step S340), and when it is determined that the angular velocity is “threshold 3” or less, the operation state Is changed to IDLE (step S341), and the sound generation timing detection process is terminated. In step S340, when the CPU 11 determines that the angular velocity is greater than “threshold 3”, the CPU 11 determines whether or not the shot passage time is equal to or less than “threshold 4” (step S342). If it is determined that it is 4 ”or less, the sound generation timing detection process is terminated. On the other hand, if the CPU 11 determines that the shot passage time has exceeded “threshold 4”, the CPU 11 changes the operation state to IDLE (step S341), and ends the sound generation timing detection process.

  That is, the DONE state is held until the angular velocity becomes “threshold 3” or less. This is to prevent malfunctions such as multiple sounds when stopping. However, when the angular velocity is slowly decelerated and the shot passing time until the angular velocity becomes “threshold 3” or less exceeds “threshold 4”, the operation state is changed from DONE to IDLE.

[Sound generation timing detection processing (second embodiment)]

FIG. 13 is a flowchart showing the flow of sound generation timing detection processing in the second embodiment.
First, the CPU 11 branches according to the operation state (step S352). There are four operating states: IDLE, ACTION, SHOT, and DONE. The CPU 11 shifts the process to step S353 in the case of IDLE, shifts the process to step S370 in the case of ACTION, shifts the process to step S380 in the case of SHOT, and shifts the process to step S380 in the case of DONE. Then, the process proceeds to step S390. Here, the transition of the standard operation state is the same as that of the first embodiment as shown in FIG.

  If the angular acceleration of the stick unit 10 is not positive, the CPU 11 ends the sound generation timing detection process (step S353). Even if the angular acceleration of the stick unit 10 is positive, if the angular velocity of the stick unit 10 is not more than “threshold 1”, the CPU 11 ends the sound generation timing detection process (step S354). When the angular velocity of the stick unit 10 exceeds the value of “threshold 1”, the CPU 11 shifts the processing to step S355 (step S354). FIG. 14 shows that “threshold value 1” was exceeded at time t2. In step S355, the CPU 11 changes the operation state to ACTION (step S355), and ends the sound generation timing detection process.

  When the operation state becomes ACTION, the stick continues to accelerate for a while. Since the angular acceleration is positive during this period, the CPU 11 determines YES in step S370 and ends the sound generation timing detection process.

When the stick unit 10 changes to the deceleration state (t3), the angular acceleration of the stick unit 10 becomes 0 or less, so the CPU 11 determines NO in step S370. In step S371, the CPU 11 estimates the time from the current stick tip angular velocity ωs _t and angular acceleration δωs _t until the angular velocity becomes zero. If the estimated time is Trem,
The approximate time can be estimated with Trem = | ωs _t | / | δωs _t |.

  Subsequently, the CPU 11 determines whether or not the estimated time is “threshold 5” or less (step S372). For example, the “threshold value 5” is about 10 milliseconds.

  This “threshold value 5” is set as a delay time from when a shot is detected until sound is generated. This is because a radio transmission delay and a later-described sound generation process are delayed, so that the sound is sounded in advance from the original sounding timing in order to guarantee the delay. The CPU 11 determines NO in step S372 and ends the sound generation timing detection process until the stick unit 10 is sufficiently decelerated. Eventually, when the angular velocity of the stick unit 10 becomes close to 0, the estimated time until the stop (angular velocity 0) becomes “threshold 5” or less, and the CPU 11 determines YES in step S372 (timing t4) and sets the operation state to SHOT. It is set (step S373), and the SHOT passage time is initialized to 0 (step S374). This SHOT state informs the sound generation timing, and is used to determine the sound generation timing in step S22 of FIG.

  If the operating state is SHOT, the CPU 11 updates the operating state to DONE in step S380, and ends the sound generation timing detection process.

  When the operation state is DONE, the CPU 11 determines whether or not the angular velocity is “threshold 3” or less (step S390), and when it is determined that the angular velocity is “threshold 3” or less, the operation state Is changed to IDLE (step S391), and the sound generation timing detection process is terminated. In step S390, if the CPU 11 determines that the angular velocity is greater than “threshold 3”, the CPU 11 determines whether or not the shot passage time is equal to or less than “threshold 4” (step S392). If it is determined that it is 4 ”or less, the sound generation timing detection process is terminated. On the other hand, if the CPU 11 determines that the shot passage time has exceeded the “threshold value 4”, the CPU 11 changes the operation state to IDLE (step S391) and ends the sound generation timing detection process.

  That is, the DONE state is held until the angular velocity becomes “threshold 3” or less. This is to prevent malfunctions such as multiple sounds when stopping. However, when the angular velocity is slowly decelerated and the shot passing time until the angular velocity becomes “threshold 3” or less exceeds “threshold 4”, the operation state is changed from DONE to IDLE.

[Sounding volume detection]
With reference to FIG. 15, the sound production amount detection process in step S <b> 21 of FIG. 6 will be described in detail.

FIG. 15 is a flowchart showing the flow of the sound generation amount detection process.
The CPU 11 determines the operation state. If the operation state is DONE or IDLE, the CPU 11 shifts the process to step S402. If the operation state is ACTION or SHOT, the CPU 11 shifts the process to step S406.

When the operation state is DONE or IDLE, the CPU 11 initializes the peak angular velocity value (scalar amount) ω _{s peak} , the shift velocity vs _t , the peak shift velocity vs _peak , and the peak shift acceleration as _peak to 0 ( Steps S402 to S405).

The peak angular velocity value ω _{s peak} is the maximum value of the angular velocity detected by the stick during the sounding swing, the shift velocity vs _t is the velocity component of the translation of the tip of the stick, and the current peak shift velocity vs _peak is the shift velocity _L Vs _t. Is the value holding the maximum value of. When the swing of the stick is detected in the sound generation timing detection (step S305 in FIG. 12 or step S355 in FIG. 13) and the operation state becomes ACTION or SHOT, the CPU 11 determines the scalar amount of the current stick tip angular velocity vector | ω _st | Is compared with the peak angular velocity value ω _{s peak,} and if | ω _st | is larger, ω _{s peak} is updated to the value of | ω _st | (step S406).

Then, CPU 11 performs the holding and updating the peak value _{the as peak} shift acceleration of the acceleration component _L As _t swing direction obtained in the step S212 of FIG. 10 (step S407).

Subsequently, CPU 11 is incomplete integrating the acceleration component _L As _t swing direction obtained in the step S212 of FIG. 10, to obtain a swing direction of the velocity _L Vs _t (step S408).

CPU11 compares the said swing direction velocity _L Vs _t and the peak velocity value _{vs peak,} when towards L Vs _t is _large, updates _{vs peak} to the value of _L Vs _t (step S409).

Next, the CPU 11 obtains the speed (volume) of the stick from the three peak values. For example, it is obtained by weighted addition as in the following equation (5).
P _shot = a ₁ · ω _{s peak} + a ₂ · vs _peak + a ₃ · as _peak (5)
Here, a ₁ , a ₂ , and a ₃ are mixing parameters, which are determined by experiments.
This P _shot is used as the loudness of the shot.

[Create note event]
Referring to FIG. 16, the note event generation process in step S23 of FIG. 6 will be described in detail.

FIG. 16 is a flowchart showing the flow of note event generation processing.
It is determined which sound source virtually arranged in space at the time of a shot, but since the shot timing is detected in advance in consideration of the delay of the system, the CPU 11 The position of the stick unit 10 is estimated (step S501). Since the angular velocity and angular acceleration at the tip of the stick unit 10 are known, the CPU 11 can estimate the position by obtaining the posture of the stick after Trem. Specifically, the CPU 11 further subdivides the time into Trem, sets the time interval as T, and obtains the angular velocity vector of each time from the previous angular velocity vector and angular acceleration vector. Further, the CPU 11 obtains a rotation angle vector at each time interval from the angular velocity vector, obtains a composite rotation matrix using the rotation angle vector, and predicts the next posture. By repeating this until Trem, the posture after Trem seconds can be predicted as _{TL → Wt + Trem} . From this posture parameter, the position P _y = (P _yX P _yY P _yZ ) ^T in the world coordinate system of the y-axis in the tip direction of the stick unit 10 can be obtained.

When six sound sources are mapped (arranged) in the world coordinate system, the position (world coordinate system) of each sound source is, for example,
Sound source 1: (0, 1, 0) forward Sound source 2: (sin 60, cos 60, 0) Forward right 60 degree direction Sound source 3: (-sin 60, cos 60, 0) Forward left 60 degree direction Sound source 4: (0, cos 45, sin 45) Front upper 45 degrees direction Sound source 5: (cos 45 sin 60, cos 45 cos 60, sin 45) Front right 60 degrees, upper 45 degrees direction Sound source 6: (-cos 45 sin 60, cos 45 cos 60, sin 45) Forward left 60 degrees, upper 45 degrees direction.

The position information of each sound source and the timbre corresponding to each sound source number are stored in advance in the memory of the ROM 14.
The CPU 11 _obtains the distance between the position P _y of the stick unit 10 at the time of sound generation estimated in step S501 and each sound source position P _si = (P _siX P _siY P _siZ ) ^T, and the closest one is subject to the current shot. (Step S502).

  The distance measurement calculates the Euclidean distance. Alternatively, it is also effective to use a weighted Euclidean distance for each component that is insensitive to the distance in the swing direction axis. Since the swing direction is a moving direction and accuracy is poor, it is desirable to use a distance with a greater weight than the rotation axis direction.

  Alternatively, the orientation can be converted to a pitch, yaw, roll angle representation. The distance may be obtained by converting the sound source position and the stick position into the angle notation by this angle notation. At this time, a distance in which the weight is changed between the pitch direction and the yaw direction may be used. The roll component weight is preferably zero.

  Assuming that the sound source number shot in step S502 or the sound source number determined in the sound search process described later with reference to FIG. 17 is i, the CPU 11 reads a note corresponding to the i-th sound source from the table ( In step S503), a note to be pronounced can be obtained.

Next, the CPU 11 checks the yaw direction correction mode determined in step S79 or S80 in FIG. 7 (step S504). When the yaw direction correction mode is the geomagnetic mode, the CPU 11 ends the note event generation process. On the other hand, when the yaw correction mode is the shot mode, the CPU 11 obtains a yaw angle error between the sound source and the stick unit 10 (step S505). Specifically, the CPU 11 obtains the direction vector of the stick from the posture parameter, uses the position information of the sound source in the world coordinate system described above as the direction vector of the sound source, and calculates the difference in the angle of the vector composed of the XY components of these two vectors. Is calculated as a yaw angle error. Since this yaw angle error is a rotation error around the Z axis, the CPU 11 obtains a rotation matrix with the Z axis as the rotation axis, and corrects the posture using this rotation matrix (step S506).
[Sound search]
With reference to FIG. 17, a detailed description will be given of a sound search process which is a modification of step S502 of FIG.

  FIG. 17 is a flowchart showing the flow of sound search processing which is a modification of step S502 of FIG. This process is executed instead of the process of step S502 in FIG. That is, this process is executed after the process of step S501 in FIG. 15. When step S613 of this process is completed, the CPU 11 shifts the process to step S503 in FIG.

First, the CPU 11 searches for the closest sound source from each sound map by the sound search method similar to step S502 in FIG. 16 (step S601).
Next, the CPU 11 stores the found sound source number i in the RAM 15 as the current sound source selection number (step S602). Subsequently, the CPU 11 reads the previous sound source number, and compares the position coordinates of the sound source number with the position coordinates of the current sound source. At this time, the X and Y coordinates excluding the vertical Z-axis are compared (step S603). The vertical direction is not corrected in this embodiment.

Further, the CPU 11 determines whether or not they have the same azimuth (step S604). When the CPU 11 determines that they have the same azimuth, the value of the horizontal component of the movement amount obtained in S202 of FIG. It is determined whether it is larger than the threshold (step S605). If the determination is YES, CPU 11 rotates the position _{P y} of the tip end direction of the horizontal rotation angle partial stick 10 of a predetermined in the positive direction (step S606). When the determination in step S605 is NO, the CPU 11 compares the negative threshold obtained by multiplying the threshold by −1 and the horizontal component of the movement amount to determine whether the value is smaller than the negative threshold (step S607). If the determination is YES, CPU 11 rotates the position _{P y} of the tip end direction of the horizontal rotation angle partial stick 10 of a previously determined in the negative direction (step S608).

  The CPU 11 corrects the azimuth in the tip direction of the stick in this way, and again searches for the closest sound source from each sound map by the sound search method similar to S502 of FIG. 16 (step S609). The CPU 11 determines whether or not a sound source has been found (step S610), and if it can be found (YES in step S610), the newly found sound source is updated to the current sound source number and stored in the RAM 15 ( Step S611). If it cannot be found (NO in step S610), the previous sound source number is held.

  Next, the CPU 11 resets the value of the movement amount to 0 in order to obtain the movement amount between the next shot and the current shot (step S612). Further, the CPU 11 stores the sound source number adopted this time in the RAM 15 as the previous sound source number (step S613).

[Pronunciation processing]
With reference to FIG. 18, the sound generation process executed in the sound generation unit 30 will be described.

  FIG. 18 is a flowchart showing the flow of sound generation processing executed in the sound generation unit 30.

  As shown in FIG. 18, the CPU 31 of the sound generation unit 30 executes initialization processing including clearing data in the RAM 34, clearing the image displayed on the screen of the display unit 35, clearing the sound source unit 371, and the like when the power is turned on. (Step S701). Next, the CPU 31 executes a switch process (step S702). In the switch processing, for example, the volume volume value and tone color desired by the performer are specified according to the switch operation of the input unit 36 and stored in the RAM 34.

  Subsequently, the CPU 31 determines whether the I / F 32 has newly received a note-on event (step S703). If it is determined Yes in step S703, the CPU 31 executes a process of instructing the sound source unit 371 to generate a musical tone having the set volume volume value and the set tone based on the received note-on event. (Step S704). When the process of step S704 ends, the CPU 31 returns the process to step S702.

The configuration and processing of the performance device 1 according to the present embodiment have been described above.
In the present embodiment, the CPU 11 acquires the acceleration detected by the acceleration sensor 122 and the angular velocity detected by the angular velocity sensor 123 at regular timings, and is detected by the acceleration sensor 122 in the initial stationary state of the stick unit 10. The posture parameter in the initial stationary state including the acceleration due to gravity is stored in the RAM 15. Then, the posture parameter is updated according to the angular velocity detected by the angular velocity sensor 123 at every fixed timing, and the acceleration in the gravity direction is calculated from the detected acceleration based on the updated posture parameter. Further, the calculated acceleration is accumulated for each predetermined section, and the accumulated acceleration is corrected so as to coincide with the acceleration stored in the RAM 15.
Therefore, when the stick unit 10 is operated at high speed, the accuracy of posture estimation of the stick unit 10 can be improved by correcting the direction of gravity.

In the present embodiment, the CPU 11 detects the geomagnetic components in the directions of the three axes orthogonal to each other including the longitudinal axis of the stick unit 10 by the magnetic sensor 121 in the initial stationary state of the stick unit 10. Then, it is converted into a geomagnetic component in another three-axis direction based on the above-mentioned posture parameter and stored in the RAM 15. Then, the geomagnetic component of each axis detected by the magnetic sensor 121 is acquired at every fixed timing, and the acquired geomagnetic component of each axis at every fixed timing is converted into the above-described different triaxial geomagnetic components based on the attitude parameter. To do. Further, the converted local magnetic component is accumulated every predetermined section, and the accumulated local magnetic component is corrected so as to coincide with the local magnetic component stored in the RAM 15.
Therefore, when the stick unit 10 is operated at high speed, the accuracy of posture estimation of the stick unit 10 can be improved by correcting the geomagnetic direction.

In the present embodiment, the CPU 11 transmits a note-on event to the sound generation unit 30 when the value of the angular velocity acquired by the angular velocity sensor 123 exceeds “threshold 1” and then falls below “threshold 2”. The “threshold value 2” is a value corresponding to the maximum value after the angular velocity value acquired by the angular velocity sensor 123 exceeds the “threshold value 1” (specifically, a value obtained by multiplying the maximum value by a certain ratio value). ).
Therefore, the sound generation accuracy can be improved by generating the sound generation timing a predetermined time before the original shot timing in consideration of the sound generation delay time.

In this embodiment, the CPU 11 estimates the time until the angular velocity value acquired by the angular velocity sensor 123 exceeds “threshold 1” and then becomes 0, and the estimated time falls below “threshold 5”. If a note-on event occurs, a note-on event is transmitted to the sound generator 30.
Therefore, the sound generation accuracy can be improved by generating the sound generation timing a predetermined time before the original shot timing in consideration of the sound generation delay time.

Further, in the present embodiment, the CPU 11 calculates the angular velocity and angular acceleration of the tip portion of the stick unit 10 based on the angular velocity acquired by the angular velocity sensor 123, and calculates the calculated angular velocity and angular acceleration of the tip portion of the stick unit 10. Based on this, the time from when the value of the angular velocity acquired by the angular velocity sensor 123 exceeds the “threshold value 1” until the value becomes 0 is estimated.
Therefore, it is possible to improve the estimation accuracy of the time until the angular velocity value becomes zero.

In the present embodiment, the CPU 11 estimates the rotation axis direction of the stick unit 10 during performance operation based on the angular velocity acquired by the angular velocity sensor 123, and the longitudinal direction of the stick unit 10 from the angular velocity acquired by the angular velocity sensor 123. Is calculated based on the angular velocity of the tip of the stick unit 10 based on the angular velocity excluding the axial component of the A position in the world coordinate system of the stick unit 10 after a predetermined time is calculated, and a note-on event of a musical sound corresponding to a region closest to the calculated position among a plurality of regions of the sound source map is transmitted to the sound generation unit 30. .
Therefore, even when the stick unit 10 is swung obliquely, it is possible to generate sound with the timbre at the position of the stick unit 10 at the original shot timing.

In the present embodiment, the CPU 11 estimates the rotation axis direction of the stick unit 10 during performance operation based on the angular velocity acquired by the angular velocity sensor 123, and the longitudinal direction of the stick unit 10 from the angular velocity acquired by the angular velocity sensor 123. Is calculated based on the angular velocity of the tip of the stick unit 10 based on the angular velocity excluding the axial component of the A position in the world coordinate system of the stick unit 10 after a predetermined time is calculated, a moving distance is calculated based on the acceleration acquired by the acceleration sensor 122, and based on the calculated position and the calculated moving distance. The position of the stick unit 10 in the world coordinate system after a predetermined time is corrected, and the area closest to the corrected position among the plurality of areas of the sound source map is corrected. Sending a note-on event of the corresponding musical tones to the sound emitting section 30.
Therefore, even when the stick unit 10 is swung obliquely and moved in parallel, sound can be generated with the tone at the position of the stick unit 10 at the original shot timing.

In the present embodiment, the CPU 11 calculates the maximum value of the angular velocity of the stick unit 10 during the performance operation based on the angular velocity acquired by the angular velocity sensor 123, and performs the performance operation based on the acceleration acquired by the acceleration sensor 122. The maximum value of the acceleration of the inside stick unit 10 is calculated, and the volume of the musical sound to be generated is controlled based on the calculated maximum value of the angular velocity and the maximum value of the acceleration.
Therefore, even when the virtual sound source is arranged perpendicular to the world coordinate system, appropriate volume control is possible reflecting the strength of the swing operation.

  As mentioned above, although embodiment of this invention was described, embodiment is only an illustration and does not limit the technical scope of this invention. The present invention can take other various embodiments, and various modifications such as omission and replacement can be made without departing from the gist of the present invention. These embodiments and modifications thereof are included in the scope and gist of the invention described in this specification and the like, and are included in the invention described in the claims and the equivalents thereof.

The invention described in the scope of claims at the beginning of the filing of the present application will be appended.
[Appendix 1]
A holding member that can be held by the user;
An acceleration sensor that detects acceleration generated in the directions of the three axes perpendicular to each other of the holding member;
An angular velocity sensor that detects an angular velocity about each of the three orthogonal axes;
First calculating means for calculating a maximum value of the angular velocity of the performance member during a performance operation based on the angular velocity detected by the angular velocity sensor;
Second calculating means for calculating a maximum value of acceleration of the holding member based on the acceleration detected by the acceleration sensor;
Volume control means for controlling the volume of a musical tone to be generated based on the maximum value of the angular velocity calculated by the first calculation means and the maximum value of the acceleration calculated by the second calculation means. ,
A performance apparatus characterized by that.
[Appendix 2]
A holding member that can be held by a user, an acceleration sensor that detects acceleration generated in each of three orthogonal directions of the holding member, an angular velocity sensor that detects an angular velocity about each of the three orthogonal axes, A method performed by a performance device comprising:
A first calculation step of calculating a maximum value of the angular velocity of the performance member during a performance operation based on the angular velocity detected by the angular velocity sensor;
A second calculation step of calculating a maximum value of acceleration of the holding member based on the acceleration detected by the acceleration sensor;
A volume control step for controlling the volume of a musical sound to be generated based on the maximum value of the angular velocity calculated in the first calculation step and the maximum value of the acceleration calculated in the second calculation step;
Including methods.
[Appendix 3]
A holding member that can be held by a user, an acceleration sensor that detects acceleration generated in each of three orthogonal directions of the holding member, an angular velocity sensor that detects an angular velocity about each of the three orthogonal axes, In a computer used as a performance device having
A first calculation step of calculating a maximum value of the angular velocity of the performance member during a performance operation based on the angular velocity detected by the angular velocity sensor;
A second calculation step of calculating a maximum value of acceleration of the holding member based on the acceleration detected by the acceleration sensor;
A volume control step for controlling the volume of a musical sound to be generated based on the maximum value of the angular velocity calculated in the first calculation step and the maximum value of the acceleration calculated in the second calculation step;
A program that executes

  DESCRIPTION OF SYMBOLS 1 ... Performance apparatus, 10 ... Stick part, 11 ... CPU, 12 ... Attitude sensor, 13 ... I / F device, 14 ... ROM, 15 ... RAM, 16. ..Input unit, 17... Wireless communication device, 18... LED, 121... Magnetic sensor, 122... Speed sensor, 123. ..CPU, 32 ... I / F, 33..ROM, 34 ... RAM, 35 ... display unit, 36 ... input unit, 37 ... sound system, 38 ... infrared communication Device, 371 ... Sound source unit, 372 ... Audio circuit, 373 ... Speaker

Claims (3)

A holding member that can be held by the user;
An acceleration sensor that detects acceleration generated in the directions of the three axes perpendicular to each other of the holding member;
An angular velocity sensor that detects an angular velocity that occurs when rotating about each of the three orthogonal axes;
First acquisition means for acquiring a maximum value of the angular velocity of the holding member during a performance operation based on an angular velocity generated corresponding to the swing-down operation of the holding member detected by the angular velocity sensor;
Second acquisition means for acquiring a maximum acceleration value of the holding member based on an acceleration generated in response to a swing-down operation of the holding member detected by the acceleration sensor;
Third acquisition means for acquiring a maximum value of the speed of the holding member based on acceleration generated in response to the swing-down operation of the holding member detected by the acceleration sensor;
Based on the maximum value of the angular velocity acquired by the first acquisition unit, the maximum value of the acceleration acquired by the second acquisition unit, and the maximum value of the velocity acquired by the third acquisition unit. Volume control means for controlling the volume of the musical sound to be generated,
A performance apparatus characterized by that. A holding member that can be held by a user, an acceleration sensor that detects acceleration generated in each of the three orthogonal axes of the holding member, and an angular velocity that is generated when the user rotates about each of the three orthogonal axes. A method performed by a performance device having an angular velocity sensor,
A first acquisition step of acquiring a maximum value of the angular velocity of the holding member during a performance operation based on an angular velocity generated corresponding to the swing-down operation of the holding member detected by the angular velocity sensor;
A second acquisition step of acquiring a maximum value of the acceleration of the holding member based on an acceleration generated in response to the swing-down operation of the holding member detected by the acceleration sensor;
A third acquisition step of acquiring a maximum value of the speed of the holding member based on an acceleration generated in response to the swing-down operation of the holding member detected by the acceleration sensor;
The maximum value of the angular velocity acquired in the first acquisition step, the maximum value of the acceleration acquired in the second acquisition step, and the maximum value of the velocity acquired in the third acquisition step. Based on the volume control step for controlling the volume of the musical sound to be generated,
Including methods. A holding member that can be held by a user, an acceleration sensor that detects acceleration generated in each of the three orthogonal axes of the holding member, and an angular velocity that is generated when the user rotates about each of the three orthogonal axes. In a computer used as a performance device having an angular velocity sensor,
A first acquisition step of acquiring a maximum value of the angular velocity of the holding member during a performance operation based on an angular velocity generated corresponding to the swing-down operation of the holding member detected by the angular velocity sensor;
A second acquisition step of acquiring a maximum value of the acceleration of the holding member based on an acceleration generated in response to the swing-down operation of the holding member detected by the acceleration sensor;
A third acquisition step of acquiring a maximum value of the speed of the holding member based on an acceleration generated in response to the swing-down operation of the holding member detected by the acceleration sensor;
The maximum value of the angular velocity acquired in the first acquisition step, the maximum value of the acceleration acquired in the second acquisition step, and the maximum value of the velocity acquired in the third acquisition step. Based on the volume control step for controlling the volume of the musical sound to be generated,
A program that executes

Images