JP5935399B2 - Music generator - Google Patents

Description

  The present invention relates to a musical sound generator.

2. Description of the Related Art Conventionally, a musical sound generating device (performance device) has been proposed that generates an electronic sound (musical sound) corresponding to a performance operation when a performance operation of a performer (user) is detected. For example, a device (air drum) that emits a percussion instrument sound when a performer performs an operation of swinging a rod-like member corresponding to a stick of a percussion instrument is known (see, for example, Patent Document 1 and Patent Document 2). In such a device, a sensor is provided on a rod-shaped member, and when a player holds and shakes the member by hand, the sensor detects the operation and generates a percussion instrument sound corresponding to the detection result. It is configured.
For example, an actual musical instrument such as a percussion instrument is heavy and requires a large space for installation, so that it is difficult to easily perform and practice at home.
In this regard, according to the musical sound generating device that detects the musical performance of the performer and generates a musical sound according to the musical performance, the musical sound of the musical instrument can be generated without the need for an actual musical instrument. You can enjoy playing easily without any restrictions on the performance space.

JP-A-63-191195 JP 2011-252994 A

By the way, as one of the performance methods of the drum which is a percussion instrument, there is a performance method called “roll performance method” (drum roll performance method) which produces a continuous sound by repeatedly hitting the drum hitting surface. In the performance by the roll performance method, usually, one swinging motion of a hand is used to produce two sounds, “ta-ta”, by using the reaction, and by repeating this alternately left and right, a strike is made. A continuous sound can be produced by repeatedly striking the surface.
Such a roll performance method is a performance method that is generally used for raising the atmosphere, and is also one of the best aspects of drum performance.

  However, in the musical sound generating device that generates the electronic sound described in Patent Document 1 and Patent Document 2 and the like, unlike the actual percussion instrument, the actual hitting surface of the instrument is not actually hit. There is no recoil. For this reason, it is impossible to perform a roll performance method in which repetitive hits are made to produce a continuous sound in a fine and fast manner, and there is a problem that the real pleasure of drum performance is lacking.

  The present invention has been made in view of the circumstances as described above, and even in the performance in the air where the reaction cannot be used, the musical sound that can make the player sound as if performing the performance by the roll technique. The object is to provide a generator.

In order to solve the above-described problem, the musical sound generating device of the present invention includes:
A performance operator for striking that the performer can hold,
Movement detecting means for detecting movement of the performance operator main body caused by the player's batting performance operation;
The maximum value and the minimum value of the acceleration generated by the vertical movement of the performance operator detected by the motion detection unit are within a predetermined range, and are detected by the motion detection unit. Rendition style determination means for determining that it is a roll rendition when the time between the maximum and minimum values of acceleration generated by the operation of the performance operator is within a predetermined range ;
A sounding instruction means for instructing the sounding means to pronounce a musical sound for roll performance at a timing determined based on the movement detected by the motion detecting means when the rendition style determining means determines that it is a roll performance;
It is characterized by having.

  According to the present invention, even in the performance in the air where the reaction cannot be used, it is possible to make a sound as if the performer is performing by a roll performance. As a result, the range of performance expressions is widened, and the effect of being able to fully enjoy the pleasure of the performance is achieved.

(A) is a figure which shows the outline | summary of one Embodiment of the musical sound generator based on this invention, (B) is the figure which showed the virtual drum set in this embodiment. It is a block diagram which shows the hardware constitutions of the stick part shown in FIG. It is a perspective view of the stick part shown in FIG. It is a block diagram which shows the hardware constitutions of the camera unit part shown in FIG. It is a block diagram which shows the hardware constitutions of the center unit part shown in FIG. It is a figure showing the change of the acceleration of the vertical direction of a motion sensor part. It is a figure showing the change of the acceleration of the vertical direction of a motion sensor part, and shows an example when it is not a roll performance method. It is a figure showing the change of the acceleration of the vertical direction of a motion sensor part, and shows an example in the case of starting a roll performance method. It is a figure showing the change of the acceleration of the vertical direction of a motion sensor part, and has shown an example in the case of complete | finishing a roll performance method. It is a figure showing the change of the acceleration of the vertical direction of a motion sensor part, and has shown an example in the case of complete | finishing a roll performance method. It is a figure showing the change of the acceleration of the vertical direction of a motion sensor part, and has shown an example in the case of complete | finishing a roll performance method. It is a figure showing the change of the acceleration of the vertical direction of a motion sensor part, and has shown an example in the case of complete | finishing a roll performance method. (A) is the figure showing the change of the acceleration of the vertical direction of the motion sensor part at the time of performing a roll performance, (B) expands the waveform of B section shown in (A), and is a note on timing. It is the figure which showed an example. It is a flowchart which shows the flow of a process of a stick part. It is a flowchart which shows the flow of a process of a camera unit part. It is a flowchart which shows the flow of a process of a center unit part. It is the figure which showed the modification of note on timing.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. The embodiments described below are given various technically preferable limitations for carrying out the present invention, but the scope of the present invention is not limited to the following embodiments and illustrated examples.

[Outline of musical tone generator]
First, with reference to FIG. 1 (A) and FIG. 1 (B), the outline | summary of one Embodiment of the musical sound generator based on this invention is demonstrated.
FIG. 1A is a diagram showing an outline of a device configuration of a musical sound generating device according to the present embodiment.
As shown in FIG. 1A, the musical sound generator 1 of the present embodiment is configured to include stick units 10A and 10B, a camera unit unit 20, and a center unit unit 30. In addition, in order to implement | achieve the virtual drum performance using two sticks, the musical tone generator 1 of this embodiment shall be provided with two stick parts 10A and 10B, However, the number of stick parts is this. The number is not limited, and may be one or three or more. Hereinafter, when the stick portions 10A and 10B are not distinguished from each other, they are collectively referred to as “stick portion 10”.

The stick portion 10 is a rod-like member extending in the longitudinal direction, and functions as a performance operator for striking that can be held by the performer. That is, the performer performs a performance operation by holding one end (base side) of the stick unit 10 and performing a swing-down operation centering on the wrist or the like.
In order to detect such a player's performance operation, the stick unit 10 as the performance operator is provided with motion detection means for detecting the movement of the stick unit 10 (performance operator) based on the player's performance operation. Has been. In the present embodiment, the other end (tip side) of the stick unit 10 is provided with various sensors such as an acceleration sensor (see motion sensor unit 14 to be described later, see FIG. 2) as motion detection means, and is detected by the various sensors. The detection results (for example, raw data acquired by various sensors) are transmitted to the center unit 30.
In addition, a marker unit 15 is provided on the distal end side of the stick unit 10 so that the camera unit unit 20 can easily determine the distal end of the stick unit 10 during imaging.

  The camera unit 20 is an imaging device that includes an imaging unit 24 (for example, an optical camera, see FIG. 4) that captures a moving image of a performer who performs a performance operation while holding the stick unit 10 at a predetermined frame rate. .

When the center unit 30 receives detection results detected by various sensors (motion sensor unit 14 described later) from the stick unit 10 as motion detection means, the position coordinates of the marker unit 15 at the time of reception of the detection results and the detection results. In accordance with the data, a predetermined musical sound is generated from the sound output unit 362 such as a speaker of the sound generation unit 36 which is a sound generation means.
Specifically, the center unit unit 30 associates with the imaging space of the camera unit unit 20 in a ROM (to be described later) or the like as its storage means, and a virtual drum set D as shown in FIG. Is stored. Based on the position coordinate data of the virtual drum set D and the position coordinate data of the marker unit 15 when the detection result is received by the motion sensor unit 14, the center unit unit 30 specifies the instrument hit by the stick unit 10, A predetermined musical sound corresponding to the musical instrument is generated.
Hereinafter, the tone generator 1 according to the present embodiment will be specifically described.

[Configuration of Musical Sound Generation Device 1]
First, with reference to FIGS. 2 to 5, the configuration of the stick unit 10, the camera unit unit 20, and the center unit unit 30 constituting the musical sound generating device 1 in the present embodiment will be described in detail.
2 is a block diagram illustrating a hardware configuration of the stick unit 10, FIG. 3 is a perspective view of the stick unit 10, and FIG. 4 is a block diagram illustrating a hardware configuration of the camera unit unit 20. FIG. 5 is a block diagram illustrating a hardware configuration of the center unit 30.

[Configuration of Stick Unit 10]
As shown in FIG. 2, the stick unit 10 includes a stick control unit 11 including a CPU (Central Processing Unit), a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, and a motion sensor unit. 14, a marker unit 15, and a data communication unit 16. In addition, the stick unit 10 includes a power supply unit (not shown) that supplies power to each unit.

The ROM 12 stores processing programs for various processes executed by the stick control unit 11.
The ROM 12 stores marker feature information used for light emission control of the marker unit 15.
Here, the marker feature information refers to each marker unit 15 (in this embodiment, the first marker of the stick unit 10A and the second of the stick unit 10B when there are a plurality of marker units 15 in the same imaging space. For example, in addition to the shape, size, hue, saturation, or luminance during light emission, the blinking speed during light emission can be used. When there are a plurality of stick portions 10, each stick portion 10 has different marker feature information, and light emission control is performed according to each marker feature information.

  The RAM 13 stores values acquired or generated in the process, such as various sensor values output by the motion sensor unit 14.

The stick control unit 11 executes control of the entire stick unit 10.
For example, when a sensor value (hereinafter referred to as “motion sensor data”) as a detection result is output from the motion sensor unit 14 which is a motion detection unit, the motion sensor data is transmitted via the data communication unit 16. Communication control with the center unit unit 30 such as transmission to the center unit unit 30.
The stick control unit 11 reads the marker feature information from the ROM 12 and performs light emission control of the marker unit 15 according to the marker feature information. As described above, when there are a plurality of stick portions 10A and 10B, each stick portion 10A and 10B performs light emission control according to different marker characteristic information. This can be detected by distinguishing the marker unit 15 (first marker) of the stick unit 10A and the marker unit 15 (second marker) of the stick unit 10B from the image captured by the imaging unit 24 of the camera unit unit 20. It is for doing so. That is, when there are a plurality of stick parts 10 and the marker part 15 emits light in the same manner, there is a possibility that it is not possible to determine which stick part 10 each marker part 15 belongs to when shooting with the camera unit part 20. is there. For this reason, the stick control unit 11 of the stick unit 10A and the stick control unit 11 of the stick unit 10B read different marker feature information, and perform light emission control of the marker unit 15 according to the read marker feature information, thereby causing each marker unit 15 to emit light. It is easy to distinguish between the two.
The control performed by the stick control unit 11 is not limited to this. For example, the marker unit is based on an instruction from the center unit unit 30 or a sensor value (motion sensor data) that is a detection result by the motion sensor unit 14. You may make it perform light emission control, such as 15 lighting / extinguishing.

The motion sensor unit 14 is a variety of sensors for detecting the state of the stick unit 10. The motion sensor unit 14 is disposed on the stick unit 10 which is a performance operator, and the stick unit 10 (performance operator) body generated by the player's batting performance operation. It functions as a motion detection means for detecting the motion of the. The motion sensor unit 14 outputs a predetermined sensor value (motion sensor data) as a detection result. A detection result (motion sensor data) detected by the motion sensor unit 14 is transmitted to the center unit unit 30 via the data communication unit 16.
Here, as a sensor which comprises the motion sensor part 14, an acceleration sensor, an angular velocity sensor, a magnetic sensor, etc. can be used, for example.

As the acceleration sensor, a three-axis sensor that outputs acceleration generated in each of the three axial directions of the X axis, the Y axis, and the Z axis can be used. For the X axis, Y axis, and Z axis, as shown in FIG. 3, the axis that coincides with the longitudinal axis of the stick 10 is defined as the Y axis, and a substrate (not shown) on which an acceleration sensor is arranged. An axis that is parallel and orthogonal to the Y-axis can be an X-axis, and an axis that is orthogonal to the X-axis and the Y-axis can be a Z-axis. The acceleration sensor may acquire the acceleration of each component of the X axis, the Y axis, and the Z axis, and may calculate a sensor composite value obtained by combining the respective accelerations.
In the present embodiment, when performing using the musical tone generator 1, the performer holds one end (the base side) of the stick unit 10 and the other end (the tip side) of the stick unit 10 around the wrist or the like. By performing a swing-up and swing-down operation that moves up and down, the stick unit 10 is caused to rotate. Here, when the stick unit 10 is stationary, the acceleration sensor calculates a value corresponding to the gravitational acceleration 1G as a sensor composite value, and when the stick unit 10 is rotating, the acceleration sensor. Calculates a value larger than the gravitational acceleration 1G as a sensor composite value. The sensor composite value is obtained, for example, by calculating the square root of the sum of the squares of the accelerations of the X-axis, Y-axis, and Z-axis components.

Moreover, as an angular velocity sensor, the sensor provided with the gyroscope, for example can be used. In the present embodiment, as shown in FIG. 3, the angular velocity sensor outputs a rotation angle 501 of the stick unit 10 in the Y-axis direction and a rotation angle 511 of the stick unit 10 in the X-axis direction.
Here, the rotation angle 501 in the Y-axis direction can be referred to as a roll angle because it is the rotation angle of the front-rear axis viewed from the player when the player holds the stick unit 10. The roll angle corresponds to an angle 502 indicating how much the XY plane is tilted with respect to the X axis, and the player holds the stick unit 10 in his hand and rotates it left and right around the wrist. Caused by.
Further, the rotation angle 511 in the X-axis direction can be referred to as a pitch angle since it is the left-right rotation angle viewed from the player when the player holds the stick unit 10. The pitch angle corresponds to an angle 512 indicating how much the XY plane is tilted with respect to the Y axis, and is generated when the player holds the stick unit 10 in his / her hand and swings his / her wrist up and down.
Although not shown, the angular velocity sensor may also output the rotation angle in the Z-axis direction. At this time, the rotation angle in the Z-axis direction has basically the same property as the rotation angle 511 in the X-axis direction, and is generated when the player holds the stick unit 10 in his hand and swings his wrist in the left-right direction. The pitch angle.

  As the magnetic sensor, a sensor capable of outputting magnetic sensor values in the three-axis directions of the X axis, the Y axis, and the Z axis shown in FIG. 3 can be used. From such a magnetic sensor, a vector indicating north (magnetic north) by a magnet is output in each of the X-axis direction, the Y-axis direction, and the Z-axis direction. Since the components in each axial direction that are output differ depending on the posture (orientation) of the stick unit 10, the stick control unit 11 determines the roll angle of the stick unit 10 and the rotation angles in the X-axis direction and the Z-axis direction from these components. Can be calculated.

Here, the detection result (motion sensor data) detected by the motion sensor unit 14 will be described with reference to FIG. Here, a detection result by an acceleration sensor among the various sensors is shown.
When the performer performs using the stick unit 10, generally, the same operation as the operation of hitting an actual musical instrument (for example, drum) is performed. In such a (performance) operation, the performer first swings up the stick unit 10 and then swings it down toward the hitting surface of the virtual instrument. Since the hitting surface does not actually exist, the performer exerts a force to stop the operation of the stick unit 10 immediately before hitting the stick unit 10 against a virtual musical instrument.
FIG. 6 is a diagram showing a change in acceleration in the vertical direction of the motion sensor unit 14 when a performance operation is performed using such a stick unit 10, and is transmitted from the stick unit 10 to the center unit unit 30. It shows an example of motion sensor data.
The vertical acceleration means the acceleration in the vertical direction with respect to the horizontal plane, and may be calculated by decomposing from the acceleration of the Y-axis component. The acceleration in the Z-axis direction (acceleration in the X-axis direction depending on the roll angle) It is good also as decomposing and calculating from. In FIG. 6, a negative acceleration indicates a downward acceleration applied to the stick unit 10, and a positive acceleration indicates an upward acceleration applied to the stick unit 10.

Even when the stick unit 10 is stationary (the portion indicated by a in FIG. 6), since the gravitational acceleration is applied to the stick unit 10, the motion of the stick unit 10 that is stationary against the gravitational acceleration. The sensor unit 14 detects a constant acceleration in the vertically downward direction, that is, in the minus direction. It should be noted that the acceleration applied to the stick unit 10 is zero when the stick unit 10 is in a free fall state.
Next, when the player lifts the stick unit 10 with the swing-up operation in a state where the stick unit 10 is stationary as in the section indicated by b in FIG. 6, the player moves in a direction more counter to gravitational acceleration. As a result, the acceleration applied to the stick portion 10 increases in the negative direction. Then, when the lifting speed is decreased to make it stand still, the acceleration changes from the minus direction to the plus direction, and the acceleration at the time when the swing-up operation reaches the highest speed (see p1 in FIG. 6) is only the gravitational acceleration ( That is, the gravitational acceleration is 1G).

Next, as in the section indicated by c in FIG. 6, when the stick unit 10 reaches the top by the swing-up operation, the performer performs the swing-down operation of the stick unit 10. In the swing-down operation, the stick unit 10 moves in a direction according to the gravitational acceleration, so that the acceleration applied to the stick unit 10 increases in a plus direction with respect to the gravitational acceleration. Thereafter, when the swing-down operation reaches the highest speed, only the gravitational acceleration (that is, the gravitational acceleration 1G) is again applied to the stick portion 10 (see p2 in FIG. 6).
Thereafter, when the swing-up operation is performed again on the stick unit 10 as in the section indicated by d in FIG. 6, the applied acceleration increases in the negative direction, and when the swing-up operation is to be stopped, the applied acceleration is in the negative direction. Turn to the plus direction.
While the performance operation continues, the change in acceleration as shown in FIG. 6 is repeated according to the player's swinging up and down operation of the stick unit 10, and this change in acceleration is detected by the motion sensor unit 14.

Returning to FIG. 2, the marker unit 15 is a light emitter such as an LED provided on the tip side of the stick unit 10, and emits light and extinguishes according to control from the stick control unit 11.
Specifically, the marker unit 15 emits light based on the marker feature information read from the ROM 12 by the stick control unit 11. As described above, the marker feature information of the stick portion 10A and the marker feature information of the stick portion 10B are set to be different. For this reason, as will be described later, the camera unit 20 can distinguish between the marker portion (first marker) of the stick portion 10A and the marker portion (second marker) of the stick portion 10B, and the respective position coordinates are determined. It is possible to distinguish and obtain which stick unit 10 is.

  The data communication unit 16 performs predetermined wireless communication with at least the center unit unit 30. The predetermined wireless communication may be performed by any method, and in the present embodiment, wireless communication with the center unit unit 30 is performed by infrared communication. Note that the data communication unit 16 may perform wireless communication with the camera unit unit 20, or may perform wireless communication with the stick unit 10A and the stick unit 10B. Further, the communication method by the data communication unit 16 is not limited to wireless communication. For example, the stick unit 10 and the center unit unit 30 may be connected by a cable, and the data communication unit 16 may perform wired communication.

[Configuration of Camera Unit 20]
Next, the configuration of the camera unit 20 will be described with reference to FIG.
The camera unit unit 20 includes a camera control unit 21 including a CPU, a ROM 22, a RAM 23, an imaging unit 24, and a data communication unit 25. The camera unit unit 20 includes a power supply unit (not shown) that supplies power to each unit.

The ROM 22 stores processing programs for various processes executed by the camera control unit 21.
The RAM 23 is data indicating the position of the marker unit 15 obtained by the imaging unit 24 (such as an image of a performer performing a performance operation), and the marker unit 15 (first number) of the stick units 10A and 10B calculated from the data indicating the position. The values acquired or generated in the processing, such as the data of the position coordinates of each of the first marker and the second marker, are stored.
The RAM 23 stores marker detection condition information transmitted from the center unit unit 30. The marker detection condition information is a condition for distinguishing and detecting each of the marker portions 15 of the stick portions 10A and 10B, and is generated from the marker feature information by the main body control portion 31 of the center unit portion 30.

The camera control unit 21 performs control of the entire camera unit unit 20.
In the present embodiment, the camera control unit 21 includes, for example, at least a marker unit such as data indicating the position of the marker unit 15 obtained by the imaging unit 24 (that is, an image of a player performing a performance operation with the stick unit 10). 15 is a marker that calculates the position coordinates of the marker portions 15 (first marker and second marker) of the stick portions 10A and 10B based on the marker detection condition information). Perform position detection processing.
That is, the camera control unit 21 detects the marker unit 15 of the stick unit 10 from the image obtained by the imaging unit 24, and the position coordinates on the space (position coordinates of the marker unit 15 of the stick unit 10 in the imaging space). ). When the marker portions 15 of the plurality of stick portions 10A and 10B are detected, the camera control portion 21 reads the marker detection condition information from the RAM 23, and each marker portion 15 reads the marker portion 15 ( It is determined whether it is the first marker) or the marker portion 15 (first marker and second marker) of the stick portion 10B.
In addition, the camera control unit 21 calculates the position coordinate data (that is, the determination result of the first marker and the second marker) of the calculated marker unit 15 (first marker and second marker) of the stick units 10A and 10B. The associated position coordinate data of each marker unit 15 (hereinafter referred to as “marker detection position information”) is transmitted to the center unit unit 30 via the data communication unit 25.

The image pickup unit 24 is, for example, an optical camera, and picks up an image of the periphery of the stick unit 10 including at least the marker unit 15 such as a moving image of a player who performs a performance operation with the stick unit 10 at a predetermined frame rate.
Imaging data for each frame captured by the imaging unit 24 is output to the camera control unit 21. The camera unit unit 20 specifies the position coordinates of the marker unit 15 of the stick unit 10 in the imaging space, but specifies the position coordinates of the marker unit 15 (that is, marker position detection processing, see FIG. 15). May be performed by the imaging unit 24, or may be performed by the camera control unit 21. Similarly, the imaging unit 24 may determine whether the marker unit 15 in the captured image is the marker unit 15 (first marker and second marker) of which stick unit 10A or 10B. It may be performed in the camera control unit 21.

The data communication unit 25 performs predetermined wireless communication (for example, infrared communication) with at least the center unit unit 30. In the present embodiment, the data communication unit 25 receives marker detection condition information from the center unit unit 30. Further, the data communication unit 25 transmits marker detection position information to the center unit unit 30.
Note that the data communication unit 16 may perform wireless communication with the stick unit 10. Further, the communication method by the data communication unit 25 is not limited to wireless communication. For example, the camera unit unit 20 and the center unit unit 30 may be connected by a cable and the data communication unit 25 may perform wired communication.

[Configuration of Center Unit 30]
Next, the configuration of the center unit 30 will be described with reference to FIG.
The center unit unit 30 includes a main body control unit 31 including a CPU, a ROM 32, a RAM 33, a switch operation detection circuit 34, a display circuit 35, a sound generation unit 36, and a data communication unit 37. Composed. The center unit 30 includes a power supply unit (not shown) that supplies power to each unit.

The ROM 32 stores processing programs for various processes executed by the main body control unit 31.
The ROM 32 includes a sound source storage area (not shown) that stores waveform data of various timbres (that is, sound source data of each musical instrument). This sound source storage area constitutes sound generation means together with a sound generation unit 36 described later. The sound source storage area stores, for example, waveform data of percussion instruments constituting a virtual drum set D (see FIG. 1B) assumed in this embodiment, such as bass drum, hi-hat, snare, cymbal, and tom, for each instrument. It is stored in association with position coordinates in space. The timbre waveform data stored in the sound source storage area of the ROM 32 is not limited to these percussion instruments. For example, timbre waveform data of wind instruments such as flutes, saxophones, and trumpets, keyboard instruments such as pianos, and stringed instruments such as guitars. May be stored in the ROM 32.
In the present embodiment, the sound source storage area of the ROM 32 functions as a storage unit that stores a roll waveform of a single tone that constitutes a musical tone when performing by a roll performance method (hereinafter also referred to as “drum roll performance method”). In the present embodiment, the single-tone roll waveform stored in the sound source storage area of the ROM 32 is obtained by sampling the hitting sound (drum roll sound) at the time of performance by the drum roll playing method different from the hitting sound at the time of normal performance. is there. Note that the single-tone roll waveform is not limited to the sampled percussion sound during the performance by the drum roll performance method, but is the percussion sound during the performance by the normal performance method (for example, the percussion sound of the snare drum during the performance by the normal performance method). However, it is preferable to sample and use the drum roll sound because it is possible to realize a sound like a performance by the drum roll playing method.

The RAM 33 receives information related to the state of the stick unit 10 received from the stick unit 10, that is, a detection result regarding the movement of the stick unit 10 (performance operator) detected by the motion sensor unit 14 that is a motion detection unit of the stick unit 10. Motion sensor data) and the position coordinate data (first marker detection position information and second marker) of the marker unit 15 (first marker and second marker) of the stick units 10A and 10B received from the camera unit unit 20 Detection position information) and the like are stored.
Further, the RAM 33 stores the marker feature information of the marker unit 15 (first marker) of the stick unit 10A and the marker unit 15 (first marker and second marker) of the stick unit 10B transmitted from the stick units 10A and 10B. Storing.

The main body control unit 31 performs control of the center unit unit 30 as a whole.
In the present embodiment, the main body control unit 31 includes a rendition style determination unit 311, a sound generation control unit 312, and the like when viewed functionally. The functions as the rendition style determination unit 311, the sound generation control unit 312 and the like are realized by the cooperation of the CPU and a program stored in the ROM 32 or the like.

Based on the movement of the stick unit 10 (performance operator) detected by the motion sensor unit 14 which is a motion detection means, the performance style determination unit 311 has a performance operation form using the stick unit 10 (performance operator) rolled. This is a performance style determination means for determining whether or not a performance style (drum roll performance style) is used.
Specifically, when a detection result (motion sensor data) relating to the movement of the stick unit 10 (performance operator) detected by the motion sensor unit 14 is transmitted from each stick unit 10 (performance operator), the performance determination unit 311 is based on this, whether the performance operation mode using the stick unit 10 (performance operator) is a drum roll performance method or a normal performance performance mode, that is, the performance operation mode by the performer is drum roll performance. It is determined whether or not the performance operation mode is unique to the performance method.
Here, the “roll performance method” (drum roll performance method) is one of performance methods of a drum that is a percussion instrument, and is a performance method that produces a continuous sound by repeatedly striking the drum surface. Normally, when playing with an actual drum, if the stick part is shaken and the drum hitting surface is hit with the tip, the stick part jumps up by reaction after hitting. The drum roll performance method is a performance method in which a continuous sound is produced by performing quick and fine hitting using the reaction of the stick portion.
Therefore, the performance operation form peculiar to the drum roll playing method is a fine and quick vertical movement of the stick part, and in this embodiment, the stick part 10 (performance operator) is finely swung up and down at a constant tempo. That is, in other words, the amount of movement due to the vertical movement of the stick unit 10 (performance operator) is within a predetermined range (first condition), and each batting performance by the stick unit 10 (performance operator) is performed. When the time between operation timings is within a predetermined range (second condition), the performance style determination unit 311 that is a performance style determination unit determines that the performance style by the performer is a roll performance style (drum roll performance style). .

Here, the amount of movement due to the vertical movement of the stick unit 10 (performance operator) is generated by, for example, a swing-down operation of the stick unit 10 (performance operator) in the detection result detected by the motion sensor unit 14. It appears as the width of the amplitude of the maximum and minimum values of acceleration (see FIG. 7). For this reason, when the maximum value and the minimum value of the acceleration are within a predetermined range, the rendition style determination unit 311 determines that the “first condition” is satisfied. The “predetermined range” in the first condition may be a value indicating that the rhythm is finer than the normal performance, and is set as appropriate. It should be noted that the numerical range of “predetermined range” may be adjusted afterwards by a player or the like.
The time between the hitting performance operation timings is the time between the timings at which the stick unit 10 (performance operator) is swung up and down. The hit performance operation timing is obtained from, for example, the movement of the stick unit 10 (performance operator) detected by the motion sensor unit 14, and the maximum value of the acceleration generated by the swing-down operation of the stick unit 10 (performance operator) Appearing as a point in time (hereinafter referred to as “maximum point”) and a point in time at which a minimum value is reached (hereinafter referred to as “minimum point”) (hereinafter, “maximum point” and “minimum point” are combined) "Pole"). Therefore, the time between hitting performance operation timings can be detected as the time between adjacent extreme points (interval s between the extreme points), and when the interval s between the extreme points is within a predetermined range. The rendition style determination unit 311 determines that the “second condition” is satisfied. The “predetermined range” in the second condition may be a value indicating that the rhythm is inscribed at a constant tempo than the normal performance. For example, the interval s between the extreme points is compared with that in the performance by the normal performance method. For example, it is appropriately set such that it is 20% or less. It should be noted that the numerical range of “predetermined range” may be adjusted afterwards by a player or the like.

  In the present embodiment, the performance style determination unit 311 determines that the performance style of the performer is the drum roll performance style when the stick part continues to move up and down finely to some extent. It is not particularly limited how much the stick portion continues to be finely and rapidly moved up and down, but it is not particularly limited. For example, the last four maximum values including the latest maximum value or the minimum value at the current time point and When all local maxima and local minima satisfy the “first condition” and local maxima corresponding to all local maxima and local minima satisfy the “second condition”. The performance style determination unit 311 determines that the performance style of the performer is the drum roll performance style.

That is, for example, in the case shown in FIG. 7, the last four local maximum values and local minimum values (in FIG. 7, the local minimum point a to the local maximum point d) including the local extreme point (the local minimum point a in FIG. 7). The values of the minimum point a, the maximum point b, and the minimum point c are all within a predetermined range, but the value of the maximum point d is outside the predetermined range. It is in. Therefore, the “first condition” is not satisfied, and the performance style determination unit 311 determines that the performance style of the performer is a performance based on the normal performance style (that is, not the drum roll performance style).
On the other hand, in the example shown in FIG. 8, the last four local maximum values and local minimum values (in FIG. 8, the local minimum point a to the local maximum point) including the local extreme point at present (the local minimum point a in FIG. 8). (value d), the values of the minimum point a to the maximum point d are all within a predetermined range and satisfy the “first condition”. In addition, the distance s between the local minimum point a and the local maximum point b, the distance s between the local maximum point b and the local minimum point c, and the distance s between the local minimum point c and the local maximum point are all short. Since it is within a predetermined range (for example, 20% or less as compared with the performance by the normal performance method), the “second condition” is also satisfied. In this case, the performance style determination unit 311 determines that the performance style of the performer is a performance based on the drum roll performance style.

The rendition style determination unit 311 always determines whether or not the last four local maximum values and minimum values including the latest extreme point at the current time point satisfy the “first condition” and the “second condition”. When any of the conditions is not satisfied, it is determined that the performance by the drum roll playing method has been completed.
That is, for example, as in the example shown in FIG. 9, when the acceleration amplitude does not satisfy the “first condition” at the current time point, as shown in FIG. As in the example, when the last four local maxima and minima (including the local minima and local maxima) including the nearest extreme point at the present time are seen, the interval s between adjacent extremes is outside the predetermined range. When the “second condition” is no longer satisfied as in the case where the performance has become, the performance style determination unit 311 determines that the performance by the drum roll performance style has ended.

Note that the method by which the performance style determination unit 311 determines that the performance by the drum roll performance style has ended is not limited to this.
For example, when a predetermined range is provided for the difference between the maximum value and the minimum value, and the difference between the maximum value and the minimum value exceeds the predetermined range, as in the example shown in FIG. Alternatively, the performance style determination unit 311 may determine that the performance by the drum roll performance style is to be ended.
When the movement of the stick unit 10 is monitored using an acceleration sensor provided in the stick unit 10, the influence of gravity is superimposed as an offset acceleration. For this reason, as described above, this offset acceleration needs to be taken into account when determining the player's performance based on whether or not the value of the extreme point falls within a predetermined range. In this regard, as shown in FIG. 11, the absolute value of the difference between the maximum value and the minimum value (level difference) and the difference between the adjacent extreme points (level difference) is detected, and this value is determined in advance. When determining whether the player's playing style is lower or higher than a predetermined value, it is not necessary to consider the offset acceleration due to the influence of gravity, and the processing becomes easy.
Further, as in the example shown in FIG. 12, for example, if the next pole cannot be detected within a predetermined time, the performance style determination unit 311 may determine that the performance by the drum roll performance style is to be ended. .

  The sound generation control unit 312 corresponds to the detection result (motion sensor data) related to the movement of the stick unit 10 (performance operator) detected by the motion sensor unit 14 which is a motion detection unit and the determination result of the rendition style determination unit 311. It is a sound generation control means for giving a sound generation instruction signal (note on event) to the sound generation section 36, which is a sound generation means, to generate a predetermined musical tone. Here, the note-on event (sound generation instruction signal) includes information such as shot timing (sound generation timing), volume, tone color (that is, instrument type), and the like.

  Specifically, first, when a detection result (motion sensor data) relating to the movement of the stick unit 10 (performance operator) detected by the motion sensor unit 14 is transmitted from each stick unit 10 (performance operator), the sound is generated. Based on this, the control unit 312 detects the state of the stick unit 10 held by the performer (in other words, the performance state of the performer). For example, the sound generation control unit 312 detects in which performance area the operation is performed based on the detection result output by the magnetic sensor or the like. Further, the sound generation control unit 312 detects a virtual instrument hitting timing (shot timing) by the stick unit 10 based on the acceleration (or sensor composite value) output from the acceleration sensor.

Here, detection of shot timing by the sound generation control unit 312 will be described with reference to FIG.
As described above, FIG. 6 is a diagram illustrating a change in acceleration in the vertical direction of the motion sensor unit 14 when a performance operation is performed using the stick unit 10.
Since the performer assumes that the musical sound is generated at the moment when the stick unit 10 is struck on the virtual musical instrument, it is desirable that the musical sound generating device 1 can generate the musical sound at the timing assumed by the performer. . Therefore, in the present embodiment, when the performance style determination unit 311 determines that the performance style of the performer is a performance based on the normal performance style, a moment when the performer strikes the stick portion on the playing surface of the virtual instrument or a slight amount thereof. The sound is to be pronounced in front.

That is, in the present embodiment, the sound generation control unit 312 of the center unit unit 30 performs the swing-up operation after the swing-down operation is performed as the moment when the performer strikes the stick unit 10 on the virtual musical instrument hitting surface. Detect the moment when it starts. That is, the sound generation control unit 312 further increases by a predetermined value in the minus direction from the stationary state, in other words, from the time point when the applied acceleration is only the gravitational acceleration (see p2 in FIG. 6) in the section indicated by d in FIG. The detected point A is detected as shot timing (sound generation timing).
The sound generation control unit 312 generates a note-on event (a sound generation instruction signal) so as to generate sound when this shot timing is detected.

Further, the sound generation control unit 312 is based on the detection result (motion sensor data) regarding the movement of the stick unit 10 (performance operator) detected by the motion sensor unit 14 which is a motion detection unit. The speed and strength of the swing-down operation, the position and angle where the stick unit 10 is swung down are detected. In the present embodiment, the sound generation control unit 312 performs a sound generation control on the sound generation unit 36, and a detection result (the performance operator) detected by the motion sensor unit 14, which is a motion detection unit, is detected. In addition to the sensor value, the marker detection position information of each of the marker units 15 (first marker and second marker) of the stick units 10A and 10B received from the camera unit unit 20 is also considered.
When the performance method determining unit 311 determines that the player's performance method is a normal performance method, the sound generation control unit 312 performs the performance in the normal performance mode (that is, the mode in which the performance method is performed). To generate a note-on event in the normal performance mode. That is, based on the motion sensor data and the marker detection position information, it is determined which instrument is struck at what speed, strength, and timing, and the instrument type (timbre) that matches this determination result. Is read out from the ROM 33, and a sound generation instruction signal (note-on event) is generated as a sound generation unit so that the predetermined musical sound is sounded at a predetermined shot timing with a volume corresponding to the hit speed and strength. To 36.

On the other hand, when the performance method determining unit 311 determines that the player's performance method is a roll performance method, the sound generation control unit 312 performs the performance in the drum roll performance mode (that is, the mode for performing the drum roll performance method). To generate a note-on event for the drum roll performance mode.
FIG. 13A shows an example of motion sensor data in the drum roll performance mode and the normal performance mode, and FIG. 13B shows the waveform of the portion B in FIG. 13A. It is the figure which expanded and showed an example of the note-on timing in the case of drum roll performance mode.
In the drum roll performance mode, the sound generation control unit 312 reads out data of a single-tone roll waveform from the ROM 33 and, for example, as shown in FIG. The local maximum and minimum points (directions, that is, peaks and valleys of the output waveform of the acceleration sensor, tp1 and tp2 in FIG. 13B) are detected, and one sound is detected at each maximum point tp1 and minimum point tp2. Set the shot timing so that the sound based on the roll sound source (drum roll sound) sounds. Then, a sound generation instruction signal (note-on event) is output to the sound generation unit 36 so that the drum roll sound is sounded at a predetermined sound volume with a predetermined sound volume according to the performer's performance.

In addition, the main body control unit 31 performs communication control between the stick unit 10 and the camera unit unit 20 via the data communication unit 37.
In the present embodiment, the main body control unit 31 receives the motion sensor data from the stick unit 10 via the data communication unit 37 and also receives the first marker and marker detection position information of the first marker from the camera unit unit 20. To do.
Further, the main body control unit 31 generates marker detection condition information based on the marker feature information, and transmits the marker detection condition information to the camera unit unit 20 via the data communication unit 37.

The switch operation detection circuit 34 is connected to the switch 341 and receives input information via the switch 341. The input information includes, for example, a change in the tone volume of the tone to be generated, a change in tone color of the tone to be pronounced, and switching of the display on the display device 351.
The display circuit 35 is connected to the display device 351 and performs display control of the display device 351.
The data communication unit 37 performs predetermined wireless communication (for example, infrared communication) between the stick unit 10 and the camera unit unit 20.

The sound generation unit 36 is a functional unit that generates predetermined musical sounds, and includes a musical sound data generation unit 361 and an audio output unit 362. The sound generation unit 36 constitutes sound generation means together with the sound source storage area of the ROM 32.
The musical sound data generation unit 361 is a functional unit that reads waveform data from the sound source storage area of the ROM 32 in accordance with a sound generation instruction from the sound generation control unit 312, generates musical sound data, and converts the generated musical sound data into an analog signal. .
The audio output unit 362 is, for example, a speaker, and generates a musical sound based on the musical sound data generated by the musical sound data generating unit 361. Note that the audio output unit 362 is not limited to a speaker, and may be an output terminal that outputs sound to headphones or the like.
In the present embodiment, the sound generation means rolls from the sound source storage area of the ROM 32 as the storage means when the performance style determination unit 311 as the performance style determination means determines that the performance action performed by the performer is the roll performance style. By reading the waveform a plurality of times in succession, the musical sound for roll performance is generated.
That is, the performance method determination unit 311 determines that the performance action performed by the performer is the roll performance method, and the sound generation control unit 312 causes the sound generation unit 36 to generate the musical sound of the performance by the drum roll performance method. When the sound generation instruction is issued, the musical sound data generation unit 361 continuously reads out the data of the single-tone roll waveform stored in the sound source storage area of the ROM 32 a plurality of times, and uses the single-tone roll waveform as a sound generation control. A plurality of musical tone data are generated on the time axis so as to generate sound at a timing according to the sound generation instruction of the unit 312. For example, a sound generation instruction signal (note-on event) is generated from the sound generation control unit 312 so that notes are turned on at the maximum and minimum points (ie, tp1 and tp2 in FIG. 13B) of the swing-up swing-down of the stick unit 10. Is sent, a single tone roll waveform is arranged on the time axis so that one drum roll sound is generated at each of the maximum point tp1 and the minimum point tp2 of the swing width. Then, by outputting the musical tone data from the audio output unit 362, a plurality of single drum roll sounds are continuously generated, and a performance by a roll performance method (drum roll performance method) in which a percussion instrument is repeatedly hit is performed. You can sound like a musical tone.

[Processing of the musical tone generator 1]
Next, processing of the musical sound generating device 1 will be described with reference to FIGS.
FIG. 14 is a flowchart showing processing of the stick unit 10, FIG. 15 is a flowchart showing processing of the camera unit unit 20, and FIG. 16 is a flowchart showing processing of the center unit unit 30.

[Processing of the stick unit 10]
As shown in FIG. 14, the stick control unit 11 of the stick unit 10 reads the marker feature information stored in the ROM 12 (step S1). In this process, the stick control units 11 of the stick units 10A and 10B read different marker feature information. Different marker characteristic information can be read out by an arbitrary method. For example, the stick units 10A and 10B may be communicated directly or through the center unit unit 30. One marker feature information may be associated in advance for every 10 and the stick control units 11 of the stick units 10A and 10B may read the unique marker feature information associated with each.

When the stick control unit 11 reads the marker feature information, the stick control unit 11 stores the marker feature information in the RAM 13 and transmits it to the center unit unit 30 via the data communication unit 16 (step S2). At this time, the stick control unit 11 transmits the marker feature information to the center unit unit 30 in association with identification information (stick identification information) that can distinguish the stick units 10A and 10B.
Moreover, the stick control part 11 lights the marker part 15 according to marker feature information (step S3).

Subsequently, the stick control unit 11 reads out motion sensor data, that is, sensor values (detection results) output from various sensors from the motion sensor unit 14, and stores them in the RAM 13 (step S4). Thereafter, the stick control unit 11 associates the read motion sensor data with the stick identification information, and transmits it to the center unit unit 30 via the data communication unit 16 (step S5).
The stick control unit 11 determines whether or not the performance operation has been completed (step S6). If it is determined that the performance operation has been completed (step S6; YES), the marker unit 15 is turned off (step S7). . Note that the stick control unit 11 determines that the performance operation has ended, for example, whether there is a certain change in the sensor value output by the motion sensor unit 14, or whether the switch (not shown) of the stick unit is OFF. The case where the performance end signal is transmitted from the center unit 30 or the like. On the other hand, when the stick control unit 11 determines that the performance operation has not ended (step S6; NO), the process returns to step S4 and the process is repeated.

[Processing of Camera Unit 20]
As shown in FIG. 15, the camera control unit 21 of the camera unit unit 20 acquires marker detection condition information transmitted from the center unit unit 30 and stores it in the RAM 23. (Step S11). The marker detection condition information is a condition for detecting each of the marker portions 15 of the stick portions 10A and 10B, and is generated from the marker feature information (see step S21 and step S22 in FIG. 16). As the marker feature information, for example, as described above, the shape, size, hue, saturation, or luminance of the marker can be used.
Subsequently, the camera control unit 21 performs various settings of the imaging unit 24 based on the marker detection condition information (step S12).

  Subsequently, the camera control unit 21 causes the camera of the imaging unit 24 to capture an image including the marker unit 15 and the like of the stick unit 10, and performs first marker position detection processing (step S13) and second marker position detection processing (step). S14) is performed. In these processes, the camera control unit 21 determines the position coordinates of the marker unit 15 (first marker) of the stick unit 10A and the marker unit 15 (second marker) of the stick unit 10B from the image captured by the imaging unit 24. Marker detection position information such as size and angle is acquired and stored in the RAM 23.

Subsequently, the camera control unit 21 transmits the marker detection position information acquired in step S13 and step S14 to the center unit unit 30 via the data communication unit 25 (step S15).
The camera control unit 21 determines whether or not the performance operation has been completed (step S16). If it is determined that the performance operation has been completed (step S16; YES), the process ends and the performance operation ends. If it is determined that it is not present (step S16; NO), the process returns to step S13 and is repeated.

[Processing of Center Unit 30]
As shown in FIG. 16, the main body control unit 31 of the center unit unit 30 receives the marker feature information from the stick unit 10 and stores it in the RAM 33 (step S21). Subsequently, the main body control unit 31 generates marker detection condition information from the marker characteristic information and the detection condition set via the switch 341, and transmits the marker detection condition information to the camera unit unit 20 via the data communication unit 37 (step). S22).

  Subsequently, the main body control unit 31 receives the marker detection position information of the first marker and the second marker from the camera unit unit 20 and stores them in the RAM 33 (step S23). Further, the main body control unit 31 receives motion sensor data such as posture information, shot information, and action information associated with the stick identification information from each of the stick units 10A and 10B, and stores it in the RAM 33 (step S24). .

Subsequently, the main body control unit 31 determines whether or not there is a shot based on the received motion sensor data (step S25). That is, for example, in the case of motion sensor data as shown in FIG. 6, when there is an operation of swinging down the point A shown in FIG. After being performed, the instant at which the swing-up operation is started is detected as a shot timing (sound generation timing). If it is determined that there is no shot (step S25; NO), the main body control unit 31 returns to step S23 and repeats the process.
On the other hand, if it is determined that there is a shot (step S25; YES), the main body control unit 31 (the rendition style determination unit 311) further has four or more swing-up or swing-down operations from the shot to the past. In other words, in the motion sensor data, it is determined whether or not there are four or more undulations (mountain and valley, maximum and minimum points) of the output waveform of the acceleration sensor as shown in FIG. (Step S26).

When it is determined that there are four or more local maximum points and local minimum points (step S26; YES), the main body control unit 31 (the rendition style determination unit 311) further determines the maximum point value (maximum value) of the acceleration. It is determined whether or not the latest four local maximum values and local minimum values among the local minimum values (local minimum values) are within a predetermined range (that is, whether or not “first condition” is satisfied) (step S27). When it is determined that the local maximum and minimum values of the latest four accelerations are within a predetermined range (step S27; YES), the main body control unit 31 (the rendition style determination unit 311) further performs the latest It is determined whether or not the interval s between the four extreme points (that is, the maximum point and the minimum point) is within a predetermined range (that is, whether or not the “second condition” is satisfied) (step S28).
If there is a shot but it is determined that there are not four or more local maximums and local minimums (step S26; NO), it is determined that the local maximum and local minimum values of the latest four accelerations are not within a predetermined range. In the case (step S27; NO), and when it is determined that the interval s between the four most recent extreme points is not within a predetermined range (step S28; NO), the main body control unit 31 (the rendition style determination unit 311). Is determined to be a performance based on a normal performance. In this case, the main body control unit 31 (sound generation control unit 312) outputs a sound generation instruction signal for performance based on the normal performance method to the sound generation unit 36 (step S29). In the normal performance mode, the main body control unit 31 (the sound generation control unit 312), based on the motion sensor data (for example, the sensor combined value of the acceleration sensor), etc., the sound generation timing, the volume of the tone to be generated, the tone color (the type of instrument) Generate a note-on event including etc. The volume of the musical sound can be obtained from the maximum value of the sensor composite value, for example. As a result, at the predetermined sounding timing in the normal performance mode, the musical sound based on the waveform of the predetermined musical instrument is generated from the sounding unit 36 at a predetermined volume.
In this case, the main body control unit 31 always determines whether or not the performance has ended (step S30). If it is determined that the performance has ended (step S30; YES), the musical tone generation process ends. . If it is determined that the performance has not ended (step S30; NO), the process returns to step S23 and the process is repeated.

On the other hand, when the main body control unit 31 (the rendition style determination unit 311) determines that the interval s between the four nearest poles is within a predetermined range (that is, both the “first condition” and the “second condition” are satisfied). If YES in step S28, the main body control unit 31 (sound generation control unit 312) outputs to the sound generation unit 36 a sound generation instruction signal based on the drum roll performance method that generates a drum roll sound based on the roll waveform. (Step S31). As a result, at a predetermined sound generation timing in the drum roll performance mode, a musical sound based on the roll waveform is generated from the sound generator 36 at a predetermined volume.
In this case, the main body control unit 31 (the rendition style determination unit 311) always determines whether the maximum and minimum values of acceleration exceed a predetermined range, or whether the interval s between adjacent extreme points exceeds a predetermined range. (That is, whether or not any of the “first condition” and “second condition” has been lost) is determined (step S32). If it is determined that none of the conditions is missing (step S32; NO), the main body control unit 31 returns to step S23 and repeats the process while continuing to perform the drum roll performance.
On the other hand, when the main body control unit 31 (playing style determination unit 311) determines that any of the conditions is missing (step S32; YES), the main body control unit 31 (sound generation control unit 312) sets the drum roll performance mode. The note is turned off and a transition is made to the normal performance mode (step S33). That is, when the next shot is detected, a sound generation instruction signal is output to the sound generation unit 36 so that a musical sound based on a sound source (waveform data) in a performance based on the normal performance method is generated at a predetermined timing in accordance with the performance operation. In this case, the main body control unit 31 always determines whether or not the performance operation has ended (step S30), in the same manner as when the sound generation instruction signal for the performance based on the normal performance method is output to the sound generation unit 36 (step S29). If it is determined that the performance operation has been completed (step S30; YES), the musical tone generation process is terminated. If it is determined that the performance operation has not ended (step S30; NO), the process returns to step S23 and the process is repeated.

As described above, according to the musical sound generating apparatus 1 of the present embodiment, the stick unit 10 (detected by the motion sensor unit 14 which is a motion detecting unit that detects the movement of the performance operator based on the performance operation of the performer. Based on the movement of the performance operator), it is determined whether the performer's performance is a normal performance or a drum roll performance, and from the sound generation unit according to the determination result and the detection result of the motion sensor unit 14. Play a specific musical tone. For this reason, if the performer wants to perform with the drum roll playing technique, the drum roll playing technique performs an operation specific to the drum roll. The sound can be generated, and the performance by the drum roll playing method can be enjoyed even in the performance in the air where there is no actual musical instrument. In addition, in this embodiment, only the operation specific to the drum roll performance is performed and the sound generation control is performed accordingly, so even a performer who does not have the technique to perform the drum roll performance with an actual drum can easily perform the drum roll himself. You can enjoy the sensation of playing a performance.
Also, in the case of performance by the drum roll performance method, sound generation control is performed according to the detection result of the motion sensor unit 14, so that the speed of the rhythm engraved by the player and the striking surface are adjusted according to the performance of the player. A drum roll sound is generated (at a timing corresponding to the strength of hitting, etc.), and the performer can enjoy the performance as if he was actually performing the drum roll performance.
The rendition style determination unit 311 performs drum roll when the movement amount due to the vertical movement of the stick unit 10 (performance operator) detected by the motion sensor unit 14 which is a motion detection unit is within a predetermined range. Judged that the performance is a performance. For this reason, the musical performance of the drum roll performance method can be generated by performing a performance operation in which the performance operator is moved up and down in small increments in the same manner as in the case of performance by the actual drum roll performance method.
In addition, the rendition style determination unit 311 performs the roll rendition style when the time between the hitting performance operation timings of the stick unit 10 (performance operator) detected by the motion sensor unit 14 serving as motion detection means is within a predetermined range. It is judged that. For this reason, as in the case of the actual drum roll performance method, the musical performance of the performance by the drum roll performance method can be generated by performing the performance operation in which the performance operator is continuously operated at a constant rhythm.
In the present embodiment, a single-note roll waveform obtained by sampling the hitting sound during the performance by the drum roll playing method different from the hitting sound during the normal performance is stored in the ROM 32, and this single note is played during the drum roll playing method. A plurality of roll waveforms are continuously generated. In the drum roll technique, the striking surface is struck quickly in small increments, and a continuous sound is produced by using the recoil that the performance operator hits the striking surface. A slightly different sound is produced. In this respect, in this embodiment, since the drum roll performance music is generated by using the hit sound during the performance by the drum roll performance, the performance is performed with the same sound as when the performance is actually performed by the drum roll performance. be able to.

  Although the embodiments of the present invention have been described above, the present invention is not limited to such embodiments, and various modifications can be made without departing from the scope of the present invention.

For example, arbitrary processing among the processing to be performed by the stick unit 10, the camera unit unit 20, and the center unit unit 30 in the above embodiment is performed by other units (stick unit 10, camera unit unit 20, and center unit unit 30. ).
That is, for example, the stick control unit 11 of the stick unit 10 (performance operator) may function as a sound generation control unit and a performance method determination unit, and the stick control unit 11 may analyze detection results of various sensors. In this case, not the detection result of the motion sensor unit 14 as the motion detection means (raw data of various sensors, etc.) but a note-on event (pronunciation instruction signal) based on the detection results of the various sensors. Sent to.
Further, for example, in the main body control unit 31 of the center unit unit 30, data indicating the position of the marker unit 15 obtained by the imaging unit 24 of the camera unit unit 20 (that is, data of an image obtained by imaging a performer or the like) and marker characteristics Based on the information, processing for calculating the respective position coordinates of the marker portions 15 (first marker and second marker) of the stick portions 10A and 10B may be performed. In this case, the camera unit unit 20 only needs to capture an image and transmit it to the center unit unit 30, so that the marker unit 15 of each of the stick units 10 </ b> A and 10 </ b> B is identified in the RAM 23 of the camera unit unit 20. It is not necessary to store the marker detection condition information.

In addition, the performance method determination unit 311 determines whether the performance performance of the performer is determined to be a performance based on a normal performance method or a performance based on a roll performance method (drum roll performance method), and the performance based on the drum roll performance method ends. The method for determining whether or not the operation has been performed is not limited to that shown in the present embodiment.
For example, only the “first condition” that the amount of movement due to the vertical movement of the stick unit 10 (performance operator) detected by the motion sensor unit 14 that is a motion detection unit is within a predetermined range can be satisfied. For example, it may be determined that it is a roll performance method (drum roll performance method), and the time between hitting performance operation timings of the stick unit 10 (performance operator) detected by the motion sensor unit 14 is within a predetermined range. If only the “second condition” is satisfied, it may be determined that the roll performance method (drum roll performance method) is used. If either one of the “first condition” and the “second condition” is satisfied, it may be determined that the roll performance method (drum roll performance method) is used.

In the present embodiment, the rendition style determination unit 311 that is a rendition style determination unit performs the vertical direction of the stick unit 10 (performance operator) based on a change in waveform (for example, a change in the output waveform of the acceleration sensor) in the motion sensor data. Whether the movement amount due to the movement of the stick is within a predetermined range (first condition), or the time between the operation timings of the stick unit 10 (performance operator) is within a predetermined range (second condition) However, the method of determining the performance method of the performer by the performance method determination unit 311 is not limited to this.
For example, the present invention is not limited to determining the player's performance based on the movement of the stick unit 10 (performance operator) detected by the motion sensor unit 14, and an image obtained by capturing the performance operation acquired by the camera unit unit 20. Based on this, by detecting the swing width of the actual stick unit 10 (performance operator) or the like, it is determined whether or not the performance style determination unit 311 satisfies the first condition and the second condition. A person's performance may be judged.

Further, in the present embodiment, the example in which the shot timing in the case of the drum roll performance is set as the maximum point tp1 and the minimum point tp2 has been described, but the shot timing in the case of the drum roll performance is limited to the maximum point tp1 and the minimum point tp2. Not.
For example, as shown in FIG. 17, when one cycle is between two maximum points tp1 or two minimum points tp2, the maximum points tp1 and minimum points tp2, and the maximum points tp1 and minimum points tp2 The shot timing may be set so that the sound is generated at approximately the middle point. Further, the sound may be generated more finely than this, for example, six times or more in one cycle. Further, the number of times the drum roll sound is generated may be dynamically increased or decreased according to the time interval between the maximum point tp1 and the minimum point tp2. As described above, when the number of times to generate the drum roll sound is increased or decreased according to the time interval between the maximum point tp1 and the minimum point tp2, for example, a performance by an actual drum roll performance method can be performed. Even performers who are not able to perform performances as fast as possible, have the experience of performing themselves like a drum roll playing technique in which fine hitting sounds are repeated by increasing the number of times of sounding in one cycle. be able to.

Further, in this embodiment, when the performance style determination unit 311 determines that the performance is based on the drum roll performance style, a case where a single roll waveform is generated at a predetermined shot timing (for example, the maximum point tp1 and the minimum point tp2) is taken as an example. However, the sound generation pattern in the case of a performance by the drum roll playing method is not limited to such that a plurality of single sound roll waveforms are continuously generated at a predetermined shot timing.
For example, waveform data of a drum roll sound as a continuous sound (a continuous sound that is generated when a performance is actually performed by a drum roll performance) is stored in the sound source storage area of the ROM 32, and the performance determination unit 311 performs drum recording. When it is determined that the performance is based on the roll performance method and the transition to the drum roll performance mode is made, the waveform data of the drum roll sound as the continuous sound is read out, and the note is turned on at the transition to the drum roll performance mode. You may make it note-off this continuous sound when it complete | finishes and it changes to normal performance mode.
As exemplified in the present embodiment, when a single-tone roll waveform is sounded at a predetermined timing in accordance with the performer's performance, the skill and expression of the performer are reflected in the drum roll sound in the drum roll performance mode. If the performer is not familiar with the performance technique, the sound in the drum roll performance mode is likely to vary. It may sound like a poor drum roll.
On the other hand, when a drum roll sound is generated as a continuous sound in the drum roll performance mode, the same sound as the performance by the actual drum roll performance is generated as long as the movement satisfies a predetermined condition. Therefore, individual differences do not appear in the performance in the drum roll performance mode, and anyone can produce a stable drum roll sound regardless of their level of performance.
For this reason, for example, both a single roll waveform and waveform data of a drum roll sound as a continuous sound are stored in the ROM 32, and any waveform data (sound source) is used when performing the drum roll performance. It may be possible for the performer to freely select whether or not to turn on the note at the timing according to his / her skill or the like.

  Further, in the present embodiment, the case where the sound generation unit 36 which is a sound generation unit is provided in the center unit unit 30 has been described as an example, but the sound generation unit may be configured separately from the center unit unit 30. Good. In this case, the sounding means and the center unit 30 are connected by wire or wirelessly, and the sounding means performs a predetermined sound according to an instruction signal from the center unit 30.

  In the present embodiment, the ROM 32 stores a single tone roll waveform obtained by sampling the hitting sound during the performance by the drum roll playing method different from the hitting sound at the time of the performance by the normal performance method. However, the sound source used in the performance by the drum roll playing method is not limited to this.

  Further, in the present embodiment, the case where the motion detection unit that detects the movement of the stick unit 10 (performance operator) based on the performance operation of the performer is the motion sensor unit 14 disposed on the stick unit 10 has been described as an example. However, the motion detection means is not limited to this. For example, the movement of the stick unit 10 (performance operator) may be detected from an image acquired by the imaging unit 24 (camera) of the camera unit unit 20. In this case, the imaging unit 24 ( Camera) functions as a motion detection means.

In the present embodiment, only the acceleration data measured by the acceleration sensor is exemplified as the motion sensor data acquired by the motion sensor unit 14 as the motion detection means, but the content of the motion sensor data is limited to this. Instead, for example, the angular acceleration may be measured by a gyro and the measured value may be used.
In the present embodiment, the rotation around the axis parallel to the stick portion 10 is not described. However, these rotations may be measured by an angular acceleration sensor or the like.

  In the present embodiment, the virtual drum set D (see FIG. 1B) is described as an example of the virtual percussion instrument. However, the present invention is not limited to this, and the present invention can swing the stick unit 10 down. It can be applied to other musical instruments such as xylophones that generate musical sounds by movement.

In the present embodiment, the performance operator is a stick-shaped stick unit 10 as an example, but the performance operator is not limited to this. For example, a portable terminal device such as a mobile phone may be used as a performance operator.
The musical sound generating device 1 generates a sound of a predetermined instrument by performing a performance operation of hitting a space with a performance operator, and does not actually hit the performance operator on the strike surface of the instrument. Even when a precision electronic device such as a device is used as a performance operator, there is no possibility that the performance operator will be damaged.

  In the present embodiment, the stick unit 10, the camera unit unit 20, and the center unit unit 30 are all provided with a power supply unit. However, the stick unit 10, the camera unit unit 20, and the center unit unit 30 are provided. May be connected to an external power source to supply power from the external power source.

Although several embodiments of the present invention have been described above, the scope of the present invention is not limited to the above-described embodiments, but includes the scope of the invention described in the claims and equivalents thereof. .
The invention described in the scope of claims attached to the application of this application will be added below. The item numbers of the claims described in the appendix are as set forth in the claims attached to the application of this application.
[Appendix]
<Claim 1>
A performance operator for striking that the performer can hold,
Movement detecting means for detecting movement of the performance operator main body caused by the player's batting performance operation;
Based on the movement detected by the movement detection means, a performance style determination means for determining whether or not the performance operation mode using the performance operator is a roll performance style;
A sounding instruction means for instructing the sounding means to pronounce a musical sound for roll performance at a timing determined based on the movement detected by the motion detecting means when the rendition style determining means determines that it is a roll performance;
A musical sound generating device comprising:
<Claim 2>
The performance style determination means determines that the performance style is a roll performance style when the amount of movement of the performance operator detected by the motion detection means is within a predetermined range. Item 2. A musical sound generator according to Item 1.
<Claim 3>
The rendition style determination means determines that it is a roll rendition style when the time between the respective hit performance operation timings obtained from the movement of the performance operator detected by the movement detection means is within a predetermined range. The musical tone generator according to claim 1 or 2, characterized in that.
<Claim 4>
The pronunciation means is:
Comprising storage means for storing a single roll waveform;
2. When the performance style determination means determines that the performance style is a roll performance style, the roll performance music is pronounced by continuously reading a roll waveform from the storage means a plurality of times. The musical sound generating device according to any one of items 3 to 4.

DESCRIPTION OF SYMBOLS 1 Musical sound generator 10 Stick part 14 Motion sensor part 15 Marker part 20 Camera unit part 24 Imaging part 30 Center unit part 31 Control part 36 Sound generation part 311 Performance method judgment part 312 Sound generation control part 361 Musical sound data generation part 362 Audio | voice output part D Drum set

Claims (2)

A performance operator for striking that the performer can hold,
Movement detecting means for detecting movement of the performance operator main body caused by the player's batting performance operation;
The maximum value and the minimum value of the acceleration generated by the vertical movement of the performance operator detected by the motion detection unit are within a predetermined range, and are detected by the motion detection unit. Rendition style determination means for determining that it is a roll rendition when the time between the maximum and minimum values of acceleration generated by the operation of the performance operator is within a predetermined range ;
A sounding instruction means for instructing the sounding means to pronounce a musical sound for roll performance at a timing determined based on the movement detected by the motion detecting means when the rendition style determining means determines that it is a roll performance;
A musical sound generating device comprising: The pronunciation means is:
Comprising storage means for storing a single roll waveform;
When it is determined that the roll performance by the rendition style determination section, by reading the roll waveform multiple times consecutively from said memory unit, according to claim 1, characterized in that the pronunciation of roll performance music sound Musical sound generator.

Images