JP7106091B2 - Performance support system and control method - Google Patents

Description

The present invention relates to a performance support system and control method.

For example, at a musical instrument practice or at a concert, music data such as MIDI standard (Musical Instrument Digital Interface) (registered trademark) is played back through a speaker or the like, and the performer plays the musical instrument along with the played music. Attempts are being made to play

The above-mentioned music data can be created in advance. In that case, even if music data rich in expression is created by adding intonation, ups and downs, dynamics, pacing, etc., when it is actually played, it will not be reproduced as it was created. only be played. For this reason, the performer may end up in a situation where he or she just needs to tune in to the reproduced music. As a result, the performance becomes mechanical and the so-called human touch is lost.

As means for arranging the tempo and dynamics of music while reproducing music data, a device as described in Patent Document 1 is known. According to this device, it is possible to freely change the tempo and dynamics of the music being reproduced using a device such as a baton.

However, it is absolutely impossible for oneself to play a musical instrument while arranging a piece of music that is being reproduced using a device such as that described in Patent Document 1.

As described above, when a performer plays a musical instrument along with a piece of music whose music data has been reproduced, it is difficult to obtain an expressive piece of music.

JP-A-8-305355

SUMMARY OF THE INVENTION It is an object of the present invention to provide a performance support system and a control method that enable a performer to play a musical instrument along with a piece of music whose music data has been reproduced, to obtain a richly expressive piece of music.

Such objects are achieved by the present invention of the following (1) to ( 9 ).
(1) a reproducing unit that reproduces music data as music;
a motion detection unit that is worn by a performer who plays a musical instrument in time with the music played back by the playback unit and detects a motion of the performer;
a receiver that receives motion information about the performer's motion detected by the motion detector;
a control unit that corrects reproduction conditions of the reproduction unit based on the operation information received by the reception unit;
The motion detection unit continuously detects the motion information while the performer is performing,
The control unit calculates an average value of the tempo played by the performer over a predetermined period of time, and if the average value is within a predetermined range, the tempo of the song data that has not yet been reproduced is calculated. A performance support system , wherein the tempo of unreproduced data is not corrected, and the tempo of the unreproduced data is corrected when the average value is out of a predetermined range .

( 2 ) The performance according to ( 1 ) above, wherein the control unit detects the tempo of the performance by the player from the motion information, and corrects the tempo of reproduction by the reproduction unit based on the detection result. support system.

( 3 ) The above (1) or (2) , wherein the control unit calculates the average value at predetermined time intervals, and uses the latest average value for determining whether or not to correct the tempo of the unreproduced data. The performance support system described in .

( 4 ) The control unit detects the intensity of the performance of the performer from the motion information, corrects the intensity of the unreproduced data based on the detection result, and causes the reproduction unit to reproduce the above ( 1 ). ) to ( 3 ).

( 5 ) The performance support system according to any one of (1) to ( 4 ) above, wherein the motion detection section has an angular velocity sensor.

( 6 ) The performance support system according to any one of (1) to ( 5 ) above, wherein the playback unit includes a performance device for playing the musical instrument.

(7) a reproducing unit that reproduces music data as music;
a motion detection unit that is worn by a performer who plays a musical instrument in time with the music played back by the playback unit and detects a motion of the performer;
a receiver that receives motion information about the performer's motion detected by the motion detector;
a control unit that corrects reproduction conditions of the reproduction unit based on the operation information received by the reception unit;
The control unit corrects unreproduced data of the music data that has not yet been reproduced, and causes the reproduction unit to reproduce the data, detects the intensity of the performance of the performer from the operation information, A musical performance support system that corrects strength of the unreproduced data based on the detection result and causes the reproduction unit to reproduce the data.
( 8 ) A reproduction unit that reproduces music data as music, and is worn by a performer who plays a musical instrument in time with the music reproduced by the reproduction unit. A control method for controlling a performance support system comprising: a motion detection unit that detects motion information about a performer's motion; and a reception unit that receives the motion information detected by the motion detection unit,
calculating the average value of the tempo played by the performer over a predetermined period of time, and if the average value is within a predetermined range, the tempo of unreproduced data of the music data that has not yet been reproduced; is not corrected, and the tempo of the unreproduced data is corrected when the average value is outside a predetermined range .
(9) a reproduction unit that reproduces music data as music; an action detection unit that is worn by a performer who plays a musical instrument in time with the music reproduced by the reproduction unit and detects an action of the performer; A control method for controlling a performance support system comprising a receiving unit for receiving motion information about the performer's motion detected by a detecting unit,
The intensity of the performance of the performer is detected from the motion information, and based on the detection result, the intensity of the unreproduced data among the music data is corrected, and the reproducing unit reproduces the data. A control method characterized by:

According to the present invention, it is possible to provide a performance support system and a control method that enable a performer to obtain a richly expressive piece of music when performing a musical instrument along with the piece of music whose music data has been reproduced.

FIG. 1 is a schematic block diagram of the performance support system of the present invention. FIG. 2 is a block diagram of the performance support system shown in FIG. 3 is a side view showing an example of the playing device shown in FIG. 1. FIG. FIG. 4 is a diagram showing a state in which the action detection section shown in FIG. 1 is worn by a player. FIG. 5 is a diagram showing a state in which a player is playing a musical instrument with the action detection unit shown in FIG. 1 attached to the player. FIG. 6 is a flow chart for explaining the control operation (control method of the present invention) performed by the control device shown in FIG. FIG. 7 is a graph in which the vertical axis represents the angular velocity detected by the motion detection unit and the horizontal axis represents time. FIG. 8 is a graph in which the vertical axis represents BPM and the horizontal axis represents time. FIG. 9 is a graph in which the vertical axis represents BPM and the horizontal axis represents time. FIG. 10 is a curve showing the BPM of music reproduced by the reproducing unit. FIG. 11 is a curve showing the BPM of music reproduced by the reproducing unit. FIG. 12 is a curve showing the BPM of music reproduced by the reproduction unit. FIG. 13 is a curve showing the BPM of music played back by the playback unit.

BEST MODE FOR CARRYING OUT THE INVENTION A performance support system and a control method according to the present invention will be described in detail below with reference to preferred embodiments shown in the accompanying drawings.

FIG. 1 is a schematic block diagram of the performance support system of the present invention. FIG. 2 is a block diagram of the performance support system shown in FIG. 3 is a side view showing an example of the playing device shown in FIG. 1. FIG. FIG. 4 is a diagram showing a state in which the action detection section shown in FIG. 1 is worn by a player. FIG. 5 is a diagram showing a state in which a player is playing a musical instrument with the action detection unit shown in FIG. 1 attached to the player. FIG. 6 is a flow chart for explaining the control operation (control method of the present invention) performed by the control device shown in FIG. FIG. 7 is a graph in which the vertical axis represents the angular velocity detected by the motion detection unit and the horizontal axis represents time. 8 and 9 are graphs in which the vertical axis represents BPM and the horizontal axis represents time. 10 to 13 are curves showing the BPM of music reproduced by the reproduction unit.

A performance support system 1 shown in FIG. 1 is a system that supports the performance of a performer 100 when the performer 100 plays a musical instrument 200 along with a piece of music to be reproduced. This musical performance support system 1 includes a reproducing section 2, a motion detecting section 3 worn by a player 100, a display device 4 for displaying the tempo of the performance performed by the player, and a control device 5 for controlling these sections. , is equipped with

Each section of the performance support system 1 will be described in detail below.
[Playback part]
The reproducing unit 2 shown in FIG. 1 has a function of reproducing (outputting) music data stored in a memory 53 (see FIG. 2) of the control device 5, which will be described later, as music.

The playback section 2 includes a speaker 21 and a performance device 22 that plays the musical instrument 300 .

The speaker 21 is electrically connected to the control device 5, converts an electric signal of music data transmitted from the control device 5 into a physical signal, emits sound, and reproduces music. The music data reproduced by the speaker 21 includes various audio data and MIDI data (music data conforming to the MIDI standard (Musical Instrument Digital Interface) (registered trademark)), which will be described in detail later.

The communication between the speaker 21 and the control device 5 may be either wireless or wired. Moreover, the speaker 21 may be provided with an adjustment unit for adjusting the volume.

Further, in the illustrated configuration, the reproducing unit 2 has one speaker 21 , but the present invention is not limited to this, and the reproducing unit 2 may have a plurality of speakers 21 .

A performance device 22 shown in FIG. 3 is a performance device that plays a musical instrument 300 .
The musical instrument 300 is not particularly limited, and includes, for example, a percussion instrument, a string instrument, a keyboard instrument, a wind instrument, and the like, which will be described later. , hi-hat, ride cymbal, crash cymbal).

In this specification, a musical instrument is one that emits sound by itself, one that amplifies the sound it emits and outputs it from a speaker or the like, or one that does not emit sound but receives an electrical signal. This includes output from speakers, etc.

The performance device 22 has a striking section 221 provided for each section (snare drum, bass drum, hi-hat, ride cymbal, crash cymbal) of the drum 301 and a driving section 222 for driving the striking section 221 .

Since each striking part 221 and each driving part 222 have substantially the same configuration except that the installation position and posture are different, the striking part 221 and the driving part 222 installed on the snare drum (hereinafter referred to as snare drum 302) will be described below. The drive unit 222 will be described representatively.

The hitting portion 221 includes a stick 221a, a rotation support portion 221b that rotatably supports the base end portion (grip) of the stick 221a, a spring 221c that is connected to the middle of the stick 221a in the longitudinal direction, and a stopper 221d. have

The stick 221a is not particularly limited as long as it can make a sound when it collides with the snare head. For example, a sponge body or the like may be attached to the tip (tip) of the drum 301 (snare drum, bass drum, hi-hat, ride cymbal, crash cymbal).

The rotation support portion 221b supports the stick 221a so as to be rotatable around the rotation shaft 221e. The rotation support portion 221b is installed above (separate side) the head 303 (striking surface). Also, the rotating shaft 221 e is substantially parallel to the head 303 of the snare drum 302 .

The rotation support portion 221b allows the stick 221a to contact (hit) the head 303 of the snare drum 302 and to separate from it. When the snare drum 302 contacts (hit) the head 303 of the snare drum 302, the tip of the stick 221a contacts the head 303, and the middle of the stick 221a in the longitudinal direction (shoulder) contacts the rim 304. is preferred. As a result, a clear striking sound can be obtained.

The spring 221c is connected in the middle of the longitudinal direction of the stick 221a and urges the stick 221a in the direction away from the snare drum 302. As shown in FIG. As a result, the stick 221a is separated from the snare drum 302 in a natural state where no external force is applied to the stick 221a.

The stopper 221d regulates the movement limit of the stick 221a. As a result, it is possible to prevent the position of the stick 221a from unintentionally deviating, and it is possible to obtain a stable hitting sound.

The driving portion 222 is composed of a solenoid having a rod 222a. The rod 222a is configured to be freely retractable by a drive source (not shown) that is driven by energization, and can take a state of striking the grip of the stick 221a and a state of being separated from the stick 221a.

When the rod 222a hits (presses) the stick 221a, the stick 221a rotates against the biasing force of the spring 221c, as indicated by the two-dot chain line in FIG. As the head 303 is hit, the middle (shoulder) of the stick 221 a in the longitudinal direction comes into contact with the rim 304 . This allows the snare drum 302 to produce an impact sound.

On the other hand, when the rod 222a is retracted and separated from the stick 221a, the stick 221a is separated from the snare drum 302 by the biasing force of the spring 221c.

The snare drum 302 can be played by repeating the appearance and disappearance of the rod 222a.

Such drive unit 222 is electrically connected to the control device 5 and its operation is controlled. The communication between the drive unit 222 and the control device 5 may be either wireless or wired.

The playback unit 2 has been described above. In particular, since the reproduction unit 2 has the performance device 22 that plays the musical instrument 300, it is possible to produce a sense of realism such as harmonic overtones of the musical instrument that cannot be reproduced by a speaker or the like.

In the present embodiment, the reproduction section 2 has the speaker 21 and the performance device 22. However, the reproduction section 2 may have only one of them.

Also, the playback unit 2 may be a device worn by the performer 100, such as earphones or headphones. As a result, for example, when the performer 100 practices the musical instrument 200 at home, he/she can enjoy playing in concert with the reproducing unit 2 without worrying about the volume. In this configuration, a jack to which an earphone or headphone is connected may be provided in a housing (not shown) in which the control device 5 is housed or in the speaker 21 .

Further, hereinafter, the music reproduced by the speaker 21 and the music played by the performance device 22 are collectively referred to simply as "music by the reproduction unit 2".

[Motion detector]
The motion detector 3 shown in FIGS. 4 and 5 has a function of detecting the motion of the performer 100 playing the musical instrument 200 .

The musical instrument 200 is not particularly limited, and includes, for example, a percussion instrument, a stringed instrument, a keyboard instrument, and a wind instrument.

The percussion instruments include drums, electronic drums, Japanese drums, cajon, djembe, tambourine, triangle, castanets, wood blocks, cowbells, clappers, claves, wooden fish, bongos, timpani, concert chimes, wind chimes, cabasas, Examples include Sleibel, Timpalace, Agogo, and the like.

Examples of stringed instruments include acoustic guitars, electric guitars, acoustic basses, electric basses, harps, mandolins, violins, violas, cellos, double basses, and the like.

Examples of keyboard instruments include pianos, honky-tonk pianos, electric pianos, keyboards, harpsichords, celestas, organs, pipe organs, reed organs, electones, keyboard harmonicas, accordions, and synthesizers.

Examples of the wind instruments include woodwind instruments such as flute, clarinet, oboe, saxophone, bassoon, sho, and bagpipes, and brass instruments such as trumpet, trumpet, horn, euphonium, tuba, whistle, and samba whistle.

Among these musical instruments, for example, percussion instruments and the like, which require a large amount of movement during performance, and instruments which exhibit a large reaction force at the moment of contact (impact) with the instrument during performance, are preferable. As a result, the motion detection unit 3 can clearly detect the motion, and the motion of the performer 100 can be detected more conspicuously.

The motion detector 3 is configured by an angular velocity sensor 31 that detects an angular velocity in this embodiment. The angular velocity sensor 31 has a housing 311, a sensor element 312 built in the housing 311, and a communication unit 313 that communicates with the control device 5 (see FIG. 2).

The housing 311 is attached to the wrist 101 of the performer 100 with the wristband 102 . It should be noted that the configuration is not limited to that shown in the drawings, and may be worn on any part such as fingers, head, abdomen, or feet. Further, a plurality of motion detection units 3 may be attached to arbitrary parts of the player 100 .

The sensor element 312 may be of any type of capacitance type, strain gauge type, piezoelectric type, eddy current type, electromagnetic induction type, or optical type. Information on the angular velocity detected by the sensor element 312 is converted into an electrical signal and transmitted to the control device 5 by the communication unit 313 . Communication between the communication unit 313 and the control device 5 may be either wireless or wired.

Here, FIG. 4 and FIG. 5 illustrate x-, y-, and z-axes that intersect (perpendicular to) each other. The x-axis is an axis along the horizontal direction and an axis along the left-right direction of the player 100 . The y-axis is the axis along the horizontal direction and the axis along the front-rear direction. The z-axis is the axis along the vertical direction.

As shown in FIG. 5, when the player 100 plays the conga 201, that is, hits the head 202 of the conga 201, the player 100 rotates the hand. During this movement, the hand and motion detector 3 rotate about the x-axis. Therefore, when the performer 100 is performing, the angular velocity around the x-axis changes significantly (see FIG. 7). Therefore, the movement of the performer 100 can be detected more reliably.

It should be noted that the axis for detecting the action of the performer 100 more conspicuously differs depending on the type of musical instrument. For example, when playing the guitar, the hand holding the pick of the player 100 rotates about the y-axis. Therefore, the change in angular velocity around the y-axis becomes remarkable.

Also, an acceleration sensor may be used in place of the angular velocity sensor depending on the type of musical instrument. For example, when the player 100 plays the violin, the hand that holds the bow makes reciprocating motions substantially along the x-axis direction. Therefore, in this case, when the acceleration sensor is used, the change in the acceleration in the x-axis direction becomes remarkable, and the movement of the performer 100 can be detected more reliably.

As described above, in the present embodiment, since the player 100 plays the conga 201, the angular velocity sensor is built in. However, the present invention is not limited to this. A sensor, pressure sensor, displacement sensor, or the like may be used. Further, a housing 311 having a plurality of these sensors built therein may be used.

In addition, in this embodiment, the action detection section 3 operates continuously while the performer 100 is performing.

[Display device]
The display device 4 shown in FIG. 1 has a display screen 41 , operation buttons 42 and a communication section 43 .

The display screen 41 is composed of, for example, a liquid crystal screen, an organic EL screen, or the like. The display screen 41 displays the performance tempo of the performer 100 detected by the motion detector 3 . In this embodiment, BPM (Beats Per Minute) is displayed. This numerical value indicates the number of times a quarter note appears in one minute. A method for calculating this BPM will be described in detail later.

Also, the display screen 41 displays the BPM of the performance in real time (for example, every bar). Therefore, when the performance is delayed at 120 BPM, for example, the number is switched to "110".

According to such a display device 4, the performer 100 can know the absolute numerical value of the tempo that he or she is playing. Furthermore, it is possible to grasp whether the speed has become relatively fast or slow, and to correct it by looking at the display screen 41 .

When the player 100 presses the operation button 42 , the information is converted into an electric signal by the communication unit 43 and transmitted to the control device 5 . As a result, as will be described later, the control device 5 drives the reproducing section 2, and the reproducing section 2 reproduces the music. Therefore, the performer 100 can start playing along with the music reproduced by the reproduction unit 2 .

The operation button 42 may be a mechanical switch or a so-called touch panel type provided on the display screen 41 .

Also, the display device 4 may have a function of tuning a guitar, bass, or the like, for example. It may also have a metronome function. As a result, the performer 100 can play at any tempo while referring to the display device 4 . As for the metronome function, in addition to the one that emits sound, it may be one that can refer to the tempo by means of light, movement of a pointing stick, or the like.

Also, the display screen 41 may display an indicator indicating the magnitude of the BPM. It is preferable that the indicator displays a BPM reference value, and that the gauge increases or decreases with respect to this reference value. As a result, it can be seen at a glance whether the BPM of the performer 100 is fast or slow. In addition, in order to enable accurate performance, the BPM of the performer 100 may be displayed in terms of frequency, percentage, etc. as to whether the BPM of the performer 100 is high, low, or the same as the reference BPM. Well, it may be configured to display the standard deviation.

[Control device]
As shown in FIG. 2 , the control device 5 has a CPU (Central Processing Unit) 51 , an I/O interface (receiving section) 52 and a memory (storage section) 53 .

A CPU (control unit) 51 executes a plurality of modules and various programs stored in a memory 53 . Modules may include embedded components in a hardware circuit including logic gates, semiconductor devices, integrated circuits, or any other discrete component. A module may also be part of a software program. Execution of the module as a software program may include a set of logical instructions to be executed by CPU 51 or any other hardware entity. A module can also be embodied in a set of instructions or programs using an interface.

The I/O interface 52 is for communicating with other communication devices (playback unit 2, motion detection unit 3, display device 4, etc.). I/O interfaces 52 include, but are not limited to, audio, analog, digital, monaural, RCA, stereo, IEEE-1394, serial bus, universal serial bus (USB), infrared, PS/2, BNC, coaxial, component , Composite, Digital Visual Interface (DVI), High-Definition Multimedia Interface (HDMI), RF Antenna, S-Video, VGA, Bluetooth, Cellular (e.g. Code-Division Multiple Access (CDMA) , HSPA+ (High-Speed Packet Access), GSM (Global System for Mobile communications) (registered trademark), LTE (Long-Term Evolution), WiMax, etc.), Wi-Fi standard, wireless LAN, etc. You may

The I/O interface 52 functions as a receiving section for receiving motion information regarding the motion of the player 100 detected by the motion detecting section 3 .

The memory 53 is composed of, for example, an EEPROM (Electrically Erasable Programmable Read-Only Memory), which is a type of nonvolatile semiconductor memory. The modules stored in memory 53 include a playback module 531 , an average BPM calculation module 532 , a correction module 533 and a communication module 534 .

The reproduction module 531 is configured to reproduce music data by the reproduction unit 2 when executed by the CPU 51 .

The average BPM calculation module 532 is configured to calculate an average BPM, which will be described later, when executed by the CPU 51 .

The correction module 533, when executed by the CPU 51, is configured to correct the music reproduced by the reproduction unit 2, as will be described later.

The communication module 534, when executed by the CPU 51, is configured to communicate with other communication devices (playback unit 2, motion detection unit 3, display device 4, etc.).

The memory 53 also stores music data for the reproduction unit 2 to reproduce music. The music data is not particularly limited, but may be audio data in an audio file format such as WAVE (RIFF waveform Audio Format) or mp3 (MPEG Audio Layer-3), or MIDI data (MIDI standard (Musical Instrument Digital Interface)). (Registered Trademark) music data), etc., but will be explained below as MIDI data. The use of MIDI data facilitates control as will be described later.

In addition to the above, the memory 53 stores a control program (control method of the present invention) and the like, which will be described later. Such a control device 5 may be built in the display device 4 or may be built in a housing different from the display device 4 .

The control device 5 has been described above. Such a control device 5 can assist the performer 100 in playing a musical instrument in time with the music of the reproduction unit 2 by performing control as described below. The control operation of the control device 5 (control method of the present invention) will be described below with reference to the flowchart shown in FIG.

First, the performer 100 gives a test performance of the conga 201 and calculates its average BPM (step S101: average BPM calculation step). This calculation method will be described using the graph shown in FIG.

In the graph shown in FIG. 8 (the same applies to FIG. 9), the vertical axis is BPM and the horizontal axis is time (sec). In this graph, when the action of the performer 100 exceeds the threshold, that is, when a strong strike exceeding the threshold is detected when striking the conga 201, each strike is extracted at quarter note timing. , and plotted as BPM by comparing one hit with the next hit.

In addition, for each time t (predetermined time), the average value of the BPM of multiple hits (three times in this embodiment) during that time is calculated by arithmetic mean, and the value is taken as the average BPM. In addition, as a method for calculating the average, a geometric average, a harmonic average, a weighted average, or the like may be used.

In this step, the BPM is displayed on the display device 4 in real time. As a result, the BPM of the performer 100 playing a trial can be grasped in real time. Therefore, the BPM desired to be reproduced by the reproduction unit 2 can be approximated. Also, when used during practice, the performer 100 can know whether or not the tempo is being kept, which contributes to improving the proficiency of performance.

Alternatively, a predetermined range may be set for the BPM of the player 100, and if the hit is out of the predetermined range, the strike may not be used for calculating the average value. For example, even if a performance is performed at a BPM of around 100 and the BPM becomes 150 due to a performance mistake, the value of 150 is not used for calculating the average value. As a result, the average BPM can be calculated more accurately, that is, according to the player's 100 intention.

In the above description, the average BPM is calculated assuming that each blow is a quarter note timing, but the present invention is not limited to this. can be calculated.

Also, in the above, the average BPM may be determined by performing statistical processing on each BMP. For example, the variance and standard deviation may be obtained, and the BPM with high frequency may be used as the average BPM.

Then, in step S102 (playback BPM determination step), the BPM (playback BPM) of the music played by the playback unit 2 is determined.

In this embodiment, the performer 100 twists the wrist on which the motion detector 3 is attached. That is, the player 100 rotates the wrist around the y-axis. When the angular velocity around the y-axis in the signal from the motion detector 3 exceeds a predetermined value, the average BPM at that time is determined as the playback BPM. That is, among the average BPM for each time t calculated in step S101, the latest value is set as the reproduction BPM.

Note that the trigger when the performer 100 determines the reproduction BPM is not particularly limited as long as it can be distinguished from the action during performance.

It should be noted that, for example, an instruction to determine the playback BPM may be given by operating a button or the like from the display device 4 .

Next, in step S103 (reproduction instruction determination step), it is determined whether or not the performer 100 has issued a reproduction instruction. When there is a reproduction instruction, the reproduction unit 2 is driven to start reproduction after a predetermined time.

In this step, it is preferable to use a motion different from the motion used as a trigger in step S102 or the motion during performance to determine whether or not there is a playback instruction. exceeds the threshold, it is assumed that there is a playback instruction. As a result, it is possible to clearly distinguish this from the instruction to determine the playback BPM, and it is possible to accurately determine whether or not there is a playback instruction.

It should be noted that, for example, an instruction to determine the playback BPM may be given by operating a button or the like from the display device 4 . Alternatively, for example, a timer may be used to determine that the preparation is completed if no performance movement is detected for a predetermined time or longer, and the last (latest) average BPM may be used as the playback BPM. Also, the order of step S102 and step S103 may be changed.

If it is determined in step S103 that a reproduction instruction has been given, the reproducing unit 2 is driven after a predetermined period of time to start reproducing the music by the reproducing unit 2 (step S104: reproducing unit driving step).

In this embodiment, as described above, the music data stored in the memory 53 is MIDI standard data, so that the speaker 21 and the performance device 22 can be synchronized with simple control.
Then, the performer 100 plays the conga 201 in time with the music played by the reproducing unit 2 .

Next, in step S105, it is determined whether or not the strength of the performance of the performer 100 is within a predetermined range (step S105: strength determination step). The predetermined _range is a value stored (set) in the memory ₅₃ in advance. (see FIG. 7). That is, the magnitude of the angular velocity detected by the motion detection unit 3 is regarded as the strength of the performance of the performer 100 .
Note that these maximum value S _max and minimum value S _min can be changed as appropriate.

If it is determined in step S105 that the value of the angular velocity detected by the motion detection unit 3 exceeds the maximum value _Smax , the music data that has not yet been reproduced (data that will be reproduced from now on) among the music data reproduced by the reproduction unit 2 is detected. Playback is performed after correcting the intensity of the playback data to increase it. That is, the music data is corrected so that the reproduction volume of the reproduction unit 2 is increased.

On the other hand, if it is determined that the value of the angular velocity detected by the motion detection unit 3 is below the minimum value S _min , the unreproduced data (data to be reproduced) among the music data reproduced by the reproduction unit 2 Play back after correcting to weaken the intensity of the That is, the music data is corrected so that the playback volume of the playback unit 2 is reduced.

The above correction can be performed, for example, by converting the detected angular velocity information into velocity signals of MIDI data in correspondence with velocity values that indicate the intensity of the performance. Note that it is also possible to correspond to MIDI expression values, volume values, and the like.

In this way, the control device 5 detects the intensity of the performance of the performer 100 from the performance information of the performer 100, corrects the intensity of the unreproduced data based on the detection result, and causes the reproducing unit 2 to reproduce the data. (Step S106: intensity correction step). As a result, the intensity of the performance of the performer 100 can be reflected in the music played by the reproducing unit 2 . Therefore, even when the performer 100 plays along with the music of the reproduction unit 2, that is, in an ensemble, the musicality that the performer 100 wants to express is not impaired, and a richly expressive music can be obtained. .

When correcting the strength of the music played by the playback unit 2, the degree to which the volume should be corrected is based on, for example, the magnitude of the detected acceleration and a calibration curve (not shown) indicating the playback volume of the playback unit 2. It can be carried out.

Further, while the reproducing unit 2 is reproducing the music, it detects the action of the performer 100 and determines whether or not the average BPM of the performance of the performer 100 is within a predetermined range (step S107: average BPM determination step). ). The predetermined range is a region between the maximum value BPM _max and the minimum value BPM _min stored (set) in the memory 53 in advance, and the maximum value BPM _max and the minimum value BPM _min can be changed as appropriate. can do.

If the average BPM of the performance of the performer 100 is outside the above range, the tempo of the unreproduced data (data to be reproduced) among the music data reproduced by the reproduction unit 2 is corrected and reproduced. (step S108: tempo correction step). As a result, a piece of music that reflects the change in the tempo of the performer 100 can be played back from the playback section 2 .

The tempo correction of the reproducing unit 2 may be made to follow the same value as the BPM of the performer 100 at any time. , the correction of the tempo of the reproducing unit 2 can be guided.

When predicting the change in the BPM of the player 100, for example, the BPM plot of the player 100 can be approximated to a straight line or curve using various functions to predict the BPM plot of the player 100. Examples of this function include n-order functions (n is a natural number), polynomials thereof, and functions obtained by transforming these functions, but it is particularly preferable to use a sigmoid function. This allows natural prediction and tracking to be achieved.
As this sigmoid function, for example, there is a function represented by the following formula (1).

A graph using this formula (1) is shown in FIG. In the graph shown in FIG. 9, the BPM of the performer 100 is indicated by dots, and the BPM of the reproducing section 2 is indicated by a solid line. By using the sigmoid function, the solid line indicating the BPM of the reproducing section 2 draws a gentle S-shaped curve. Therefore, by correcting the tempo of the music of the reproducing unit 2 along this curve, it is possible to realize natural prediction and tracking as described above.

Values can be appropriately set for a ₁ to a ₇ in formula (1), and by changing each value, the solid line indicating the BPM of the reproducing unit 2 can be changed as follows.

As shown in FIG. ₁₀ , when the value of a1 is increased, the position after the curve portion in the graph can be shifted downward. By decreasing the value of _a1 , the position after the curve portion in the graph can be shifted upward.

Further, as shown in FIG. 10, when the value of _a2 is increased, the position after the curve portion in the graph can be shifted upward. By decreasing the value of _a2 , the position after the curve portion in the graph can be shifted downward.

Further, as shown in FIG. 11, when the value of _a3 is increased, the curve portion in the graph can be shifted to the right, and when the value of _a3 is decreased, the curve portion in the graph can be shifted to the left. .

Also, as shown in FIG. ₁₂ , when the value of a4 is increased, the start position of the curve portion in the graph can be shifted to the right, and the end position of the curve portion in the graph can be shifted to the left. By decreasing the value of _a4 , the start position of the curve portion in the graph can be shifted to the left, and the end position of the curve portion in the graph can be shifted to the right.

Further, as shown in FIG. ₁₁ , when the value of a5 is increased, the curve portion in the graph can be shifted to the right, and when the value of _a5 is decreased, the curve portion in the graph can be shifted to the left. .

Further, as shown in FIG. ₁₁ , when the value of a6 is increased, the curve portion in the graph can be shifted to the right, and when the value of _a6 is decreased, the curve portion in the graph can be shifted to the left. .

Further, as shown in FIG. ₁₃ , when the value of a7 is increased, the entire graph can be shifted upward, and when the value of _a7 is decreased, the entire graph can be shifted downward.

Such a formula (function) is stored in the memory 53 in advance. If the average BPM of the performer 100 gradually increases (accelerando), control can be performed using an equation obtained by multiplying the right side of equation (1) by -1.

In such correction guidance, the performer 100 can perform additional correction in multiple steps by changing the BPM. For example, in the case of the ritardando, the intention was to guide the BPM correction so that it would gradually slow down. Correction is performed, and correction guidance is provided again.

More specifically, for example, by performing a correction by changing a ₁ to a ₇ in formula (1) while performing correction using formula (1), the BPM at the end of guidance, and It is also possible to change the timing and the degree of change in BPM. Also, for example, when guidance is provided using a linear function, it is also possible to change the inclination in the middle.

Correction data is calculated in this manner, and the resulting signal is transmitted to the reproduction unit 2 . Further, since the music data is MIDI standard data, the above correction can be performed by a simple control of correcting the MIDI clock signal and transmitting it to the reproducing unit 2 .

If the average BPM of the performance of the performer 100 is within the above range, the tempo of the unreproduced data is not corrected, and the process proceeds to step S111, which will be described later.

In this way, the control device 5 does not correct the tempo of the unreproduced data when the average BPM (average value) is within a predetermined range, and when the average BPM (average value) is outside the predetermined range. , to correct the tempo of unplayed data. As a result, it is possible to prevent the music of the reproducing unit 2 from following a slight deviation in the performance tempo of the performer 100 from time to time, thereby preventing the performance itself from becoming unnatural. Therefore, it is possible to provide a natural performance (ensemble performance).

Further, the control device 5 calculates the average value every predetermined time (every time t in the illustrated configuration), and uses the latest average value to determine whether or not to correct the tempo of the unreproduced data. As a result, the latest intention of the performer can be reflected in the music played by the reproducing unit 2 .

Although the maximum value BPM _max is not particularly limited, it is preferably about 5% to 20% of the average BPM, more preferably about 8% to 11% of the average BPM. Although the minimum value BPM _min is not particularly limited, it is preferably about -20% to -5% of the average BPM, and more preferably about -11% to -8% of the average BPM. Thereby, the above effect can be exhibited more reliably.

Then, in step S109 (correction cancellation instruction judging step), it is judged whether or not the performer 100 has given an instruction to cancel the tempo correction.

In this step, determination is made by a signal from the action detection section 3 worn by the player 100 . It is preferable to use a motion different from the motion used as a trigger in steps S102 and S103 and the motion during performance to determine whether or not there is a correction cancellation instruction. As a result, the correction cancellation instruction can be clearly distinguished from the reproduction BPM determination instruction and the reproduction instruction, and the presence or absence of the correction cancellation instruction can be accurately determined.

If it is determined in step S109 that there has been an instruction to cancel the correction, the tempo correction is canceled, that is, the original BPM (latest average BPM before tempo correction) is restored, and the music is reproduced from the reproduction unit 2 (step S110). : correction release step).

If it is determined in step S109 that there is no correction cancellation instruction, the process proceeds to step S111.

Then, it is determined whether or not there is a reproduction end instruction (step S111: reproduction end instruction judgment step).

In this step, determination is made by a signal from the action detection section 3 worn by the player 100 . It is preferable to use an action different from the action used as a trigger in steps S102, S103 and S109 and the action during performance for determining whether or not there is an instruction to end reproduction. As a result, the reproduction end instruction can be clearly distinguished from the reproduction BPM determination instruction, the reproduction instruction, and the correction cancellation instruction, and the presence or absence of the reproduction end instruction can be accurately determined.

In step S111, if there is a reproduction end instruction, the reproduction by the reproduction unit 2 is ended. If it is determined in step S111 that there is no reproduction end instruction, the process returns to step S105, and the following steps are sequentially repeated.

It should be noted that if the music data is reproduced to the end during steps S104 to S111, the process is forcibly terminated at that point.

As described above, the performance support system is attached to the player 100 who plays the musical instrument 200 along with the playing unit 2 for playing back music data as a song, and the player 100 playing the musical piece played back by the playing unit 2. A motion detector 3 that detects a motion, a receiver (I/O interface 52) that receives motion information about the motion of the player 100 detected by the motion detector 3, and a receiver (I/O interface 52) receive and a CPU 51 (control unit) that corrects the reproduction conditions of the reproduction unit 2 based on the obtained operation information.

According to the present invention as described above, it is possible to obtain information on the movement of the performer 100 during the performance, and to reflect the information on the movement in the music reproduced by the reproduction section 2 in real time. Therefore, even when the performer 100 plays along with the music of the reproducing unit 2, that is, when playing in concert, the musicality that the performer 100 wants to express is not spoiled, and a richly expressive music can be obtained.

Further, in this embodiment, the control device 5 detects the tempo of the performance of the player 100 from the performance information of the player 100, and corrects the tempo of reproduction by the reproduction section 2 based on the detection result. As a result, even if the performer 100 changes the tempo during the performance, the music piece by the reproducing unit 2 is also reproduced at the changed tempo. Therefore, more expressive music can be obtained.

For this reason, for example, even if the band members are not all together, it is possible to perform an expressive band performance at a live house, etc. It can be carried out. In addition, people who have given up playing musical instruments (ensemble) due to their physical disabilities can experience the joy of playing in an ensemble.

Further, according to the present invention, when the player 100 is practicing the musical instrument 200, even if the performer 100 cannot keep up with the music of the reproducing unit 2 and falls behind, the music of the reproducing unit 2 can be played by the player 100. You can slow down the tempo by following As a result, the performer 100 can be prevented from giving up playing during practice, and can continue practicing the musical instrument 200 happily.

For this reason, for example, by introducing this system in a music classroom or the like, students regardless of age or gender can enjoy taking lessons for a long time.

The musical performance support system and control method of the present invention have been described above with reference to the illustrated embodiments, but the present invention is not limited to this, and each part (step) constituting the musical performance support system and control method is the same. can be replaced with any configuration that can exhibit the function of Also, optional components and steps may be added.

Further, in the above-described embodiment, the control unit corrects music data and reproduces the corrected data when correcting the reproduction conditions of the reproduction unit based on the performance information of the performer. However, the present invention is not limited to this, and for example, the configuration may be such that the reproduction speed or the reproduction volume is corrected without correcting the music data.

Further, when correcting the reproduction conditions of the reproduction unit based on the performance information of the performer, the control unit corrects unreproduced data of the music data that has not yet been reproduced, and causes the reproduction unit to reproduce the data. Although the configuration has been described, the present invention is not limited to this, and for example, the same correction may be performed on already reproduced portions of music data. In this case, the performer's movement information is reflected in the music data, and when the music data is reproduced from the next time onwards, the music data incorporates the performer's movement information, performance intentions, and the like. In other words, it is possible to easily perform the work of editing the music data, and by listening to the music data by the performer himself/herself, it is possible to contribute to the improvement of the proficiency of the performance.

Further, in the above embodiment, in steps S102, S103, S109 and S111, as an example, the case where the motion detected by the motion detection unit is used as the trigger for the determination criteria has been described, but the present invention is not limited to this. Alternatively, for example, an operation button of the display device or an operation using a separately provided foot switch or the like may be used.

Note that steps S101 and S102 may be omitted, and the BPM to be reproduced by the reproduction unit may be determined based on past data.

Further, in the above embodiment, the storage unit (memory) storing the music data was built in the control device, but the present invention is not limited to this, and an external device (for example, a USB flash memory) can be stored. ). In this case, by connecting an external device to the control device, the control described in the embodiment can be performed.

1 Performance support system 2 Playing unit 21 Speaker 22 Musical performance device 221 Striking unit 221a Stick 221b Rotation support unit 221c Spring 221d Stopper 221e Rotation shaft 222 Driving unit 222a Rod 3 Motion detection unit 4 Display device 41 Display screen 42 Operation button 43 Communication Part 5 Control device 31 Angular velocity sensor 311 Housing 312 Sensor element 313 Communication part 51 CPU
52 I/O interface 53 memory 531 playback module 532 average BPM calculation module 533 correction module 100 performer 101 wrist 102 wristband 200 instrument 201 conga 202 head 300 instrument 301 drum 302 snare drum 303 head 304 rim BPM _max maximum BPM _min minimum value S _max maximum value S _min minimum value

Claims (9)

a reproducing unit that reproduces music data as music;
a motion detection unit that is worn by a performer who plays a musical instrument in time with the music played back by the playback unit and detects a motion of the performer;
a receiver that receives motion information about the performer's motion detected by the motion detector;
a control unit that corrects reproduction conditions of the reproduction unit based on the operation information received by the reception unit;
The motion detection unit continuously detects the motion information while the performer is performing,
The control unit calculates an average value of the tempo played by the performer over a predetermined period of time, and if the average value is within a predetermined range, the tempo of the song data that has not yet been reproduced is calculated. A performance support system , wherein the tempo of unreproduced data is not corrected, and the tempo of the unreproduced data is corrected when the average value is out of a predetermined range . 2. The performance support system according to claim 1 , wherein the control unit detects the tempo of the performance by the performer from the motion information, and corrects the tempo of reproduction by the reproduction unit based on the detection result. 3. The musical performance support system according to claim 1, wherein the control section calculates the average value at predetermined time intervals, and uses the latest average value for determining whether or not to correct the tempo of the unreproduced data. . 4. The controller of claim 1 , wherein the control unit detects the strength of the performance of the performer from the motion information, corrects the strength of the unreproduced data based on the detection result, and causes the reproduction unit to reproduce the data. The performance support system according to any one of items 1 and 2. 5. The musical performance support system according to any one of claims 1 to 4 , wherein the motion detection section has an angular velocity sensor. 6. The performance support system according to any one of claims 1 to 5 , wherein the playback section includes a performance device for playing the musical instrument. a reproducing unit that reproduces music data as music;
a motion detection unit that is worn by a performer who plays a musical instrument in time with the music played back by the playback unit and detects a motion of the performer;
a receiver that receives motion information about the performer's motion detected by the motion detector;
a control unit that corrects reproduction conditions of the reproduction unit based on the operation information received by the reception unit;
The control unit corrects unreproduced data that has not yet been reproduced in the music data, and causes the reproduction unit to reproduce the data, and detects the intensity of the performer's performance from the operation information, A musical performance support system that corrects strength of the unreproduced data based on the detection result and causes the reproduction unit to reproduce the data. A reproducing unit that reproduces music data as music, and a player that plays a musical instrument to the music reproduced by the reproducing unit . A control method for controlling a performance support system comprising: a motion detection unit that detects motion information about a motion; and a reception unit that receives the motion information detected by the motion detection unit,
An average value of the tempo played by the performer over a predetermined time period is calculated, and if the average value is within a predetermined range, the tempo of unreproduced data of the music data that has not yet been reproduced. is not corrected, and the tempo of the unreproduced data is corrected when the average value is outside a predetermined range . A reproduction unit that reproduces music data as music, a motion detection unit that is worn by a performer who plays a musical instrument in time with the music reproduced by the reproduction unit and detects the motion of the performer, and the motion detection unit. A control method for controlling a performance support system comprising a receiving unit that receives motion information about the detected motion of the performer,
The intensity of the performance of the performer is detected from the motion information, and based on the detection result, the intensity of the unreproduced data among the music data is corrected, and the reproducing unit reproduces the data. A control method characterized by:

Images