JP4368222B2 - Concert support system - Google Patents

Description

  The present invention relates to an ensemble system, and in particular, to a novel ensemble support system that enables even beginners to easily enjoy ensembles and renditions.

  There are many parents who want to enjoy ensembles with their children, such as when they are taking piano lessons.

  For this reason, recently music schools and music software for adults as well as children are popular. In particular, there are many lessons and software that allow each adult to play the song they want to play in as short a time as possible. These seem to have given priority to expressing the music in the earliest rather than learning from the basics like a child. However, in order to be able to perform with various songs with children, it is necessary to learn from the basics of playing musical instruments according to the notes on the score, which takes a long time.

  Also, piano lessons are encouraged to incorporate a series of ensembles. For example, when a child in the first week of learning piano learns a song with just a few “dos” arranged, the teacher is attending a lesson that accompanies various chords as a partner in the series. . There are reports that students' motivation to practice has increased through the bullets, and that children have naturally expressed their "expressions they want to do" themselves by listening to their partner's performances. No matter how novice children are doing their essential activities to nurture their musicality, they can “breath” with the teacher in the bullet.

  However, it is very difficult for a pair of a child who has begun to attend a lesson at home and a parent who has little experience of playing musical instruments to practice and play as a series. This can also be explained by the fact that most of the piano renditions on the market are supposed to be at least one intermediate pair. Recently, for piano learning, a lot of minus ones with partner performance data prepared in advance have been seen. However, even if Minus One is used, this method does not have a real relationship with each person's performance, so it is not possible to “breath” with the partner.

  On the other hand, in recent years, research on a system for performing with a human performer has progressed. Non-Patent Document 1 and Non-Patent Document 2 disclose an automatic accompaniment system for music for which a score is given in advance. The non-patent document 1 shows an algorithm using only pitch information output by the performer, and the non-patent document 2 recognizes a pitch and a performance time every predetermined time and changes an accompaniment. A method is proposed.

  Furthermore, Non-Patent Document 3 proposes a model that analyzes the ensemble between a person and a computer and predicts the next “change in time length” from the “deviation” between the person and the computer, aiming at a more natural accompaniment system performance. is doing.

Non-Patent Document 4 proposes a model that takes into account the interaction between a human and a computer.
Dannenberg, R.D. : An On-Line Algorithm for Real-Time Accompaniment, Proc. ICMC 1984, pp. 193-198, 1984 Vercoe, B.E. : The Synthetic Performer in the Context of Live Performance, Proc. ICMC 1984, pp. 200, 1984 Horiuchi Ikuo, Sakamoto Koji, Ichikawa Satoshi: Estimation of human pronunciation time control model in ensemble, IPSJ Transactions, Vol. 43, no. 2, pp. 260-267, 2002 Takayuki Igawa, Kuniaki Naoi, Osamu Oteru, Shuji Hashimoto: Real-time adaptive automatic accompaniment and its analysis by interaction model, Proceedings of the IEICE Spring National Conference, pp. 1. 389-390, 1992

  However, all of the above studies are aimed at ensembles between humans and systems (computers), and are aimed at systems that support the ensemble between humans, especially beginners, to which this invention is directed. is not.

  Therefore, the main object of the present invention is to provide a novel ensemble support system.

  Another object of the present invention is to provide an ensemble support system that allows a pair such as a beginner's parent of a musical instrument performance and a beginner's child to perform immediately.

  According to the first aspect of the present invention, a first input interface that is operated by the first player and outputs the first performance data according to the performance, and a second input that is operated by the second player and outputs the second performance data according to the performance. Stores musical score data including an input interface, performance positions, pitch data of sounds to be played by the first player at each performance position, and pitch data of sounds to be played by the second player at each performance position A musical score database, performance position determination means for determining the performance position of the first player based on the first performance data and the score data, and the performance position of the first player determined by the performance position determination means with reference to the score data. Pitch data acquisition means for acquiring pitch data of a sound to be played by the second player, and a pitch for replacing only pitch data in the second performance data with pitch data acquired by the pitch data acquisition means. Chromatography comprising data replacement means and and the first performance data, and pronounce sound generating means in accordance with the second performance data that only pitch data has been replaced by the pitch data replacement means is a ensemble support system.

  In the first aspect of the invention, an input interface (reference numeral 12 corresponding to the corresponding portion in the embodiment) is operated by a first player (primo), and MIDI data is output as first performance data according to the performance. . The input interface (14) is operated by the second player (second), and MIDI data is output as the second performance data according to the performance. The MIDI data from the first input interface is sounded as it is from the sounding means (16, 18), but this MIDI is also given to the performance position judging means.

  In the performance position judging means (20, S1, S101-S133), on the score of the first player based on the first performance data (MIDI data) from the first input interface and the score data of the score database (22). The performance position at is determined.

  The pitch data acquisition means (20, S3, S301-S311) acquires the pitch data of the sound to be played by the second player from the score data based on the performance position of the first player determined as described above. To do. The pitch data replacement means (20, S5, S501 to S503) converts only the pitch data in the second performance data input from the second input interface into the pitch data acquired by the pitch data acquisition means. Replace and give to the pronunciation means. Therefore, the sound generation means outputs music in accordance with the first performance data performed by the first player and the second performance data which has been performed by the second player but has only the pitch data replaced.

  According to this invention, since only the pitch of the performance data of the second player at the performance position of the first player is replaced, the second player can easily perform the correct performance. Therefore, for example, even if the first player is a beginner and the second player is a beginner, it is possible to easily perform ensembles and renditions.

  Also, if it is possible to cope with replaying, it is possible to quickly determine the correct performance position even when replaying during the performance, which is often seen by beginners, occurs.

  The above object, other objects, features, and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

  The ensemble support system 10 of this embodiment shown in FIG. 1 includes a primo input interface 12 and a second input interface 14. However, in FIG. 1, the two interfaces 12 and 14 are separately illustrated. However, in the case of continuous shots, the same interface (keyboard) is divided into two areas and used. Further, a keyboard (keyboard) is used as the input interface of this embodiment, but an interface of another musical instrument may be used.

  Note that “primo” refers to a performer who is in charge of the treble part in the case of a piano series, and “second” refers to a performer who performs the bass part. In the case of an ensemble, such a designation is not necessarily used, but here, for convenience, the player who leads the ensemble is called “primo”, and the other player is called “second”. However, the former may be referred to as “first player” and the latter as “second player”.

  Both the primo input interface 12 and the second input interface 14 output MIDI (Musical Instrument Digital Interface) data according to the performance. Then, the MIDI data output from the input interface 12 in response to the performance of the primo is input as it is to the MIDI sound source 16 without any processing, and is output as a sound from the speaker 18. Performance data (MIDI) from the primo input interface 12 is also input to the computer 20 at the same time. Similarly, MIDI data output from the input interface 14 in response to the second performance is also input to the computer 20.

  In short, the computer 20 supports only the performance of the second while referring to the performance data of the primo and the score data. No performance support for Primo.

  The musical score data to be referred to by the computer 20 is stored in the musical score database 22.

  FIG. 2 shows an example of a description format on which such score data is based. In this example, the performance data itself is not described in the line beginning with “*”, but a marker indicating a position that is easy to return to when a beginner plays the song.

According to Bloch & Dannenberg (Bloch, J. and Dannenberg, RB: Real-Time Computer Accompaniment of Keyboard Performances, Proc. ICMC 1985, pp.279-289, 1985), performance errors are classified into the following three categories: Is done.
(1) Extra: The played sound does not match the sound on the score (extra number of notes).
(2) Pitch error (wrong): The sound is produced according to the number of notes on the score, but the pitch is incorrect.
(3) Missing: The number of played sounds is less than the number of sounds on the score.

  However, especially for beginners, it is thought that “replay” will be added to these. Replaying is a case where more notes are played than the number of notes on the score, as in “Insert”, but it is not a note or pitch sequence that is not indicated on the score. This refers to the case where a sound or pitch sequence is played in duplicate. Beyond intermediate, there is a tendency to continue to the end without stopping the flow of performance, even if it fails, except for replaying it during practice. On the other hand, beginners tend to return if they fail on the way.

  With subject A who is 4th and 4th grade elementary school students for the first time in piano lessons and subject B who is a nursery school student for the first year after attending piano lessons, as a primo, various practice songs are first seen. According to the experiments performed, the results shown in Table 1 were obtained. However, in the experiment, one of the inventors was in charge of the second. Table 1 shows the characteristics of the number of replays and the points returned to replay.

  “First beat” means returning to the first beat of the measure. The “second strong beat” refers to the third beat of 4 beats and the second beat of 2 beats. The “last beat” refers to the third beat of the third beat and the fourth beat of the fourth beat. The above three items are beats within the same bar as the sound played immediately before replaying. “Phrase start” refers to the beginning of a slur or the beginning of a weak melody. “Change hand” means that the playing hand returns to the place where the right hand changes to the left hand in one melody. The above two items are not counted in the categories of “first beat”, “second strong beat”, and “last beat”. A “column” refers to a group of several bars printed in a horizontal row on a musical score. Note that “one measure before” means that the sound has returned to one measure before the sound immediately before being replayed.

  The number of replays was 377 for subject A and 181 for subject B. In the return position, the first beat of the measure occupies 80%.

  On the other hand, “insertion” was seen by the subject A with 1 sound and the subject B with 6 sounds. “Pitch error” was found in subject B by 37 sounds (total of 111 sounds). “Dropped” was seen by subject A in 3 sounds.

  In addition, since subject A tended to see and play the score for each measure, a blank time not described on the score was likely to occur immediately before the first beat of the measure. For this reason, there were many scenes where the second sounded the first beat in the rendition, and then sounded with the subject A again. In addition, because the level of the solo solo and the number of pieces of the twin pieces are different from each other, the subject A tends to be replayed less frequently.

  In this embodiment, referring to the result of the experiment as described above, a position where a beginner can easily return to a song is described as a marker, such as a line marked with “*” in FIG. did. The following four types of markers can be set in the experiments by the inventors. However, a plurality of types of markers can be simultaneously set in one place.

B: Indicates the beginning of a measure P: Indicates the beginning of a phrase R: Indicates the beginning of a “column” of a printed score S: Other markers that can be arbitrarily set In the embodiment of FIG. The operation is not switched depending on the type, and only the presence or absence of the marker is checked. In other words, in this embodiment, a plurality of types of markers are prepared only for the convenience of a person when creating or reading musical score data, but the frequency of replay points shown in Table 1 is considered. And it is also possible to implement a more accurate replay determination by adjusting the weighting according to these types.

Further, in the description format of FIG. 2, the score data is described as follows.
(1) All note data is described except for the marker line that starts with “*”. One line lists the sounds that are to be pronounced "simultaneously" on the score.
(2) In the “sound name”, only the pitch is described, ignoring the “note value” of the note (that is, the length of the sound such as a quarter note or an eighth note). For example, “C5”, “A3”, etc.
(3) When a single primo or second performer plays multiple sounds at the same time (plays chords), the names of those notes are connected by “, (comma)”.
(4) Connect primo and second with ": (colon)".
(5) When only one of them produces a sound at a certain moment and there is no sound to be pronounced, the person who does not have a sound to pronounce does not specify a pitch name.
(6) “Comments” may or may not be described.

  As a specific example, the score shown in FIG. 3 is the first four bars of the nursery rhyme “bonfire”, and FIG. 4 shows the score converted into data in the description format of FIG. However, in FIG. 3, the upper score is a primo and the lower score is a second score.

  In FIG. 4, “* S” is an example in which it is neither the beginning of a measure nor the beginning of a phrase, but is set as a particularly easy-to-return portion. Since the score data is in a format that can be easily edited with a normal text editor as described above, the “* S” marker or the like is used in this way when there is a particularly easy-to-return part depending on the individual player. It is also possible for a person to embed freely and optimize it for himself.

  In accordance with the description data of FIG. 4, the score data shown in FIG. 5 is stored in the score database 22 of FIG.

  In the score data in FIG. 5, the position indicates the note number through both the primo and the second from the beginning of the score. Where a plurality of sounds are played simultaneously, the same position number is assigned to all of them. For example, for positions 1 and 5, etc., the same position number is assigned to each of three sounds. There is a sound to be played in one of the primo or the second, but the other part is blank if there is no sound to be played in the corresponding part. For example, second positions 2, 4 etc. In the return field, “*” symbols are set only at positions where any kind of marker is set in the musical score data, and the others are blank. In reality, such a score data table is constituted by links of C language structures.

  In the embodiment shown in FIG. 1, the computer 20 executes the performance position determination process shown in FIG. In this performance position determination process, the score data for the primo acquired from the score database 22 is collated with the performance data (MIDI) of the primo that is sequentially input to determine which position (position) is currently being played on the score. This result is notified to the performance pitch acquisition process shown in FIG.

  In the performance pitch acquisition process, the second musical score data obtained from the musical score database is referred to, the part corresponding to the current performance position of the primo is found on the second musical score, and the pitch of the sound that the second should play now is determined from there. Data is acquired and passed to the pitch data replacement process shown in FIG.

  In the pitch data replacement process, only the value (MIDI note number) that specifies the pitch data among the MIDI data of the performance sound input from the second input interface is the sound acquired by the performance pitch acquisition process. Replace with high data. At this time, all other data (such as the sound generation time and the velocity value corresponding to the sound intensity) hold the values input by the seconds.

  Thus, the second performance data (second performance data) in which only the pitch data is replaced is input to the MIDI sound source 16 (FIG. 1) together with the primo performance data (first performance data), and is output from the speaker 18 as sound. The That is, the MIDI sound source 16 and the speaker 18 function as sounding means.

  Therefore, regardless of which key on the keyboard is pressed, the second sound is output at the correct pitch to be played at each time point. On the other hand, all elements related to musical expressions such as sound intensity and pronunciation timing are output while the seconds are played. As a result, the second can easily reproduce the melody according to the score, and can be reflected in the performance as expressed.

  Even when the primo is playing, even if the second pauses for a while (when no key is pressed), the computer 20 recognizes the performance position from the performance data of the primo, so when the second resumes. But you can match the primo with the correct pitch. On the other hand, if the second key is played even though the primo has stopped playing, the first keystroke will produce a pitch that corresponds to the point where the primo will play next, but after that the primo will play. It is set so that no sound is output until it starts.

Here, the performance position determination process shown in FIG. 6 will be described in detail. However, FIG. 6 is intended to be driven in the event that keying (operation of the input interface 12) Primo, that are represented as Primo in playing time t _j is played sounds P _j, previously Please keep in mind.

  A great deal of research on performance position detection (so-called Score Following / Tracking) has been made since the 1980s. Past systems have dealt with three types of errors: “insertion”, “pitch error”, and “dropout”, but all of them are basically premised on performances that do not cause major errors.

  On the other hand, in the ensemble support system 10 of the present invention, a very incomplete performance at the practice stage by a beginner is handled. In this case, as described above, replaying occurs very frequently, and the performance position frequently goes back many times repeatedly. Therefore, the conventional system cannot cope with such replay.

  Therefore, in this embodiment, the DP matching method proposed by Dannenberg in the previous Non-Patent Document 1 is extended to devise a performance position detection method capable of coping with replay.

  In the performance position detection method of this embodiment, as in Dannenberg's method, the performance position is judged by matching only the pitch, ignoring the note value of each sound on the score and the length of the performance sound. This is based on the judgment that the note value and the tone length cannot be handled as matching targets because the variation of the tone length during performance is extremely large in the performance of the beginner and long stops that are not in the score frequently occur.

Let N be the number of sounds included in the primo score. However, when a plurality of sounds such as chords are pronounced at the same time, the number of sounds at that point is set to “1” regardless of the number of sounds that are sounded simultaneously, and only the highest sound is targeted for matching. Now, when the primo has played the j-th note P _j from the start of performance (this time is referred to as “performance time t _j ”), the performance position on the score at the performance time t _j-1 is S in the previous matching. It is assumed that the sound is _i (1 ≦ i ≦ N). At this time, it is determined by the following algorithm which sound on the musical score P _j corresponds to.

(1) and pitch Pitch of all sounds S _k of the score (1 ≦ k ≦ N) (S _k), compared with the P _j of pitch Pitch (P _j), playing Primo for all sounds The weight W (S _k , t _j ) at the time point t _j is evaluated by the method shown in Equation 1 below.

If this is shown in FIG. 6, the number k of sounds is set to “1” in step S101. Since initially at k <N (Sooto number), "NO" next in step S103, in a next step S105, the computer 20 (FIG. 1) includes a pitch Pitch in sound _{S k (S k),} a sound comparing the pitch pitch _{(P j)} in P _j. If “YES” in this step S105, the computer 20 uses the weight W (S _k , t _j ) of the note S _k at the time point t _j as W (S _k−1 , t _j−1 ) in the next step S107. ) Set +1. In step S109, the number of sounds k is incremented and the process returns to the previous step S103. However, in this embodiment, the value added in step S107 is “1”, but this value does not have to be “1”, and other appropriate values may be added.

If “NO” in the step S105, the computer 20 determines whether or not the weight W (S _k−1 , t _j−1 ) of the note S _k−1 at the time point t _j−1 is larger than “1” in the step S111. That is, it is determined whether W (S _k−1 , t _j−1 ) −1> 0. If “YES”, W (S _k−1 , t _j−1 ) −1 is set as the weight W (S _k , t _j ) of the note S _k at the time t _j in the next step S113. However, if “NO” in the step S111, “0” is set as the weight W (S _k , t _j ) of the note S _k at the time point t _j in the next step S115.

  Such steps S101 to S115 are repeatedly executed until “NO” is determined in step S103, that is, until the weights are evaluated for all sounds. After the weight is evaluated at all positions, in principle, the performance position showing the maximum weight is identified as the next performance position.

That is, if and rated the weight W for all positions, "YES", and the computer 20 in step S103 continues with step S117, the weights in playing position _{S i + 1} at time _{t j} is at time _{t j-1} It is determined whether or not the weight at the performance position S _i is equal to the weight obtained by adding “1”, that is, whether W (S _{i + 1} , t _j ) = W (S _i , t _j−1 ) +1. That is, it is determined whether or not the performance position is correctly updated. However, in this embodiment, the value added in step S117 is “1”, but this value does not have to be “1”, and other appropriate predetermined values may be added.

Here, the condition determination in step S117 will be described in detail. In the step S117, the a state of attempting to determine the performance position in playing time t _j. The performance position at the performance time t _j−1 is S _i . Therefore, W (S _i , t _j−1 ) is the weight of the portion determined as “this is the performance position” in the determination at the time of the previous determination (t _j−1 ). W (S _{i + 1} , t _j ) is a weight obtained as a result of the weighting process at this time (time t _j ) of the sound S _{i + 1} next to S _i on the score. _{Then, W} if the value of _{(S i + 1, t j} ) had become to a value obtained by adding "1" to the value of _{W (S i, t j-} 1) (W (S i, t j-1) is S _{i + 1} is determined as the performance position at time t _j when 1 is added by the process of S107 in FIG.

If “YES” in this step S117, the performance position is correct, so that the performance position determination routine of FIG. 6 is terminated with the next performance position S _{i + 1} as the performance position at the time point t _j .

(2) If W (S _{i + 1} , t _j ) ≠ W (S _i , t _j−1 ) +1, step S121 and subsequent steps are executed to deal with performance errors (including replay). . That is, if “NO” is determined in step S117, it means that the primo has made a mistake in performance, and hereinafter, processing for identifying the correct performance position is executed.

First, replaying will be described in detail. The “playing” obtained in the above-mentioned experiment (Table 1) in the range of “1” to “i” and including measures, phrases, columns, and page heads on the score. The weights of all the locations previously designated on the musical score data as “locations where correction is likely to occur” are W (S _{i + 1} , t _j ) −m (m is a positive constant). That is, the weights of the performance positions where the replay position markers are set are all set to a value “m” smaller than the maximum. However, if the original weight at that location is larger than the weight obtained by this calculation, the value is not changed. In the experiments by the inventors, the value of m is empirically set to “2”. Decreasing the value of m improves the ability to follow replays. On the other hand, in the case of music in which similar patterns appear repeatedly, there is a high possibility of inaccurately recognizing performance points with minor errors (such as missing sounds). Become. In other words, when there are m sounds from a part that seems to be a “replayed part”, it is determined as “replayed” for the first time when m pieces of sound from a part that is considered to be “replayed part” are played continuously. This prevents excessive reaction to minor errors such as missing or insertion of about one sound.

Next, how to handle insertion, pitch error, and dropout will be described. For all sounds in the range from the position S _i-r to S _{i + r} , the weight is the same as W (S _{i + 1} , t _j ). In other words, it is possible to deal with up to r insertions, pitch errors, and omissions by highly evaluating the possibility that each r sound before and after the current performance position at time t _j-1 is the performance position at time t _j. ing. Note that the number r of sounds is a positive constant, and the value of r is empirically set to “2” in the experiment. Increasing this value can deal with errors over a larger number of sounds, but increasing it too much widens the range of possibilities, and conversely increases the risk of finding incorrect matching points.

(3) And, the place where the weight W (S _k , t _j ) (where 1 ≦ k ≦ N) takes the maximum value is the current performance position. If there are a plurality of places with the same weight, Prioritize in the following order:
(a) the weight _{W (S} k, _{t j)} if it is the only that portion having the maximum value, the weight _{W (S} k, _{t j)} at the time the current performance position location _{S k} with the maximum value And
(b) If there are a plurality of places where the weight W (S _k , t _j ) takes the maximum value, the current performance position is determined as follows.

(i) If the point S _k taking the maximum value closest to S _i is unique, that point S _k is set as the current performance position.

(ii) if the point where the maximum value closest to S _i there is a plurality, that is, the maximum value and the position equidistant from S _i S _i-d and S _{i + d} occurs, the S _i-d The current performance position.

This will be described with reference to FIG. 6. If “NO” is determined in the step S117 in FIG. 6, the computer 20 determines in the next step S121 all replay positions S _h (return markers in Table 1). W (S _{i + 1} , t _j ) −m is set as the weight of the portion to which the value is assigned) in order to make those weights smaller by m than the maximum. Next, in step S123, the computer 20 sets the weight W (S _g , t _j ) at the position Sg (= i−r ≦ g ≦ i + r) to W (S _{i + 1} , t _j ) in step S123. And That is, the weights of the performance positions before and after the performance position are made equal.

In step S125, it is determined whether there are a plurality of places where the weight W (S _k , t _j ) (k = 1 to N) takes the maximum value. If “NO”, in step S127, the computer 20 sets the position S _{k at} which the weight W (S _k , t _j ) is the maximum value as the performance position of the current t _j .

When “YES” is determined in step S125, that is, when there are two or more places where the weight W (S _k , t _j ) takes the maximum value, in the subsequent step S129, the computer 20 is the most at the position S _i . playing point S _k of the maximum value is whether or multiple exist, to determine close. When "YES" in the step S129, the computer 20, at step S131, the to k <playing point _{S k} of i and the current playing position. If “NO” in the step S129, the computer 20 sets the position _Sk to the current performance position in a step S133.

  The above algorithm realizes a robust performance tracking process capable of following the “replay” that frequently occurs in beginner performances, in addition to the three types of performance errors that have been handled conventionally.

  According to the experiment, in the case where “responding to replay” is performed and in the case where it is not performed, the former detects a correct position after erroneous recognition of, for example, three sounds. On the other hand, in the latter case of the same song, 13 sounds were erroneously recognized until the correct position was detected after replaying. As described above, when the performance position determination algorithm of this embodiment is employed, it is possible to follow the replay at a sufficiently practical level.

  Next, the performance pitch acquisition process will be described with reference to FIG.

The performance pitch acquisition process is also driven by an event called primo keystroke, as in FIG. Therefore, in the first step S301 in FIG. 7, the computer 20 receives the data of the performance position S _i of the primo at that time. Then, after the elapse of the fixed time Tw milliseconds in step S303, in step S305, the computer 20 determines whether or not the sound to be played by the second at the position S _i +1 is on the score. If “NO”, in step S307, the computer 20 sets the sound to be played by the second to “none (0)”. On the other hand, when “YES” is determined in step S305, that is, when it is determined that there is a sound to be played by the second at the position S _i +1, in the next step S309, the computer 20 For example, the sound that can be played is set to the sound specified in the second column of FIG.

Therefore, if the Primo is playing without error, the position (position) S _i is 1, 2, 3, 4, playing position in ... and the order is entered into this routine. For example, when S _i = 9 and the performance position of the primo is notified, after taking Tw milliseconds, it is checked whether or not the sound played by the second is designated at the position S _i + 1 = 10. In the example of FIG. 5, since no sound is designated at the second position 10, the determination in step S305 is “NO”. Therefore, the sound that can be played by the second is set as “none”. In this state, no sound is produced even if the second key is pressed.

If the primo continues to perform without error, the same processing is performed even when S _i = 10, and the second cannot continue to sound. Further, when the performance position of the primo becomes S _i = 11, after a time interval of Tw milliseconds, the second performance sound at the position S _i +1 (= 12) is confirmed. In the example of Table 1, since the sound “C5” is designated at the second position 12, the result of the condition determination in step S305 is “YES”. Is set, and the sound is ready. With the above procedure, the second sound that can be played when there is no error in the primo is switched.

Next, a case where the second forgets to pronounce a sound that should be pronounced by mistake will be described. The playable sound of a second is not actually played unless there is a second keystroke. In a state that is not the pronunciation by seconds, Primo is playing the next position, and a new playing position information S _i are input to the algorithm of FIG. 7, regardless of whether the seconds has actually pronounce the previous sound First, this algorithm is executed, and the sound to be played by the second is also updated. For example, assume that a second plays at position 3 and produces a G4 sound. Thereafter, when the primo advances to the performance of positions 4 and 5, the playable sound of the second is updated to B4. However, if the primo further advances to positions 6 and 7 without the second being mistakenly played in this state, the second playable sound is switched to G4 while the B4 sound is not produced. Therefore, when the second plays in this state, the sound of G4 is generated instead of B4 forgot to perform. In this way, even if the second mistakenly skips some performance sounds, the second performance sound will always correspond to the primo performance position, so even if the second resumes playing in the middle The sound you should play now is pronounced correctly, not the skipped sound.

  On the other hand, a case will be described in which the second is played in an extra portion where it should not be played by mistake. For example, assume that a second plays at position 9 and produces the sound of A4. Immediately after this, consider the case where the second key is pressed again before the primo plays the sound at position 10. If this re-keying is within Tw milliseconds from the keying of the primo position 9 sound (C5), the A4 sound is generated again. On the other hand, when Tw milliseconds or more have passed since the key of the sound (C5) at the position 9 of the primo, the sound at the next position 10 is generated. However, in the present example, since no playable sound is designated at the second position 10, no sound is produced in this case. If the primo continues to play at position 10, the second playable sound is not designated at position 11. Therefore, even if the second key is played here, nothing is still pronounced. Furthermore, when the primo sounds the sound at position 11 and Tw milliseconds have passed, the playable sound of the second is finally set to C5 at position 12. As described above, no matter how many times a second plays an extra key, only the second performance position does not unilaterally advance. Extra sound may be inserted, but this will not shift the primo and second performance positions.

The above is a description of the flow of processing when the primo does not make a mistake in performance. Next, processing when the primo makes a mistake in performance and skips several sounds will be described. Also in this case, the outline of the processing is basically the same. If Primo has wrong playing position data S _i to be notified from the performance position determining routine is only becomes previous discontinuous, that it is not related to the processing of the algorithm. For example, consider the case where Primo returns to position 9 after playing the sound at position 12. In this case, the performance position information notified from the performance position determination module is notified to the pitch acquisition algorithm shown in FIG. 7 when S _i = 12 and then S _i = 9 if the performance position determination is ideally operated. Is done. When S _i = 12 is notified and Tw milliseconds have passed, the playable sound of the second is B4 at position 13. In this state, if the primo plays the sound at position 9 and the second key is pressed almost simultaneously, the second performance sound becomes B4 at position 13 as it is. This is an unavoidable error. However, since S _i = 9 is notified as the performance position, the sound that can be played in the next second is the sound at position 10. In this example, since the position of the second position 10 is blank, in this case, the playable sound is “none”. Thereafter, if the primo continues to perform without error, the process proceeds in the same manner as described above.

Here, the meaning of the delay of Tw milliseconds in step S303 will be described. If the score is simply used, for example, at the moment when the primo plays the sound of S _i = 13 (ie, D5), the playable sound of the second may be set to “B4” at position 13 at the same time. However, in reality, the primo and second performance timings cannot coincide with each other without any deviation. In some cases, the primo is slightly ahead, and in other cases, the second is slightly ahead. When the primo is ahead and the second is delayed, there is no problem even if the above simultaneous switching is performed, and the second can play the sound of B4 immediately after the primo plays the sound of D5 at the position of position 13. However, if the second precedes and the primo is delayed, the second will play the “previous sound” that is not switched. In other words, if the second precedes at the position 13, the second plays the C5 sound, which is the previous sound, instead of the B4 sound that should be played, which is inconvenient.

In order to avoid this problem, a second playable sound is switched after Tw milliseconds. The time of Tw milliseconds is the maximum value of the time width in which the second may be delayed from the primo when the primo and the second should sound simultaneously, and is 200 milliseconds in the current system. In other words, in this algorithm, for example, the sound that the second can play is set to the C5 sound at position 12 after Tw milliseconds when the primo has played the E5 sound with S _i = 11. In this state, when the primo plays E5 with S _i = 12, the second can maintain the currently set C5 for Tw milliseconds. In other words, if the second is delayed from the primo by less than Tw milliseconds, the correct sound to be played here can be played C5. However, if there is no second pronunciation for Tw milliseconds or longer, the system assumes that the second sound has been “forgotten to play” and sets the B4 sound at the next position 13 as the sound to be played. To prepare for the pronunciation of S _i = 13. For this reason, even if the primo is not playing the sound of D5 with S _i = 13, the second performance sound is already the sound at position 13, so the second will sound slightly ahead of the primo. However, B4 at position 13 to be played is correctly played. With the above mechanism, the problem of inconsistency in performance sound due to a difference in performance timing between the primo and the second is absorbed.

Next, the pitch data replacement process shown in FIG. 8 will be described in detail. The processing of FIG. 8 is driven by an event of keystroke by a second, and in the first step S501, the computer 20 refers to the score data and determines whether or not there is a sound to be played by the primo at the position S _i +1. . If “YES”, the routine is exited as it is. If “NO”, in the next step S503, the computer 20 designates a sound that can be played by the second in the second column of the position S _i +1 of the score data. Change to sound.

  More specifically, the pitch data replacement algorithm shown in FIG. 8 performs processing when there is no sound to be played in the primo and sound to be played only in the second. In the example handled here, the locations of positions 14 to 16 correspond to this case. When Primo plays D5 at position 13 and Tw milliseconds have elapsed, the second playable sound is A4 at position 14. When the second key is pressed in this state, the A4 sound is generated and the algorithm shown in FIG. 8 operates.

Since the performance position S _i on the primo side is still 13, the score data of the primo at the position S _i +1 (= 14) is checked to check whether there is a sound that the primo plays at the position 14. In this example, since the primo has no data to be played at position 14, the condition determination is “NO”. In this case, the current performance position is set to S _i = S _i +1 (= 14), and the second performance possible sound is set to G4 of S _i +1 (= 15). When the second key is pressed again in this state, the G4 sound is generated, and at the same time, the second algorithm operates again, and the same processing as described above is performed. In this way, when there is no performance data in the primo, the second, not the primo, takes the initiative of the performance progression, and the performance can proceed independently regardless of the performance of the primo.

It should be noted that, if the Primo in a state in which the seconds is not finished playing yet to the position 16 has been playing sound E5 of position 17, playing position S _i is moved to the stretch position 17, to be non-performance sound of seconds are ignored Become. This is because the performance position determination routine does not consider the second performance position, and refers only to the performance data and score data of the primo. However, this is not a flaw in the algorithm. This is because the system of this embodiment is a primo-driven system to the last, and the second is led in a state that occurs exceptionally in a limited part. Therefore, even if only the second is to be played, if the primo resumes playing, it is necessary to immediately give the initiative to the primo. As a result, even if the primo mistakenly moves to the next performance position early, the second can correctly produce the performance sound corresponding to the performance position of the primo.

By the way, if there is a second keystroke, the algorithm shown in FIG. 8 is started unconditionally, so this figure is not only when there is no playable sound in the primo but also when there is a playable sound in the primo. The algorithm of 8 is activated. In this case, however, the condition determination as to whether or not there is a sound to be played by the primo at the performance position S _i +1 is “YES”, so neither the performance position S _i nor the second performance sound is changed. In other words, in this case, the algorithm ends without doing anything.

  As described above, the two simple algorithms shown in FIGS. 7 and 8 operate asynchronously so that the playing positions of the primo and the second always coincide.

  According to experiments conducted by the inventors asking several pairs, it was possible to perform ensembles (repetitions) immediately by using any of these pairs.

FIG. 1 is a schematic block diagram showing an ensemble support system according to an embodiment of the present invention. FIG. 2 is an illustrative view showing an example of a description format of a score. FIG. 3 is an illustrative view showing an example of a score. FIG. 4 is an illustrative view showing a description format in which the score of FIG. 3 is described. FIG. 5 is an illustrative view showing one example of score data according to the score of FIG. FIG. 6 is a flowchart showing the operation of the performance position determination process in the embodiment of FIG. FIG. 7 is a flowchart showing the operation of the pitch data acquisition process in the FIG. 1 embodiment. FIG. 8 is a flowchart showing the operation of the pitch data replacement process in the embodiment of FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Concert support system 12 ... Primo input interface 14 ... Second input interface 16 ... MIDI sound source 18 ... Speaker 20 ... Computer 22 ... Musical score database

Claims (7)

A first input interface operated by the first player and outputting first performance data in accordance with the performance;
A second input interface operated by the second player and outputting second performance data in accordance with the performance;
Music score data including performance positions, pitch data of sounds to be played by the first player at each performance position, and pitch data of sounds to be played by the second player at each performance position is stored. Music score database,
Performance position determination means for determining the performance position of the first player based on the first performance data and the score data;
Pitch data acquisition means for acquiring pitch data of a sound to be played by the second player at the performance position of the first player determined by the performance position determination means with reference to the score data;
Pitch data replacement means for replacing only pitch data in the second performance data with pitch data acquired by the pitch data acquisition means, and pitches by the first performance data and the pitch data replacement means An ensemble support system comprising sound generation means for generating sound in response to the second performance data in which only data is replaced. The performance position determination means includes
Weight evaluation means for evaluating the current weights of all sounds to be played by the first player on the score data by comparing the pitch of the first performance data with the pitch of the score data;
Whether or not the weight at that time of the previous performance position and the current weight of the next performance position on the musical score data at the previous performance position satisfy a predetermined condition Satisfaction judgment means for judging, and
2. The ensemble support system according to claim 1, further comprising: a first determination unit that determines the next performance position on the score data as a current performance position when the satisfaction determination unit determines that the predetermined condition is satisfied. . The weight evaluation means includes
Comparing means for comparing the current pitch of the first performance data with the pitches of all performance positions to be played by the first player of the score data; and
When the comparison results by the comparison means match, the weight at the present time of the performance position on the score data whose pitch matches is a predetermined value as the weight at the previous time of the previous performance position. If the comparison results by the comparison means do not match, the current weight of the performance position on the score data where the pitches do not match is the previous one before the performance position. Including a weight calculation means that subtracts a predetermined value from the weight at the time,
The sufficiency determination means adds the predetermined value to the weight at the time of the previous performance point and adds the predetermined value to the current position of the next performance position on the score data of the previous performance point. The ensemble support system according to claim 2, wherein it is determined whether or not a condition of being equal to the weight of the ensemble is satisfied . The musical score data has a performance position where replaying is likely to occur as a designated performance position,
The performance position determination means includes
When the satisfaction determination means determines that the condition is not satisfied, the weight values of all the designated performance positions from the beginning to the current performance position are set to the next on the score data of the performance position of the previous time point. First weight setting means for setting a predetermined value smaller than the current weight value of the performance position ;
Based on the performance position at the previous time point, the current weight of the performance position from the performance position before the predetermined value on the musical score data by the predetermined value is used as the score at the performance position at the previous time point. Second weight setting means for setting the same value as the current weight of the next performance position on the data; and
The ensemble support system according to claim 2 or 3, further comprising second determining means for determining a performance position having the maximum weight as a current performance position . The performance position determination means includes first presence determination means for determining whether or not there are a plurality of performance positions having the maximum weight ,
Said second determining means, wherein the first existence determining means, when Starring Kanade position that have a maximum weight has been determined that no more exist, playing position of the current the Starring Kanade position that having a weight of said maximum The ensemble support system according to claim 4, which is determined as: When the first presence determination unit determines that there are a plurality of performance positions having the maximum weight, the performance position determination unit has the maximum weight and the performance position of the previous time point on the score data. A second presence determining means for determining whether or not there are a plurality of closest performance positions;
The second determination means has the maximum weight when the second presence determination means determines that there are a plurality of performance positions closest to the previous performance position on the score data having the maximum weight. and among the closest performance position in playing position of the previous point on the score data, determines a playing position is before playing position of the previous time as the performance position of the current, according to claim 5, wherein Ensemble support system. When the first presence determination unit determines that there are a plurality of performance positions having the maximum weight, the performance position determination unit has the maximum weight and the performance position of the previous time point on the score data. A second presence determining means for determining whether or not there are a plurality of closest performance positions;
When the second determining means determines that there are not a plurality of performance positions closest to the previous performance position on the score data having the maximum weight, the second determination means 6. The ensemble support system according to claim 5, wherein a performance position that has a weight and is closest to the previous performance position on the score data is determined as the current performance position.

Images