JP2006301019A - Pitch-notifying device and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a means for supporting improvement in performance ability, by notifying a user about the performance of other parts affecting the performance of the user. <P>SOLUTION: A pitch identifying section 1013 identifies the pitch of user's singing sounds, based on the pitch of guide sounds indicated by reference pitch data 1021. A performing part identifying section 1014 estimates, when the pitch of the singing sounds deviates from the pitch of the guide sounds, the part of accompaniment sounds causing the deviation, based on the relation between the pitch of singing sounds identified by the pitch identifying section 1013, the pitch of guide sounds, and the pitch of accompaniment sounds indicated by the reference pitch data 1021. An image signal generating section 1015 generates image signals, indicating the performing part estimated by the performing part identifying section 1014 and displays the image signals on a display 13. Thus, the user can conduct singing training, while checking the other performing parts that affect the user's singing performance. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a pitch notification technique for improving performance.

There is a technique for evaluating the singing ability of a singer in karaoke. For example, Patent Document 1 discloses a music evaluation apparatus that more accurately evaluates singing ability and musical instrument performance by performing fuzzy inference.
Japanese Patent No. 3074693

  By the way, when performing a song or a performance using a fretless instrument such as a violin together with other performance parts, the performer is affected by the sound of the other performance and produces a sound with a pitch that deviates from the sound that should originally be pronounced. May end up. In the case of a music evaluation apparatus according to the prior art, when such an erroneous performance is performed, for example, the player is notified of a bad evaluation as a whole performance, or the player is notified of an evaluation that the pitch is not correct. Or However, since the performer is not notified of the reason that the pitch is not correct, the performer cannot efficiently improve his performance.

  In view of the above situation, an object of the present invention is to provide means for supporting the improvement of performance by notifying the performer of the performance of other performance parts that affect his performance. To do.

  In order to achieve the above object, the present invention provides a reference pitch that indicates in time series the pitch of a sound to be produced by one performance part of the plurality of performance parts in a music played by a plurality of performance parts. Storage means for storing data; input means for acquiring a sound signal indicating a sound produced by the one performance part; and pitch specifying means for specifying a pitch of a sound indicated by the sound signal acquired by the input means The pitch of the sound indicated by the reference pitch data of the one performance part at one timing, the pitch of the sound of the one performance part specified by the pitch specifying means at the one timing, and the one Based on the relationship with the pitch of each of the performance parts other than the one performance part of the plurality of performance parts at the timing of A performance part specifying means for specifying another performance part that affects the performance of the one performance part at the one timing, and a performance part specified by the performance part specifying means or a sound produced by the performance part An output means for outputting data indicating the pitch of the generated sound is provided.

  According to the pitch notification device having such a configuration, the user can know which of the other performance parts affects his / her performance.

  In a preferred aspect, the reference pitch data stored by the storage means is data indicating time-sequentially the pitch of the sound to be generated by each of the plurality of performance parts, When specifying other performance parts, the pitch of the sound indicated by the reference pitch data of each performance part other than the one performance part at one timing is set to the performance other than the one performance part at the one timing. Used as the pitch of each sound of the part.

  In another preferred embodiment, the input means acquires a sound signal indicating a sound produced by each of the plurality of performance parts, and the pitch specifying means is indicated by a sound signal acquired by the input means. The pitch of the sound is specified for each of the sounds produced by the plurality of performance parts, and the performance part specifying means is specified by the pitch specifying means at one timing when specifying the other performance parts. The pitch of the sound of each performance part other than the one performance part is used as the pitch of the sound of each performance part other than the one performance part at the one timing.

  In another preferred embodiment, the storage means stores actual performance pitch data indicating the pitch specified by the pitch specifying means in time series, and the performance part specifying means is currently performing the performance. The pitch of the sound of the one performance part specified by the pitch specifying means at one timing and the actual performance pitch data of the one performance part at the one timing during the performance performed in the past. And the pitch of the sound at the same timing during the performance being performed by a performance part that has participated in the current performance and has not participated in the performance performed in the past. The other performance part is specified based on the relationship with the pitch.

  In another preferred embodiment, the storage means stores actual performance pitch data indicating the pitch specified by the pitch specifying means in time series, and the performance part specifying means is currently performing the performance. The pitch of the sound of the one performance part specified by the pitch specifying means at one timing and the actual performance pitch data of the one performance part at the one timing during the performance performed in the past. And the pitch of the sound at the same timing during the performance performed in the past by the performance part that did not participate in the performance performed in the past and participated in the performance performed in the past. The other performance part is specified based on the relationship with the pitch.

  In another preferred embodiment, the storage means stores real performance pitch data indicating the pitches specified by the pitch specification means in a time series, and the performance part specification means is configured to store the performance parts of the one performance part. The difference between the pitch of the sound indicated by the reference pitch data and the pitch of the sound indicated by the actual performance pitch data of the one performance part, the pitch of the sound indicated by the reference pitch data of the one performance part, and the The other performance parts are identified based on the correlation with the difference between the pitches of the sounds of the performance parts other than the performance part.

  In another preferred embodiment, the pitch specifying means includes a sound pitch candidate indicated by the sound signal acquired by the input means and a sound pitch indicated by the reference pitch data of the one performance part. A value of a function including the difference as a variable is calculated, and a pitch of a sound produced at the one timing by the one performance part is specified based on the value of the function.

  Further, in the above preferred aspect, the input means is a plurality of input means provided in association with each of the plurality of performance parts, and the function is applied to the one performance part with respect to one frequency. The amplitude of the frequency component at the one frequency of the sound signal acquired by the corresponding input means, and the sound signal acquired by at least one input means among the input means corresponding to the performance parts other than the one performance part A numerical value indicating a difference from the amplitude of the frequency component at the one frequency may be included as a variable.

  The present invention also provides a program that causes a computer to execute processing performed by the pitch notification device.

[1. First Embodiment]
Hereinafter, the first embodiment of the present invention will be described by taking as an example a case where singing training is performed in accordance with an accompaniment in which a singer (hereinafter referred to as “user”) is automatically played. FIG. 1 is a diagram showing a configuration of a performance training system 1 according to the first embodiment. The performance training system 1 outputs a sound signal indicating an accompaniment sound of a music, and outputs an image signal for displaying a graph representing a pitch of the user's singing sound and a pitch of the model singing sound, and a pitch A speaker 11, a microphone 12, a display 13, and a keyboard 14 connected to the notification device 10 are provided.

  In the following description, the pitch is represented by a frequency, and the unit is [Hz]. Therefore, the pitch of a certain sound is the fundamental frequency of the waveform of that sound. However, the numerical value indicating the pitch is not limited to the frequency, and may be expressed by a numerical value converted so that the interval between semitones corresponds to 100, for example. In that case, the unit is [cent].

  The speaker 11 receives a sound signal from the pitch notification device 10, converts the received sound signal into a sound, and generates a sound. The microphone 12 collects ambient sounds and outputs a sound signal indicating the collected sounds to the pitch notification device 10. The display 13 receives an image signal from the pitch notification device 10 and displays a screen including graphics and characters according to the received image signal. The keyboard 14 includes a plurality of keys and transmits a signal corresponding to the user's key operation to the pitch notification device 10.

  The pitch notification device 10 stores a control unit 101 that controls the components of the pitch notification device 10, a program that instructs various processes by the control unit 101, and various data that is used by the control unit 101, and a work of the control unit 101. A storage unit 102 used as an area, an input / output interface 103 through which the pitch notification device 10 transmits and receives signals to and from an external device, and an oscillator 104 that generates a clock signal at a predetermined time interval and delivers it to the control unit 101 are provided. ing. The clock signal generated by the oscillator 104 is used for timing of processing synchronization between components of the pitch notification device 10 and elapsed time from the beginning of the music.

  In the storage unit 102, reference pitch data 1021 indicating performance information of music is stored in advance. FIG. 2 is a diagram illustrating the contents of the reference pitch data 1021. The reference pitch data 1021 is related to one piece of music, and data such as the pitch of the sound to be generated by each of a plurality of performance parts playing the music and the lyrics to be displayed to the user is processed. This data includes data indicating the timing to be performed. The example of FIG. 2 shows a part of the content of the reference pitch data 1021 regarding the music played by each performance part of a vocal part, a flute part, and a piano part.

  Each row of data included in the reference pitch data 1021 (hereinafter referred to as “event data with timing”) has items of “timing”, “type”, and “content”, and “timing” indicates the timing of the music. “Type” indicates the type of the process to be executed at the timing indicated by “Timing”, and “Content” indicates the content of the process to be executed.

  For example, the timed event data illustrated in FIG. 2 [03: 01: 00 / Lyrics display / "Why are you not doing]" is the lyrics at the timing of the tick "000" of the first beat of the third measure of the song. It indicates that “why not” should be displayed. “Tick” is a time interval obtained by dividing the length of one beat into a predetermined number, and the elapsed time from the beginning of the beat is represented by the number of ticks, thereby indicating a specific timing in the music. Therefore, for example, [03: 01: 000] indicates the start timing of the first beat of the third measure. Hereinafter, as an example, one tick is set to 1/480 beat.

  The event data with timing [00: 01: 00 / tone / flute] illustrated in FIG. 2 indicates that the tone color of the sound produced in the performance part is “flute”. Here, [00: 01: 000] indicating the timing indicates before the start of the music or at the time of disclosure.

  Also, the event data with timing [03: 01: 240 / note-on / C4 (72)] indicates that the sound of the pitch “C4” is intensified at the timing of the tick “240” of the first beat of the third measure of the music. 72 "indicates that the sound should be pronounced. The event data with timing [03: 01: 420 / note off / C4] silences the sound of the pitch “C4” at the timing of the tick “420” of the third beat and the first beat of the music. Instructed that it should.

  As illustrated in the event data with timing [03: 01: 240 / note on / C4 (72)], in the reference pitch data 1021, the pitch of the sound to be generated is not a frequency, but “C4” or the like. It is expressed by the pitch name. This is for improving the readability of the data. For example, data directly indicating the pitch such as the frequency “261.626” (Hz) may be used instead of the pitch name “C4”.

  The format of the reference pitch data 1021 is not limited to that illustrated in FIG. 2, and may be in accordance with, for example, the format of SMF (Standard Musical Instrument Digital Interface File).

  The control unit 101 includes a playback unit 1011 that issues a music playback instruction according to the reference pitch data 1021. When the playback unit 1011 receives an instruction to start playback of a song by an operation using the keyboard 14 by the user, the playback unit 1011 continuously counts the elapsed time from the start timing of the song by counting the number of clock signals received from the oscillator 104. When the timing indicated by the “timing” column of the event data with timing included in the reference pitch data 1021 arrives, data indicating the performance part is added to the data in the “type” column and “content” column of the event data with timing. After the addition, it is handed over to other components included in the control unit 101 according to the contents of the “type” column. Hereinafter, data constituted by the “performance part” column, the “type” column, and the “content” column delivered to the other components by the playback unit 1011 is referred to as “event data”. The event data is data in a format such as [Piano / Note On / D4 (63)].

  The control unit 101 includes a sound source unit 1012 that generates a sound signal indicating a sound having a specified pitch and a specified pitch according to event data received from the playback unit 1011. The sound source unit 1012 can generate sound signals indicating different timbres, pitches, and sound intensities at the same time by, for example, FM (Frequency Modulation). However, the method of the sound source unit 1012 is not limited to the FM method, and may be any other method such as a PCM (Pulse Code Modulation) method or a physical model method. The sound source unit 1012 receives event data whose “type” column is “tone”, “note on”, or “note off” from the reproduction unit 1011. The sound source unit 1012 generates a sound signal indicating a designated sound for each performance part in accordance with the received event data, and delivers the generated sound signal to the input / output interface 103.

  The input / output interface 103 includes a sound signal output unit 1031 that outputs a sound signal to an external device. The sound signal output unit 1031 receives the sound signal from the sound source unit 1012 and outputs the received sound signal to the speaker 11. When the speaker 11 receives the sound signal from the sound signal output unit 1031, the speaker 11 converts the received sound signal into a sound and generates a sound. As a result, the user can hear the contents of the performance indicated by the reference pitch data 1021. Here, the performance sound of the vocal part is a guide sound indicating the pitch of the sound to be sung by the user.

  The user can select which performance part sound is to be generated from the speaker 11 prior to the start of the music performance. For example, when the user gives an instruction to perform with the vocal and flute sound muted, the playback unit 1011 delivers the event data to the sound source unit 1012 in accordance with only the piano part data in the reference pitch data 1021. As a result, the guide sound of the vocal part and the accompaniment sound of the flute part are not generated from the speaker 11.

  The input / output interface 103 includes a sound signal input unit 1032 that receives a sound signal from an external device. When the user instructs the pitch notification device 10 to start playing and sings according to the accompaniment sound or the guide sound of the music sounded from the speaker 11, the microphone 12 collects surrounding sounds including the user's singing sound. The sound signal indicating the collected sound is output to the sound signal input unit 1032. When the sound signal input unit 1032 receives a sound signal from the microphone 12, the received sound signal is stored in the storage unit 102 as actual performance waveform data 1022.

  The control unit 101 includes a pitch specifying unit 1013 that specifies the pitch of the user's singing sound almost in real time. The pitch specifying unit 1013 extracts the waveform data for a predetermined time recently stored from the actual performance waveform data 1022 sequentially stored in the storage unit 102. Hereinafter, waveform data for a predetermined time is referred to as a “frame”. The length of one frame is sufficient to specify the pitch, and the time difference between the specified pitch and the pitch of the currently sounding sound needs to be short enough to be acceptable to the user. Hereinafter, as an example, the length of one frame is assumed to be 100 milliseconds.

  Various methods for specifying the pitch of the sound from the sound waveform data have been proposed. Hereinafter, as an example, the pitch specifying unit 1013 specifies the pitch based on the power spectrum of the sound waveform data. However, the method used when the pitch specifying unit 1013 specifies the pitch may use any existing method as long as the pitch can be specified almost in real time.

  The pitch specifying unit 1013 performs processing such as fast Fourier transform on one frame extracted from the actual performance waveform data 1022, and calculates the distribution of frequency components included in the waveform indicated by the extracted one frame. Subsequently, the pitch specifying unit 1013 obtains a graph indicating the envelope of the graph indicating the distribution of the calculated frequency components. FIG. 3 is a diagram illustrating a graph showing an envelope of the distribution of frequency components calculated by the pitch specifying unit 1013. In FIG. 3, the frequency band indicated by W is a range in which the center frequency of a general human voice is distributed.

According to the distribution of frequency components shown in the graph of FIG. 3, the amplitude of the frequency component has a maximum value at _six frequencies ω ₁ to ω ₆ . Thus, the frequency at which the amplitude of the frequency component takes the maximum value is a frequency that is a candidate for the fundamental frequency of the sound indicated by the sound waveform. However, the sound usually includes not only the fundamental frequency component but also many harmonic components of the fundamental frequency. The sound collected by the microphone 12 in the performance training system 1 is mixed with accompaniment sounds and guide sounds generated by the sound source unit 1012 of the pitch notification device 10 and generated from the speakers 11 in addition to the user's singing sound. . Therefore, for example, it cannot be easily specified which of the frequencies ω _{1 to} ω ₆ shown in FIG. 3 is the fundamental frequency of the user's singing sound.

  Therefore, the pitch specifying unit 1013 receives the event data of the performance part “vocal” and the type “note on” from the reproduction unit 1011, and the pitch indicated by the received event data, that is, the singing sound that the vocal part should be pronounced at that time Is used to identify which fundamental frequency candidate is the correct fundamental frequency.

Specifically, the pitch specifying unit 1013 converts the pitch name shown in the “content” column of the event data of the performance part “vocal” and type “note on” received last from the playback unit 1011 into a frequency. For example, if the pitch name indicated by the event data is “A4”, the pitch specifying unit 1013 obtains “440.000” as the frequency corresponding to the pitch name. Hereinafter, the frequency thus converted is _denoted by ω ₀ . In addition, the pitch of the pitch indicated by the event data is hereinafter referred to as “reference pitch”.

Subsequently, the pitch specifying unit 1013 extracts the frequency ω _{1 to} ω ₆ shown in FIG. 3 that is included in the frequency band W. In this example, the frequencies ω ₂ and ω ₃ are extracted. Subsequently, the pitch specifying unit 1013 calculates, for example, a function value represented by the following (Equation 1) for each of the extracted frequencies.

However, s (ω) in (Equation 1) indicates the amplitude of the frequency component at the frequency ω. Further, k is an arbitrary natural number, and ω _k indicates a candidate for the fundamental frequency of the user's singing sound. In the example of FIG. 3, k = 2 or k = 3. The first term on the right side of (Expression 1) is a term indicating the amplitude of the frequency component in the fundamental frequency candidate. The second term on the right side of (Equation 1) is a term indicating the difference between the reference pitch frequency indicated by the reference pitch data 1021 and the fundamental frequency candidate, and the difference is always taken to be a value of 0 or more. Has been squared. C in (Expression 1) is a variable for determining a coefficient indicating the contribution in the function f (ω _k ) of the amplitude of the frequency component in the fundamental frequency candidate, and is empirically appropriate in the range of 0 to 1. A value is selected.

The value of the function f (ω _k ) expressed by (Equation 1) is the difference between the frequency of the reference pitch indicated by the reference pitch data 1021 and the basic frequency candidate with respect to the amplitude of the frequency component in the basic frequency candidate. This is the value to which the bias based is added. Pitch identification unit 1013 calculates the value of k = 2 and k = 3 for each relates to the function f (ω _k), the omega _k when these values is the maximum, the singing sound that the user is currently Pronunciation Identified as the fundamental frequency.

When the frequency having the maximum frequency component is simply selected as the fundamental frequency of the sound waveform according to the prior art, for example, in FIG. 3, ω ₃ is the correct fundamental frequency of the singing sound, and ω ₂ is the basic accompaniment sound by the flute part. If it is a frequency, ω ₂ having a larger amplitude is erroneously specified as the fundamental frequency of the singing sound. On the other hand, in the function value as in the above (formula 1) used by the pitch specifying unit 1013, as a result of the absolute value of the second term being increased with respect to the fundamental frequency candidate greatly deviating from the frequency of the reference pitch, The value of the function f (ω _k ) becomes small and is not finally selected as the fundamental frequency, and the possibility that the correct fundamental frequency is specified increases.

  However, the above (Formula 1) is an example of a functional formula used by the pitch identifying unit 1013 to identify the fundamental frequency of the singing sound, and it goes without saying that various other functional formulas can be used. In short, any method can be used as long as the correct fundamental frequency is identified from a plurality of fundamental frequency candidates by reflecting the difference between the reference pitch frequency and the fundamental frequency candidate. 1013 can be employed as a method for specifying the pitch.

Further, instead of using the frequency when the amplitude of the frequency distribution takes an extreme value as the fundamental frequency candidate, for example, the pitch specifying unit 1013 sequentially takes out the frequencies included in the frequency band W every 1 Hz to make the fundamental frequency candidate, The value of the function (ω _k ) of (Equation 1) may be calculated for these fundamental frequency candidates, and ω _k that maximizes the calculated value may be specified as the fundamental frequency.

  The pitch specifying unit 1013 performs the above-described basic frequency specifying process every 10 milliseconds, for example, and stores the data indicating the result in the storage unit 102 sequentially in association with the data indicating the timing of the music at that time. To do. When the performance is finished, the pitch specifying unit 1013 associates the data indicating the performance part where the performance sound was generated during the performance with the collection of data indicating the fundamental frequency and timing stored in sequence, and the actual performance pitch data. 1023 is stored in the storage unit 102. FIG. 4 is a diagram illustrating the contents of the actual performance pitch data 1023. In addition, when the singing practice using the performance training system 1 is performed a plurality of times by the user, actual performance pitch data 1023 corresponding to each singing practice is stored in the storage unit 102.

  The control unit 101 includes a performance part specifying unit 1014 that specifies another performance part that affects the user's singing sound. The performance part specifying unit 1014 processes whether or not the actual performance pitch data 1023 related to the past performance is stored in the storage unit 102 and, if stored, the actual performance pitch data 1023 related to the performance by any performance part is stored. It depends on which part is being played or which part of the current performance is sounding. Hereinafter, the processing of the performance part specifying unit 1014 will be described for each variation.

(Case 1: There is no actual performance pitch data related to past performances, and the accompaniment part being pronounced is a single voice)
First, the actual performance pitch data 1023 relating to past performances is not yet stored in the storage unit 102, and there is one performance part other than the vocal part, that is, one that is sounding among the accompaniment parts, and the flute Thus, in the case of a monophonic instrument, the performance part specifying unit 1014 receives event data of the type “note on” regarding the vocal part and the accompaniment part that is sounding from the playback unit 1011. Further, the performance part specifying unit 1014 sequentially receives data indicating the pitch of the current singing sound specified by the pitch specifying unit 1013 from the pitch specifying unit 1013 (hereinafter referred to as “current actual performance pitch data”).

The performance part specifying unit 1014 is a place where the singing sound is affected by the accompaniment sound during the performance based on the pitch relationship between the guide sound pitch, the accompaniment sound pitch, and the user singing sound pitch indicated by the data. Is identified. Hereinafter, the process will be described using a specific example. It is assumed that the event data and the current actual performance pitch data regarding each performance part received last by the performance part specifying unit 1014 are as follows at a certain timing during performance.
(A) Event data [Vocal / Note On / C4]
(B) Event data [flute / note on / A5]
(C) Current performance pitch data [258.415]

  Here, (a) indicates the pitch of the guide sound, and (b) indicates the pitch of the accompaniment sound. The performance part specifying unit 1014 shifts the pitch of the accompaniment sound in units of octaves so that the pitch of the accompaniment sound falls within a range of one octave before and after the pitch of the guide sound. In the above example, since the pitch of the guide sound is [C4] and the pitch of the accompaniment sound is [A5], the performance part specifying unit 1014 shifts down the pitch of the accompaniment sound to [A4]. . The reason why the pitch of the accompaniment sound is pitch-shifted in this way is because the pitch of the scale in the music is more meaningful than the absolute pitch of the sound. Hereinafter, the pitch shift processing for bringing the pitch of the accompaniment sound closer to the guide sound is referred to as “pitch shift processing”.

Subsequently, the performance part specifying unit 1014 converts the pitch of the guide sound and the pitch of the accompaniment sound after the pitch shift (hereinafter simply referred to as “accompaniment sound”) into a corresponding frequency. For example, pitch [C4] is converted to frequency [261.626] Hz, and pitch [A4] is converted to frequency [220.000] Hz. Hereinafter, the process of converting the pitch indicated by the pitch name into a frequency is referred to as “pitch conversion process”. The performance part specifying unit 1014 obtains the guide sound frequency [261.626] Hz, the accompaniment sound frequency [220.000] Hz, and the user singing sound frequency [258.415] Hz by pitch conversion processing. Hereinafter, these values are referred to as f _a , f _b, and f _c .

Subsequently, the playing part specification section 1014 as an index of deviation from the guide sound of the singing _sound, d = calculates a _{/ f a × 100 (%)} (| | f c -f a). The performance part specifying unit 1014 determines that the singing sound has deviated from the guide sound to the extent that correction is required when the deviation index calculated in this way is a predetermined threshold, for example, 1.00 (%) or more. Hereinafter, the process of detecting that the singing sound is deviated to an extent that requires correction by the index d is referred to as “deviation detection process”.

  When a deviation requiring correction is detected in the deviation detection process, the performance part specifying unit 1014 determines whether or not the accompaniment sound affects the deviation. Specifically, the performance part specifying unit 1014 has a gap in the singing sound when the pitch of the accompaniment sound is on the same side as the pitch of the singing sound as viewed from the pitch of the guide sound and is far from the pitch of the singing sound. It is determined that the sound is influenced by the accompaniment sound.

For example, in the case of the above example, the deviation index is d = about 1.23, which is equal to or greater than the predetermined threshold value 1.00 (%). Further, the frequency of the accompaniment sound (f _b = 220.000) is on the same side as the frequency of the singing sound (f _c = 258.415) when viewed from the frequency of the guide sound (f _a = 261.626), and the guide sound the absolute value of the difference between the frequency of the accompaniment sound _{(| f a -f b | =} 41.626) is the frequency of the guide sound absolute value of the difference between the frequency of the singing sound _(| f a -f _c | = 3.211). Accordingly, the performance part specifying unit 1014 estimates that the deviation of the singing sound is affected by the accompaniment sound. Hereinafter, the process of estimating that the accompaniment sound has an influence on the shift of the singing sound based on the relationship between the frequency of the accompaniment sound, the frequency of the guide sound, and the frequency of the singing sound is referred to as “influence estimation processing”.

  The control unit 101 includes an image signal generation unit 1015 that generates a screen image signal indicating the pitch of the guide sound and the accompaniment sound indicated by the event data and the pitch of the user singing sound in a graph. When the performance part specifying unit 1014 estimates that the accompaniment sound affects the deviation of the singing sound in the effect estimation process, the performance part specifying unit 1014 indicates data indicating the performance part that is producing the accompaniment sound (hereinafter, “performance part data”). Is called) to the image signal generation unit 1015. When the performance part specifying unit 1014 estimates that the accompaniment sound does not affect the singing sound, for example, performance part data “none” is used as data indicating that none of the performance parts has an influence on the singing. Delivered to the image signal generation unit 1015.

(Case 2: When there is no actual performance pitch data related to past performances and there are multiple or polyphonic accompaniment parts)
When the actual performance pitch data 1023 relating to the past performance is not yet stored in the storage unit 102 and there are a plurality of accompaniment parts that are sounding, or when the accompaniment part is a polyphonic instrument, the performance part specifying unit 1014 performs playback. Event data of the type “Note On” relating to the vocal part and all the accompaniment parts being pronounced is received from the section 1011. The performance part specifying unit 1014 sequentially receives the current actual performance pitch data from the pitch specifying unit 1013.

  When the accompaniment part is a polyphonic instrument such as a piano, the performance part specifying unit 1014 separates a chord string that is simultaneously generated by the accompaniment part into a plurality of melody lines. Treated as a sequence of sounds played by two monophonic instruments.

  Subsequently, the performance part specifying unit 1014 performs a pitch shift process on each accompaniment sound, and shifts the pitch of the accompaniment sound to the vicinity of the pitch of the guide sound. Thereafter, the performance part specifying unit 1014 performs pitch conversion processing on the pitch of the guide sound and the pitch of the accompaniment sound after the shift (hereinafter simply referred to as “accompaniment sound”) to obtain each frequency.

  The performance part specifying unit 1014 performs a deviation detection process using the frequency obtained as described above. The performance part specifying unit 1014 performs the above-described series of processing every time the actual performance pitch data is received from the pitch specifying unit 1013. When performing the deviation detection process, the performance part specifying unit 1014 generates data in which the data indicating the presence / absence of the deviation is associated with the frequency indicating the pitch of the guide sound, each accompaniment sound, and the singing sound. The work pitch data 1024 is temporarily stored in the storage unit 102 sequentially. FIG. 5 is a diagram illustrating the contents of the work pitch data 1024. In FIG. 5, accompaniment sound 1 indicates the pitch of the sound of the accompaniment part “flute”, and accompaniment sounds 2 to 4 indicate the pitches of the first to third sounds of the accompaniment part “piano”, respectively. Yes.

  When the performance part specifying unit 1014 determines that there is a shift that needs to be corrected in the shift detection process, the performance part specifying unit 1014 estimates which of the plurality of accompaniment sounds causes the shift. Therefore, the performance part specifying unit 1014 first performs an influence estimation process for each accompaniment sound, and extracts an accompaniment sound that has yielded a positive result as a candidate for an accompaniment sound that causes the user's singing error. To do. In other words, an accompaniment sound that is in the same range as the singing sound and is farther from the guide sound than the singing sound is extracted as a candidate for an accompaniment sound that affects the singing sound. Hereinafter, the process of extracting accompaniment sound candidates that affect the singing sound is referred to as “candidate sound extraction process”.

  If there is still a small number of data in which “presence / absence of deviation” is “present” among the data stored in the work pitch data 1024, such as immediately after the start of the performance, the performance part specifying unit 1014 selects the candidate sound. A performance part that produces an accompaniment sound having a pitch closest to the pitch of the singing sound among the accompaniment sounds extracted by the extraction process is estimated to be a performance part that affects the deviation of the singing sound.

  On the other hand, when the number of data having “existence of deviation” is “present” among the data stored in the work pitch data 1024 is sufficiently large, the performance part specifying unit 1014 is stored in the work pitch data 1024. For each of the performance parts of the accompaniment sounds extracted in the candidate sound extraction process, data with “presence / absence” of “exist” is extracted from the recorded data, and the pitch and singing of the guide sound indicated by the extracted data A correlation coefficient between the difference between the pitch of the sound and the difference between the pitch of the guide sound and the pitch of the accompaniment sound is calculated.

For example, when accompaniment sound 1 (flute) is extracted as a candidate for an accompaniment sound that affects the singing sound at a certain timing, “presence / absence of deviation” is “present” in the work pitch data 1024. The frequency sequence of the guide sound included in the object is f _ak , the frequency sequence of the accompaniment sound 1 is f _bk , and the frequency sequence of the singing sound is f _ck (where k = 1 to n is a natural number, n is Assuming that “presence / absence” is “number of data”, the performance part specifying unit 1014 calculates a correlation coefficient for the combination of (f _bk −f _ak ) and (f _ck −f _ak ) that changes according to k. calculate. The correlation coefficient thus calculated is an index indicating the correlation between the change of the singing sound and the change of the accompaniment sound when the singing sound has deviated from the guide sound in the past and the pitch of the guide sound is used as a reference. It is.

  When the performance part specifying unit 1014 calculates the correlation coefficient for all the performance parts of the accompaniment sounds extracted in the candidate sound extraction process, the performance part having the largest correlation coefficient is affected by the deviation of the singing sound. It is presumed that the performance part is.

  When the performance part specifying unit 1014 estimates the performance part that affects the deviation of the singing sound as described above, the performance part specifying unit 1014 passes the performance part data indicating the estimated performance part to the image signal generation unit 1015. If the performance part specifying unit 1014 determines that none of the performance parts has an effect on the singing sound shift, the performance part specifying unit 1014 delivers, for example, performance part data “none” to the image signal generation unit 1015.

(Case 3: When actual performance pitch data relating to past performances in which an accompaniment part participating in the current performance does not participate is stored)
When actual performance pitch data 1023 relating to a past performance that does not participate in any of the accompaniment parts that are sounding in the currently performed performance is stored in the storage unit 102, the performance part specifying unit 1014 is a playback unit. From 1011, event data of the type “Note On” relating to the vocal part and all the accompaniment parts being sounded are received. The performance part specifying unit 1014 sequentially receives the current actual performance pitch data from the pitch specifying unit 1013.

  The performance part specifying unit 1014 performs processing for separating a chord string of an accompaniment part of a polyphonic instrument into a plurality of melody lines, pitch shift processing for accompaniment sounds, pitch conversion processing for guide sounds and accompaniment sounds, and guide sound , The frequency of each accompaniment sound, and the frequency of the singing sound in the current performance.

Subsequently, the performance part specifying unit 1014 reads out, from the storage unit 102, data of timing corresponding to the timing in the currently played music from the data included in the actual performance pitch data 1023 regarding the past performance. Hereinafter, the data thus read is referred to as “past actual performance pitch data”. It is assumed that the data indicating the frequency obtained as described above is as follows.
(A) Guide sound [261.626]
(B1) Accompaniment sound 1 (flute) [220.000]
(B2) Accompaniment sound 2 (piano 1) [196.000]
(B3) Accompaniment sound 3 (piano 2) [2466.942]
(B4) Accompaniment sound 4 (piano 3) [293.665]
(C) Current performance pitch data [258.415]
(D) Past actual performance pitch data [260.521]

Here, when the performance part sounded in the performance indicated by the actual performance pitch data 1023 related to the past performance stored in the storage unit 102 is “piano”, the above data indicates the following facts. ing.
(A) In the performance currently being performed, the singing sound (258.415 Hz) deviates from the guide sound (261.626 Hz) to the extent that requires correction.
(B) In performances performed in the past, the singing sound (260.521 Hz) was closer to the guide sound (261.626 Hz) than the current singing sound.
(C) The accompaniment part “flute” was not pronounced in the performances performed in the past, but the accompaniment part “flute” is pronounced in the current performance.

  From the above, it is inferred that there is a possibility that the user is producing a sound with a pitch lower than that of the sound that should be originally produced by being dragged by the sound of the accompaniment part “flute” in the current song. Therefore, the performance part specifying unit 1014 estimates whether or not the accompaniment sound 1 of the accompaniment part “flute” affects the deviation of the current singing sound. The estimation method is the same as that in Case 1 described above.

On the other hand, when the performance part sounded in the performance indicated by the actual performance pitch data 1023 related to the past performance stored in the storage unit 102 is “flute”, the above data indicates the following facts: Yes.
(A) In the performance currently being performed, the singing sound (258.415 Hz) deviates from the guide sound (261.626 Hz) to the extent that requires correction.
(B) In performances performed in the past, the singing sound (260.521 Hz) was closer to the guide sound (261.626 Hz) than the current singing sound.
(C) The accompaniment part “piano” was not pronounced in the performances performed in the past, but the accompaniment part “piano” is pronounced in the current performance.

  From the above, it is inferred that the user is making a sound with a pitch lower than the pitch of the sound that should be originally produced by being dragged by any sound of the accompaniment part “piano” in the current song. Therefore, the performance part specifying unit 1014 estimates which of the accompaniment sounds 2 to 4 separated from the accompaniment part “piano” is an accompaniment sound that affects the deviation of the current singing sound. The estimation method is the same as that in Case 2 described above.

  When the performance part specifying unit 1014 estimates the performance part of the accompaniment sound that affects the deviation of the singing sound as described above, the performance part specifying unit 1014 hands over the performance part data indicating the estimated performance part to the image signal generation unit 1015. If the performance part specifying unit 1014 determines that none of the performance parts has an effect on the singing sound shift, the performance part specifying unit 1014 delivers, for example, performance part data “none” to the image signal generation unit 1015.

(Case 4: Actual performance pitch data related to past performances in which accompaniment parts not participating in the current performance participated are stored)
When actual performance pitch data 1023 relating to a past performance in which any of the accompaniment parts that are not sounding in the currently performed performance is participating is stored in the storage unit 102, the performance part specifying unit 1014 is a playback unit. From 1011, event data of the type “note on” regarding the vocal part and all the accompaniment parts that have been pronounced in the past performance are received. The performance part specifying unit 1014 sequentially receives the current actual performance pitch data from the pitch specifying unit 1013.

  The performance part specifying unit 1014 performs processing for separating a chord string of an accompaniment part of a polyphonic instrument into a plurality of melody lines, pitch shift processing for accompaniment sounds, pitch conversion processing for guide sounds and accompaniment sounds, and guide sound , The frequency of each accompaniment sound, and the frequency of the singing sound in the current performance.

Subsequently, the performance part specifying unit 1014 reads the past actual performance pitch data from the actual performance pitch data 1023 related to the past performance, that is, the timing data corresponding to the timing in the currently played music from the storage unit 102. It is assumed that the data indicating the frequency obtained as described above is as follows.
(A) Guide sound [261.626]
(B1) Accompaniment sound 1 (flute) [220.000]
(B2) Accompaniment sound 2 (piano 1) [196.000]
(B3) Accompaniment sound 3 (piano 2) [2466.942]
(B4) Accompaniment sound 4 (piano 3) [293.665]
(C) Current actual performance pitch data [261.102]
(D) Past actual performance pitch data [265.314]

Here, when the accompaniment part that produces pronunciation in the current performance is only “flute”, the above data indicates the following facts.
(A) In the performance currently being performed, the singing sound (261.102 Hz) is close to the guide sound (261.626 Hz) to the extent that no correction is required.
(B) In performances performed in the past, the singing sound (265.314 Hz) deviated from the guide sound (261.626 Hz) to the extent that correction is required.
(C) The accompaniment part “piano” was pronounced in the performances performed in the past, but the accompaniment part “piano” is not pronounced in the current performance.

  From the above, it is inferred that the user has been producing a sound with a pitch higher than the pitch of the sound that should be originally produced by being dragged by any sound of the accompaniment part “piano” in the past performance. Therefore, the performance part specifying unit 1014 estimates which of the accompaniment sounds 2 to 4 separated from the accompaniment part “piano” in the past performance is the accompaniment sound that has affected the deviation of the past singing sound. The estimation method is the same as that in Case 2 described above.

On the other hand, when the accompaniment part that produces pronunciation in the current performance is only “piano”, the above data shows the following facts.
(A) In the performance currently being performed, the singing sound (261.102 Hz) is close to the guide sound (261.626 Hz) to the extent that no correction is required.
(B) In performances performed in the past, the singing sound (265.314 Hz) deviated from the guide sound (261.626 Hz) to the extent that correction is required.
(C) The accompaniment part “flute” was pronounced in the performances performed in the past, but the accompaniment part “flute” was not pronounced in the current performance.

  From the above, it is inferred that in the past performance, the user might have been dragged by the sound of the accompaniment part “flute” and had produced a sound with a pitch higher than the pitch of the sound that should be originally produced. Therefore, the performance part specifying unit 1014 estimates whether or not the accompaniment sound 1 of the accompaniment part “flute” in the past performance was an accompaniment sound that had affected the deviation of the past singing sound. The estimation method is the same as that in Case 1 described above.

  When the performance part specifying unit 1014 estimates the performance part of the accompaniment sound that has affected the deviation of the singing sound as described above, the performance part specifying unit 1014 delivers performance part data indicating the estimated performance part to the image signal generation unit 1015. When the performance part specifying unit 1014 determines that none of the performance parts has affected the deviation of the singing sound, the performance part data “None” is transferred to the image signal generation unit 1015, for example.

  As described above, the image signal generation unit 1015 receives performance part data indicating a performance part that is estimated to be affecting the deviation of the singing sound from the performance part specifying unit 1014. Further, the image signal generation unit 1015 receives event data indicating the pitch of the sound produced by each performance part and the lyrics to be displayed from the reproduction unit 1011. Further, the image signal generation unit 1015 receives the current actual performance pitch data from the pitch specifying unit 1013. The image signal generation unit 1015 uses these data to inform the user of the lyrics to be sung, the pitch of the guide sound, the pitch of the accompaniment sound, the pitch of the singing sound, the location where the singing sound is deviated from the guide sound, and the deviation thereof. An image signal for indicating a performance part that is estimated to have been affected is generated.

  The input / output interface 103 includes an image signal output unit 1033 that outputs the image signal generated by the image signal generation unit 1015 to an external device. The image signal generation unit 1015 transmits the generated image signal to the display 13 via the image signal output unit 1033.

  FIG. 6 is a diagram showing an image displayed on the display 13 in accordance with the image signal transmitted from the image signal output unit 1033. In the image shown in FIG. 6, the horizontal axis indicates the time that elapses as the music progresses, and the vertical axis indicates the pitch of the sound that is sounded. Character 131 indicates the lyrics. Line 132 indicates the current timing of the music.

  The broken lines 133 to 135 indicate the pitches of the guide sound, accompaniment part 1 (flute) and accompaniment part 2 (piano 1), respectively. A curve 136 indicates the pitch of the singing sound. In FIG. 6, a broken line indicating the pitch of accompaniment parts 3 and 4 (pianos 2 and 3) is omitted for simplification of the drawing. In addition, the broken line indicating the pitch of the accompaniment part that has participated in the past performance and does not participate in the current performance is distinguished from the broken line that indicates the pitch of the accompaniment part participating in the current performance, for example, by a dotted line. Is done.

  Markers 137 and 138 are attached to the polygonal lines 134 and 135, respectively. These markers indicate the period during which the pitch of the singing sound is greatly deviated from the pitch of the guide sound, and the markers that indicate the performance part that is estimated to have caused the deviation and the pitch of the sound produced by the performance part. is there. However, it is indicated that the accompaniment part 1 (flute) indicated by the broken line 134 does not participate in the current performance and has influenced the singing sound during the period indicated by the marker 137 of the past performance. Yes. Therefore, the pitch of the current singing sound is not greatly deviated from the pitch of the guide sound during the period indicated by the marker 137. That is, the performance during that period is improved by the loss of accompaniment part 1 (flute). Arrows 139 and 140 are arrows indicating which accompaniment part affects the singing sound.

  As described above, according to the performance training system 1, the user can confirm in real time how much the pitch of his current performance sound is deviated from the pitch of the sound that should be originally pronounced, and the deviation. Can be found by the sound of any other performance part. Therefore, the user can effectively improve performance performance by repeating performance training while being conscious not to be dragged by the sound of the performance part thus notified.

  In the above description, the performance part of the user is a vocal part, but the performance part of the user may be an instrument such as a violin. In that case, the pitch identification unit 1013 is different from the above except that the range of the frequency band W in the extraction of the fundamental frequency candidates is different.

  In the above description, the reference pitch data 1021 is in the form of performance control data as illustrated in FIG. 2, but is generated at each timing as illustrated in FIG. 5, for example. Data of a format that indicates the pitch of the power sound by frequency or the like may be used. In that case, a pitch conversion process for converting the pitch into a frequency is not necessary.

  Further, the image shown in FIG. 6 is an example of an image for notifying the user of other performance parts that have an influence on the performance by the performance training system 1, and may have various display modes. Needless to say. For example, the color or thickness of the broken line may be changed instead of the marker, or the state where the user's performance sound is dragged by the sound of another performance part may be indicated by an arrow.

  The pitch notification device 10 may be realized by dedicated hardware or may be realized by causing a general-purpose computer to execute processing according to an application program. When the pitch notification device 10 is realized by a general-purpose computer, each component of the control unit 101 includes a CPU (Central Processing Unit) included in the general-purpose computer and a DSP (Digital Signal Processor) operating under the control of the CPU. Are performed as functions of a general-purpose computer by performing processing in accordance with each module included in the above in parallel.

[2. Second Embodiment]
Hereinafter, the second embodiment of the present invention will be described using a case where a plurality of users perform chorus training as an example. FIG. 7 is a diagram showing a configuration of the performance training system 2 according to the second embodiment. The performance training system 2 is different from the performance training system 1 in that a plurality of microphones 12 are provided according to the number of users, but the other configurations are the same as the performance training system 1. Accordingly, in FIG. 7, the same or similar components as those of the performance training system 1 are denoted by the same reference numerals as those used in FIG. However, the pitch notification device included in the performance training system 2 is distinguished from the pitch notification device 10 as the pitch notification device 20 because the processing of the control unit is partly different from that of the pitch notification device 10. In addition, the operation of the performance training system 2 is common in many respects to the operation of the performance training system 1. Accordingly, only the differences between them will be described below.

  Hereinafter, for explanation, the number of users who use the performance training system 2 is three, and these are users 19-1 to 19-3. The performance training system 2 is assumed to include microphones 12-1 to 12-3 used by the users 19-1 to 19-3. Furthermore, it is assumed that the users 19-1 to 19-3 are in charge of the chorus parts 1 to 3, respectively, in the choral music for training.

  The pitch specifying unit 1013 included in the control unit 101 of the pitch notification device 20 receives sound signals indicating sounds collected by the microphones 12-1 to 12-3, and the users 19-1 to 19-1 from the three sets of sound signals. The fundamental frequency of each singing sound of 3 is specified. First, the sound collected by each of the microphones 12-1 to 12 includes the sound of the accompaniment part output from the sound source unit 1012 and pronounced by the speaker 11, as in the performance training system 1. . In addition, the frequency component of the singing sound of each user of the microphones 12-1 to 12 includes a harmonic component of the fundamental frequency.

  In addition to the above, in the performance training system 2, since the microphones 12-1 to 12-3 are arranged in the same space, for example, the microphone 12-1 mainly collects the singing sound of the user 19-1, but at the same time. The singing sounds of the nearby users 19-2 and 3 are also collected. Therefore, when the pitch specifying unit 1013 of the pitch notification device 20 specifies the fundamental frequency of the singing sound of the user 19-1 from the sound signal acquired from the microphone 12-1, for example, the other microphone 12, that is, the microphone 12-2 and 3 calculates a function value including the difference in amplitude between the frequency component of the sound signal acquired from 3 and the frequency component of the sound signal acquired from the microphone 12-1 in the fundamental frequency candidate as a variable, and calculates the fundamental frequency based on the value. Identify.

Specifically, the pitch identifying unit 1013 first calculates the frequency ω ₀ indicating the pitch of the guide sound of the chorus part 1 indicated by the event data received from the playback unit 1011 as in the performance training system 1. . Subsequently, the pitch specifying unit 1013 performs frequency analysis on the actual performance waveform data 1022 indicating the sound signal input from the microphone 12-1, and obtains a graph indicating the envelope. The pitch specifying unit 1013 extracts, as a fundamental frequency candidate, a frequency having a maximum amplitude included in the frequency band W in which the fundamental frequency of human speech is distributed in the graph thus obtained. Hereinafter, these fundamental frequency candidates are denoted by ω ₁ to ω _k (where k is the number of extracted fundamental frequency candidates).

Subsequently, the pitch specifying unit 1013 performs frequency analysis on the actual performance waveform data 1022 indicating the sound signals input from the microphones 12-2 and 3, and calculates the amplitude of the frequency component at each frequency. Hereinafter, s _i (ω) is the frequency component at the frequency ω of the sound signal input from each of the microphones 12-1 to 12 (where i is any one of the numerical values 1 to 3 corresponding to the microphones 12-1 to 12). And

Subsequently, the pitch specifying unit 1013 calculates a value of a function represented by the following (Equation 2), for example.

The first term and the third term on the right side of (Formula 2) are terms that play the same role as the first term and the second term on the right side of (Formula 1) in the performance training system 1, respectively. The second term of the right side of (Equation 2) is the sum of the difference between the frequency components of the sound collected by the frequency component and the other microphone 12 of collected sounds by the microphone 12-1 in the candidate omega _k of the fundamental frequency It is a term to show. The second term on the right side of (Expression 2) is large when ω _k is a component of sound collected with a large volume by the microphone 12-1 among the same sounds collected by the microphones 12-1 to 12-1. Value. Therefore, f ₁ (ω _k ) has a larger value at the fundamental frequency of the singing sound of the user 19-1 singing near the microphone 12-1 than at the fundamental frequency of the singing sound of the users 19-2 and 3. Take. Pitch specifying unit 1013, the value of _{f 1} (ω _k) is identified as the fundamental frequency of the singing sound of the user 19-1 omega _k that maximizes.

The pitch specifying unit 1013 also calculates a similar function value for the actual performance waveform data 1022 indicating the sound signals input from the microphones 12-2 and 3, and sets the frequency at which the calculated function value is maximum to the user. It is specified as the fundamental frequency of 19-2 and 3 singing sounds. That is, (Expression 2) is generalized as (Expression 3).

  Note that a and b in (Equation 2) or (Equation 3) are variables for determining a coefficient indicating the degree of contribution of each term, and appropriate values are selected empirically. Here, (Equation 3) is an example of a functional equation used by the pitch identification unit 1013 to identify the fundamental frequency of the singing sound, and that various other functional equations can be used is that the performance training system 1 It is the same as in the case of. If necessary, the evaluation index reflects the difference between the frequency component of the sound signal of the sound collected by the microphone 12 of interest and the frequency component of the sound signal of the sound collected by the other microphone 12. The fundamental frequency of the singing sound of the user 19 who is singing toward the microphone 12 that is not performing is specified as the fundamental frequency of the singing sound of the user 19 who is singing toward the microphone 12 that is erroneously paying attention. Any method can be used as a method for the pitch specifying unit 1013 to specify the pitch.

  The pitch specifying unit 1013 generates data indicating the fundamental frequency of the singing sound of the users 19-1 to 19-3 specified as described above, that is, the pitch of the chorus parts 1 to 3, and sequentially performs the generated data on the actual performance. The pitch data 1023 is stored in the storage unit 102 and is also transferred to the performance part specifying unit 1014 and the image signal generation unit 1015.

  The performance part specifying unit 1014 of the pitch notification device 20 receives the event data indicating the pitch of the guide sounds of the chorus parts 1 to 3 from the playback unit 1011, and instead of receiving the event data indicating the pitch of the chorus parts 1 to 3, Data indicating the pitch is received, and the performance part is specified using the data. The contents of the processing are the same as those in the performance training system 1.

  The image signal generation unit 1015 includes performance part data indicating the performance part specified by the performance part specification unit 1014, event data indicating the pitch of the guide sound delivered from the reproduction unit 1011, and a user delivered from the pitch specification unit 1013. Using the current actual performance pitch data relating to each of the singing sounds 19-1 to 19-3, an image signal indicating which of the other singing sounds is dragged by each of the singing sounds of the users 19-1 to 3 is generated. The generated image signal is transmitted to the display 13.

  FIG. 8 is a diagram exemplifying an image displayed on the display 13 in the performance training system 2. 8, broken lines 231 to 233 indicate the pitches of the guide sounds of the chorus parts 1 to 3, respectively, and curves 234 and 235 indicate the pitches of the singing sounds of the users 19-2 and 3, respectively. In addition, since the user 19-1 has not participated in the present performance, the curve which shows the pitch of the song sound is not shown. Moreover, the markers 236 to 238 and the arrows 239 to 241 are displays showing a state in which the singing sound is influenced by the sounds of other chorus parts.

  As described above, with the performance training system 2, as with the performance training system 1, the user can effectively improve the performance ability while being conscious of not being dragged by the sound of other performance parts. it can. Moreover, according to the performance training system 2, even when a performance is performed simultaneously by a plurality of users, the plurality of users can simultaneously improve their performance capabilities.

It is the figure which showed the structure of the performance training system concerning 1st Embodiment. It is the figure which illustrated the contents of the reference pitch data concerning a 1st embodiment. It is the figure which illustrated the graph produced | generated by the pitch specific | specification part concerning 1st Embodiment. It is the figure which illustrated the contents of the actual performance pitch data concerning a 1st embodiment. It is the figure which illustrated the contents of work pitch data concerning a 1st embodiment. It is the figure which showed the image displayed by the display concerning 1st Embodiment. It is the figure which showed the structure of the performance training system concerning 2nd Embodiment. It is the figure which showed the image displayed by the display concerning 2nd Embodiment.

Explanation of symbols

1... 2, performance training system, 10 20, pitch notification device, 11 speaker, 12 microphone, 13 display, 14 keyboard, 101 control unit, 102 storage unit, 103 input / output interface, 104 Oscillator, 1011... Playback unit, 1012... Sound generator unit, 1013... Pitch specification unit, 1014... Performance part specification unit, 1015. Data, 1024 ... Work pitch data, 1031 ... Sound signal output unit, 1032 ... Sound signal input unit, 1033 ... Image signal output unit

Claims (9)

Storage means for storing reference pitch data indicating time-sequentially the pitch of the sound to be pronounced by one of the plurality of performance parts in a music played by a plurality of performance parts;
Input means for obtaining a sound signal indicating a sound produced by the one performance part;
Pitch specifying means for specifying the pitch of the sound indicated by the sound signal acquired by the input means;
The pitch of the sound indicated by the reference pitch data of the one performance part at one timing, the pitch of the sound of the one performance part specified by the pitch specifying means at the one timing, and the one timing Other performance parts other than the one performance part among the plurality of performance parts in FIG. 5 are affected by the performance of the one performance part at the one timing. A performance part identifying means for identifying a performance part;
A pitch notification device comprising: output means for outputting data indicating the pitch of the performance part specified by the performance part specifying means or the sound produced by the performance part. The reference pitch data stored by the storage means is data indicating in time series the pitch of the sound to be generated by each of the plurality of performance parts.
The performance part specifying means specifies the pitch of the sound indicated by the reference pitch data of each performance part other than the one performance part at one timing when specifying the other performance part. The pitch notification device according to claim 1, wherein the pitch notification device is used as a pitch of each sound of a performance part other than the one performance part. The input means obtains a sound signal indicating a sound produced by each of the plurality of performance parts;
The pitch specifying means specifies the pitch of the sound for each of the sounds produced by the plurality of performance parts indicated by the sound signal acquired by the input means,
When the performance part specifying means specifies the other performance part, the pitch of each sound of the performance parts other than the one performance part specified by the pitch specifying means at one timing is used as the one timing. The pitch notification device according to claim 1, wherein the pitch notification device is used as a pitch of each sound of a performance part other than the one performance part. The storage means stores actual performance pitch data indicating the pitch specified by the pitch specifying means in time series,
The performance part specifying means includes the pitch of the sound of the one performance part specified by the pitch specifying means at one timing during the performance being performed and the one timing during the performance performed in the past. The pitch of the sound indicated by the actual performance pitch data of the one performance part and the performance part that has participated in the current performance and has not participated in the performance performed in the past. The pitch notification device according to claim 1, wherein the other performance part is specified based on a relationship with a pitch of the sound at the one timing during the performance being performed. The storage means stores actual performance pitch data indicating the pitch specified by the pitch specifying means in time series,
The performance part specifying means includes the pitch of the sound of the one performance part specified by the pitch specifying means at one timing during the performance being performed and the one timing during the performance performed in the past. In the past, the pitch of the sound indicated by the actual performance pitch data of the one performance part and the performance part that has not participated in the currently performed performance and participated in the performance performed in the past The pitch notification device according to claim 1, wherein the other performance part is specified based on a relationship with a pitch of the sound at the one timing during the performance performed at the same time. The storage means stores actual performance pitch data indicating the pitch specified by the pitch specifying means in time series,
The performance part specifying means includes a difference between a pitch of a sound indicated by the reference pitch data of the one performance part and a pitch of a sound indicated by the actual performance pitch data of the one performance part, and the one performance The other performance part is specified based on the correlation between the pitch of the sound indicated by the reference pitch data of the part and the difference between the pitches of the sounds of the performance parts other than the one performance part. The pitch notification device according to claim 1. The pitch specifying means is a function value including, as a variable, a difference between a sound pitch candidate indicated by the sound signal acquired by the input means and a sound pitch indicated by the reference pitch data of the one performance part. The pitch notification device according to claim 1, wherein the pitch of the sound produced at the one timing by the one performance part is specified based on the value of the function. The input means is a plurality of input means provided in association with each of the plurality of performance parts,
The function is related to one frequency, the amplitude of the frequency component at the one frequency of the sound signal acquired by the input means corresponding to the one performance part, and the input corresponding to the performance part other than the one performance part. The pitch notification device according to claim 7, wherein a numerical value indicating a difference between the amplitude of the frequency component at the one frequency of the sound signal acquired by at least one of the means is included as a variable. A process of storing reference pitch data indicating time-sequentially the pitch of the sound to be produced by one of the plurality of performance parts in a music played by a plurality of performance parts;
Processing for obtaining a sound signal indicating a sound produced by the one performance part;
A process of identifying the pitch of the sound indicated by the acquired sound signal;
The pitch of the sound indicated by the reference pitch data of the one performance part at one timing, the pitch of the sound of the one performance part specified at the one timing, and the plurality of performance parts at the one timing A process of identifying another performance part that affects the performance of the one performance part at the one timing based on the relationship with the pitch of each sound of the performance parts other than the one performance part When,
A program for causing a computer to execute a process of outputting data indicating a specified performance part or a pitch of a sound produced by the performance part.

Images