JP4865607B2 - Karaoke apparatus, singing evaluation method and program - Google Patents

Description

  The present invention relates to a technique for evaluating a special singing technique in a karaoke apparatus for scoring a song.

Some karaoke apparatuses have a scoring function for displaying the skill of a singer's singing in points. Of these scoring functions, actual singing skill and scoring results correspond as much as possible, and data such as pitch data and volume data extracted from the singer's singing voice signal and karaoke song melody ( Some have a comparison function between the guide melody) and the corresponding data (for example, Patent Document 1).
Japanese Patent Laid-Open No. 10-69216

  The karaoke apparatus provided with such a scoring function makes it possible to compare and score changes in notes for each note, and this scoring function is in the MIDI (Musical Instrument Digital Interface) format. Since it was compared with the singer's singing based on the guide melody that was converted into data, the scoring was based on the notes on the score. Therefore, when such a scoring is performed, the scoring result may differ from the actual skillful impression. For example, if you sing a song that sings with a lower pitch at the end of the sound (hereinafter referred to as the Fall song), you may be able to hear it skillfully. The result of scoring may be lowered due to deviation from the pitch that should be performed.

  This invention is made | formed in view of the above-mentioned situation, and it aims at providing the karaoke apparatus, singing evaluation method, and program which detect and evaluate a fall song.

In order to solve the above-described problems, the present invention provides a playback unit that plays back a song, a voice input unit that generates singer voice data based on a song of a singer input during playback of the song, If there is a pitch extraction means for extracting the pitch of the singer's singing based on the singer's voice data and a period during which the pitch extraction means cannot extract the pitch continuously for a predetermined time or longer, a predetermined time before the period A first specifying means for specifying the pitch extracted by the pitch extracting means at the timing as a first pitch, and a preset setting of the first timing at which the first pitch is extracted by the pitch extracting means second specifying means for specifying a pitch extracted by the pitch extracting means at the second timing corresponding to previous time as the second pitch, the second When the pitch is larger than the first pitch by a predetermined pitch or more, an identification unit that outputs an identification signal and an evaluation unit that performs a predetermined process based on the identification signal output from the identification unit A karaoke apparatus is provided.

Further, the present invention is based on storage means for storing guide melody data indicating a melody to be sung by a singer corresponding to the progress of the music, and a singer's singing input corresponding to the progress of the music. Voice input means for generating singer voice data, pitch extraction means for extracting the pitch of the singer's singing corresponding to the progress of the music, based on the singer voice data, and the pitch extraction means When there is a period during which the pitch cannot be extracted continuously for a predetermined time or more with respect to the progress of the music, the pitch extracted by the pitch extracting means at a predetermined timing before the period is specified as the first pitch. Of the sounds constituting the melody indicated by the guide melody data, the pitch extracting means extracts the first pitch from the sound that constitutes the melody indicated by the guide melody data. A pitch calculation unit that calculates a pitch of a sound corresponding to the timing at which the first pitch is lower than the calculated pitch, and a pitch calculation unit that calculates a pitch that is lower than the calculated pitch as a second pitch; If the pitch the pitch extraction unit and extracted in a preset period of time from the previous corresponding to the second timing to the first timing of the first timing for the progression includes a larger pitch than said second pitch And when the second pitch is larger than the first pitch by a predetermined pitch or more, the identification means for outputting the identification signal and the identification signal output from the identification means are determined in advance. There is provided a karaoke apparatus comprising an evaluation means for performing the processing.

Further, in another preferred embodiment, the identification unit further, when the pitch of the pitch extraction unit and extracted in a predetermined time earlier than the first timing is included in the pitch range of Tokoro constant width is The identification signal may be output.

Further, in another preferred embodiment, the identification unit further, when the pitch extracting means in a predetermined time of the first timing earlier variations in pitch was extracted, have become monotonically decrease, the identification signal May be output.

  Moreover, in another preferable aspect, the predetermined process in the evaluation means measures the number of times that the identification signal is output from the identification means, and evaluates the song of the singer based on the measured number of times. It may be a process of performing.

Further, in another preferred embodiment, in response to the progress of the music further comprising a range storage means for storing a range specifying information indicating a specific range of musical, said identification means further said first timing is, If it is included in the specific range indicated by the range specifying information, an identification signal may be output.

  In another preferred embodiment, setting storage means for storing setting information indicating a setting value of the predetermined time, the predetermined timing, the setting time or the predetermined pitch, and the predetermined time based on the setting information, You may further comprise the setting means which sets the value of the said predetermined timing, the said setting time, or the said predetermined pitch.

Further, the present invention is based on a playback process of playing back a song, a voice input process of generating singer voice data based on a singer's song input during playback of the song, and the singer voice data When there is a pitch extraction process for extracting the pitch of the singer's song and a period during which the pitch cannot be extracted continuously for a predetermined time or more in the pitch extraction process, the pitch extraction is performed at a predetermined timing before the period. first specific process and, said first timing preset time before it falls second timing of said first pitch in the pitch extraction process is extracted to identify the pitch extracted by the process as the first pitch wherein the second specific procedure for identifying the pitch extracted by the pitch extraction process as a second pitch, the second pitch of the first in An identification process for outputting an identification signal and an evaluation process for performing a predetermined process based on the identification signal output by the identification process. A singing evaluation method is provided.

The present invention also provides a voice input process for generating singer voice data based on a singer's singing input corresponding to the progress of the music, and the singer's singing based on the singer voice data. If there is a pitch extraction process for extracting the pitch corresponding to the progress of the music and a period during which the pitch cannot be extracted continuously for a predetermined time or more with respect to the progress of the music in the pitch extraction process, A first identification process that identifies the pitch extracted by the pitch extraction process at a predetermined timing as a first pitch, and a melody that the singer should sing in response to the progress of the music stored in the storage means Among the sounds constituting the melody indicated by the indicated guide melody data, the timing for the progression of the music in which the first pitch is extracted in the pitch extraction process. Calculating a pitch of the response to the sound, the pitch calculation step of calculating a predetermined amount lower pitch than the pitch the calculated as the second pitch, the second on the progression of the music extracted the first pitch in the pitch extraction process a pitch the pitch extraction process is extracted from the second timing before falls preset time in the period until the first timing of the first timing, when contains greater pitch than the second pitch, and, When the second pitch is larger than the first pitch by a predetermined pitch or more, an identification process for outputting an identification signal and a predetermined process based on the identification signal output by the identification process are performed. A singing evaluation method characterized by comprising an evaluation process to be performed.

The present invention also provides a computer with a playback function for playing back music, a voice input function for generating singer voice data based on a song of a singer input during playback of the music, and the singer voice Based on the data, if there is a pitch extraction function for extracting the pitch of the singer's singing and a period in which the pitch cannot be extracted continuously for a predetermined time or more in the pitch extraction function, at a predetermined timing before the period hits before the first and the specific function of specifying a pitch extracted in the pitch extracting function as a first pitch, said first timing a preset time of the said first pitch in the pitch extraction function is extracted a second specifying function for specifying a more extracted pitch to the pitch extraction function in the second time as the second pitch, the second When the pitch is larger than the first pitch by a predetermined pitch or more, an identification function for outputting an identification signal and an evaluation function for performing a predetermined process based on the identification signal output by the identification function A program for realizing the above is provided.

Further, the present invention provides a storage function for storing guide melody data indicating a melody to be sung by a singer in accordance with the progress of the music, and a singer's singing input corresponding to the progress of the music. A voice input function for generating singer voice data based on the singer voice data, a pitch extraction function for extracting the singer's singing pitch corresponding to the progress of the music, and the pitch extraction function based on the singer voice data When there is a period during which the pitch cannot be extracted continuously for a predetermined time or more with respect to the progress of the music, the pitch extracted by the pitch extraction function at a predetermined timing before the period is specified as the first pitch Among the sounds constituting the melody indicated by the first specific function and the guide melody data, the pitch extraction function extracts the first pitch. A pitch calculation function for calculating a pitch of a sound corresponding to the timing with respect to the progress of the music, a pitch lower than the calculated pitch by a predetermined amount as a second pitch, and the first pitch in the pitch extraction function. the extracted pitch the pitch extraction is extracted in the period from the second timing falls before a preset time of the first timing until the first timing on the progression of the song, it included larger pitch than said second pitch And when the second pitch is larger than the first pitch by a predetermined pitch or more, based on the identification function for outputting the identification signal and the identification signal output by the identification function A program for realizing an evaluation function for performing a predetermined process is provided.

ADVANTAGE OF THE INVENTION According to this invention, the karaoke apparatus which detects and evaluates a fall song, the song evaluation method, and a program can be provided.

  Hereinafter, an embodiment of the present invention will be described.

<Embodiment>
In the present embodiment, a karaoke apparatus 1 capable of evaluating a fall song will be described. First, the hardware configuration of the karaoke apparatus 1 will be described with reference to FIG. FIG. 1 is a block diagram showing a hardware configuration of a karaoke apparatus 1 according to the first embodiment of the present invention.

  A CPU (Central Processing Unit) 11 reads out a program stored in a ROM (Read Only Memory) 12, loads it into a RAM (Random Access Memory) 13, and executes it, thereby executing a bus for each part of the karaoke apparatus 1. 10 to control. The RAM 13 functions as a work area when the CPU 11 performs data processing.

  The storage unit 14 is, for example, a large-capacity storage unit such as a hard disk, and includes a music data storage area 14a and a singer voice data storage area 14b. The song data storage area 14a stores a plurality of song data of karaoke songs, and each song data has a guide melody track, an accompaniment data track, and a lyrics data track.

  The guide melody track is data that indicates the melody of the vocal part of the music, and reads and executes event data such as note-on that indicates a sound generation command, note-off that indicates a mute command, and control change, and the next event data. Delta time data indicating the time until. With this delta time, the time of event data to be executed can be associated with the passage of time since the progression of music began. Note on and note off each have a note number indicating the pitch of the sound to be sounded and muted. Thereby, each sound which comprises the melody of the vocal part of a music can be prescribed | regulated by note-on, note-off, and delta time. The accompaniment data track is composed of a plurality of tracks of each accompaniment instrument, and each instrument track has the same data structure as the above-described guide melody track. In the case of the present embodiment, MIDI format data is stored.

  The lyric data track changes the color of text data indicating the lyrics of music, display timing data indicating the timing of displaying lyrics telop on the display unit 15 to be described later according to the progress of the music, Wipe timing data indicating timing for performing wipe. Then, the CPU 11 reproduces the music data stored in the music data storage area 14a, outputs the audio data generated based on the accompaniment data track of the music data to the audio processing unit 18 described later, and outputs it to the lyrics data track. Based on this, a lyrics telop is displayed on the display unit 15.

  In the singer voice data storage area 14b, voice data (hereinafter referred to as singer voice data) A / D converted from the microphone 17 (to be described later) via the voice processing unit 18 is time-sequentially in, for example, the WAVE format or the MP3 format. Is remembered. By storing in chronological order in this way, it is possible to associate the time elapsed since the progression of the music started with each frame having a predetermined time length of the singer's voice data.

  The display unit 15 is a display device such as a liquid crystal display. The display unit 15 is controlled by the CPU 11, and based on the lyrics data track stored in the music data storage area 14a of the storage unit 14, a background image or the like according to the progress of the music. A lyrics telop is also displayed. In addition, various screens such as a menu screen for operating the karaoke apparatus 1 and a singing evaluation result screen are displayed. The operation unit 16 is, for example, a keyboard, a mouse, a remote controller, and the like. When the user of the karaoke apparatus 1 operates the operation unit 16, data representing the operation content is output to the CPU 11.

  The microphone 17 picks up a singer's song. The voice processing unit 18 performs A / D conversion on the voice collected by the microphone 17 to generate singer voice data. The singer voice data is stored in the singer voice data storage area 14b of the storage unit 14 as described above. The audio processing unit 18 D / A converts the audio data input by the CPU 11 and emits the sound from the speaker 19.

  Next, functions realized by the CPU 11 executing programs stored in the ROM 12 will be described. FIG. 2 is a block diagram showing a software configuration showing the functions realized by the CPU 11.

  The pitch extraction unit 101 reads out the singer's voice data stored in the singer's voice data storage area 14b, and extracts the pitch of the singer's voice data related to the singer's voice data in units of frames having a predetermined time length. Then, the singing pitch data indicating the singing pitch extracted in units of frames is output to the normal evaluation unit 103 and the fall singing evaluation unit 105. The pitch may be extracted from a spectrum generated by FFT (Fast Fourier Transform) or may be extracted by other known methods. Here, the pitch of the frame unit extracted by the pitch extraction unit 101 is expressed as Pitch (f), and f is the number of frames counted from the first frame. For example, Pitch (1) is the pitch extracted from the first frame, and Pitch (2), Pitch (3),. If the pitch cannot be extracted, such as when there is no song related to the singer's voice data, Pitch (f) = 0 is set.

  The pitch calculation unit 102 reads the guide melody track of the music to be evaluated from the music data storage area 14a, and recognizes the melody of the music from the read guide melody track. In addition, for each sound constituting the recognized melody, a pitch is calculated in units of frames having a predetermined time length. Then, the melody pitch data indicating the pitch of the guide melody calculated for each frame is output to the normal evaluation unit 103. In addition, since the pitch of each sound which comprises a melody is prescribed | regulated by the note number, a pitch will be determined corresponding to a note number. For example, when the note number is 69 (A4), the pitch is 440 Hz. At this time, if a table that associates the note number with the pitch is stored in the storage unit 14, the pitch calculation unit 102 may calculate the pitch with reference to the table.

  The normal evaluation unit 103 compares the singing pitch data output from the pitch extraction unit 101 and the melody pitch data output from the pitch calculation unit 102 in units of frames, and generates normal evaluation data indicating the degree of pitch matching. , Output to the scoring unit 104. Here, the degree of coincidence may be calculated from the difference between the pitch of the sound that composes the melody and the pitch of the singing in each frame, or the pitch of the sound that composes the melody and the pitch of the singing substantially match. That is, it may be calculated from a time ratio within a predetermined pitch range with respect to the pitch of the sound constituting the melody. Note that the normal evaluation unit 103 may not only evaluate the singing pitch but also evaluate the sound volume and other feature quantities. In this case, an extraction unit that extracts each necessary feature amount from the singing is provided, and a feature amount serving as a reference for evaluation may be stored in the storage unit 14.

  The fall song evaluation unit 105 includes a detection unit 1051, a measurement unit 1052, an accumulation unit 1053, and an evaluation unit 1054. The fall song evaluation unit 105 indicates the fall song evaluation result based on the song pitch data output from the pitch extraction unit 101. Singing evaluation data is generated and output to the scoring unit 104. Hereinafter, each part which comprises the fall song evaluation part 105 is demonstrated.

  The detecting unit 1051 buffers a predetermined amount of the singing pitch data output from the pitch extracting unit 101, analyzes the singing pitch data in units of frames, and determines determination information indicating whether or not the singing pitch is extracted for each frame. The data is output to the measurement unit 1052. When the comparison is completed for all the frames, the end information indicating the completion and the information indicating the total number Tall of the compared frames are output to the evaluation unit 1054.

  The measuring unit 1052 is the number of frames in which the pitch related to the singing pitch data has not been continuously extracted based on the determination information output from the detecting unit 1051, that is, Pitch (f) has been continuously zero. The number of Tcounts is measured, and when a predetermined condition is satisfied, an identification signal indicating that the condition is satisfied is output to the accumulation unit 1053. Here, the predetermined conditions are the following two conditions. First, when Tcount reaches a predetermined number m, that is, the number of frames in which the pitch related to the singing pitch data is not continuously extracted in the detection unit 1051 reaches the predetermined number m. Second, Pitch (α) −Pitch (β) is larger than a preset setting value k (for example, 300 cent). Here, β indicates a frame that the detection unit 1051 has finally determined that Pitch (f) ≠ 0. In addition, α indicates a frame that is a predetermined number of frames before β from β. The measurement unit 1052 acquires Pitch (α) and Pitch (β) from the singing pitch data buffered by the detection unit 1051. On the other hand, when the determination information indicating that the pitch related to the singing pitch data is extracted from the detection unit 1051, the count of Tcount is reset to “0”. The predetermined number m indicates the number of frames and can be converted into time since the time of one frame is determined. The predetermined number m may be changed by operating the operation unit 16. Similarly, the number of frames n and the set value k may be changed by operating the operation unit 16.

  The accumulation unit 1053 measures the number of identification signals output from the measurement unit 1052. The measured value is referred to as Ttotal. In a case where the end information and the information indicating the total number of frames Tall are output from the detection unit 1051, the evaluation unit 1054 reads Ttotal that is a value accumulated by the accumulation unit 1053 from the accumulation unit 1053, and the Ttotal Fall singing evaluation data indicating the ratio to the total number of frames Tall is output to the scoring unit 104. In addition, if fall song evaluation data are produced | generated based on Ttotal, it will not be restricted to what shows Ttotal / Tall. For example, it may indicate Ttotal itself.

  The scoring unit 104 calculates the evaluation score of the singer's song based on the normal evaluation data output from the normal evaluation unit 103 and the Fall song evaluation data output from the Fall song evaluation unit 105. The calculated evaluation score is displayed on the display unit 15 by the CPU 11.

  Next, the operation of the karaoke apparatus 1 will be described. First, the singer operates the operation unit 16 to select a song to be sung. The CPU 11 reads the music data corresponding to the music selected by the singer from the music data storage area 14a, and releases the music accompaniment from the speaker 19 based on the accompaniment data track of the read music data as the music progresses. The lyric is wiped on the display unit 15 based on the lyric data track of the read music data. When the singer sings along with the progress of the music, the singing is picked up by the microphone 17 and stored in the singer voice data storage area 14b as singer voice data.

  When the music is finished by proceeding to the end, the CPU 11 starts to evaluate the singer's singing. The pitch extraction unit 101 reads the singer voice data stored in the singer voice data storage area 14 b and outputs the singing pitch data to the normal evaluation unit 103 and the detection unit 1051 of the fall singing evaluation unit 105. The pitch calculation unit 102 reads the guide melody track of the music serving as the evaluation reference from the music data storage area 14 a and outputs the melody pitch data to the normal evaluation unit 103.

  The normal evaluation unit 103 compares the singing pitch data output from the pitch extraction unit 101 and the melody pitch data output from the pitch calculation unit 102 in units of frames, and generates normal evaluation data indicating the degree of pitch matching. , Output to the scoring unit 104. The fall song evaluation unit 105 generates fall song evaluation data indicating the evaluation result based on the song pitch data output from the pitch extraction unit 101, and outputs the fall song evaluation data to the scoring unit 104.

  Here, the flow of evaluation of the fall singing evaluation unit 105 will be described in detail with reference to FIG. FIG. 3 is a flowchart showing a flow of evaluation performed by the fall singing evaluation unit 105.

  First, when the evaluation in the fall singing evaluation unit 105 is started, the parameters Tall, Ttotal, and Tcount used in the fall singing evaluation unit 105 are initialized to all “0” (step S1).

  Next, the detection unit 1051 increments the value of Tall indicating the number of compared frames by “1” and counts up (step S2), and the first pitch among the pitches related to the singing pitch data output from the pitch extraction unit 101 is counted. It is determined by detecting whether or not the pitch of the frame has been extracted, that is, whether Pitch (Tall) ≠ 0 (Tall = 1 at this time) (step S3).

  When it is determined that the pitch of the first frame among the pitches related to the singing pitch data is extracted (Pitch (1) ≠ 0) (step S3; Yes), the detecting unit 1051 extracts the pitch. The determination information indicating that is output to the measurement unit 1052. Then, the measurement unit 1052 resets the count of Tcount to “0” (step S4). And it progresses to step S9 mentioned later.

  On the other hand, if it is determined that the pitch related to the singing pitch data has not been extracted (Pitch (1) = 0) (step S3; No), the detection unit 1051 has determined that the pitch has not been extracted. The determination information shown is output to the measurement unit 1052. The measuring unit 1052 then increments Tcount by “1” and counts up (step S5), and determines whether or not Tcount has reached a predetermined number m (step S6). Hereinafter, the description is not limited to the first frame (Tall = 1).

  When Tcount = m is not satisfied, that is, when Tcount is smaller or larger than the predetermined number m (step S6; No), the process proceeds to step S9 described later. On the other hand, if Tcount = m (step S6; Yes), it is determined whether Pitch (α) −Pitch (β) is larger than a preset set value k (step S7). Here, α = Tall-m and β = Tall-mn.

  When Pitch (α) −Pitch (β) is equal to or less than the set value k (step S7; No), the process proceeds to step S9 described later. On the other hand, when Pitch (α) −Pitch (β) is larger than the set value k (step S7; Yes), the measurement unit 1052 outputs an identification signal to the accumulation unit 1053. Then, the accumulating unit 1053 increases Ttotal by “1” and counts up (step S8). And it progresses to step S9 mentioned later.

  When the processing in steps S4, S6, S7, and S8 ends, the detection unit 1051 determines whether or not the music has ended based on the singing pitch data output from the pitch extraction unit 101 (step S9). If it is determined that the music has not ended (step S9; No), the above-described processing is repeated from step S2. On the other hand, when it is determined that the music has ended (step S9; Yes), the detection unit 1051 outputs the end information and information indicating Tall to the evaluation unit 1054. When the end information is output from the detection unit 1051, the evaluation unit 1054 reads Ttotal that is a result of accumulation every time an identification signal is output in the accumulation unit 1053. And the evaluation part 1054 outputs the fall song evaluation data which shows Ttotal / Tall to the scoring part 104 (step S10). In addition, as above-mentioned, if fall song evaluation data are produced | generated based on Ttotal, it will not be restricted to what shows Ttotal / Tall. For example, it may indicate Ttotal itself.

  Here, as a specific example of the processing in the fall singing evaluation unit 105, a case where the music progresses as shown in FIG. 4 will be described. FIG. 4 is an explanatory diagram for explaining singing pitch data, α and β. Here, the vertical axis indicates the pitch height, the horizontal axis indicates the time, and the solid line indicates the pitch indicated by the singing pitch data. Further, k described in the figure indicates a preset set value k. M and n are a predetermined number m and a preset number of frames n, respectively, which are expressed by converting the number of frames into time. In addition, the content regarding the frame described above will be described as being converted into time in the following description. Moreover, Ttotal before evaluating the said part shall be (gamma).

  First, after the time t1, the pitch indicated by the singing pitch data becomes 0, and the measuring unit 1052 counts up Tcount. When Tcount = m, that is, when the time becomes t1 + m, the measurement unit 1052 acquires the pitch Pitch (t1) at t1 corresponding to β and the pitch Pitch (t1-n) at t1-n corresponding to α. Here, since P1 = Pitch (t1-n) −Pitch (t1) is smaller than the set value k, the measurement unit 1052 does not output an identification signal. At time t2, since the pitch indicated by the singing pitch data is not 0, the count of Tcount is reset and Tcount = 0.

  Next, after the time t3, the pitch indicated by the singing pitch data becomes 0 again, and the measuring unit 1052 counts up Tcount. Here, at time t4 before Tcount = m, the pitch indicated by the singing pitch data is no longer 0, and Tcount = 0 again. Then, after the time t5, the pitch indicated by the singing pitch data becomes 0 again, and the measuring unit 1052 counts up Tcount. When the time t5 + m is reached, the pitch Pitch (t5) at t5 corresponding to β described above and The pitch Pitch (t5-n) at t5-n corresponding to α is acquired. Here, since P5 = Pitch (t5-n) −Pitch (t5) is larger than the set value k, the measurement unit 1052 outputs an identification signal to the accumulation unit 1053. As a result, the accumulation unit 1053 counts up Ttotal, and Ttotal = γ + 1. At time t6, since the pitch indicated by the singing pitch data is not 0, the count of Tcount is reset and Tcount = 0. Thus, in the fall singing evaluation unit 105 in the portion given as an example, Ttotal increases from γ to γ + 1.

  In this way, the processing in the fall singing evaluation unit 105 is performed for all frames. Then, the evaluation unit 1054 outputs fall song evaluation data indicating Ttotal / Tall to the scoring unit 104.

  Then, the scoring unit 104 determines the evaluation score of the singer's song by a predetermined algorithm based on the normal evaluation data output from the normal evaluation unit 103 and the Fall song evaluation data output from the Fall song evaluation unit 105. calculate. Then, the calculation result is displayed on the display unit 15.

  As described above, the karaoke apparatus 1 according to the present embodiment, when the pitch cannot be extracted due to silence for a predetermined time such as the end of a singing phrase, and the pitch Pitch (β) extracted by the pitch extraction unit 101 at the end, Further, it is possible to recognize the pitch Pitch (α) that is the previous number n of frames. When Pitch (α) −Pitch (β) is larger than the set value k, the pitch has fallen larger than the set value k within the time corresponding to the number of frames n, and a fall singing state is assumed. Since it can be said, the influence of the Fall singing can be added to the scoring result by the evaluation of the singing of the singer.

  As mentioned above, although embodiment of this invention was described, this invention can be implemented in various aspects as follows.

<Modification 1>
Although the fall song evaluation in the fall song evaluation unit 105 in the embodiment is performed using the song pitch data output from the pitch extraction unit 101, the fall song evaluation unit 105 further uses the melody pitch data output from the pitch calculation unit 102 to Singing may be evaluated. In this case, the following may be performed. As shown in FIG. 5, the pitch calculation unit 102 outputs the melody pitch data to the normal evaluation unit 103 and also outputs it to the measurement unit 1052 of the fall song evaluation unit 105. And fall song evaluation part 105 performs processing as shown in FIG. The flowchart shown in FIG. 6 is the process of steps S71 and S72 instead of the process of step S7 in the flowchart shown in FIG. Hereinafter, the process of steps S71 and S72 in the measurement unit 1052 will be described.

  If Tcount = m (step S6; Yes), it is determined whether Pitch2 (α) −Pitch (β) is larger than a preset set value k (step S71). Here, Pitch2 (f) is a pitch corresponding to the pitch indicated by the melody pitch data in the frame number f. Here, “corresponding to Pitch2 (f)” means that the pitch is slightly lower than the pitch indicated by the melody pitch data in the frame number f (for example, −10 cent). Note that the pitch indicated by the melody pitch data is obtained by calculating the pitch of each sound indicated by the guide melody track and does not change within the same sound. There is no problem even if the pitch is slightly shifted in the frame. In other words, any method may be used as long as the sound corresponding to the number of frames α can be recognized, and the pitch of the sound may be regarded as Pitch2 (α).

  If Pitch2 (α) −Pitch (β) is smaller than the preset setting value k (step S71; No), the process proceeds to step S9. On the other hand, when Pitch2 (α) −Pitch (β) is larger than a preset set value k (step S71; Yes), among the pitches related to the singing pitch data, the pitch2 (α) It is determined whether or not there is a pitch (Pitch (h) = Pitch2 (α), where α <h <β) corresponding to α) (step S72). Here, since Pitch2 (α) is a pitch slightly lower than the pitch indicated by the melody pitch data in the frame number α, when the singing is performed at a pitch slightly lower than the pitch of the sound indicated by the guide melody track, the fall is performed. Even if it is in the state of singing, it can prevent that the said conditions are no longer satisfied.

  If the above condition is not satisfied (step S72; No), the process proceeds to step S9. On the other hand, if the above condition is satisfied (step S72; Yes), the process proceeds to step S8. Here, FIG. 7A shows a case where the condition in step S72 is satisfied with respect to the pitch related to the melody pitch data and the pitch related to the singing pitch data in a part of the progression of the music, and FIG. ). In FIG. 7 (a), h is located between α and β. However, in FIG. 7 (b), since the singing pitch declines slowly, h is located before α. It is not recognized as a fall song. Even in the above configuration, the fall song can be evaluated in the same manner as in the embodiment.

<Modification 2>
In the embodiment, it has been recognized and evaluated that the state of fall singing has been reached by determining that the pitch has decreased by the set value k within the time corresponding to the number of frames n. On the other hand, the fall song may be recognized in a specific period. In this case, event data of information indicating that a fall song should be recognized may be provided at a data position indicating a specific period so as to be included in the music data. When the CPU 11 identifies the above information when reproducing the music data, the information is output to the measuring unit 1052 or the like so that the measuring unit 1052 recognizes the specific period indicated by the event data. For example, since the identification signal can be prevented from being output outside the specific period, it is not recognized as a fall song. Note that data indicating a specific period may be separately stored in the storage unit 14 and may be recognized by the CPU 11 reading it.

  Another method is to analyze the guide melody track and provide analysis means to determine a specific period based on the relationship between adjacent pitches in each sound that composes the melody, sound length, silent time, etc. The unit 1052 may be made to recognize. For example, when comparing the pitch of one sound that composes a melody with the pitch of the next sound, if the pitch of the next sound is more than a certain amount away, the last part of the certain sound is specified. The period of time may be used. Similarly, when the sound length of a certain sound is longer than a certain time, or when the silent time until a next sound is longer than a certain time, the last part of the certain sound may be set as a specific period. If it does in this way, since a fall song comes to be recognized in a specific period, it can also be made not to be an object of evaluation in a portion where fall song is unnecessary.

<Modification 3>
In the embodiment, it was recognized and evaluated that the state of fall singing was reached by judging that the pitch was lowered by the set value k within the time corresponding to the number of frames n, but further conditions were added. When the measurement unit 1052 analyzes the singing pitch data buffered by the detection unit 1051 and determines that the condition is satisfied, the measurement unit 1052 may output an identification signal. Here, the conditions may be set as follows. For example, as a condition, if the pitch of a certain number of frames corresponding to the part before the frame number α is within a certain pitch range, the condition is limited to a state where a fall singing state is obtained from a stable pitch state. Can evaluate the fall song. At this time, the fixed pitch range may be determined by referring to the melody pitch data as in Modification 1, and a range of 20 cents in the vertical direction, for example, centering on Pitch2 (α) may be used. Further, as another condition, pitch (β) is not included in a certain range of pitch with respect to the pitch of a sound next to a certain sound constituting the melody corresponding to the above Pitch 2 (α). . In this way, it is possible to distinguish between the singing (portamento) and the fall singing that are continuously changed to the pitch of the next sound.

  Moreover, it is good also as conditions on the condition that the vibrato detection means which detects the song (vibrato) which shakes a pitch from song pitch data is provided, and a vibrato detection means does not detect vibrato. That is, when vibrato is detected, the measurement unit 1052 may not output an identification signal. Since vibrato fluctuates up and down, it may be recognized as a fall song when the pitch fluctuates downward, but this can be prevented. As described above, the measurement unit 1052 can evaluate the fall singing with higher accuracy by outputting the identification signal when each set condition is satisfied.

<Modification 4>
In the embodiment, parameters such as the set value k, the count number m, and the frame number n are set in advance. However, these parameters may be changed during the progress of the music. In this case, information indicating the parameter value may be included in the music data by event data or the like. Then, when the CPU 11 identifies the above information when reproducing the music data, setting means for outputting the information to the detection unit 1051, the measurement unit 1052, and the like, respectively, may be provided. If it does in this way, fall song can be evaluated in a different mode to progress of music. Moreover, it is also possible to control the portion that is not subject to evaluation even in the fall singing state in the music according to this setting mode (for example, increasing the set value k, the count number m, etc.) The effects described in the second modification can also be obtained. Note that data indicating parameter values may be separately stored in the storage unit 14 and recognized by the CPU 11 reading the data, and setting means for setting values for each parameter may be provided. In addition, which mode is used among the mode that is fixedly used with each preset parameter as in the embodiment, or the mode that is used with each parameter set based on music data as in this modification The user may be able to select by operating the operation unit 16. Each parameter may be changed according to the level of the singer input by the user operating the operation unit 16.

<Modification 5>
In the embodiment, since β = Tall−m, Pitch (β) indicates the pitch of the frame immediately before the pitch extraction unit 101 cannot extract the pitch from the singer voice data. May be the pitch. In this case, β = Tall−m−10 and a pitch Pitch (β) of a certain number of frames before (in this case, 10 frames before) may be used, or frames from Tall−m−10 to Tall−m A pitch determined based on the pitch at, such as an average value, may be regarded as Pitch (β). In this way, since Pitch (β) is a pitch that is actually heard compared to the last pitch that can be extracted, the fall singing can be evaluated more accurately. It is desirable that the fixed number of frames is smaller than the preset number of frames n. In addition, since the interval between α and β decreases when the number of fixed frames is increased, the measurement unit 1052 may set the value of the set value k so as to decrease according to the interval. The number of frames n may be set so that the interval between α and β does not change without being changed.

<Modification 6>
In the embodiment, if Pitch (α) is larger than Pitch (β) than the set value k, it is determined that the state is in the fall singing state, but the pitch of the intermediate frame (between α and β) is determined. A condition may be provided for the mode of change, and the measurement unit 1052 may output an identification signal when the condition is satisfied. In this case, the measurement unit 1052 recognizes a change in pitch between the frame numbers α and β related to the singing pitch data buffered by the detection unit 1051. And what is necessary is just to judge whether the aspect of the change satisfy | fills a predetermined condition. Here, the predetermined condition may be, for example, that the pitch monotonously decreases from the number of frames α to β (for example, the first derivative is negative). If it does in this way, the state of the singing that the pitch goes up and goes down in the middle can be removed, and the fall singing with higher accuracy can be evaluated.

<Modification 7>
In the embodiment, the processing by the fall singing evaluation unit 105 is performed after the music sung by the singer is finished, but the processing may be sequentially performed during the singing. In this case, the pitch extraction unit 101 sequentially extracts the singing pitch from the singer's voice data, which is the data of the already sung portion, as the music progresses, and the singing pitch data is sent to the fall singing evaluation unit 105. Just output it. And the fall song evaluation part 105 should just process sequentially according to the song pitch data sequentially output from the pitch extraction part 101. FIG. In this way, since the process is completed in a short time after the music is completed, the evaluation result can be quickly displayed on the display unit 15. Further, when the timing at which the measurement unit 1052 outputs Tcount, that is, when a fall song is detected, the CPU 11 can also display on the display unit 15 that the detection has been performed.

It is a block diagram which shows the structure of the hardware of the karaoke apparatus which concerns on embodiment. It is a block diagram which shows the structure of the software of the karaoke apparatus which concerns on embodiment. It is a flowchart which shows the flow of evaluation of the fall song in the fall song evaluation part which concerns on embodiment. It is explanatory drawing which shows a part of evaluation of the fall song in the fall song evaluation part which concerns on embodiment. It is a block diagram which shows the structure of the software of the karaoke apparatus which concerns on the modification 1. It is a flowchart which shows the flow of evaluation of the fall song in the fall song evaluation part which concerns on the modification 1. FIG. It is explanatory drawing which shows a part of fall song evaluation in the fall song evaluation part which concerns on the modification 1. FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Karaoke apparatus, 10 ... Bus, 11 ... CPU, 12 ... ROM, 13 ... RAM, 14 ... Storage part, 14a ... Music data storage area, 14b ... Singer voice data storage area, 15 ... Display part, 16 ... Operation , 17 ... Microphone, 18 ... Audio processing part, 19 ... Speaker, 101 ... Pitch extraction part, 102 ... Pitch calculation part, 103 ... Normal evaluation part, 104 ... Scoring part, 105 ... Fall song evaluation part, 1051 ... Detection part , 1052 ... Measurement unit, 1053 ... Accumulation unit, 1054 ... Evaluation unit

Claims (11)

Playback means for playing back music;
Voice input means for generating singer voice data based on a singer's singing input during playback of the music;
Based on the singer voice data, pitch extraction means for extracting the singer's singing pitch;
When there is a period during which the pitch extraction means cannot extract the pitch continuously for a predetermined time or more, the first pitch that is extracted by the pitch extraction means at a predetermined timing before the period is specified as the first pitch. Specific means,
Second identifying a pitch extracted by the pitch extracting means at the second timing corresponding to before a preset time of the first timing at which the first pitch is extracted as a second pitch by the pitch extracting means Specific means,
An identification means for outputting an identification signal when the second pitch is larger than the first pitch by a predetermined pitch or more;
A karaoke apparatus comprising: an evaluation unit that performs a predetermined process based on an identification signal output from the identification unit. Storage means for storing guide melody data indicating a melody to be sung by a singer in accordance with the progress of the music;
Voice input means for generating singer voice data based on a singer's singing input corresponding to the progress of the music;
Based on the singer voice data, pitch extraction means for extracting the pitch of the singer's singing corresponding to the progress of the music;
If there is a period during which the pitch extraction means cannot extract the pitch continuously for a predetermined time or more with respect to the progress of the music, the pitch extracted by the pitch extraction means at a predetermined timing before the period is set to the first First specifying means for specifying as a pitch;
Of the sounds constituting the melody indicated by the guide melody data, the pitch extraction means calculates the pitch of the sound corresponding to the timing of the progression of the music from which the first pitch has been extracted, and based on the calculated pitch A pitch calculating means for calculating a fixed low pitch as the second pitch;
The pitch extracted by the pitch extractor during a period from a second timing that is a preset set time before the first timing with respect to the progress of the music from which the pitch extractor has extracted the first pitch to the first timing. And when the second pitch is larger than the first pitch, and when the second pitch is larger than the first pitch by a predetermined pitch or more, an identification unit that outputs an identification signal;
A karaoke apparatus comprising: an evaluation unit that performs a predetermined process based on an identification signal output from the identification unit. Said identification means further that the pitch the pitch extraction unit and extracted in a predetermined time earlier than the first timing is when included in the pitch range of Tokoro constant width is for outputting an identification signal The karaoke apparatus according to claim 1 or 2, characterized by the above. It said identification means further variation of pitch the pitch extracting means has extracted a given interval of the first timing previously, if it has been monotonically decrease, and outputs an identification signal The karaoke apparatus in any one of Claims 1 thru | or 3. The predetermined process in the evaluation unit is a process of measuring the number of times that the identification signal is output from the identification unit, and evaluating the singing of the singer based on the measured number of times. The karaoke apparatus according to any one of claims 1 to 4. Further comprising range storage means for storing range specifying information indicating a specific range of the music corresponding to the progress of the music,
It said identification means further wherein the first timing is when included in a specific range the range specifying information indicates the of claims 1 to 5, characterized in that outputs the identification signal Karaoke apparatus in any one. Setting storage means for storing setting information indicating a setting value of the predetermined time, the predetermined timing, the setting time or the predetermined pitch;
7. The apparatus according to claim 1, further comprising a setting unit configured to set a value of the predetermined time, the predetermined timing, the set time, or the predetermined pitch based on the setting information. Karaoke apparatus as described in 1. A playback process for playing music,
A voice input process for generating singer voice data based on a singer's singing input during playback of the music;
Based on the singer voice data, a pitch extraction process of extracting the singer's singing pitch;
If there is a period during which the pitch cannot be extracted continuously for a predetermined time or more in the pitch extraction process, the first pitch that identifies the pitch extracted by the pitch extraction process at a predetermined timing before the period as the first pitch Specific process,
Second identifying a pitch extracted by the pitch extraction process at the second timing corresponding to before a preset time of the first timing at which the first pitch is extracted at the pitch extraction process as a second pitch Specific process,
An identification process for outputting an identification signal when the second pitch is larger than the first pitch by a predetermined pitch or more;
A singing evaluation method comprising: an evaluation process for performing a predetermined process based on the identification signal output in the identification process. A voice input process for generating singer voice data based on a singer's singing input corresponding to the progress of the music;
Based on the singer voice data, a pitch extraction process of extracting the pitch of the singer's singing corresponding to the progress of the music;
If there is a period of time can not be extracted pitch continuously over a predetermined time with respect to the progression of the music piece in the pitch extraction process, the pitch extracted by the pitch extraction process at a predetermined timing before the period first A first identifying process identified as a pitch;
The first pitch is extracted in the pitch extraction process among the sounds constituting the melody indicated by the guide melody data indicating the melody that the singer should sing corresponding to the progress of the music stored in the storage means. Calculating a pitch of a sound corresponding to the timing with respect to the progress of the music, and calculating a pitch lower than the calculated pitch by a predetermined amount as a second pitch;
The pitch extracted by the pitch extraction process in the period from the second timing before the first set time to the first timing with respect to the progression of the music from which the first pitch was extracted in the pitch extraction process. And when a pitch larger than the second pitch is included, and when the second pitch is larger than the first pitch by a predetermined pitch or more, an identification process of outputting an identification signal;
A singing evaluation method comprising: an evaluation process for performing a predetermined process based on the identification signal output in the identification process. On the computer,
A playback function to play music,
A voice input function for generating singer voice data based on a singer's singing input during playback of the music;
Based on the singer voice data, a pitch extraction function for extracting the pitch of the singer's song;
When there is a period during which the pitch cannot be extracted continuously for a predetermined time or longer in the pitch extraction function, the pitch extracted by the pitch extraction function at a predetermined timing before the period is specified as the first pitch. Specific functions,
Second identifying a more extracted pitch to the pitch extraction function in the second timing falls before a preset time of the first timing at which the first pitch is extracted at the pitch extraction function as a second pitch Specific features of
An identification function for outputting an identification signal when the second pitch is larger than the first pitch by a predetermined pitch or more;
A program for realizing an evaluation function for performing a predetermined process based on an identification signal output by the identification function. On the computer,
A memory function for storing guide melody data indicating a melody that a singer should sing in response to the progress of the music;
A voice input function for generating singer voice data based on a singer's singing input corresponding to the progress of the music;
Based on the singer voice data, a pitch extraction function for extracting the pitch of the singer's singing corresponding to the progress of the music;
If there is a period during which the pitch cannot be extracted continuously for a predetermined time or more with respect to the progress of the music in the pitch extraction function, the pitch extracted by the pitch extraction function at a predetermined timing before the period is set to the first A first specific function for specifying as a pitch;
Of the sounds constituting the melody indicated by the guide melody data, the pitch of the sound corresponding to the timing of the progression of the music from which the first pitch has been extracted by the pitch extraction function is calculated. A pitch calculation function for calculating a fixed low pitch as the second pitch;
A pitch the pitch extraction is extracted from the second timing before falls preset time in the period until the first timing of the first timing on the progression of the music extracted the first pitch in the pitch extraction function An identification function for outputting an identification signal when a pitch larger than the second pitch is included, and when the second pitch is larger than the first pitch by a predetermined pitch,
The program for implement | achieving the evaluation function which performs a predetermined process based on the identification signal output by the said identification function.

Images