JP2010217382A - Following performance evaluation system, karaoke system, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To determine how much it can sing according to tempo. <P>SOLUTION: Each of voicing time vlen[k] of each composition sound k is specified by collating a singing voice and a model voice (s180), and a non-reproducibility nvlen[k] is specified by contrasting the voicing time with corresponding model voicing time clen[k] (s190). A higher evaluation value is outputted as following performance of singing to an object musical piece when a periodicity included in a series of the non-reproducibility nvlen[k] is lower (s210). <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a follow-up evaluation system for evaluating the follow-up performance of a song with respect to the target song for a singing voice when a user sings the target song.

  In recent years, a technique has been proposed in which, based on the tendency of pitch change extracted from the singing voice formed by singing the target music and the tendency of pitch change in the target music, the slowness of singing the target music is determined. (See Patent Document 1).

JP-A-10-149180

  However, in the above technology, since it is only possible to determine whether the singing is delayed, the singing is too fast, or just right for the target song, how much the song can follow the target song Or, more specifically, how much can you sing at that tempo.

  The present invention has been made to solve such problems, and an object of the present invention is to provide a technique for determining how much singing can be performed at the tempo.

  In order to solve the above problem, the first configuration is that each of the timings at which the constituent sounds constituting the singing voice when the user sings the target music (hereinafter referred to as “singing change timing”) appropriately changes the target music. Timing collating means for collating which timing corresponds to cst [k] (k = 1 to n) at which the constituent sound in the exemplary voice changes when singing (hereinafter referred to as “exemplary change timing”), and the singing change For each timing, a deviation specifying means for specifying a timing deviation amount vdt [k] based on the model change timing checked by the timing checking means as corresponding to the singing change timing, and the adjacent examples The change timings cst [k], cst [k + 1], and the exemplary change timing are described above. Based on the deviation amounts vdt [k], vdt [k + 1] specified by the re-identifying means, the utterance specifying means for specifying the utterance time vlen [k] of each constituent sound in the singing voice, and the utterance specifying means The utterance time clen [k] when the utterance time vlen [k] is appropriately uttered in the section of the example change timings cst [k] and cst [k + 1] referred to when the utterance time vlen [k] is specified. ], The reproducibility specifying means for specifying the non-reproducibility nvlen [k] of the exemplary utterance time clen [k] at the utterance time vlen [k], and the non-reproducibility specified by the reproducibility specifying means When each of the nvlen [k] is distributed in the order of arrival of the model change timing cst [k] referred to in the identification, Based on the non-reproducibility series, the evaluation output means for outputting a higher evaluation value as the follow-up performance of the song to the target music as the periodicity included in the non-reproducibility change pattern in the series is lower. Yes.

  In the follow-up evaluation system configured as described above, the utterance time of each constituent sound is specified by comparing the singing voice and the model voice, and the non-reproducibility is specified by comparing it with the corresponding model voice time. As the periodicity included in the non-reproducibility series is lower, a higher evaluation value is output as the followability of the song to the target music.

  If the non-reproducible sequence can follow the singing of the target music, that is, if it can sing appropriately according to the tempo, a large periodicity does not appear in the change pattern in the sequence. If you can sing appropriately according to the tempo of the target song, the non-reproducibility of the model utterance time will not increase, and it will change at a certain size, so what is a change pattern with a large periodicity? Because it cannot be.

  On the other hand, if you cannot sing properly according to the tempo of the target song and sing late or sing faster than the actual tempo, the non-reproducible It is expected that a singer who notices a deviation in singing timing due to the size repeats the singing behavior of changing the pitch of the constituent sounds in accordance with the model change timing.

  In this case, a change pattern in which the non-reproducibility of the model utterance time increases and then becomes smaller than before is repeated, and this appears as a large periodicity in the sequence. And this periodicity becomes so large that it cannot follow the singing with respect to the object music, that is, it cannot sing according to the tempo.

  Therefore, as described above, the lower the periodicity included in the non-reproducible sequence of model utterance times, the better the singing of the target music can be in accordance with the tempo, and the higher the followability to singing. Can do.

  In other words, as in the above configuration, the lower the periodicity is, the higher the evaluation value is output as the followability of the singing with respect to the target music, and how much the singing can follow the target music. That is, it can be the result of determining how much singing can be performed at the tempo.

  In this configuration, “output the evaluation value” means that, for example, a message indicating the evaluation value is output from the display unit or the speaker, or the evaluation value is passed to another device such as a karaoke device described later to be processed. That is.

  Further, in this configuration, when specifying the utterance time vlen [k] of each component sound in the singing voice, for example, a mathematical expression such as {cst [k + 1] + vdt [k + 1]} − {cst [k] + vdt [k]} It is conceivable that the value calculated according to the above is specified as the utterance time vlen [k].

  The “non-reproducibility” in the above configuration indicates how much the actual utterance time in the singing voice cannot reproduce the model utterance time of the same component sound. For example, the utterance time vlen [ k] and the ratio of the exemplary utterance time clen [k] (= vlen [k] / clen [k], or clen [k] / vlen [k]), the value calculated increases as the distance from “1” increases. It is conceivable to specify as non-reproducibility nvlen [k].

  In the above configuration, when specifying which model change timing in the model voice corresponds to the song change timing in the singing voice, the model voice and the singing voice may be collated by any method. .

  As a specific example, for example, the transition timing of the pitch along the time axis of each of the model voice and the singing voice (as a specific example, a waveform indicating the transition of the pitch, etc.) is collated to determine the change timing. It is possible to specify.

For this purpose, the first configuration is preferably a second configuration (claim 2) shown below.
In this configuration, the timing collating unit collates the transition patterns of the pitch along the time axis of each of the singing voice and the exemplary voice, so that each of the singing change timings at which the pitch changes in the singing voice is obtained. And which of the model change timings cst [k] in the model voice corresponds to.

  If it is this composition, by comparing the transition pattern of the pitch in each of the model voice and the singing voice, for example, among the transition patterns of the pitch along the time axis of the model voice, the singing change of the constituent sound in the singing voice The model change timing that approximates the transition pattern in the timing by a predetermined threshold or more can be specified, and this can be specified as the model change timing corresponding to the approximate song change timing.

  In this configuration, as a transition pattern of the pitch in the model voice used for collation, a waveform showing the transition of the actual pitch along the time axis may be used. An information string indicating a price (specific example, musical score data) may be used.

  Note that with this configuration, if the model speech contains constituent sounds that are continuous at the same pitch, it will be difficult to specify the correspondence relationship of the model change timing only by the transition of the pitch. It is desirable to use the methods together. As “another collation method” in this case, for example, a collation method based on a voice level transition pattern can be considered.

  In this configuration, the timing collating means is sandwiched by the singing change timing specifying the correspondence relationship with the model change timing in the section along the time axis of the singing voice, and specifies the correspondence relationship. For the section corresponding to the non-exemplary change timing cst [k], the singing change corresponding to the exemplary change timing cst [k] for which the correspondence relationship has not been specified is the timing at which the sound level becomes lower than a certain level in the section. Specify as timing.

  If it is this structure, after identifying the correspondence relationship of change timing by collating the transition pattern of the pitch in each of the model voice and the singing voice, the transition pattern of the voice level for the section in which the correspondence relationship is not specified here Can be identified as the model change timing corresponding to any of the singing change timings in the singing voice.

  Therefore, even if constituent sounds that are continuous at the same pitch are included in the model voice, it is possible to appropriately identify which model change timing corresponds to the song change timing in the singing voice. .

  In each of the above configurations, when determining the evaluation value indicating the followability of the singing, it is necessary to specify the periodicity included in the “non-reproducibility series”, and the determination determines the evaluation value. It may be performed at the timing, or may be performed prior to the determination.

For this latter, each of the above-mentioned configurations should be as a fourth configuration (claim 4) shown below.
In this configuration, there is provided period specifying means for specifying the periodicity included in the non-reproducible series. And the said evaluation output means outputs a high evaluation value as followability of the singing with respect to the said target music, so that the periodicity specified by the said period specific means is low.

If it is this structure, prior to determining the evaluation value which shows the followability of a song, the periodicity contained in the non-reproducibility series can be specified.
The method for specifying periodicity in this configuration is not particularly limited. For example, the non-reproducibility sequence is regarded as a waveform that changes with the magnitude of non-reproducibility as an amplitude, and is defined by the distribution of frequency components of the waveform. It is conceivable to be able to specify the periodicity.

As a configuration for this purpose, it is conceivable that the fourth configuration is changed to a fifth configuration (claim 5) shown below.
In this configuration, the period specifying unit regards the non-reproducibility series as a waveform that changes in order of arrival of the model change timing cst [k] with the non-reproducibility magnitude as an amplitude, and a frequency component of the waveform The periodicity defined by the distribution is specified, and the evaluation output means determines the frequency component distributed based on the distribution of the frequency components calculated by the period specifying means. As the sharpness is smaller, a higher evaluation value is output on the assumption that the periodicity included in the change pattern of the time difference is lower.

  With this configuration, the “non-reproducibility series” is regarded as a waveform whose magnitude of non-reproducibility changes as an amplitude, and after calculating the frequency component distribution of the waveform, the sharpness ( As the periodicity included in the change pattern is lower as the so-called Q value is smaller, a higher evaluation value is output in such a case.

  If the periodicity in the non-reproducible sequence is large in the frequency component distribution, naturally the spectrum intensity of the specific frequency component should be large, and a peak appears in the frequency component distribution. In this case, the frequency component having such a large spectrum intensity should show a large value as the sharpness. Conversely, if the periodicity in a non-reproducible sequence is small, the sharpness shows a small value.

  Therefore, as in the above configuration, if the sharpness is small, the periodicity included in the change pattern is low, and if it is a configuration that outputs a high evaluation value in such a case, the evaluation value is sung to the target song. It can be set as high evaluation as followability of.

  In this configuration, as long as the spectrum intensity is high in the distribution of frequency components, the evaluation value may be determined based on the sharpness of any frequency component, but the spectrum intensity is the highest. What is necessary is just to make it determine based on the sharpness of a large frequency component.

For this purpose, the above-described configuration is preferably a sixth configuration (claim 6) described below.
In this configuration, the evaluation output means, based on the distribution of frequency components calculated by the period specifying means, for the frequency component having the highest spectral intensity in the distribution, the smaller the sharpness of the frequency component, the smaller the time difference. A high evaluation value is determined on the assumption that the periodicity included in the change pattern is low.

With this configuration, the evaluation value can be determined based on the sharpness of the frequency component having the highest spectral intensity in the frequency component distribution.
Each of the above-described configurations may be a seventh configuration (claim 7) described below.

  In this structure, the singing data acquisition means which acquires the singing data which shows the singing sound at the time of the singing of the object music by a user with the identification information which can identify this object music sung is provided. And the said timing collation means collates the song audio | voice shown by song data by the said song data acquisition means with the model audio | voice of the target music identified with the identification information acquired with the said song data.

If it is this structure, while obtaining song data for every song of the object music by a user, an evaluation value can be determined and output based on the song data.
The followability evaluation system in each of the above configurations may be configured as a single device, or may be configured such that a plurality of devices that are communicably connected operate in cooperation with each other.

Moreover, as a structure for solving the said subject, you may make it like the 8th structure (Claim 8) which shows a karaoke system below.
In this configuration, for each unit section obtained by dividing the follow-up evaluation system according to any one of claims 1 to 7 and the singing voice indicated by the singing data for each predetermined section along a time series, the unit By comparing the singing parameters relating to the voice of the section with the ideal parameters based on the correct voice to be uttered in the unit section, the singing scoring means for scoring the singing song, and the scoring results scored by the singing scoring means are reported. And a result notification means.

  The singing scoring means determines the final scoring result by adding or subtracting the scoring result based on the comparison between the singing parameter and the ideal parameter according to the evaluation value output by the evaluation output means. .

If it is this structure, the effect | action and effect similar to said each structure can be acquired. Furthermore, the scoring result considering the evaluation value output as described above can be notified.
Moreover, in order to solve the said subject, it is good also as a program (Claim 9) for making a computer system perform the various process procedures for functioning as all the means in any of the said 1st-8th.

If it is a computer system that executes this program, it can constitute a part of the follow-up evaluation system according to any one of the first to eighth aspects.
The above-described program is composed of an ordered sequence of instructions suitable for processing by a computer system, and uses a tracking evaluation system, a karaoke system, and the like via various recording media and communication lines. It is provided to users and the like.

Block diagram showing the overall configuration of the karaoke system Flow chart showing follow-up evaluation process The figure which shows a mode that the correspondence of a change timing is specified based on the transition pattern of a pitch The figure which shows a mode that the correspondence of a change timing is specified based on the transition pattern of an audio | voice level The figure which shows a mode that the periodicity of the pattern of a change in a non-reproducibility series is specified. Flow chart showing music performance processing

Embodiments of the present invention will be described below with reference to the drawings.
(1) Overall Configuration The karaoke system 1 is configured such that a server 2 composed of a well-known computer system and each of one or more karaoke apparatuses 3 are communicably connected via a network 100.

  The server 2 includes a control unit 21 that controls the entire server, a storage unit 23 that stores various information, a communication unit 25 that controls communication via the network 100, and a user interface (U / I) unit 27 including a keyboard and a display. , A media drive 29 for inputting / outputting information via a recording medium. The server 2 functions as a followability evaluation system in the present invention.

The karaoke device 3 includes a control unit 31 that controls the entire device, a storage unit 33 that stores music data and video data indicating the accompaniment content and lyrics of the performance music, a communication unit 35 that controls communication via the network 100, and the like. A display unit 41 for displaying video, an operation unit 43 including a plurality of keys and switches, a voice input / output unit 49 for controlling voice input from the microphone 45 and voice output from the speaker 47, and the like are provided. .
(2) Follow-up evaluation process by server 2 Hereinafter, the process procedure of the follow-up evaluation process executed by the control unit 21 of the server 2 according to a program stored in the built-in memory or the storage unit 23 will be described with reference to FIG. To do.

This follow-up evaluation process is started by acquiring singing data from any karaoke apparatus 3 (s110).
This singing data is data that is transmitted after the user sings a song using the karaoke device 3, and indicates an audio signal along a time series of audio related to the singing as a digital signal. The song data is transmitted with identification information (music number) of the music related to the song added. In addition, this song data is good also as being acquired irrespective of the song by the karaoke apparatus 3. FIG.

  When the follow-up evaluation process is activated, first, the speech waveform is converted into a discrete frequency spectrum based on the speech waveform indicated by the singing data received at the time of activation (s120).

Here, first, the voice waveform v [i] (i: time index) (see FIG. 3A) is shifted by a predetermined number n ₀ from the sampling point as a digital signal to obtain a time length N ₀ (for example, A waveform segment v _w [p] (p = 1, 2,..., N ₀ ) cut out in order in a time window w [n] of several tens of ms is obtained by the following equation 1.

The time segment v _w [p] is a time region t [m] determined by the following equation 2 based on the order (number) m of the time window w [n] and the sampling frequency F _s in the digital signal. ] Is shown.

Then, the waveform segment v _w [p] thus obtained is subjected to discrete Fourier transform by the following expression 3, thereby obtaining a discrete frequency spectrum V [i ′] obtained by converting the speech waveform v [i]. It is done.

Next, based on the discrete frequency spectrum V [i ′] converted in s120, the fundamental frequency in the harmonic structure component included in the discrete frequency spectrum is estimated (s130). Here, the harmonic structure model V _HM [i ′] (formula 4 below) composed of the fundamental frequency F ₀ and its harmonic component (harmonic component), and the discrete frequency spectrum V [i ′ converted in the above s120. ] (I ′: frequency index) and the maximum correlation F ₀ is obtained for the time domain t [m] described above, and F ₀ thus obtained is estimated as the fundamental frequency vf ₀ [m].

  When the fundamental frequency vf0 [m] estimated in this way is distributed over the frequency corresponding to each time window, as shown in FIG. 3B, the transition of the fundamental frequency included in the speech waveform indicated by the song data, that is, It shows the transition pattern of the pitch.

  Next, based on the identification information (music number) of the music (hereinafter referred to as “song music”) added to the song data received in s110, the correct sound (hereinafter referred to as “model voice”) to be uttered in the music. Is read from a plurality of types of model data stored in advance in the storage area for model data in the storage unit 23 (s140).

  This model data includes the pitch transition pattern along the time axis of the model voice in the song composition, the utterance start timing cst [k], the sound of each of the constituent sounds k (= 1, 2,...) As the model voice. It is defined by a high cf0 [k], a note value clen [k], and a voice level cvol [k]. In the present embodiment, the score data represents each component sound as a note.

  Next, the pitch transition pattern along the time axis of the model voice indicated by the model data read out at s140 and the song voice indicated by the song data received at s110 is checked. Each change timing (hereinafter referred to as “exemplary change timing”) in which the constituent sound continues in the model voice is changed by a change timing (hereinafter referred to as “singing change timing”) in the singing voice cst [k ] (K = 1 to n) is identified (s150).

  In s150, first, based on the pitch transition pattern in the model voice, each predetermined timing (intermediate point in the present embodiment) from the start to the end of the continuous change in the constituent voice in the model voice is determined. Specified as an example change timing.

  Subsequently, in each of the singing voice and the model voice, the pitch transition patterns for the reference period (for example, the period to each adjacent constituent sound) centered on each model change timing are collated in the same reference period ( (Refer FIG.3 (c)).

  Here, the transition pattern of the same reference section in the singing voice is changed to the transition pattern of each reference period centering on the model change timing in which the pitch of the continuous constituent sounds is changing among the reference periods in the model voice. By moving along the axis, the similarity when the similarity (correlation) between both transition patterns becomes maximum and the time difference vdt [k] along the time axis are calculated. The method for calculating the similarity (correlation) and the time difference here is not particularly limited. For example, it is conceivable to use the method described in JP-A-2005-107330.

  Then, each model change timing at which the similarity and time difference are calculated by the above collation is specified as corresponding to the song change timing included in the reference period of the singing voice subjected to collation with this model change timing. . Here, with respect to the model change timings of the constituent sounds that are continuous at the same pitch, the transition pattern is not collated and the time difference vdt [k] is not calculated, but these time differences vdt [k] are set to the initial value “0”. Is done.

Next, based on the speech waveform indicated by the singing data received at s110, the speech waveform is converted into a level waveform indicating the transition of the speech level (s160).
Here, first, similarly to s120, the speech waveform v [i] (see FIG. 3 (a)) is shifted by a predetermined number n ₀ from the sampling point as a digital signal, and a time window of time length N ₀ is obtained. The waveform segment v _w [p] cut out in order by w [n] is obtained by the above-described equation 1.

Then, the waveform segment v _w [p] obtained in this way is converted into a level waveform v _p [m] indicating the transition of the voice level by the following Expression 5.

When the level waveform v _p [m] thus converted distributes the sound level corresponding to each time window, as shown in FIG. 4A, the transition pattern of the sound level in the sound waveform indicated by the song data is shown. It will be shown.

  Next, among the sections along the time axis of each of the singing voices indicated by the singing data received at s110, the singing change timing between which the correspondence relationship with the model changing timing is specified at s150 is sandwiched, and , For the section corresponding to the model change timing for which the correspondence relationship has not been specified, the timing at which the audio level falls below a certain level in this section is specified as the song change timing corresponding to the model change timing for which the correspondence relationship has not been specified. (S170).

  Here, as shown in FIG. 4B, the timing at which the audio level is minimized in the corresponding section is specified in the level waveform converted in s160, and this timing is not specified in the corresponding section. It is specified as the song change timing corresponding to the model change timing.

Although this timing may be specified in any way, in the present embodiment, based on the level waveform v _p [m] converted by Equation 5 described above, the first-order differential value v _p ′ [m] of this level waveform is used. ] = 0 and the timing corresponding to “m” that simultaneously satisfies the condition of the secondary differential value v _p ″ [m]> 0.

Each differential value here is approximated by the following formulas 6 and 7.
First derivative value v _p '[m] = v _p [m + 1] −v _p [m] (Expression 6)
Second derivative value v _p ″ [m] = v _p ′ [m + 1] −v _p ′ [m] (Expression 7)
In s170, the time difference vdt [k] with respect to the model change timing cst [k] at the singing change timing for which the correspondence relationship is specified is further calculated and specified by the following Expression 8. In the following Expression 8, m specified as described above is represented as m0.

  Next, each utterance time vlen [k] of each constituent sound k in the singing voice is specified based on the time difference vdt [k] for each of the constituent sounds k specified in s150 and s170 (s180). Here, the time difference vdt [k] for a certain component sound k and the time difference vdt [k + 1] for the component sound k + 1 adjacent thereto are expressed by the following Expression 9 to give the utterance time length vlen [k] for the component sound k. Is calculated and specified.

  Next, for each utterance time vlen [k] calculated in s180, each non-reproducible nvlen [k] of the exemplary utterance time clen [k] in the same section is specified (s190). This “non-reproducibility” indicates how much the actual utterance time of the singing voice cannot reproduce the model utterance time of the same constituent sound.

  Here, each utterance time vlen [k] specified in s180 is appropriately uttered in the section of the model change timings cst [k] and cst [k + 1] referred to when the utterance time vlen [k] is calculated. The non-reproducibility nvlen [k] (= vlen [k] / clen [k]) of the exemplary utterance time clen [k] at the utterance time vlen [k] by comparing with the exemplary utterance time clen [k] Is calculated.

Next, based on the non-reproducible nvlen [k] series specified in s190, the periodicity included in the non-reproducible change pattern in this series is specified (s200).
Here, the non-reproducibility series is regarded as a waveform that changes with the non-reproducibility magnitude as an amplitude in the order of arrival of the change timing referred to in the identification (see FIG. 5A), and the frequency spectrum of the waveform. The distribution NVLEN [k] is calculated by the following equation 10 to identify the periodicity defined by this distribution.

  As shown in FIG. 5 (b), the frequency spectrum distribution NVLEN [k] identified in this way distributes the spectrum intensity for each change timing, and as shown in FIG. 5B, the higher the periodicity included in the non-reproducible change pattern, The spectral intensity of the frequency component corresponding to the periodicity increases. That is, this frequency spectrum distribution NVLEN [k] indicates that the greater the spectrum intensity, the higher the periodicity of the frequency component corresponding to the spectrum intensity.

  Then, based on the periodicity specified in s200, an evaluation value obtained by evaluating the followability of singing in the singing voice indicated by the singing data received in s110 is determined (s210).

  Here, in the frequency spectrum distribution NVLEN [k] specified in s200, the smaller the sharpness Q of the predetermined frequency component (for example, the frequency component having the highest spectral intensity) distributed, the less the reproduction is performed. A high evaluation value is determined on the assumption that the periodicity included in the sex change pattern is low.

  Specifically, when the time index k that is a peak in the frequency component is “k0” and the width of the time index k that is ½ of the peak is “Δk”, k0 and Δk The sharpness Q is obtained from the ratio (k0 / Δk), and the reciprocal of this sharpness Q is determined as the evaluation value SC (= 1 / Q).

  In s210, not only the above-described evaluation value SC is determined, but also a well-known scoring is performed based on the singing data, and the scoring result is added or subtracted according to the evaluation value SC, thereby obtaining a final scoring result. May be determined. The scoring here is, for example, for each unit section obtained by dividing the singing voice indicated by the singing data for each predetermined section along the time series, and singing parameters related to the sound of the unit section should be uttered in the unit section. By comparing with an ideal parameter based on correct speech, a value corresponding to a parameter error in each unit interval may be used as a scoring result.

  Then, after the evaluation value SC (or evaluation value and scoring result; hereinafter referred to as “evaluation value etc.”) determined in s210 is returned to the karaoke device 3 that is the music data transmission source (s220), The followability evaluation process ends.

In the karaoke apparatus 3 that has received the evaluation value or the like, the evaluation value or the like is displayed on the display unit 41 by a music performance process described later.
(3) Music performance processing by the karaoke apparatus 3 Hereinafter, a processing procedure of music performance processing executed by the control unit 31 of the karaoke apparatus 3 according to a program stored in the built-in memory or the storage unit 33 will be described with reference to FIG. This music performance process is repeatedly executed after the karaoke apparatus 3 is activated.

When the music performance process is activated, the user enters a standby state until an operation for selecting a music to be sung is performed by the user (s310: NO).
Thereafter, when an operation for selecting a song is performed (s310: YES), the song number of the song (designated song) thus selected is acquired (s320).

  Next, as information for requesting the karaoke apparatus 3 for music data for playing the designated music identified by the music number based on the music number acquired in s320, the music number, and this A notification request with the user ID acquired together is generated (s330), and is transmitted to the server 2 (s340).

The server 2 that has received the notification request is configured to return music data for playing the designated music identified by the music number associated with the notification request.
Thus, after the notification request is transmitted in s340, when the music data returned from the server 2 is received (s350), the music data is stored in the storage unit 33 (s360).

  Next, the performance of the designated music based on the music data stored in the storage unit 33 is started in s360 (s380), and the voice input from the microphone 45 during the performance, that is, the designated music is sung. Generation of song data indicating voice is started (s390).

  In this way, after the performance of the designated music is started, it is in a standby state until the performance is completed (s400: NO), and when the performance is completed (s400: YES), the song data started in s390 is stored. The generation is finished, and the song data generated up to that point is acquired (s410).

  Next, the singing data acquired in s410 is transmitted to the server 2 (s420). The server 2 that has received this singing data returns the evaluation value or scoring result (evaluation value, etc.), which is the evaluation result, after evaluating the followability by the followability evaluation process described above.

  Although the singing data itself is transmitted to the server 2 here, only the parameters necessary for determining the evaluation value and the like on the server 2 side may be transmitted to the server 2.

Then, after the singing data is transmitted to the server 2 by the above s420, the evaluation value and the like transmitted from the server 2 are received (s430), and the evaluation value and the like are displayed on the display unit 41 (s440). ), The music performance process ends.
(4) Actions and effects In the karaoke system 1 configured as described above, the singing voice and the model voice are collated to specify the utterance time vlen [k] of each constituent sound k (s180 in FIG. 2), and this Is compared with the corresponding exemplary utterance time clen [k] (s190 in the figure), and the lower the periodicity included in this non-reproducible nvlen [k] series, the more A high evaluation value is determined as the followability of the singing to the music (s210 in the figure).

  If the non-reproducible sequence can follow the singing of the target music, that is, if it can sing appropriately according to the tempo, a large periodicity does not appear in the change pattern in the sequence. That is, if the singing can be appropriately performed according to the tempo of the target music, the non-reproducibility nvlen [k] of the exemplary utterance time clen [k] does not become large and changes with a certain size, so a large cycle This is because it cannot be a change pattern with sex.

  On the other hand, when the singing cannot be performed properly in accordance with the tempo of the target music and the singing is delayed or sung quickly from the actual tempo, the non-reproducibility nvlen [k] of the model utterance time clen [k] is large. After that, it is expected that the singer who noticed the deviation of the singing timing due to the size of the non-reproducibility nvlen [k] will repeat the singing behavior of changing the pitch of the constituent sound in accordance with the model change timing. Is done.

  In this case, the non-reproducibility nvlen [k] of the exemplary utterance time clen [k] repeats a change pattern in which the non-reproducibility nvlen [k] becomes smaller than before after increasing, and this appears as a large periodicity in the sequence . And this periodicity becomes so large that it cannot follow the singing with respect to the object music, that is, it cannot sing according to the tempo.

  Therefore, as described above, it can be said that the lower the periodicity included in the non-reproducible nvlen [k] series of the model utterance time clen [k], the more suitable the target music can be sung according to the tempo. It can be said that followability to singing is high.

  In other words, as in the above configuration, the lower the periodicity is, the higher the evaluation value is output as the followability of the singing with respect to the target music, and how much the singing can follow the target music. That is, it can be the result of determining how much singing can be performed at the tempo.

  Moreover, in the said embodiment, the singing change of the structure sound in a singing voice is compared among the transition patterns of the pitch along the time axis of a model voice by collating the transition pattern of the pitch in each of a model voice and a singing voice. The model change timing that approximates the transition pattern in the timing more than a predetermined threshold (having the maximum similarity) is specified, and this is the model change timing corresponding to the singing change timing to be approximated It can be identified (s150 in FIG. 2).

  Moreover, in the said embodiment, after identifying the correspondence relationship of a change timing by collating the transition pattern of the pitch in each of model voice and singing voice (s150 of FIG. 2), the correspondence relationship was not specified here. By checking the transition pattern of the voice level for the section, it is possible to specify which of the singing change timings in the singing voice corresponds to the exemplar change timing of the constituent sounds that continue at the same pitch in the singing voice ( Fig. 170).

  Therefore, even if constituent sounds that are continuous at the same pitch are included in the model voice, it is possible to appropriately identify which model change timing corresponds to the song change timing in the singing voice. .

Moreover, in the said embodiment, prior to determining the evaluation value which shows the followability of a song, the periodicity contained in a non-reproducibility series can be specified (s200 of FIG. 2).
Further, in the above embodiment, the “non-reproducibility series” is regarded as a waveform whose magnitude of non-reproducibility changes as an amplitude, and after calculating the distribution of frequency components of the waveform, the sharpness in the frequency component is calculated. As the degree (so-called Q value) is small, the periodicity included in the change pattern is low, and in such a case, a high evaluation value is output (s200 and s210 in FIG. 2).

  If the periodicity in the non-reproducible sequence is large in the frequency component distribution, naturally the spectrum intensity of the specific frequency component should be large, and a peak appears in the frequency component distribution. In this case, the frequency component having such a large spectrum intensity should show a large value as the sharpness. Conversely, if the periodicity in the non-reproducible series is small, the sharpness shows a small value (see FIG. 5).

  Therefore, as in the above-described embodiment, assuming that the periodicity included in the change pattern is lower as the sharpness is smaller, and in such a case, a configuration that outputs a high evaluation value, the evaluation value for the target music It can be set as high evaluation as followability of a song.

  Moreover, in the said embodiment, an evaluation value can be determined based on the sharpness of the frequency component with the largest spectrum intensity in distribution of a frequency component (210 of FIG. 2).

  Moreover, in the said embodiment, while acquiring song data for every song of the object music by a user (s110 of FIG. 2), an evaluation value can be determined and output based on the song data (s120 of the same figure). ~ S210).

Moreover, in the said embodiment, if it is a case where it is comprised so that the result of having performed well-known scoring may be added or subtracted according to the evaluation value SC, the scoring result which considered the evaluation value determined as follow-up evaluation is taken. Notification can be made (s220 in FIG. 2).
(5) Modifications Embodiments of the present invention have been described above, but the present invention is not limited to the above-described embodiments, and can take various forms as long as they belong to the technical scope of the present invention. Needless to say.

  For example, in the said embodiment, it is comprised so that an evaluation value may be output by the aspect of the display on the display part 41 of the karaoke apparatus 3 (s440 of FIG. 6). However, the output of the evaluation value may be realized, for example, by displaying a message indicating the evaluation value on the display unit or the speaker of the server 2.

  Moreover, in the said embodiment, the case where model data was the musical score data which represented each component sound of model voice as a musical note was illustrated. However, the model data may be data indicating a waveform of a pitch or a sound level in the model voice.

  Moreover, in the said embodiment, the case where the server 2 and the karaoke apparatus 3 were comprised so that it might operate | move cooperatively as the karaoke system 1 was illustrated. However, this system may be configured by the server 2 alone by causing the server 2 to implement the functions implemented on the karaoke device 3 side.

  In addition, the server 2 in the above embodiment can function as the server 2 as a whole including other devices by performing part or all of the processing by the server 2 in cooperation with the other devices. Needless to say.

  Moreover, in the said embodiment, when calculating the time difference of model change timing and song change timing, what was comprised so that a transition pattern might be collated was illustrated. However, in calculating the time difference between the two change timings, by comparing the characters formed by voice recognition of the singing voice and the timing of the singing with the characters constituting the lyrics of the target song and the timing of the singing, The time difference may be calculated.

  Moreover, in the said embodiment, it is comprised so that the value calculated by vlen [k] / clen [k] may be used as non-reproducibility nvlen [k]. However, this non-reproducibility nvlen [k] only needs to indicate that the non-reproducibility increases as the distance from “1” increases. For example, the value calculated by clen [k] / vlen [k] is used. Also good.

  Moreover, in the said embodiment, when estimating a fundamental frequency by s130 of FIG. 2, what was comprised so that the model VHM [i '] of the said Formula 4 might be used was illustrated. However, the model used when estimating the fundamental frequency is not limited to this model. For example, it is conceivable to use the model of Equation 11 shown below.

Note that “σ” in Equation 7 is a parameter for adjusting the spread of the spectrum, and is a value of a predetermined ratio X% (several tens%; approximately 37% under the conditions of the present embodiment) from the peak value of the distribution. The deviation of the frequency index i until it becomes smaller is shown. The smaller the value, the finer the components of the harmonic structure, and the larger the value, the thicker and smoother the shape. The value of “σ” may be set to a value smaller than the predetermined ratio X% (specifically, about 10 to 20%).
(6) Correspondence with the Present Invention In the embodiment described above, s150 and s170 in FIG. 2 are deviation specifying means in the present invention, s180 in FIG. 2 is utterance specifying means in the present invention, and s190 in FIG. S220 is an evaluation output means in the present invention, s200 is a period specifying means in the present invention, and s110 is a song data acquiring means in the present invention. FIG. S210 is singing scoring means in the present invention, and s440 in FIG. 6 is result notifying means in the present invention.

  DESCRIPTION OF SYMBOLS 1 ... Karaoke system, 2 ... Server, 21 ... Control part, 23 ... Storage part, 25 ... Communication part, 27 ... User interface part, 29 ... Media drive, 3 ... Karaoke apparatus, 31 ... Control part, 33 ... Storage part, 35 ... Communication unit, 41 ... Display unit, 43 ... Operation unit, 45 ... Microphone, 47 ... Speaker, 49 ... Audio input / output unit, 100 ... Network.

Claims (9)

Each of the timings at which the constituent sounds constituting the singing voice change when the user sings the target music (hereinafter referred to as “singing change timing”) changes the constituent sounds in the model voice when the target music is appropriately sung. Timing collating means for collating which timing (hereinafter referred to as “model change timing”) cst [k] (k = 1 to n) corresponds to;
For each of the singing change timings, a deviation specifying means for specifying a timing deviation amount vdt [k] based on the model change timing collated with the timing collating means as corresponding to the singing change timing;
Based on the model change timings cst [k] and cst [k + 1] adjacent to each other and the shift amounts vdt [k] and vdt [k + 1] specified by the shift specifying means for the model change timings, Utterance specifying means for specifying the utterance time vlen [k] of each component sound;
When each utterance time vlen [k] specified by the utterance specifying means is appropriately uttered in the section of the model change timings cst [k] and cst [k + 1] referred to when the utterance time vlen [k] is specified Reproducibility specifying means for specifying non-reproducibility nvlen [k] of the exemplary utterance time clen [k] in the utterance time vlen [k] by comparing with the exemplary utterance time clen [k] in
Based on the non-reproducibility sequence when the non-reproducibility nvlen [k] specified by the reproducibility specifying means is distributed in the order of arrival of the model change timing cst [k] referred to in the specification. An evaluation output means for outputting a higher evaluation value as the followability of the singing to the target music as the periodicity included in the non-reproducible change pattern in the sequence is lower. Evaluation system. The timing collating means collates pitch transition patterns along the time axis of the singing voice and the exemplary voice, respectively, so that each of the singing change timings at which the pitch changes in the singing voice is the model voice. The followability evaluation system according to claim 1, wherein it corresponds to which of the model change timings cst [k]. The timing matching means is sandwiched between singing change timings that specify a correspondence relationship with the model change timing in a section along the time axis of the singing voice, and the model change that does not specify the correspondence relationship For the section corresponding to the timing cst [k], the timing at which the audio level becomes lower than a certain level in the section is specified as the singing change timing corresponding to the model change timing cst [k] for which the correspondence relationship has not been specified. The followability evaluation system according to claim 2. A period specifying means for specifying the periodicity included in the non-reproducible series,
The said evaluation output means outputs a high evaluation value as followability of the singing with respect to the said target music, so that the periodicity specified by the said period specific | specification means is low. Follow-up evaluation system. The period specifying unit regards the non-reproducible series as a waveform that changes in order of arrival of the model change timing cst [k] with the non-reproducible magnitude as an amplitude, and calculates a distribution of frequency components of the waveform By specifying the periodicity specified by the distribution,
The evaluation output means is based on the distribution of frequency components calculated by the period specifying means, and the higher the sharpness of the distributed frequency components, the lower the periodicity included in the time difference change pattern. The follow-up evaluation system according to claim 4, wherein an evaluation value is output. The evaluation output means is based on the frequency component distribution calculated by the period specifying means, and the frequency component having the highest spectral intensity in the distribution is included in the time difference change pattern as the sharpness of the frequency component is smaller. The followability evaluation system according to claim 5, wherein a high evaluation value is determined as having a low periodicity. Singing data acquisition means for acquiring singing data indicating singing voice at the time of singing the target music by the user together with identification information capable of identifying the target music sung;
The said timing collation means collates the singing voice shown by song data by the said singing data acquisition means with the model audio | voice of the target music identified by the identification information acquired with the said singing data. To 6. The following evaluation system according to 6. The followability evaluation system according to any one of claims 1 to 7,
For each unit section obtained by dividing the singing voice indicated by the singing data for each predetermined section along a time series, the singing parameters relating to the sound of the unit section are ideal parameters based on the correct voice to be uttered in the unit section, and By contrast, a singing scoring means for scoring the singing song,
A result informing means for informing the scoring result scored by the singing scoring means,
The singing scoring means determines the final scoring result by adding or subtracting the scoring result based on the comparison between the singing parameter and the ideal parameter according to the evaluation value output by the evaluation output means. A characteristic karaoke system.   A program for causing a computer system to execute various processing procedures for causing the computer system to function as all means according to claim 1.