JP2015069082A - Information processing device, data generation method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique for generating evaluation data.SOLUTION: In evaluation data generation processing, a control unit generates information including a weight which becomes larger as a singing skill in a musical piece is more characteristic, as evaluation data, on the basis of an average value and standard deviation of a skill characteristic amount for each singing skill in one musical piece and a standard skill characteristic amount in a plurality of musical pieces (S280), generates reference data (S290), and stores it together with the evaluation data in a storage unit 14 of an information processing server 10 (S300). In a karaoke scoring processing, the control unit calculates a skill evaluation point by performing weighted calculation based on the evaluation data. As a result obtained by comparing pitch transition in a singing waveform with that in the reference data, the control unit calculates a higher reference evaluation point with a higher matching degree and calculates a total amount with the skill evaluation point as a total evaluation point.

Description

  The present invention relates to an information processing apparatus that generates evaluation data, a data generation method, and a program.

  Conventionally, in a karaoke apparatus, a score obtained by adding an additional score obtained by evaluating a singing technique used during singing to a reference score scored based on a pitch transition in a singing voice has been calculated as an evaluation score ( Patent Document 1).

  In such a karaoke device, the reference data that represents the melody to be sung and prepared in advance for each song and the difference between the pitch transitions in the voice when singing the song is smaller, the higher the score Calculate the score. Furthermore, in the karaoke apparatus described in Patent Document 1, the singing voice is analyzed to detect various singing techniques used during the singing, and the larger the number of times the singing technique is used, the larger the added value is calculated. doing.

JP 2007-233303 A

In general pop music, the type of singing technique (hereinafter referred to as “characteristic singing technique”) used as a center when singing a music differs depending on the genre or singer of the music.
For this reason, in the karaoke apparatus described in Patent Document 1, it is conceivable that an additional score is given when the singing technique detected from the singing voice matches the evaluation data generated in advance. As the evaluation data referred to here, for example, it is conceivable that the content of the singing technique to be used when singing a song is associated with the timing at which the singing technique is to be used. Such evaluation data usually needs to be generated in advance for each piece of music by a human hand, and the conventional technology has a problem that it is difficult to automatically generate evaluation data based on music data. It was.

  Therefore, an object of the present invention is to provide a technique for generating evaluation data.

The present invention made to achieve the above object is an information processing apparatus comprising music data acquisition means, extraction means, first determination means, second determination means, and execution means.
The music data acquisition means of this invention acquires music data from the 1st memory | storage part in which the music data containing the singing song sound was memorize | stored. The extraction means extracts vocal data representing the sung singing sound from the music data acquired by the music data acquisition means.

  And a 1st determination means determines the technique feature-value showing evaluation about several singing techniques about the vocal data extracted by the extraction means. The second determining means is based on the evaluation information stored in the second storage unit that stores the vocal data of the plurality of songs and the evaluation information evaluated on the plurality of singing techniques for the vocal data of the plurality of songs. Thus, the characteristic singing technique satisfying a predetermined condition is determined among the technical feature quantities determined by the first determining means.

Furthermore, the execution means executes the evaluation of the singing for the music using the characteristic singing technique determined by the second determining means.
According to the information processing apparatus of the present invention, it is possible to automatically generate a characteristic singing technique as evaluation data. In addition, in the information processing apparatus of the present invention, if a predetermined condition is a characteristic singing technique in music, the characteristic singing technique as evaluation data to be generated may be a characteristic singing technique in music. it can.

  Therefore, if singing is evaluated using such a characteristic singing technique, it can be evaluated whether or not the singing technique used when the user sings the music is a characteristic singing technique. It can be reduced that the person who listens to the song feels uncomfortable.

Moreover, the information processing apparatus of this invention may be provided with the song acquisition means which acquires the song data showing the audio | voice input during the performance of the music.
In this case, the execution means in the present invention may perform singing evaluation on the music using the characteristic singing technique determined by the second determination means for the singing data acquired by the singing acquisition means.

According to such an information processing apparatus, when a song is sung, the singing can be evaluated.
Furthermore, the extracting means in the present invention may extract vocal data and accompaniment data representing accompaniment sounds in the music from the music data.

  In this case, the information processing apparatus of the present invention may further include performance means for playing a music piece based on the accompaniment data extracted by the extraction means. And the song acquisition means of this invention may acquire the audio | voice input during the performance of the music by the performance means as song data.

According to such an information processing device, music can be played based on the music data stored in the first storage unit, and the evaluation of the voice (singing) input during the performance can be performed.
Moreover, the execution means in this invention may derive | lead-out the difference of the technique feature-value and characteristic song technique in song data, and may raise the evaluation of the song with respect to a music, so that the derived difference is large.

According to such an information processing device, the higher the difference between the skill feature amount and the characteristic singing skill in the song data, the higher the evaluation.
That is, in the information processing apparatus, when the difference between the skill feature quantity and the characteristic singing technique in the song data is large, it can be said that the technique at the time of singing appears more strongly than the characteristic singing technique.

  For this reason, according to the information processing device, a song that strongly expresses a characteristic technique in music can be highly evaluated, and it can be reduced that the person who listened to the user's song feels uncomfortable about the evaluation result.

  Furthermore, the 1st determination means in this invention may determine a technique feature-value for every singing technique, and a 2nd determination means may determine a characteristic singing technique for every singing technique. In this case, the execution means in the present invention derives the difference between the skill feature quantity and the feature singing technique in the singing data for each singing technique, assigns a larger weight to the singing technique that strongly expresses the feature in the music, and derives the difference. The song singing may be evaluated by averaging the differences with weights.

According to such an information processing device, in the evaluation of song singing, the rate of evaluation with respect to singing techniques that strongly express characteristics can be increased.
The information processing apparatus of the present invention may further include standard calculation means, evaluation information generation means, and storage control means.

  The standard calculation means in the present invention calculates a standard feature amount representing a standard evaluation of the singing skill for vocal data of a plurality of music pieces. The evaluation information generation means numerically processes the standard feature amount derived by the standard calculation means, and calculates the result of the numerical processing as evaluation information. Further, the storage control means stores the evaluation information calculated by the evaluation information generating means in the second storage unit in association with vocal data of a plurality of music pieces.

According to such an information processing apparatus, evaluation information can be derived from vocal data of a plurality of music pieces.
Note that the numerical processing referred to here is, for example, obtaining an arithmetic mean or standard deviation.

By the way, this invention may be made | formed as a data generation method which evaluates a song.
In this case, the data generation method includes a music data acquisition process for acquiring music data from the first storage unit, an extraction process for extracting vocal data from the acquired music data, and a plurality of extracted vocal data. Based on the first determination process for determining the technique feature amount for the singing technique and the evaluation information stored in the second storage unit, the technique feature amount determined by the first determination process satisfies a predetermined condition. A second determination process for determining the characteristic singing technique and an execution process for performing an evaluation of the singing on the music using the determined characteristic singing technique.

According to such a data generation method, an effect similar to that of the information processing apparatus according to claim 1 can be obtained.
Further, the present invention may be made as a program executed by a computer.

  In this case, the program of the present invention includes a plurality of song data acquisition procedures for acquiring song data from the first storage unit, an extraction procedure for extracting vocal data from the acquired song data, and the extracted vocal data. Based on the first determination procedure for determining the skill feature amount for the singing technique and the evaluation information stored in the second storage unit, a predetermined condition is determined among the skill feature amounts determined by the first determination procedure. The computer is made to perform the 2nd determination procedure which determines the characteristic song technique to satisfy | fill, and the execution procedure which performs evaluation of the song with respect to a music using the determined characteristic song technique.

  If the present invention is implemented as a program, it can be used by loading it into a computer from a recording medium as necessary and starting it, or by acquiring it and starting it through a communication line as necessary. And by making a computer perform each procedure, the computer can be functioned as an information processing apparatus described in claim 1.

  The recording medium referred to here includes, for example, a computer-readable electronic medium such as a DVD-ROM, a CD-ROM, and a hard disk.

It is a block diagram which shows schematic structure of the karaoke system containing the information processing apparatus with which this invention was applied. It is a flowchart which shows the process sequence of a standard feature-value calculation process. It is a flowchart which shows the process sequence of an evaluation data generation process. It is a flowchart which shows the process sequence of a karaoke scoring process. It is a figure which shows an example of evaluation data.

Embodiments of the present invention will be described below with reference to the drawings.
<Configuration of karaoke system>
The karaoke system 1 shown in FIG. 1 is a system in which a music specified by a user (user) is played and the user sings along with the performance.

  In order to realize this, the karaoke system 1 includes an information processing server 10 and at least one karaoke apparatus 30. The information processing server 10 and the karaoke apparatus 30 are connected via a communication network. The communication network referred to here may be a wired communication network or a wireless communication network.

  The information processing server 10 stores music data MD-1 to MD-N prepared for each music. The karaoke apparatus 30 acquires the music data MD corresponding to the music specified by the user (user) from the information processing server 10 and plays the music based on the music data MD. Accept input.

The code “N” is an identifier for identifying the music data MD, and “N” is a natural number of 2 or more.
<Information processing server>
The information processing server 10 includes a communication unit 12, a storage unit 14, and a control unit 16.

Among these, the communication unit 12 performs communication between the information processing server 10 and the outside via a communication network.
The control unit 16 is a known control device that is configured around a known microcomputer including a ROM 18, a RAM 20, and a CPU 22. The ROM 18 stores processing programs and data that need to retain stored contents even when the power is turned off. The RAM 20 temporarily stores processing programs and data. The CPU 22 executes each process (various calculations) in accordance with a processing program stored in the ROM 18 or the RAM 20.

That is, the control unit 16 controls each unit constituting the information processing server 10 and executes data communication with the karaoke apparatus 30.
The storage unit 14 is a known storage device configured to be able to read and write stored contents. The storage unit 14 stores at least a plurality of music data MD.

  The music data MD includes music management information in which information related to music is described, master waveform data representing the performance sound of the music, and lyric data representing the lyrics of the music. The music management information includes at least music identification information (for example, music number) for identifying music.

  The master waveform data of the present embodiment is sound data including performance sounds of a plurality of musical instruments and singing sounds singing the main melody. The audio data may be data constituted by an audio file in an uncompressed audio file format, or data constituted by an audio file in an audio compression format.

  In the following, voice waveform data representing performance sound included in the master waveform data is referred to as accompaniment data, and voice waveform data representing singing sound included in the master waveform data is referred to as vocal data.

Musical instrument performance sounds included in the accompaniment data of the present embodiment include percussion instrument (eg, drum, drum, cymbal, etc.) performance sounds, stringed instrument (eg, guitar, bass, etc.) performance sounds, percussion instrument (eg, piano) ) And wind instruments (eg, trumpet, clarinet, etc.). In general music, percussion instruments and bass are usually used as rhythm instruments.

The music data MD stored in the storage unit 14 includes music data MD of music composed by a general user of the karaoke system 1 in addition to music data MD of music composed by a professional. The music data MD of the music composed by the general user is created by a known information processing terminal (for example, a personal computer or a portable terminal) and uploaded to the information processing server 10.
<Karaoke equipment>
The karaoke apparatus 30 includes a communication unit 32, an input reception unit 34, a music playback unit 36, a storage unit 38, an audio control unit 40, a video control unit 46, and a control unit 50.

  In the communication unit 32, the karaoke apparatus 30 communicates with the outside via a communication network. The input receiving unit 34 is an input device that receives input of information and commands in accordance with external operations. The input device in the present embodiment is, for example, a key, a switch, a remote control receiving unit, or the like.

  The music playback unit 36 plays back music based on the music data MD stored in the storage unit 38 or the music data MD downloaded from the information processing server 10. The voice control unit 40 is a device that controls voice input / output, and includes an output unit 42 and a microphone input unit 44.

  A microphone 62 is connected to the microphone input unit 44. Thereby, the microphone input part 44 acquires a user's song sound. A speaker 60 is connected to the output unit 42. The output unit 42 outputs the sound source signal of the music reproduced by the music reproducing unit 36 and the sound source signal of the singing sound from the microphone input unit 44 to the speaker 60. The speaker 60 outputs the sound source signal output from the output unit 42 instead of sound.

The video control unit 46 outputs video based on video data sent from the control unit 50. A display unit 64 for displaying video is connected to the video control unit 46.
The control unit 50 is configured around a known computer having at least a ROM 52, a RAM 54, and a CPU 56. The ROM 52 stores processing programs and data that need to retain stored contents even when the power is turned off. The RAM 54 temporarily stores processing programs and data. The CPU 56 executes each process (various calculations) in accordance with a processing program stored in the ROM 52 or the RAM 54.

  The ROM 52 includes a processing program for the control unit 50 to execute the karaoke scoring process, a processing program for executing the evaluation data generation process, and a processing program for executing the standard feature amount calculation process. Stored.

  The karaoke scoring process is a process of playing music designated by the user and evaluating the voice input through the microphone 62 during the performance period. The evaluation data generation process is a process of generating evaluation data necessary for the karaoke scoring process for each piece of music data MD. The standard feature quantity calculation process is a process for calculating a standard feature quantity used for generating evaluation data.

That is, the karaoke apparatus 30 calculates standard feature values as evaluation information according to the standard feature value calculation processing, and generates evaluation data as feature singing techniques for each piece of music data MD according to the evaluation data generation processing. And in the karaoke apparatus 30, according to a karaoke scoring process, a music is performed based on the music data MD corresponding to an object music,
The voice input through the microphone 62 is acquired as song data. Further, the karaoke apparatus 30 scores and evaluates the acquired singing data according to the karaoke scoring process.

That is, the karaoke apparatus 30 functions as an information processing apparatus that executes standard feature value calculation processing, evaluation data generation processing, and karaoke scoring processing.
<Standard feature calculation processing>
Next, a standard feature amount calculation process executed by the control unit 50 of the karaoke apparatus 30 will be described.

  This standard feature amount calculation process is started at a predetermined time interval. Note that the start timing of the standard feature amount calculation process is not limited to a predetermined time interval. For example, the timing at which a start command for starting a processing program (application) is input via the input receiving unit 34 may be used.

  In the standard feature amount calculation process, as shown in FIG. 2, when activated, the control unit 50 firstly selects one piece of music data from all the music data MD stored in the information processing server 10. The MD is acquired (S110). Subsequently, the control unit 50 acquires master waveform data included in the music data MD acquired in S110 (S120).

  Further, in the standard feature amount calculation process, the control unit 50 separates and extracts accompaniment data and vocal data from the master disk waveform data acquired in S120 (S130). In S <b> 130, a known method (for example, “PreFEst” described in JP-A-2008-134606) is conceivable as a method by which the control unit 50 separates accompaniment data and vocal data. Note that PreFEst is a method of separating the most prevalent voice waveform in the master waveform data from the master waveform data as vocal data and separating the remaining voice waveform as accompaniment data.

  Subsequently, in the standard feature value calculation process, the control unit 50 performs a musical score process on the vocal data extracted in S130 (S140). The music recording process in S140 is a well-known method of recording music based on the temporal change in sound pressure in vocal data and the temporal change in pitch in vocal data.

  That is, in the music recording process, the control unit 50 specifies the timing at which the temporal change of the sound pressure in the vocal data becomes equal to or greater than the specified threshold as the start timing nnt (a, i) of each note constituting the song melody in the music. . Further, in the music recording process, the control unit 50 specifies the timing at which the time change of the sound pressure in the vocal data is equal to or less than the specified threshold as the end timing nft (a, i) of each note constituting the song melody in the music. .

  In the music recording process, the control unit 50 specifies a section specified by the start timing nnt (a, i) and the end timing nft (a, i) corresponding to each other as a note section of each note. At the same time, in the music recording process, the control unit 50 specifies the pitch in each note interval based on the time change of the pitch in the vocal data, and the pitch nn (a, i) in each note interval and the note interval. Is associated.

In addition, the code | symbol a is a code | symbol which identifies a music, and the code | symbol i is a code | symbol which identifies the musical note area of the song melody in a music.
In the standard feature value calculation process, the control unit 50 further determines a skill feature value representing an evaluation of a plurality of singing skills for each note section in the music (S150). The plurality of singing techniques mentioned here include “vibrato”, “for”, “shrimp”, “fall”, and “fist”.

  Among these, in calculating the technical feature amount (hereinafter referred to as “vibrato feature amount”) vib (a, i) for “vibrato”, the control unit 50 firstly calculates a speech waveform corresponding to each note interval from vocal data. , And frequency analysis (DFT) is performed on the speech waveform of each note interval. And the control part 50 calculates the vibrato feature-value vib (a, i) according to following (1) Formula.

ただし、上記（１）式におけるｖｉｂ＿ｐｅｒ（ａ，ｉ）は、各音符区間の音声波形におけるスペクトルピークの突出精度を表す指標である。このｖｉｂ＿ｐｅｒ（ａ，ｉ）は、周波数解析結果（即ち、振幅スペクトル）のピーク値を、周波数解析結果の平均値で除すことで求めれば良い。また、上記（１）式におけるｖｉｐ＿ｄｅｐ（ａ，ｉ）は、各音符区間の音声波形の標準偏差である。

However, vib_per (a, i) in the above equation (1) is an index representing the protruding accuracy of the spectrum peak in the speech waveform of each note interval. The vib_per (a, i) may be obtained by dividing the peak value of the frequency analysis result (that is, the amplitude spectrum) by the average value of the frequency analysis result. Also, vip_dep (a, i) in the above equation (1) is the standard deviation of the speech waveform in each note interval.

  In calculating the technical feature amount (hereinafter referred to as “for feature amount”) tt (a, i) for “for”, the control unit 50 first converts the sound waveform of the non-harmonic component in the accompaniment data to the rhythm instrument. Extract as performance sound waveform. A known method may be used as the method for extracting the inharmonic component. Specific examples of the method include passing accompaniment sound data through a filter prepared in advance as a filter representing a speech waveform of a non-harmonic component, or “harmonic sound based on the anisotropy of spectrogram smoothness / The method described in “Separation of percussion instrument sounds” (Proceedings of the Spring Meeting of the Acoustical Society of Japan, 2-5-8, p.903-904 (2008.03)) can be considered.

  Further, in calculating the feature quantity tt (a, i), the control unit 50 estimates the timing at which the sound pressure becomes equal to or higher than a specified value in the performance sound waveform of the rhythm instrument as the beat position. Subsequently, the control unit 50 extracts a note having the shortest note value (hereinafter referred to as “shortest note”) from the notes constituting the song melody of the music. Then, the control unit 50 specifies the utterance timing by dividing the interval between the beat positions by the note value of the extracted shortest note. The utterance timing here is a timing at which singing may be started for each note i.

  Therefore, in the calculation of the feature quantity tt (a, i), the control unit 50 further specifies an utterance timing that satisfies a specified condition. Satisfying the specified condition here is an utterance timing that is later than the start timing nnt (a, i), and that the absolute value of the value subtracted from the start timing nnt (a, i) is minimum. It is. Then, a time length obtained by subtracting the specified utterance timing from the start timing nnt (a, i) is calculated as a feature quantity tt (a, i).

  In calculating the skill feature amount (hereinafter referred to as “shackle feature amount”) rise (a, i) for “shrimp”, the control unit 50 first derives a differential change obtained by differentiating the pitch time change of vocal data. To do. Subsequently, the control unit 50 specifies the timing at which the differential change becomes a positive value along the time axis before the start timing nnt (a, i) of each note. Further, the control unit 50 obtains the cross-correlation value between the pitch time change of the vocal data and the predefined exemplary curve in the section from the identified timing to the start timing nnt (a, i), and the scribing feature amount rise. Derived as (a, i).

In calculating the technical feature amount (hereinafter referred to as “fall feature amount”) fall (a, i) for “fall”, the control unit 50 performs differentiation after the end timing nft (a, i) of each note interval. The first timing when the change becomes a positive value along the time axis is specified. Further, the control unit 50 changes the pitch time of the vocal data in the section from the end timing nft (a, i) of each note section constituting the singing melody to the specified timing, and a prescribed model curve. Are derived as the fall feature value fall (a, i).

  In calculating the technical feature amount (hereinafter referred to as “fist feature amount”) kob (a, i) for “fist”, the control unit 50 first specifies a fist section. Here, the fist section is a section in which each note over a plurality of pitches is sung while changing the pitch with the same vowel.

  For this reason, in calculating the fist feature value kob (a, i), the control unit 50 identifies a section uttered by the same vowel (hereinafter referred to as “same vowel section”). As a method for specifying the same vowel interval, a method of deriving similarity of average values of mel frequency cepstrum (MFCC) of each note interval by cross-correlation and specifying a note interval whose cross-correlation value is equal to or greater than a threshold as the same vowel interval Is used.

  In addition, the control unit 50 identifies only the same vowel section that satisfies the setting condition as the fist section in the same vowel section. Satisfying the setting condition here means that the time interval between the end timing nft (a-1, i) and the start timing nnt (a, i) of the note intervals adjacent to each other along the time axis is equal to or less than a threshold value, and The pitches of adjacent note intervals are all different.

  In calculating the fist feature value kob (a, i), the control unit 50 calculates a chroma vector from the vocal waveform in the fist section. Further, the control unit 50 calculates a chroma vector of accompaniment data in the same fist section, and calculates a cross-correlation value with the chroma vector of the vocal waveform as a fist feature value kob (a, i).

  In the standard feature value calculation process, the control unit 50 calculates a music feature value as evaluation information evaluated for a plurality of singing techniques (S160). The music feature amount is an average value in one piece of music for each of the vibrato feature amount vib, the feature amount tt, the shawl feature amount rise, the fall feature amount fall, and the fist feature amount kob.

  Therefore, the control unit 50 calculates the music feature amount sviv of the vibrato feature amount vib according to the following equation (2). Therefore, the control unit 50 calculates the music feature amount stt of the feature amount tt according to the following equation (3). The control unit 50 calculates the music feature amount srise of the scribbling feature amount rise according to the following equation (4). Further, the control unit 50 calculates the music feature amount sfall of the fall feature amount fall according to the following equation (5). The control unit 50 calculates the music feature amount skob of the fist feature amount kob according to the following equation (6).

なお、（２）〜（５）式における符号Ｎｉは、楽曲の歌唱旋律を構成する音符区間の個数である。また、（６）式における符号Ｎｊは、ボーカルデータに含まれるこぶし区間の個数である。

In addition, the code | symbol Ni in (2)-(5) Formula is the number of the note intervals which comprise the song melody of a music. Moreover, the code | symbol Nj in (6) Formula is the number of fist areas contained in vocal data.

  In the standard feature value calculation process, the control unit 50 subsequently determines whether or not the process until the music feature value is calculated (ie, S110 to S160) has been executed for all the music data MD ( S170). As a result of the determination in S170, if the process until calculating the music feature amount is not executed for all the music data MD (S170: NO), the control unit 50 performs the standard feature amount calculation process in S110. Return to. And the control part 50 acquires the music data MD which has not performed the process from the information processing server 10, and transfers to S120.

  On the other hand, as a result of the determination in S170, if the process until calculating the music feature amount is executed for all the music data MD (S170: YES), the control unit 50 performs the standard feature amount calculation process. Proceed to S180.

  In S180, the control unit 50 calculates a standard feature amount. This standard feature amount includes a music feature amount sviv, stt, srise, sfall, and skov, an average value obtained by averaging all the song data MD, and a standard deviation.

  Among these, the average value msvib of the music feature amount sviv is calculated by the control unit 50 according to the following equation (7). Further, the control unit 50 calculates the average value mstt of the music feature amount stt according to the following equation (8), and calculates the average value msrise of the music feature amount srise according to the following equation (9). The control unit 50 calculates the average value msfall of the music feature value sfall according to the following equation (10), and calculates the average value mskob of the music feature value skob according to the following equation (11).

また、制御部５０は、楽曲特徴量ｓｖｉｂの標準偏差ｓｄｖｉｂを下記（１２）式に従って算出し、楽曲特徴量ｓｔｔの標準偏差ｓｄｔｔを下記（１３）式に従って算出し、楽曲特徴量ｓｒｉｓｅの標準偏差ｓｄｒｉｓｅを下記（１４）式に従って算出する。さらに、制御部５０は、楽曲特徴量ｓｆａｌｌの標準偏差ｓｄｆａｌｌを下記（１５）式に従って算出し、楽曲特徴量ｓｋｏｂの標準偏差ｓｄｋｏｂを下記（１６）式に従って算出する。

Further, the control unit 50 calculates the standard deviation sdvib of the music feature quantity sviv according to the following formula (12), calculates the standard deviation sdtt of the music feature quantity stt according to the following formula (13), and the standard deviation of the music feature quantity srise. sdrise is calculated according to the following equation (14). Further, the control unit 50 calculates the standard deviation sdfall of the music feature value sfall according to the following equation (15), and calculates the standard deviation sdkob of the music feature value skov according to the following equation (16).

なお、（７）式〜（１６）式における符号ＮＳは、楽曲データの個数を表す。

Note that the symbol NS in the equations (7) to (16) represents the number of music data.

  In the standard feature value calculation process, the control unit 50 uses the average values msvib, mstt, msrise, msfall, mskob and standard deviations sdvib, sdtt, sdrise, sdfall, sdkov calculated in S180 as standard feature values. 10 is uploaded to the storage unit 14 (S190). Further, in S <b> 190, the control unit 50 stores the vocal data of the music in association with the music feature amount for the vocal data of the plurality of music in the storage unit 14 of the information processing server 10.

Thereafter, the standard feature amount calculation process is terminated.
That is, in the standard feature value calculation process of the present embodiment, the control unit 50 calculates a skill feature value representing evaluation of each singing skill for vocal data of a plurality of music pieces. Further, in the standard feature amount calculation process, the control unit 50 performs numerical processing on the calculated technical feature amount, and stores the result of the numerical processing in the storage unit 14 of the information processing server 10 as a standard feature amount.
<Evaluation data generation process>
Next, an evaluation data generation process executed by the control unit 50 of the karaoke apparatus 30 will be described.

The evaluation data generation process is started when a start command for executing the evaluation data generation process is input.
Then, in the evaluation data generation process, as shown in FIG. 3, when activated, the control unit 50 firstly selects one piece of music data MD from all the music data MD stored in the information processing server 10. Is acquired (S210). Subsequently, the control unit 50 acquires master waveform data included in the music data MD acquired in S210 (S220).

  Further, in the evaluation data generation process, the control unit 50 separates the accompaniment data and the vocal data from the original disc waveform data acquired in S220, and extracts the accompaniment data and the vocal data (S230). The technique for separating accompaniment data and vocal data in S230 may be the same technique as in S130 in the standard feature value calculation process.

  Subsequently, in the evaluation data generation process, the control unit 50 performs a musical score process on the vocal data extracted in S230 (S240). The musical score processing in S240 may use the same method as the musical score processing in S140 of the standard feature amount calculation process.

  In the evaluation data generation process, the control unit 50 further determines each technical feature amount of the vocal data extracted in S230 for each note section in the music (S250). As a technique for determining the skill feature amount in S250, a method similar to S150 in the standard feature amount calculation process may be used.

  In the evaluation data generation process, subsequently, the control unit 50 calculates a music feature amount (S260). The music feature values calculated in S260 include the music (vibrato) feature value nsvib, the music (for) feature value nstt, the music (shackle) feature value nsrise, and the music (fall) feature value in the vocal data extracted in S230. nsfall, music (fist) feature value nskob. Similar to S160 in the standard feature value calculation process, the derivation of these music feature values may be realized by obtaining the average value of the skill features calculated in S250 for each singing skill or obtaining the standard deviation. .

  Further, in the evaluation data generation process, the control unit 50 generates evaluation data (S270). The evaluation data referred to here is a weight defined for each singing technique so that the characteristic singing technique in the music has a larger value.

  Specifically, in S270, first, the control unit 50 calculates evaluation values dviv, dtt, drise, dfall, dkob according to the following formulas (17) to (21).

そして、Ｓ２７０では、制御部５０は、下記（２２）式から（２６）式に従って、各歌唱技巧に対する重みｗｖｉｂ，ｗｔｔ，ｗｒｉｓｅ，ｗｆａｌｌ，ｗｋｏｂを算出する。すなわち、本実施形態の評価データ生成処理では、技巧特徴量の中で所定の条件を満たす特徴歌唱技巧を、重みによって表している。

In S270, the control unit 50 calculates weights wvib, wtt, write, wfall, and wkob for each singing technique according to the following equations (22) to (26). That is, in the evaluation data generation process of the present embodiment, the feature singing technique that satisfies a predetermined condition in the technique feature amount is represented by a weight.

これと共に、Ｓ２７０では、制御部５０は、各楽曲特徴量ｎｓｖｉｂ，ｎｓｔｔ，ｎｓｒｉｓｅ，ｎｓｆａｌｌ，ｎｓｋｏｂについて、標準特徴量からの向きｄｉｒｖｉｂ，ｄｉｒｔｔ，ｄｉｒｒｉｓｅ，ｄｉｒｆａｌｌ，ｄｉｒｋｏｂを求める。

At the same time, in S270, the control unit 50 obtains the directions dirviv, dirtt, dirrise, dirfall, and dirkov from the standard feature values for the music feature values nsvib, nstt, nsrise, nsfall, and nskob.

Specifically, the control unit 50 calculates each of dirviv, dirtt, dirrise, dirfall, and dirkob according to the following equations.
dirvib = sign (nsvib-msvib)
dirtt = sign (nstt−mst)
dirrise = sign (nsrise-mrise)
dirfall = sign (nsfall-msfall)
dirkob = sign (nskob-mskob)
However, “sign” here is a sign function, and is a function that returns “1” and “−1” according to the sign in parentheses. That is, the directions dirviv, dirtt, dirrise, dirfall, and dirkob are “1” or “−1”, respectively.

  In S270, the control unit 50, as shown in FIG. 5, the music identification information, the standard feature, the weights wvib, wtt, wrise, wfall, wkob, the direction dirviv, dirtt, dirrise, dirfall, dirkob, Is associated with each singing skill to generate evaluation data. Note that the numerical values in “” in FIG. 5 show examples of standard feature amounts, weights, and directions.

  Subsequently, in the evaluation data generation process, the control unit 50 generates the result of the music transcription process in S240 as reference data (S290). The reference data referred to here is data representing notes (that is, pitches and note values) constituting a melody to be sung.

  In the evaluation data generation process, the control unit 50 uploads the music identification information, the evaluation data, and the reference data in association with each other to the storage unit 14 of the information processing server 10 (S300).

Thereafter, the evaluation data generation process ends.
In other words, in the evaluation data generation process, the control unit 50 generates information including a weight that becomes a larger value as the characteristic singing technique in the music is an evaluation feature amount. Further, in the evaluation data generation process, the control unit 50 generates reference data and stores it in the storage unit 14 of the information processing server 10 together with the evaluation data.
<Karaoke scoring>
Next, the karaoke scoring process which the control part 50 of the karaoke apparatus 30 performs is demonstrated.

The karaoke scoring process is activated when a command for activating a processing program for executing the karaoke scoring process is input.
In the karaoke scoring process, when activated, as shown in FIG. 4, the control unit 50 first stores the music data MD corresponding to the music specified via the input receiving unit 34 in the information processing server 10. Obtained from the storage unit 14 (S510). Subsequently, the control unit 50 extracts accompaniment data included in the music data MD acquired in S510 (S520).

  In the karaoke scoring process, the control unit 50 reproduces the accompaniment data and plays the music (S530). Specifically, in S530, the control unit 50 outputs the accompaniment data to the music reproducing unit 36, and the music reproducing unit 36 that has acquired the accompaniment data reproduces the music. Then, a sound source signal of the music reproduced by the music reproducing unit 36 is output to the speaker 60 via the output unit 42. Then, the speaker 60 outputs the sound source signal instead of sound.

  Further, in the karaoke scoring process, the control unit 50 acquires the voice input through the microphone 62 and the microphone input unit 44 as song data (S540). And the control part 50 stores the song data acquired in S540 in the memory | storage part 38 (S550).

  Subsequently, in the karaoke scoring process, the control unit 50 determines whether or not the performance of the music has ended (S560). If the result of this determination is that the music performance has not ended (S560: NO), the control unit 50 returns the karaoke scoring process to S540. On the other hand, if the result of determination in S560 is that the music has been played (S560: YES), the control unit 50 shifts the karaoke scoring process to S570.

  In S <b> 570, the control unit 50 acquires all song data stored in the storage unit 38. And the control part 50 extracts each song waveform of the area (henceforth a "musical note song area") which sang each note which comprises a song melody from the song data along the time-axis in a music (S580). . The note singing section may be specified by the same method as S140 in the standard feature amount calculation process.

  Subsequently, in the karaoke scoring process, the control unit 50 calculates a skill feature amount (hereinafter referred to as “singing feature amount”) that evaluates the singing skill of the song data (S590). The singing feature value calculation method is the same as S150 and S160 in the standard feature value calculation process, except that “vocal data” is replaced with “singing data”, and detailed description thereof will be omitted.

Further, in the karaoke scoring process, the control unit 50 calculates a reference evaluation score (S600). In the calculation of the reference evaluation score in S600, the control unit 50 first collates the pitch transition of the singing waveform in each note singing section with the pitch transition in the reference data. And the control part 50 calculates the reference | standard evaluation score made into the high score, so that a matching degree is high as a result of collation.

  Subsequently, in the karaoke scoring process, the control unit 50 calculates a skill evaluation score (S610). In the calculation of the skill evaluation score in S610, the control unit 50 first averages the singing feature amount calculated in S590 for each singing skill. Subsequently, the control unit 50 calculates a skill evaluation score according to the following formula.

Technical evaluation score = α × Σ (weight × direction × (singing feature value−average value of music feature value) / standard deviation of music feature value)
However, in the formula for obtaining the skill evaluation score, the object of Σ is a singing skill, and the direction in the formula is “1” or “−1”. In addition, the symbol α in the formula for obtaining the skill evaluation point is a weight given to the skill evaluation point, and is a predetermined constant.

  In other words, the formula for obtaining the skill evaluation point is obtained by adding a weight to the difference between the singing feature value and the average value of the song feature value and giving a greater weight to the singing skill that strongly expresses the feature of the song. In addition, the method for obtaining the skill evaluation score is not limited to this, and may be a weighted average in which a larger weight is given to a singing technique that strongly expresses the feature of the music.

  Further, in the karaoke scoring process, the control unit 50 calculates the overall evaluation score by adding the skill evaluation score calculated in S610 to the reference evaluation score calculated in S600 (S620). And the control part 50 displays the comprehensive evaluation score calculated in S620 on the display part 64 (S630). The display in S630 is realized by the control unit 50 outputting a control signal to the display unit 64 via the video control unit 46. Note that the evaluation points displayed on the display unit 64 are not limited to the overall evaluation points, and may include at least one of a reference evaluation point and a skill evaluation point.

Thereafter, the karaoke scoring process is terminated, and the system waits until the next activation timing.
That is, in the karaoke scoring process, the control unit 50 stores the voice input during the performance of the music as singing data. In the karaoke scoring process, the control unit 50 analyzes the stored singing data and calculates the singing feature amount. Further, in the karaoke scoring process, the control unit 50 performs a skill by weighted calculation in which a weight is given so that a larger score is obtained as the characteristic skill in the music is strongly expressed based on the weight as the evaluation data. An evaluation score is calculated.

Further, in the karaoke scoring process, the control unit 50 collates the pitch transition of the singing waveform in each note singing section with the pitch transition in the reference data, and calculates a reference evaluation score having a higher score as the matching degree is higher. . And the control part 50 calculates the sum total of a skill evaluation score and a reference | standard evaluation score as a comprehensive evaluation score.
[Effect of the embodiment]
As described above, according to the karaoke system 1, the evaluation data can be automatically generated by analyzing the music data MD.

Moreover, according to the karaoke system 1, it is possible to generate evaluation data including a weight that becomes a larger value as a characteristic singing technique in music.
And according to the karaoke system 1, singing can be evaluated using such evaluation data, and it can be evaluated whether the singing technique used when the user sang music is a characteristic technique.

Moreover, in the karaoke scoring process of this embodiment, the skill evaluation score is implemented so that the higher the difference between the singing feature value and the average value of the song feature value, the higher the evaluation.
That is, it can be said that as the difference between the singing feature quantity and the average value of the song feature quantity is larger, the characteristic technique is more strongly expressed in the song.

For this reason, according to the karaoke scoring process of this embodiment, the song which strongly expressed the characteristic technique in the music can be highly evaluated. Therefore, according to the karaoke system 1, it can reduce that the person who listened to the user's song feels strange about an evaluation result.
[Other Embodiments]
As mentioned above, although embodiment of this invention was described, this invention is not limited to the said embodiment, In the range which does not deviate from the summary of this invention, it is possible to implement in various aspects.

  For example, in S270 in the evaluation data generation process of the above embodiment, the weights wvib, wtt, wrise, wfall, and wkob are expressed as characteristic singing techniques that satisfy a predetermined condition in the technical feature quantities. , Is not limited to weights.

  That is, in the present invention, a singing technique that maximizes the evaluation values dviv, dtt, drise, dfall, dkob may be specified as the characteristic singing technique. In this case, even if the control part 50 produces | generates the information which matched the average value and standard deviation of the music feature-value about characteristic singing technique, the direction from a standard feature-value, and music identification information as evaluation data. good.

  In the above embodiment, the karaoke device 30 executes the standard feature quantity calculation process, the evaluation data generation process, and the karaoke scoring process. However, the standard feature quantity calculation process, the evaluation data generation process, and the karaoke scoring process are executed. The device is not limited to the karaoke device 30. For example, the information processing server 10 or another information processing device may be used as the device that executes the standard feature amount calculation process, the evaluation data generation process, and the karaoke scoring process.

In addition, the aspect which abbreviate | omitted a part of structure of the said embodiment as long as the subject could be solved is also embodiment of this invention. Further, an aspect configured by appropriately combining the above embodiment and the modification is also an embodiment of the present invention. Moreover, all the aspects which can be considered in the limit which does not deviate from the essence of the invention specified by the wording described in the claims are the embodiments of the present invention. [Correspondence between Embodiment and Claims]
Finally, the relationship between the description of the above embodiment and the description of the scope of claims will be described.

  The function obtained by executing S210 in the evaluation data generation process of the above embodiment corresponds to the music data acquisition means described in the claims, and the function obtained by executing S220 corresponds to the extraction means. To do. The function obtained by executing S240 to S260 in the evaluation data generation process corresponds to the first determining means in the claims, and the function obtained by executing S270 and S280 is the second determining means. It corresponds to.

  And the function obtained by performing the karaoke process of the said embodiment is equivalent to the execution means in description of a claim, and the function obtained by performing S540 of karaoke process is equivalent to a song acquisition means. The function obtained by executing S530 corresponds to the performance means.

  Further, the function obtained by executing S110 to S160 in the standard feature amount calculation processing of the above embodiment corresponds to the standard calculation means in the description of the claims, and the function obtained by executing S180 is The function corresponding to the evaluation information generating means and the function obtained by executing S190 corresponds to the storage control means.

DESCRIPTION OF SYMBOLS 1 ... Karaoke system 10 ... Information processing server 12 ... Communication part 14 ... Memory | storage part 16 ... Control part 18 ... ROM 20 ... RAM 22 ... CPU 30 ... Karaoke apparatus 32 ... Communication part 34 ... Input reception part 36 ... Music reproduction part 38 ... Storage unit 40 ... Audio control unit 42 ... Output unit 44 ... Microphone input unit 46 ... Video control unit 50 ... Control unit 52 ... ROM 54 ... RAM 56 ... CPU 60 ... Speaker 62 ... Microphone 64 ... Display unit

Claims (8)

Music data acquisition means for acquiring the music data from the first storage unit storing music data including the sung singing sound;
Extraction means for extracting vocal data representing the sung sound from the music data acquired by the music data acquisition means;
A first determining means for determining a technique feature amount representing an evaluation of a plurality of singing techniques for the vocal data extracted by the extracting means;
Based on the evaluation information stored in the second storage unit that stores the vocal data of a plurality of pieces of music and the evaluation data evaluated about the plurality of singing techniques for the vocal data of the plurality of pieces of music. A second deciding means for deciding a characteristic singing technique that satisfies a predetermined condition among the skill features determined by the first deciding means;
Execution means for performing evaluation of singing on the music using the characteristic singing technique determined by the second determining means;
An information processing apparatus comprising: Singing acquisition means for acquiring singing data representing the voice input during the performance of the music,
The execution means includes
2. The information processing according to claim 1, wherein the song data acquired by the song acquisition unit is subjected to song evaluation using a characteristic song technique determined by the second determination unit. 3. apparatus. The extraction means extracts the vocal data and accompaniment data representing an accompaniment sound in the music from the music data,
The information processing apparatus further includes:
Based on the accompaniment data extracted by the extraction means, comprising performance means for playing the music,
The information processing apparatus according to claim 2, wherein the singing acquisition unit acquires, as the singing data, a voice input during the performance of a music piece by the performance unit. The execution means includes
The difference between the skill feature amount in the singing data and the characteristic singing skill is derived, and the larger the derived difference is, the higher the evaluation of the song is. The information processing apparatus according to claim 1. The first determining means determines the technique feature amount for each singing technique,
The second determining means determines the characteristic singing technique for each singing technique,
The execution means includes
The difference between the technical feature amount in the singing data and the characteristic singing skill is derived for each singing skill, the singing technique that strongly expresses the feature in the music is given a greater weight, and the derived difference is weighted averaged The information processing apparatus according to claim 4, wherein the singing of the music is evaluated. A standard calculation means for calculating a standard feature amount representing a standard evaluation of a singing skill for vocal data of a plurality of songs;
Evaluation information generation means for numerically processing the standard feature amount derived by the standard calculation means, and calculating a result of the numerical processing as the evaluation information;
6. A storage control unit that stores the evaluation information calculated by the evaluation information generation unit in association with vocal data of a plurality of music pieces and stores it in the second storage unit. The information processing apparatus according to any one of the above. A music data acquisition process for acquiring the music data from the first storage unit storing the music data including the sung singing sound;
An extraction process for extracting vocal data representing the sung sound from the music data acquired by the music data acquisition process;
A first determination step for determining a technique feature amount representing an evaluation of a plurality of singing techniques for the vocal data extracted by the extraction process;
Based on the evaluation information stored in the second storage unit that stores the vocal data of a plurality of pieces of music and the evaluation data evaluated about the plurality of singing techniques for the vocal data of the plurality of pieces of music. A second determination process for determining a feature singing technique that satisfies a predetermined condition among the skill features determined in the first determination process;
Using the characteristic singing technique determined in the second determining process, performing an evaluation of singing the music;
A data generation method comprising: A music data acquisition procedure for acquiring the music data from the first storage unit storing the music data including the sung singing sound;
Extraction procedure for extracting vocal data representing the singing sound from the song data acquired by the song data acquisition procedure;
A first determination procedure for determining a technique feature amount representing an evaluation of a plurality of singing techniques for the vocal data extracted by the extraction procedure;
Based on the evaluation information stored in the second storage unit that stores the vocal data of a plurality of pieces of music and the evaluation data evaluated about the plurality of singing techniques for the vocal data of the plurality of pieces of music. A second determination procedure for determining a feature singing technique that satisfies a predetermined condition among the skill features determined by the first determination procedure;
A program for causing a computer to execute an execution procedure for performing evaluation of singing a song using the characteristic singing technique determined by the second determination procedure.

Images