JP2015069083A - Information processing device, data generation method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique for generating evaluation data.SOLUTION: In evaluation data generation processing, a control unit 50 extracts vocal data representing voice sound from musical piece data (S230), and determines a skill characteristic amount representing evaluation of a plurality of singing skills for each note section which is a predetermined note section composing the musical piece data (S240). Furthermore, in evaluation data generation processing, a skill characteristic amount which satisfies a predetermined condition is determined among the skill characteristic amounts in the respective note sections, and then data in which a note section corresponding to the determined skill characteristic amount is associated with the determined skill characteristic amount is generated as evaluation data used for singing evaluation (S280).

Description

  The present invention relates to an information processing apparatus that generates evaluation data, a data generation method, and a program.

  Conventionally, in a karaoke apparatus, a score obtained by adding an additional score obtained by evaluating a singing technique used during singing to a reference score scored based on a pitch transition in a singing voice has been calculated as an evaluation score ( Patent Document 1).

  In such a karaoke device, the reference data that represents the melody to be sung and prepared in advance for each song and the difference between the pitch transitions in the voice when singing the song is smaller, the higher the score Calculate the score. Furthermore, in the karaoke apparatus described in Patent Document 1, the singing voice is analyzed to detect various singing techniques used during the singing, and the larger the number of times the singing technique is used, the larger the added value is calculated. doing.

JP 2007-233303 A

In general pop music, the type of singing technique (hereinafter referred to as “feature technique”) used as a center when singing a music differs depending on the genre of the music or the singer.
For this reason, in the karaoke apparatus described in Patent Document 1, it is conceivable that an additional score is given when the singing technique detected from the singing voice matches the evaluation data generated in advance. As the evaluation data referred to here, for example, it is conceivable that the content of the singing technique to be used when singing a song is associated with the timing at which the singing technique is to be used. Such evaluation data usually needs to be generated in advance for each piece of music by a human hand, and the conventional technology has a problem that it is difficult to automatically generate evaluation data based on music data. It was.

  Therefore, an object of the present invention is to provide a technique for generating evaluation data.

In order to achieve the above object, the present invention is an information processing apparatus including music data acquisition means, extraction means, determination means, and generation means.
The music data acquisition means in this invention acquires music data from the 1st memory | storage part in which the music data containing the sung singing sound were memorize | stored. The extraction means extracts vocal data representing the sung singing sound from the music data acquired by the music data acquisition means.

  Further, the determining means determines a technique feature amount representing an evaluation of a plurality of singing techniques for each musical note section which is a predetermined musical note section constituting the music data for the vocal data extracted by the extracting means. Then, the generating means determines the skill feature amount satisfying a predetermined condition among the skill feature quantities of the note interval determined by the determination means, and the note interval corresponding to the determined skill feature amount is determined. Data in which the technical feature quantity is associated is generated as evaluation data used for singing evaluation.

According to such an information processing apparatus, evaluation data corresponding to music data can be automatically generated.
Therefore, in the present invention, evaluation data for music can be automatically generated even for music data of music composed by a general human.

The information processing apparatus of the present invention may further include a song acquisition unit and an evaluation unit.
The singing acquisition means in the present invention acquires singing data representing the voice input during the performance of the music. An evaluation means evaluates the singing skill in the input audio | voice using the evaluation data produced | generated by the production | generation means about the song data acquired by the song acquisition means.

According to such an information processing apparatus, the singing skill in the singing voice singing the music can be evaluated.
Furthermore, the extracting means in the present invention may extract vocal data and accompaniment data representing accompaniment sounds in the music from the music data.

  In this case, the information processing apparatus of the present invention may further include performance means for playing music based on the accompaniment data extracted by the extraction means. And a song acquisition means may acquire the audio | voice input during the performance of the music by the performance means as song data.

According to such an information processing apparatus, music can be played based on the music data stored in the first storage unit, and the singing skill in the voice (singing voice) input during the performance can be evaluated.
Further, the generation means in the present invention, the technical features corresponding to the singing technique used characteristically in the music among the technical features in the note interval determined by the determining means, the technical features that satisfy a predetermined condition It may be determined as an amount.

According to such an information processing apparatus, the technique feature quantity corresponding to the singing technique used characteristically in the music can be determined as the technique feature quantity satisfying a predetermined condition.
As a result, if singing is evaluated using an information processing device created by such an information processing device, it can be evaluated so as to match the impression that many people have with respect to the music.

  The information processing apparatus according to the present invention includes a standard acquisition unit that acquires a standard feature amount from a second storage unit that stores a standard feature amount representing a standard evaluation of a singing technique used in a plurality of songs. Also good.

  In this case, the generation means according to the present invention, if the difference between the technical feature amount in the note interval determined by the determination means and the standard feature amount acquired by the standard acquisition means is outside the reference range, the technical skill that falls outside the reference range. The feature amount may be determined as a skill feature amount that satisfies a predetermined condition.

According to such an information processing apparatus, evaluation data can be generated based on the difference between the standard feature value and the skill feature value.
Further, the generation means in the present invention may include distribution calculation means for calculating a feature quantity distribution obtained by tabulating the technical feature quantities in the note section determined by the determination means for each pitch and note value in the note section. good.

  In this case, in the feature amount distribution calculated by the distribution calculating unit, the generation unit, when included in a significant range as a feature in the music, sets the technical feature amount included in the significant range as a technical feature amount satisfying a predetermined condition. You may decide.

According to such an information processing apparatus, data necessary for generating evaluation data can be only vocal data in music.
By the way, this invention may be made | formed as a data generation method which produces | generates evaluation data.

  The data generation method in this case includes a music data acquisition process for acquiring music data from the first storage unit, an extraction process for extracting vocal data from the acquired music data, and music data for the extracted vocal data A predetermined condition is satisfied in the determination process for determining the technique feature quantity for a plurality of singing techniques for each note section, which is a predetermined note section that constitutes, and the technique feature quantity of the determined note section And a generation process for determining a skill feature amount and generating data associating the note interval corresponding to the determined skill feature amount and the determined skill feature amount as evaluation data used for singing evaluation. May be.

According to such a data generation method, an effect similar to that of the information processing apparatus according to claim 1 can be obtained.
Further, the present invention may be made as a program executed by a computer.

  The program in this case constitutes music data for the music data acquisition procedure for acquiring music data from the first storage unit, the extraction procedure for extracting vocal data from the acquired music data, and the extracted vocal data A determination procedure for determining a technique feature amount for a plurality of singing techniques for each note section, which is a predetermined note section, and a technique feature that satisfies a predetermined condition among the determined technique feature quantities of the note section The amount is determined, and the computer is caused to execute a generation procedure for generating data associating the note interval corresponding to the determined skill feature amount and the determined skill feature amount as evaluation data used for singing evaluation .

  If the present invention is implemented as a program, it can be used by loading it into a computer from a recording medium as necessary and starting it, or by acquiring it and starting it through a communication line as necessary. And by making a computer perform each procedure, the computer can be functioned as an information processing apparatus described in claim 1.

  The recording medium referred to here includes, for example, a computer-readable electronic medium such as a DVD-ROM, a CD-ROM, and a hard disk.

It is a block diagram which shows schematic structure of the karaoke system containing the information processing apparatus with which this invention was applied. It is a flowchart which shows the process sequence of a standard feature-value calculation process. It is a flowchart which shows the process sequence of the evaluation data generation process in 1st embodiment. It is a flowchart which shows the process sequence of a karaoke scoring process. It is a flowchart which shows the process sequence of the evaluation data generation process in 2nd embodiment.

Embodiments of the present invention will be described below with reference to the drawings.
[First embodiment]
<Configuration of karaoke system>
The karaoke system 1 shown in FIG. 1 is a system in which a music specified by a user (user) is played and the user sings along with the performance.

  In order to realize this, the karaoke system 1 includes an information processing server 10 and at least one karaoke apparatus 30. The information processing server 10 and the karaoke apparatus 30 are connected via a communication network. The communication network referred to here may be a wired communication network or a wireless communication network.

  The information processing server 10 stores music data MD-1 to MD-N prepared for each music. The karaoke apparatus 30 acquires the music data MD corresponding to the music specified by the user (user) from the information processing server 10 and plays the music based on the music data MD. Accept input.

The code “N” is an identifier for identifying the music data MD, and “N” is a natural number of 2 or more.
<Information processing server>
The information processing server 10 includes a communication unit 12, a storage unit 14, and a control unit 16.

Among these, the communication unit 12 performs communication between the information processing server 10 and the outside via a communication network.
The control unit 16 is a known control device that is configured around a known microcomputer including a ROM 18, a RAM 20, and a CPU 22. The ROM 18 stores processing programs and data that need to retain stored contents even when the power is turned off. The RAM 20 temporarily stores processing programs and data. The CPU 22 executes each process (various calculations) in accordance with a processing program stored in the ROM 18 or the RAM 20.

That is, the control unit 16 controls each unit constituting the information processing server 10 and executes data communication with the karaoke apparatus 30.
The storage unit 14 is a known storage device configured to be able to read and write stored contents. The storage unit 14 stores at least a plurality of music data MD.

  The music data MD includes music management information in which information related to music is described, master waveform data representing the performance sound of the music, and lyric data representing the lyrics of the music. The music management information includes at least music identification information (for example, music number) for identifying music.

  The master waveform data of the present embodiment is sound data including performance sounds of a plurality of musical instruments and singing sounds singing the main melody. The audio data may be data constituted by an audio file in an uncompressed audio file format, or data constituted by an audio file in an audio compression format.

In the following description, the sound data representing the performance sound included in the master waveform data is referred to as accompaniment data, and the sound data indicating the singing sound included in the master waveform data is referred to as vocal data.
Musical instrument performance sounds included in the accompaniment data of the present embodiment include percussion instrument (eg, drum, drum, cymbal, etc.) performance sounds, stringed instrument (eg, guitar, bass, etc.) performance sounds, percussion instrument (eg, piano) ) And wind instruments (eg, trumpet, clarinet, etc.). In general music, a percussion instrument and a bass are usually rhythm instruments.

The music data MD stored in the storage unit 14 includes music data MD of music composed and written by a general user of the karaoke system 1 in addition to music data MD of music composed by a professional. The song data MD of the song composed / written by the general user is created by a known information processing terminal (for example, a personal computer or a portable terminal) and uploaded to the information processing server 10.
<Karaoke equipment>
The karaoke apparatus 30 includes a communication unit 32, an input reception unit 34, a music playback unit 36, a storage unit 38, an audio control unit 40, a video control unit 46, and a control unit 50.

  In the communication unit 32, the karaoke apparatus 30 communicates with the outside via a communication network. The input receiving unit 34 is an input device that receives input of information and commands in accordance with external operations. The input device in the present embodiment is, for example, a key, a switch, a remote control receiving unit, or the like.

  The music playback unit 36 plays back music based on the music data MD stored in the storage unit 38 or the music data MD downloaded from the information processing server 10. The voice control unit 40 is a device that controls voice input / output, and includes an output unit 42 and a microphone input unit 44.

  A microphone 62 is connected to the microphone input unit 44. Thereby, the microphone input part 44 acquires a user's song sound. A speaker 60 is connected to the output unit 42. The output unit 42 outputs the sound source signal of the music reproduced by the music reproducing unit 36 and the sound source signal of the singing sound from the microphone input unit 44 to the speaker 60. The speaker 60 outputs the sound source signal output from the output unit 42 instead of sound.

The video control unit 46 outputs video based on video data sent from the control unit 50. A display unit 64 for displaying video is connected to the video control unit 46.
The control unit 50 is configured around a known computer having at least a ROM 52, a RAM 54, and a CPU 56. The ROM 52 stores processing programs and data that need to retain stored contents even when the power is turned off. The RAM 54 temporarily stores processing programs and data. The CPU 56 executes each process (various calculations) in accordance with a processing program stored in the ROM 52 or the RAM 54.

  The ROM 52 includes a processing program for the control unit 50 to execute the karaoke scoring process, a processing program for executing the evaluation data generation process, and a processing program for executing the standard feature amount calculation process. Stored.

  The karaoke scoring process is a process of playing music designated by the user and evaluating the voice input through the microphone 62 during the performance period. The evaluation data generation process is a process of generating evaluation data necessary for the karaoke scoring process for each piece of music data MD. The standard feature quantity calculation process is a process for calculating a standard feature quantity used for generating evaluation data.

  That is, the karaoke apparatus 30 calculates standard feature values as evaluation information in accordance with the standard feature value calculation processing, and generates evaluation data for each piece of music data MD in accordance with the evaluation data generation processing. And the karaoke apparatus 30 performs a music based on the music data MD corresponding to an object music according to a karaoke scoring process, and acquires the audio | voice input via the microphone 62 as song data during the performance. Furthermore, the karaoke apparatus 30 scores and evaluates the acquired singing data according to the karaoke scoring process.

That is, the karaoke apparatus 30 functions as an information processing apparatus that executes standard feature value calculation processing, evaluation data generation processing, and karaoke scoring processing.
<Standard feature calculation processing>
Next, a standard feature amount calculation process executed by the control unit 50 of the karaoke apparatus 30 will be described.

  This standard feature amount calculation process is started at a predetermined time interval. Note that the start timing of the standard feature value calculation process is not limited to a predetermined time interval, and an input command for starting a processing program (application) for executing the standard feature value calculation process is input The timing input via 34 may be used.

  In the standard feature amount calculation process, as shown in FIG. 2, when activated, the control unit 50 firstly selects one piece of music data from all the music data MD stored in the information processing server 10. The MD is acquired (S110). Subsequently, the control unit 50 acquires master waveform data included in the music data MD acquired in S110 (S120).

  Further, in the standard feature amount calculation process, the control unit 50 separates the accompaniment data and the vocal data from the master waveform data acquired in S120, and extracts the accompaniment data and the vocal data (S130). In S <b> 130, a known method (for example, “PreFEst” described in JP-A-2008-134606) is conceivable as a method by which the control unit 50 separates accompaniment data and vocal data. Note that PreFEst is a method of separating the most prevalent voice waveform in the master waveform data from the master waveform data as vocal data and separating the remaining voice waveform as accompaniment data.

  Subsequently, in the standard feature value calculation process, the control unit 50 performs a musical score process on the vocal data extracted in S130 (S140). The music recording process in S140 is a well-known method of recording music based on the temporal change in sound pressure in vocal data and the temporal change in pitch in vocal data.

  That is, in the music recording process, the control unit 50 specifies the timing at which the temporal change of the sound pressure in the vocal data becomes equal to or greater than the specified threshold as the start timing nnt (a, i) of each note constituting the song melody in the music. . Further, in the music recording process, the control unit 50 specifies the timing at which the time change of the sound pressure in the vocal data is equal to or less than the specified threshold as the end timing nft (a, i) of each note constituting the song melody in the music. .

  In the music recording process, the control unit 50 specifies a section specified by the start timing nnt (a, i) and the end timing nft (a, i) corresponding to each other as a note section of each note. At the same time, in the music recording process, the control unit 50 specifies the pitch in each note interval based on the time change of the pitch in the vocal data, and the pitch nn (a, i) in each note interval and the note interval. Is associated.

In addition, the code | symbol a is a code | symbol which identifies a music, and the code | symbol i is a code | symbol which identifies the musical note area of the song melody in a music.
In the standard feature value calculation process, the control unit 50 further determines a skill feature value representing an evaluation of a plurality of singing skills for each note section in the music (S150). The plurality of singing techniques mentioned here include “vibrato”, “for”, “shrimp”, “fall”, and “fist”.

  Among these, in calculating the technical feature amount (hereinafter referred to as “vibrato feature amount”) vib (a, i) for “vibrato”, the control unit 50 firstly calculates a speech waveform corresponding to each note interval from vocal data. , And frequency analysis (DFT) is performed on the speech waveform of each note interval. And the control part 50 calculates the vibrato feature-value vib (a, i) according to following (1) Formula.

ただし、上記（１）式におけるｖｉｂ＿ｐｅｒ（ａ，ｉ）は、各音符区間の音声波形におけるスペクトルピークの突出精度を表す指標である。このｖｉｂ＿ｐｅｒ（ａ，ｉ）は、周波数解析結果（即ち、振幅スペクトル）のピーク値を、周波数解析結果の平均値で除すことで求めれば良い。また、上記（１）式におけるｖｉｐ＿ｄｅｐ（ａ，ｉ）は、各音符区間の音声波形の標準偏差である。

However, vib_per (a, i) in the above equation (1) is an index representing the protruding accuracy of the spectrum peak in the speech waveform of each note interval. The vib_per (a, i) may be obtained by dividing the peak value of the frequency analysis result (that is, the amplitude spectrum) by the average value of the frequency analysis result. Also, vip_dep (a, i) in the above equation (1) is the standard deviation of the speech waveform in each note interval.

  In calculating the technical feature amount (hereinafter referred to as “for feature amount”) tt (a, i) for “for”, the control unit 50 first converts the sound waveform of the non-harmonic component in the accompaniment data to the rhythm instrument. Extract as performance sound waveform. As a method for extracting the non-harmonic component, the accompaniment sound data is passed through a filter prepared in advance as a filter representing the sound waveform of the non-harmonic component, or the harmonic sound based on the anisotropy of the spectrogram smoothness is used. It is conceivable to use the method described in “Separation of percussion instrument sounds” (Proceedings of the Spring Meeting of the Acoustical Society of Japan, 2-5-8, p.903-904 (2008.03)).

  Further, in calculating the feature quantity tt (a, i), the control unit 50 estimates the timing at which the sound pressure becomes equal to or higher than a specified value in the performance sound waveform of the rhythm instrument as the beat position. Subsequently, the control unit 50 extracts a note having the shortest note value (hereinafter referred to as “shortest note”) from the notes constituting the song melody of the music. Then, the control unit 50 specifies the utterance timing by dividing the interval between the beat positions by the note value of the extracted shortest note. The utterance timing here is a timing at which singing may be started for each note i.

  Therefore, in the calculation of the feature quantity tt (a, i), the control unit 50 further specifies an utterance timing that satisfies a specified condition. Satisfying the specified condition here is an utterance timing that is later than the start timing nnt (a, i), and that the absolute value of the value subtracted from the start timing nnt (a, i) is minimum. It is. Then, a time length obtained by subtracting the specified utterance timing from the start timing nnt (a, i) is calculated as a feature quantity tt (a, i).

  In calculating the skill feature amount (hereinafter referred to as “shackle feature amount”) rise (a, i) for “shrimp”, the control unit 50 first derives a differential change obtained by differentiating the pitch time change of vocal data. To do. Subsequently, the control unit 50 specifies the timing at which the differential change becomes a positive value along the time axis before the start timing nnt (a, i) of each note interval. Further, the control unit 50 obtains the cross-correlation value between the pitch time change of the vocal data and the predefined exemplary curve in the section from the identified timing to the start timing nnt (a, i), and the scribing feature amount rise. Derived as (a, i).

  In calculating the technical feature amount (hereinafter referred to as “fall feature amount”) fall (a, i) for “fall”, the control unit 50 performs differentiation after the end timing nft (a, i) of each note interval. The first timing when the change becomes a positive value along the time axis is specified. Further, the control unit 50 changes the pitch time of the vocal data in the section from the end timing nft (a, i) of each note section constituting the singing melody to the specified timing, and a prescribed model curve. Are derived as the fall feature value fall (a, i).

  In calculating the technical feature amount (hereinafter referred to as “fist feature amount”) kob (a, i) for “fist”, the control unit 50 first specifies a fist section. The fist section here is a section in which each note section over a plurality of pitches is sung while changing the pitch with the same vowel.

  For this reason, in calculating the fist feature value kob (a, i), the control unit 50 identifies a section uttered by the same vowel (hereinafter referred to as “same vowel section”). As a method for specifying the same vowel section, a method is used in which a cross-correlation of average values of Mel frequency cepstrum (MFCC) of each note section is derived and a note section having a cross-correlation value equal to or greater than a threshold is specified as the same vowel section.

  In addition, the control unit 50 identifies only the same vowel section that satisfies the setting condition as the fist section in the same vowel section. Satisfying the setting condition here means that the time interval between the end timing nft (a-1, i) and the start timing nnt (a, i) of the note intervals adjacent to each other along the time axis is equal to or less than a threshold value, and The pitches of adjacent note intervals are all different.

  In calculating the fist feature value kob (a, i), the control unit 50 calculates a chroma vector from the vocal waveform in the fist section. Further, the control unit 50 calculates a chroma vector of accompaniment data in the same fist section, and calculates a cross-correlation value with the chroma vector of the vocal waveform as a fist feature value kob (a, i).

  In the standard feature value calculation process, the control unit 50 calculates a music feature value as evaluation information evaluated for a plurality of singing techniques (S160). The music feature value is an average value in the music for each of the vibrato feature value vib, the feature value tt, the shawl feature value rise, the fall feature value fall, and the fist feature value kob. However, in S160, the control part 50 calculates the music feature-value of each singing skill for every combination of the note value and pitch of the note interval which comprises a song melody.

  In the standard feature value calculation process, the control unit 50 subsequently determines whether or not the process until the music feature value is calculated (ie, S110 to S160) has been executed for all the music data MD ( S170). As a result of the determination in S170, if the process until calculating the music feature amount is not executed for all the music data MD (S170: NO), the control unit 50 performs the standard feature amount calculation process in S110. Return to. And the control part 50 acquires the music data MD which has not performed the process from the information processing server 10, and transfers to S120.

  On the other hand, as a result of the determination in S170, if the process until calculating the music feature amount is executed for all the music data MD (S170: YES), the control unit 50 performs the standard feature amount calculation process. Proceed to S180.

  In S180, the control unit 50 calculates, as standard feature values, the average value and standard deviation of all the song data of the song feature values calculated for each combination of note value and pitch of the note interval. To do. In S180, the control unit 50 calculates the standard feature value for each combination of note value and pitch of the note interval.

  In the standard feature amount calculation process, the control unit 50 uploads the standard feature amount calculated for each combination of note value and pitch of the note interval calculated in S180 to the storage unit 14 of the information processing server 10 (S190). ).

Thereafter, the standard feature amount calculation process is terminated.
That is, in the standard feature value calculation process of the present embodiment, the control unit 50 calculates a skill feature value representing evaluation of each singing skill for vocal data of a plurality of music pieces. Further, in the standard feature value calculation process, the control unit 50 calculates an average value and a standard deviation obtained by averaging the calculated skill feature value for each combination of the note value and pitch of the note melody. Ask. In the standard feature value calculation process, the control unit 50 stores the standard feature value obtained for each note value and pitch in the note section in the storage unit 14 of the information processing server 10.
<Evaluation data generation process>
Next, an evaluation data generation process executed by the control unit 50 of the karaoke apparatus 30 will be described.

The evaluation data generation process is started when a start command for executing the evaluation data generation process is input.
Then, in the evaluation data generation process, as shown in FIG. 3, when activated, the control unit 50 firstly selects one piece of music data MD from all the music data MD stored in the information processing server 10. Is acquired (S210). Subsequently, the control unit 50 acquires master waveform data included in the music data MD acquired in S210 (S220).

  Further, in the evaluation data generation process, the control unit 50 separates and extracts accompaniment data and vocal data from the master disk waveform data acquired in S220 (S230). The technique for separating accompaniment data and vocal data in S230 may be the same technique as in S130 in the standard feature value calculation process.

  Subsequently, in the evaluation data generation process, the control unit 50 performs a musical score process on the vocal data extracted in S230 (S240). The musical score processing in S240 may use the same method as the musical score processing in S140 of the standard feature amount calculation process.

  In the evaluation data generation process, the control unit 50 further determines each technical feature amount of the vocal data extracted in S230 for each note section in the music (S250). As a technique for determining the skill feature amount in S250, a method similar to S150 in the standard feature amount calculation process may be used.

  In the evaluation data generation process, the control unit 50 acquires a standard feature amount stored in the storage unit 14 of the information processing server 10 (S260). Subsequently, the control unit 50 calculates the feature amount distance for each singing skill (S270).

  In S270, specifically, the control unit 50 calculates the feature value by dividing the absolute value of the difference between the technical feature value determined in S250 and the average value in the standard feature value by the standard deviation in the standard feature value. Calculate as distance. However, in S270, the control unit 50 calculates the feature amount distance for each combination of the note value and pitch of the note interval and for each singing skill.

  Further, in the evaluation data generation process, the control unit 50 generates evaluation data (S280). In S280, the control unit 50 first specifies the singing technique corresponding to the feature distance that satisfies the specific condition for each note interval. Satisfying the specific condition mentioned here means that the feature amount distance is equal to or greater than a predetermined reference value among the feature amount distances of each singing technique calculated in S270.

  Then, the control unit 50 generates, as evaluation data, information that associates each technical feature amount (hereinafter referred to as “specific feature amount”) that satisfies the specific feature with each note interval corresponding to the specific feature amount. .

  That is, in S270, when the difference between the technical feature amount and the standard feature amount in each note section constituting the singing melody is outside the reference range, the control unit 50 determines the technical feature amount outside the reference range, It is determined as a technical feature amount (that is, a specific feature amount) that satisfies a predetermined condition. And the control part 50 produces | generates the information which matched the note area in which the characteristic song technique is used in the note area, and the technique feature-value of the characteristic song technique as evaluation data.

  Subsequently, in the evaluation data generation process, the control unit 50 generates the result of the music transcription process in S240 as reference data (S290). The reference data referred to here is data representing note intervals (that is, pitches and note values) constituting a melody to be sung.

  In the evaluation data generation process, the control unit 50 uploads the music identification information, the evaluation data, and the reference data in association with each other to the storage unit 14 of the information processing server 10 (S300).

Thereafter, the evaluation data generation process ends.
That is, in the evaluation data generation process, the control unit 50 generates evaluation data that evaluates the singing technique for each note section in which a characteristic singing technique is used in the music. Further, in the evaluation data generation process, the control unit 50 generates reference data and stores it in the storage unit 14 of the information processing server 10 together with the evaluation data.
<Karaoke scoring>
Next, the karaoke scoring process which the control part 50 of the karaoke apparatus 30 performs is demonstrated.

This karaoke scoring process is started when a command for starting a processing program for executing the karaoke scoring process is input via the input receiving unit 34.
In the karaoke scoring process, when activated, as shown in FIG. 4, the control unit 50 first stores the music data MD corresponding to the music specified via the input receiving unit 34 in the information processing server 10. Obtained from the storage unit 14 (S510). Subsequently, the control unit 50 extracts accompaniment data included in the music data MD acquired in S510 (S520).

  In the karaoke scoring process, the control unit 50 reproduces the accompaniment data and plays the music (S530). Specifically, in S530, the control unit 50 outputs the accompaniment data to the music reproducing unit 36, and the music reproducing unit 36 that has acquired the accompaniment data reproduces the music. Then, a sound source signal of the music reproduced by the music reproducing unit 36 is output to the speaker 60 via the output unit 42. Then, the speaker 60 outputs the sound source signal instead of sound.

  Further, in the karaoke scoring process, the control unit 50 acquires the voice input through the microphone 62 and the microphone input unit 44 as song data (S540). And the control part 50 stores the song data acquired in S540 in the memory | storage part 38 (S550).

  Subsequently, in the karaoke scoring process, the control unit 50 determines whether or not the performance of the music has ended (S560). If the result of this determination is that the music performance has not ended (S560: NO), the control unit 50 returns the karaoke scoring process to S540. On the other hand, if the result of determination in S560 is that the music has been played (S560: YES), the control unit 50 shifts the karaoke scoring process to S570.

  In S <b> 570, the control unit 50 acquires all song data stored in the storage unit 38. And the control part 50 extracts each song waveform of the area (henceforth a "musical note song area") which sang each note which comprises a song melody from the song data along the time-axis in a music (S580). . The note singing section may be specified by the same method as S140 in the standard feature amount calculation process, except that “vocal data” is replaced with “singing data”.

Subsequently, in the karaoke scoring process, the control unit 50 calculates a skill feature amount (hereinafter referred to as “singing feature amount”) that evaluates the singing skill of the song data (S590). The singing feature value calculation method is the same as S150 and S160 in the standard feature value calculation process, except that “vocal data” is replaced with “singing data”, and detailed description thereof will be omitted.

  Further, in the karaoke scoring process, the control unit 50 calculates a reference evaluation score (S600). In the calculation of the reference evaluation score in S600, the control unit 50 collates the pitch transition of the singing waveform in each note singing section with the pitch transition in the reference data. And the control part 50 calculates the reference | standard evaluation score made into high score, so that a matching degree is high as a result of collation.

  Subsequently, in the karaoke scoring process, the control unit 50 calculates a skill evaluation score (S610). In the calculation of the skill evaluation score in S610, the control unit 50 first acquires the standard feature value and the evaluation data of the music data MD acquired in S510. Then, based on the singing feature value calculated in S590, the acquired evaluation data, and the standard feature value, the control unit 50 calculates an evaluation score for each note according to the following formula for each note interval.

Evaluation score for each note = α × direction × (average value in singing feature value−standard feature value) / standard deviation in standard feature value However, in the above formula for calculating the evaluation score for each note, the direction is included in the evaluation data. It is positive or negative in the difference between the specific feature value and the singing feature value, and is “1” or “−1”.

Further, the symbol α in the equation for obtaining the evaluation score for each note is a weight of the score for the singing skill, and is a predetermined constant.
Furthermore, in the calculation of the skill evaluation score, the control unit 50 calculates the average value of the evaluation scores for each note over the entire music as the skill evaluation score.

  Further, in the karaoke scoring process, the control unit 50 calculates the overall evaluation score by adding the skill evaluation score calculated in S610 to the reference evaluation score calculated in S600 (S620). And the control part 50 displays the comprehensive evaluation score calculated in S620 on the display part 64 (S630). The display in S630 is realized by the control unit 50 outputting a control signal to the display unit 64 via the video control unit 46. Note that the evaluation points displayed on the display unit 64 are not limited to the overall evaluation points, and may include at least one of a reference evaluation point and a skill evaluation point.

Thereafter, the karaoke scoring process is terminated, and the system waits until the next activation timing.
That is, in the karaoke scoring process, the control unit 50 stores the voice input during the performance of the music as singing data. In the karaoke scoring process, the control unit 50 analyzes the stored singing data and calculates the singing feature amount. Further, in the karaoke scoring process, the control unit 50 calculates the skill evaluation score so that the higher the characteristic skill in the music, the higher the score.

Further, in the karaoke scoring process, the control unit 50 collates the pitch transition of the singing waveform in each note singing section with the pitch transition in the reference data, and calculates a reference evaluation score having a higher score as the matching degree is higher. . And the control part 50 calculates the sum total of a skill evaluation score and a reference | standard evaluation score as a comprehensive evaluation score.
[Effect of the first embodiment]
As described above, according to the karaoke system 1, the evaluation data corresponding to the music data MD can be automatically generated.

Therefore, in the karaoke system 1, even if it is the music data MD of the music composed by a general human, evaluation data for the music can be automatically generated.
Furthermore, in the karaoke system 1, the singing technique used characteristically in the note interval which comprises a singing melody is specified as a characteristic technique. Then, in the karaoke scoring process, if the characteristic skill is strongly expressed in the voice in which the note interval is sung, a large skill evaluation score is added to the reference evaluation score.

As a result, if singing is evaluated using the karaoke system 1, it can be evaluated so as to match the impression many people have with respect to the music.
[Second Embodiment]
The karaoke system of the second embodiment is mainly different from the karaoke system 1 of the first embodiment in the content of the evaluation data generation process. For this reason, in the present embodiment, the same configurations and processes as those in the first embodiment are denoted by the same reference numerals, description thereof will be omitted, and evaluation data processing different from that in the first embodiment will be mainly described.
<Evaluation data generation process>
The evaluation data generation process in the present embodiment is started when a start command for executing the evaluation data generation process is input.

  In the evaluation data generation process, as shown in FIG. 5, when activated, the control unit 50 firstly selects one piece of music data MD from all the music data MD stored in the information processing server 10. Is acquired (S710). Subsequently, the control unit 50 acquires master waveform data included in the music data MD acquired in S710 (S720).

  Further, in the evaluation data generation process, the control unit 50 separates accompaniment data and vocal data from the master waveform data acquired in S720, and extracts accompaniment data and vocal data (S730). The technique for separating accompaniment data and vocal data in S730 may be the same technique as S130 in the standard feature value calculation process.

  Subsequently, in the evaluation data generation process, the control unit 50 performs a musical score process on the vocal data extracted in S730 (S740). The musical score processing in S740 may use the same method as the musical score processing in S140 of the standard feature amount calculation process.

  In the evaluation data generation process, the control unit 50 further determines each technical feature amount of the vocal data extracted in S730 for each note section in the music (S750). As a technique for determining the skill feature amount in S750, a method similar to S150 in the standard feature amount calculation process may be used.

  In the evaluation data generation process, subsequently, the control unit 50 calculates a feature amount distribution (S760). The feature amount distribution calculated in S760 is a distribution in which the technical feature amounts calculated in S750 are tabulated for each pitch and note value of the note interval constituting the singing melody. In S760, control part 50 computes feature-value distribution for every singing technique.

  Further, in the evaluation data generation process, the control unit 50 generates evaluation data (S770). In S770, the control unit 50 is a technique feature quantity included in a significant range as a feature in the music in the feature quantity distribution calculated in S760, and among the technique feature quantities for each singing technique in each note section. The technical feature amount that is the maximum is specified as the specific feature amount. And the control part 50 produces | generates the information which matched the specified specific feature-value and the note area corresponding to the specific feature-value as evaluation data.

Subsequently, in the evaluation data generation process, the control unit 50 generates the result of the music recording process in S740 as reference data (S780).
In the evaluation data generation process, the control unit 50 associates the music identification information, the evaluation data, and the reference data and uploads them to the storage unit 14 of the information processing server 10 (S790).

Thereafter, the evaluation data generation process ends.
[Effects of Second Embodiment]
According to the karaoke system 1 of the second embodiment, the data necessary for generating the evaluation data can be only vocal data in the music.
[Other Embodiments]
As mentioned above, although embodiment of this invention was described, this invention is not limited to the said embodiment, In the range which does not deviate from the summary of this invention, it is possible to implement in various aspects.

  In the above embodiment, the karaoke device 30 executes the standard feature quantity calculation process, the evaluation data generation process, and the karaoke scoring process. However, the standard feature quantity calculation process, the evaluation data generation process, and the karaoke scoring process are executed. The device is not limited to the karaoke device 30. For example, the information processing server 10 or another information processing device may be used as the device that executes the standard feature amount calculation process, the evaluation data generation process, and the karaoke scoring process.

  In the evaluation data generation process of the above embodiment, each specific feature amount and information associated with each note interval corresponding to the specific feature amount are generated as evaluation data. However, the evaluation data is limited to this. It is not a thing.

  For example, the evaluation data may include information on a note interval in which the skill feature amount is smaller than a predetermined reference threshold value, that is, information indicating a note interval in which it is inappropriate to use a predetermined singing skill. Thereby, the information showing the note area which prohibits using a singing technique can be included in evaluation data.

  The evaluation data generated by the evaluation data generation process is data representing the note interval in which the feature technique should be used, the contents of the feature technique, and the technique feature amount, but is generated by the evaluation data generation process. The evaluation data to be performed is not limited to this. The evaluation data may be, for example, data representing a weight to be given to each singing technique for each note interval.

In this case, in the karaoke scoring process, the control unit 50 may calculate the skill evaluation score according to the following procedure.
First, the control unit 50 calculates an evaluation score for each note according to the following formula.

Evaluation score for each note = α × Σ × (weight × direction × (singing feature value−average value in standard feature value) / standard deviation in standard feature value)
However, in the above formula for obtaining the evaluation score for each note, the object for obtaining the sum (that is, the object of sigma) is a singing technique. The direction is positive or negative in the difference between the specific feature amount and the singing feature amount included in the evaluation data, and is “1” or “−1”.

Further, the symbol α in the equation for obtaining the evaluation score for each note is a weight of the score for the singing skill, and is a predetermined constant.
Furthermore, in the calculation of the skill evaluation score, the control unit 50 calculates the average value of the evaluation scores for each note over the entire music as the skill evaluation score.

In addition, the aspect which abbreviate | omitted a part of structure of the said embodiment as long as the subject could be solved is also embodiment of this invention. Further, an aspect configured by appropriately combining the above embodiment and the modification is also an embodiment of the present invention. Moreover, all the aspects which can be considered in the limit which does not deviate from the essence of the invention specified by the wording described in the claims are the embodiments of the present invention. [Correspondence between Embodiment and Claims]
Finally, the relationship between the description of the above embodiment and the description of the scope of claims will be described.

  The function obtained by executing S210 and S710 in the evaluation data generation process of the above embodiment corresponds to the music data acquisition means described in the claims, and obtained by executing S220, S230, S720, and S730. The function to be performed corresponds to the extraction means. Further, the function obtained by executing S250 and S750 corresponds to the determining means, and the function obtained by executing S270, S280, S770, and S780 corresponds to the generating means.

  Furthermore, the function obtained by executing S540, S550, S570 in the karaoke scoring process of the above embodiment corresponds to the singing acquisition means in the description of the claims, and the function obtained by executing S610, Corresponds to evaluation means. The function obtained by executing S530 corresponds to a performance means.

  Note that the function obtained by executing S260 in the evaluation data generation process of the first embodiment corresponds to the standard acquisition means in the description of the claims, and executes S760 in the evaluation data generation process of the second embodiment. The function obtained by doing this corresponds to the distribution calculating means.

  DESCRIPTION OF SYMBOLS 1 ... Karaoke system 10 ... Information processing server 12 ... Communication part 14 ... Memory | storage part 16 ... Control part 18 ... ROM 20 ... RAM 22 ... CPU 30 ... Karaoke apparatus 32 ... Communication part 34 ... Input reception part 36 ... Music reproduction part 38 ... Storage unit 40 ... Audio control unit 42 ... Output unit 44 ... Microphone input unit 46 ... Video control unit 50 ... Control unit 52 ... ROM 54 ... RAM 56 ... CPU 60 ... Speaker 62 ... Microphone 64 ... Display unit

Claims (8)

Music data acquisition means for acquiring the music data from the first storage unit storing music data including the sung singing sound;
Extraction means for extracting vocal data representing the sung sound from the music data acquired by the music data acquisition means;
For vocal data extracted by the extraction means, a determination means for determining a skill feature amount representing an evaluation of a plurality of singing techniques for each note section that is a predetermined note section constituting the music data;
Among the technical features of the note interval determined by the determining means, determine a technical feature amount satisfying a predetermined condition, and a note interval corresponding to the determined technical feature amount, and the determined technical feature amount An information processing apparatus comprising: generating means for generating the associated data as evaluation data used for singing evaluation. Singing acquisition means for acquiring singing data representing voice input during the performance of the music;
An evaluation means for evaluating the singing skill in the input voice using the evaluation data generated by the generation means for the singing data acquired by the singing acquisition means. The information processing apparatus described in 1. The extraction means extracts the vocal data and accompaniment data representing an accompaniment sound in the music from the music data,
The information processing apparatus further includes:
Based on the accompaniment data extracted by the extraction means, comprising performance means for playing the music,
The information processing apparatus according to claim 2, wherein the singing acquisition unit acquires, as the singing data, a voice input during the performance of a music piece by the performance unit. The generating means includes
Of the skill features in the note interval determined by the determination means, a skill feature amount corresponding to a singing skill that is characteristically used in the music is determined as a skill feature amount satisfying the predetermined condition. The information processing apparatus according to any one of claims 1 to 3, wherein: Standard acquisition means for acquiring the standard feature value from a second storage unit storing a standard feature value representing a standard evaluation of the singing technique used in a plurality of songs,
The generating means includes
If the difference between the technical feature amount in the note interval determined by the determination unit and the standard feature amount acquired by the standard acquisition unit is outside the reference range, the technical feature amount outside the reference range is determined as the predetermined feature amount. The information processing device according to claim 4, wherein the information processing device is determined as a technique feature amount that satisfies the following condition. The generating means includes
Distribution calculating means for calculating a feature quantity distribution in which the technical feature quantities determined in the note section determined by the determining means are tabulated for each pitch and note value in the note section;
When the feature amount distribution calculated by the distribution calculating means is included in a significant range as a feature in music, the skill feature amount included in the significant range is determined as a skill feature amount satisfying the predetermined condition. The information processing apparatus according to claim 4. A music data acquisition process for acquiring the music data from the first storage unit storing the music data including the sung singing sound;
An extraction process for extracting vocal data representing the sung sound from the music data acquired by the music data acquisition process;
For the vocal data extracted by the extraction process, for each note interval that is a predetermined note interval constituting the music data, a determination step for determining a technique feature amount representing evaluation of a plurality of singing techniques;
Among the technical features of the note section determined by the determination process, a technical feature amount satisfying a predetermined condition is determined, and a note section corresponding to the determined technical feature amount, and the determined technical feature amount A data generation method comprising: generating the associated data as evaluation data used for singing evaluation. A music data acquisition procedure for acquiring the music data from the first storage unit storing the music data including the sung singing sound;
Extraction procedure for extracting vocal data representing the singing sound from the song data acquired by the song data acquisition procedure;
For the vocal data extracted by the extraction procedure, a determination procedure for determining a technique feature amount representing evaluation of a plurality of singing techniques for each note section that is a predetermined note section constituting the music data;
Among the technical features of the note section determined by the determination procedure, a technical feature amount satisfying a predetermined condition is determined, and a musical note section corresponding to the determined technical feature amount and the determined technical feature amount A program for causing a computer to execute a generation procedure for generating the associated data as evaluation data used for song evaluation.

Images