JP2013195738A - Singing evaluation device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To evaluate a sense of rhythm in user's singing while taking beats into consideration.SOLUTION: Once a user specifies a song (Sa1), a control section acquires guide melody data and musical sound data representing an accompaniment sound of the specified song from a server device (Sa2). The control section creates an evaluation table based upon the acquired guide melody data (Sa3). The control section classifies respective reference component sounds of the acquired guide melody data by types associated with beats (Sa4). Once the song is reproduced (Sa5), a voice signal representing user's singing is acquired (Sa6). The control section specifies the pitch of the user's singing from the voice signal (Sa7). The control section specifies user's singing start time of each reference component sound (Sa8). The control section evaluates singing to each reference component sound (Sa9). The control section calculates average points by the types associated with the beats (Sa10) and evaluates a total sense of rhythm (Sa11).

Description

  The present invention relates to a technique for evaluating a rhythm feeling in a user's song.

  Conventionally, in a karaoke apparatus, what evaluates a user's rhythm sense is widely known. For example, Patent Literature 1 discloses a technique for evaluating a user's singing rhythm with respect to a guide melody.

JP 2004-102148 A

  In the technique described in Patent Document 1, the rhythm feeling of the user's singing is evaluated equally for each sound constituting the guide melody. On the other hand, there are many popular songs called J-POP in recent years in which syncopation, anticipation, etc. are used for the rhythm, and the relationship with the beat influences the sense of rhythm in singing. Moreover, in this kind of popular song, since the number of characters of the lyrics per unit time may be large, it is required to sing not only at the table beat but also at the back beat.

  The present invention has been made in view of the above-described background, and an object thereof is to evaluate a user's rhythm feeling in singing in consideration of beats.

  In order to solve the above-described problem, the present invention relates to each of a plurality of reference constituent sounds constituting a model sound that is referred to when a user plays a musical piece, and the pitch of the reference constituent sound and the reference constituent sound. Model sound data acquisition means for acquiring model sound data representing the pronunciation start time, and each of a plurality of reference constituent sounds whose pitch and pronunciation start time are represented by the model sound data acquired by the model sound data acquisition means Classifying means for classifying the sound into one of a plurality of types related to beats, and a performance sound signal acquisition means for acquiring a performance sound signal representing a performance sound that is a sound generated by a user's performance A pitch specifying means for specifying the pitch of the user's performance sound from the performance sound signal acquired by the performance sound signal acquisition means, and a plurality of pitches and pronunciation start times represented by the model sound data With respect to each of the reference component sounds, the pronunciation start time specifying means for specifying the pronunciation start time at which the reference component sound was performed according to a predetermined rule, and each of the plurality of reference component sounds represented by the exemplary sound data, A configuration for generating component sound evaluation data indicating a higher evaluation as a difference between the sound generation start time of the reference component sound and the sound generation start time of the performance specified by the sound generation start time specifying unit with respect to the reference component sound is smaller Based on the component sound evaluation data generated by the component sound evaluation data generation unit for one or more reference component sounds classified into the type by the classification unit, for each of the types relating to the beat evaluation data generation unit, There is provided a singing evaluation device including evaluation data generating means for each beat type for generating evaluation data for each beat type indicating evaluation for each type.

  In a preferred aspect, the present invention provides that the sound generation start time specifying means relates to each of a plurality of reference component sounds whose pitches and sound generation start times are represented by the model sound data, and uses the sound generation start time of the reference component sound as a reference The time when the pitch of the performance sound of the user specified by the pitch specifying means falls within the range of the pitch determined on the basis of the pitch of the reference constituent sound within the period determined as It is specified as the sounding start time when the constituent sound is played.

  In another preferred aspect, the present invention provides that the model sound data includes timing data indicating a timing at which the reference constituent sound is generated with respect to each of at least one or more reference constituent sounds, and the classification unit includes: The classification is performed based on the timing data.

  In another preferred embodiment, the present invention provides the model sound data, the data specifying the start time of each measure obtained by dividing the performance period of the music into a plurality of times, and the time signature relating to each of the plurality of measures. The classification means performs the classification based on data specifying the start time of each of the bars and data specifying the time signature of each of the plurality of bars. .

  In another preferred embodiment, the present invention provides that the model sound data includes, for each of at least one or more of the reference component sounds, a coefficient to be multiplied by an evaluation point regarding the reference component sound, and the component sound evaluation data generation The means generates the constituent sound evaluation data indicating an evaluation point obtained by multiplying an evaluation point determined based on a difference between the sounding start time of the reference constituent sound and the sounding start time of the performance by the user by the coefficient It is characterized by doing.

  In another preferred embodiment, in the present invention, the beat type evaluation data generated by the beat type evaluation data generation unit is data indicating a numerical value of the evaluation of each type related to the beat, and the beat type evaluation data Multiplying the numerical value represented by the above by a coefficient of a ratio predetermined for each type, and adding a calculation means for calculating a numerical value obtained by adding them.

  ADVANTAGE OF THE INVENTION According to this invention, a rhythm feeling in a user's song can be evaluated in consideration of a beat.

The figure which shows the structure of the singing evaluation system Block diagram showing hardware configuration of karaoke equipment Schematic diagram showing classification of types related to beats A figure showing an example of guide melody data A figure showing an example of an evaluation table Block diagram showing functional configuration of karaoke equipment Processing flow diagram of karaoke equipment Schematic diagram for explaining detection of singing by the user The figure showing an example of an evaluation standard table The figure showing an example of the evaluation table after evaluation is performed Figure showing an example of the frequency distribution table A figure showing an example of an evaluation comment table A figure showing an example of guide melody data concerning modification 1 The figure showing an example of the guide melody data which concerns on the modification 2. The figure showing an example of the guide melody data which concerns on the modification 3. The figure showing an example of the evaluation comment table which concerns on the modification 4.

<Embodiment>
<Configuration>
FIG. 1 is a diagram showing a configuration of a singing evaluation system 100 according to an embodiment of the present invention. The singing evaluation system 100 includes a karaoke device 10 and a server device 20. The server device 20 stores musical sound data representing accompaniment sounds of music and information on guide melody (referred to as guide melody data) to be referred to when the user sings the music and supplies them to the karaoke device 10. It is a server device. When the user designates a song to be sung in the karaoke device 10, the karaoke device 10 acquires musical sound data and guide melody data representing the accompaniment sound of the designated song from the server device 20. When the user sings according to the accompaniment sound based on the musical sound data, the karaoke apparatus 10 evaluates the rhythm feeling in the user's singing based on the audio signal representing the user's singing and the guide melody data.

  FIG. 2 is a block diagram illustrating a hardware configuration of the karaoke apparatus 10. The karaoke apparatus 10 includes a control unit 11, a storage unit 12, a communication unit 13, a UI (User Interface) unit 14, a voice input unit 15, and a voice output unit 16. The control unit 11 is an example of a song evaluation apparatus according to the present invention, and includes a CPU (Central Processing Unit), a RAM (Random Access Memory), and a ROM (Read Only Memory), and controls the entire karaoke apparatus 10. The CPU performs various controls by reading the program stored in the ROM or the storage unit 12 into the RAM and executing it. The storage unit 12 is, for example, a large-capacity storage device such as a hard disk, and stores various programs, the above-described musical tone data and guide melody data, an evaluation table described later, a threshold value, and the like. The communication unit 13 communicates with the server device 20 by wire or wireless. The UI unit 14 includes, for example, a touch screen and a key, and accepts an operation by a user and displays an image based on the operation on the touch screen. The voice input unit 15 includes voice pickup means such as a microphone, acquires a voice signal representing the user's voice input from the microphone, and inputs the acquired voice signal to the control unit 11. The sound output unit 16 has sound emission means such as a speaker, for example, and converts the musical sound data representing the accompaniment sound of the music and the sound signal representing the user's voice from digital to analog and outputs the result from the speaker. When the user designates a song to be sung using the UI unit 14, the communication unit 13 notifies the server device 30 of the designated song under the control of the control unit 11, and accordingly, from the server device 30. Acquired musical tone data and guide melody data. The control unit 11 causes the storage unit 12 to store the acquired musical sound data and guide melody data.

In the guide melody data described above, information such as pitch and pronunciation start time is described for each constituent sound with respect to the guide melody that the user refers to when singing. Information such as the pitch of each constituent sound and the sounding start time is information on a sound that serves as an example in the user's singing. Guide melody data is an example of model sound data according to the present invention. The control unit 11 classifies each constituent sound indicated by the guide melody data into one of a plurality of types related to a predetermined number of beats according to a predetermined rule, and the rhythm feeling in the user's singing for each type of classified beats. To evaluate. Hereinafter, the classification of types related to beats will be described with reference to FIG.
FIG. 3 is a schematic diagram showing classification of types related to beats. FIG. 3 shows an example of one measure in a 4/4 time signature. In this example, because the time is 4/4, the notes from the first beat to the fourth beat are “table beats”, the notes between the beats are “back beats”, and the other notes are “other than that”. Become. In other words, here, when one measure is equally divided into eight sections in time, the start position of the odd-numbered section corresponds to “front beat” and the start position of the even-numbered section becomes “back beat”. Equivalent to. For the triplet shown in the figure, the first note is a table beat, but the remaining two notes are “other” that is neither a table beat nor a back beat.

In the present embodiment, the types related to beats are classified into three: “table beats”, “back beats”, and “others”. In this embodiment, the type related to the beat of “other than that” is a concept including the remaining two notes excluding the first note in a triplet, notes corresponding to syncopation, anticipation, shuffle, and the like. is there. Here, definitions of “strong beat”, “weak beat”, “syncopation”, “anticipation”, “triplet”, and “shuffle” other than the above-described “table beat” and “back beat” will be described. . “Strong beat” refers to the first note of the first beat in a certain time signature. On the other hand, “weak beat” indicates a note other than a strong beat in a certain time signature. For example, in the case of “4/4 time”, the note of the first beat is “strong beat”, and the notes of the second to fourth beats are “weak beat”. “Syncopation” is a rhythm produced by a certain sound that continues to sound from a lesser beat to a more dominant beat. For example, when a tie is connected as one sound from a weak beat of a measure to a strong beat positioned next in the measure, this is called a syncopation rhythm. In the present embodiment, when notes connected in this way are classified by the type related to beats, it becomes “other”, and when it is classified in more detail, it becomes “syncopation”. “Anticipation” is a syncopation when straddling a bar line. For example, when a tie is connected as a single sound from a weak beat of a bar to a strong beat located at the head of the next bar across the bar line, this is called an anti-citation rhythm. In the present embodiment, when the notes connected in this way are classified by the type related to beats, it becomes “other”, and when it is classified in more detail, it becomes “anticipation”. The “triplet” indicates a note obtained by dividing a certain reference note such as a quarter note into three equal parts. In the present embodiment, among the triplets, the remaining notes other than the notes that can be classified as “table beat” or “back beat” are classified as “others” as the type related to beats, and are classified in more detail. In this case "triplet". “Shuffle” is a rhythm in which the length of the first note is long and the length of the second note is short among two consecutive notes. The shuffle rhythm is similar to a triplet with the middle note rested. In the present embodiment, among such two consecutive notes, when the second note is classified by the type relating to beats, it becomes “other”, and when it is classified in more detail, it becomes “shuffle”.
Next, guide melody data will be described with reference to the drawings.

  FIG. 4 is a diagram illustrating an example of guide melody data. The guide melody data is data in which information on each component sound of the guide melody that the user refers to when singing is described in time series. The guide melody data includes a plurality of items such as “No.”, “pitch”, “pronunciation start time”, “pronunciation end time”, and “beat type”. Each guide melody data is associated with each constituent sound in the guide melody. Hereinafter, the constituent sounds of the guide melody indicated by the guide melody data are referred to as reference constituent sounds. “No.” is an identifier for uniquely identifying each guide melody data. For example, a four-digit number is assigned in ascending order from the earliest pronunciation start time. “Pitch” represents the pitch of each reference component sound. The “pronunciation start time” represents the time when the pronunciation of each reference constituent sound is started with reference to the start point of the music. The “pronunciation end time” represents the time at which the pronunciation of each reference constituent sound ends with the start point of the music as a reference. The “beat type” indicates a type related to the beat of each reference constituent sound. In the example shown in the figure, the “beat type” of the reference constituent sound corresponding to the table beat that is the rhythm reference in the progression of the music describes the table beat. The “beat type” indicates a timing at which the reference component sound should be generated since the timing at which the reference component sound should be generated is a specific beat. The “beat type” is an example of timing data according to the present invention. Next, the above-described evaluation table will be described with reference to the drawings.

  FIG. 5 is a diagram illustrating an example of the evaluation table. The evaluation table is a table in which guide melody data and information related to the user's singing are associated with each corresponding reference constituent sound, and an evaluation point of rhythm feeling is described for each reference constituent sound. Each time the user designates a song to be sung using the UI unit 14, the control unit 11 creates an evaluation table based on the guide melody data of the designated song and stores it in the storage unit 12. The evaluation table in FIG. 5 shows a state immediately after being created. The evaluation table includes a plurality of items such as “No.”, “pitch”, “pronunciation start time”, “pronunciation end time”, “beat classification information”, “singing start time”, and “evaluation point”. “No.”, “pitch”, “pronunciation start time”, and “pronunciation end time” are the same items as those in the guide melody data. Since the control unit 11 creates the evaluation table based on the guide melody data, “No.”, “pitch”, “sounding start time”, and “sounding end time” are created in the evaluation table. The same contents as the guide melody data are described. “Beat classification information” represents the result of classifying each reference component sound by the type related to the beat. The control unit 11 sets a table beat in the “beat classification information” of the evaluation table for the reference constituent sound whose “beat type” is a table beat in the guide melody data. In the evaluation table at the time of creation, “beat classification information” other than the table beat is blank. "Singing start time" represents the time when the user is considered to have started singing for each reference constituent sound. The “evaluation score” represents an evaluation score of a rhythm feeling in the user's song for each reference constituent sound. In the evaluation table at the time of creation, “singing start time” and “evaluation point” are blank.

  FIG. 6 is a block diagram illustrating a functional configuration of the karaoke apparatus 10. The model sound data acquisition unit 111, the classification unit 112, the performance sound signal acquisition unit 113, the pitch specification unit 114, the sound generation start time specification unit 115, the constituent sound evaluation data generation unit 116, and the evaluation data generation unit 117 for each beat type are controlled. This is realized by the unit 11. The model sound data acquisition unit 111 relates to each of a plurality of reference component sounds constituting a model sound to be referred to when the user plays a musical piece, and the pitch of the reference component sound and the pronunciation start time of the reference component sound The model sound data representing is acquired. The classification unit 112 classifies each of the plurality of reference constituent sounds represented by the model sound data acquired by the model sound data acquisition unit 111 into one of the types related to a plurality of beats determined in advance according to a predetermined rule. . The performance sound signal acquisition unit 113 acquires a performance sound signal representing a performance sound that is a sound generated by a user's performance. The pitch specifying unit 114 specifies the pitch of the user's performance sound from the performance sound signal acquired by the performance sound signal acquisition unit 113. The sound generation start time specifying unit 115 relates to each of the plurality of reference constituent sounds represented by the model sound data, and within a period determined in accordance with a predetermined rule based on the sound generation start time of the reference constituent sounds. The time when the pitch of the performance sound of the user specified by the step is within the range of the pitch determined according to a predetermined rule based on the pitch of the reference configuration sound is started. Specify as time. The component sound evaluation data generation unit 116 specifies the sound generation start time of the reference component sound and the sound generation start time specifying unit 115 for the reference component sound for each of the plurality of reference component sounds represented by the model sound data. Evaluation data indicating higher evaluation as the difference from the sound generation start time of the performance by the user is smaller is generated as component sound evaluation data related to the reference component sound. The beat type evaluation data generation unit 117 generates component sound evaluation data generated by the component sound evaluation data generation unit 116 for one or more reference component sounds classified into the type by the classification unit 112 for each type related to a plurality of beats. Based on the above, the evaluation data for each beat type indicating the evaluation for each type of beat is generated.

<Operation>
Next, the operation in this embodiment will be described.
FIG. 7 is a process flow diagram of the karaoke apparatus 10. First, when the user designates a piece of music to be sung using the UI unit 14, the control unit 11 accepts this (step Sa1). Next, when the control unit 11 uses the communication unit 13 to acquire the musical tone data representing the accompaniment sound of the designated song and the guide melody data of the designated song from the server device 20, these are stored in the storage unit 22. Store (step Sa2). If the control part 11 produces an evaluation table based on the acquired guide melody data, it will memorize | store this in the memory | storage part 22 (step Sa3). Next, the control unit 11 classifies each reference component sound in the acquired guide melody data according to the type relating to the beat (step Sa4). Next, the process of step Sa4 will be described in detail.

  As described above, in the guide melody data, the pronunciation start time of each reference component sound is described, and the reference component sound corresponding to the table beat is associated with “table beat” as the “beat type”. . In step Sa4, the control unit 11 uses the reference component sounds A and B of the table beat for the reference component sound C existing between the reference component sound A of a certain table beat and the reference component sound B of the next table beat. Based on the time difference of the sound generation start time of the reference component sound C with respect to the sound generation start time, the beat type is classified for the beat corresponding to the reference component sound C. For example, the pronunciation start time of the reference constituent sound A of the table beat is “00:04:00”, the pronunciation start time of the reference constituent sound B of the table beat is “00:06:00”, and the reference constituent sound C Suppose that the pronunciation start time of “00:05:00”. In this case, the sound generation start time of the reference component sound C is located exactly in the middle between the sound generation start time of the reference component sound A which is a table beat and the sound generation start time of the reference component sound B which is a table beat. Therefore, the control unit 11 classifies the beat of the reference component sound C as a back beat. In short, based on the difference between the pronunciation start times in the reference constituent sounds of consecutive table beats, based on the time difference between the pronunciation start time of the reference constituent sounds of the table beats and the pronunciation start times of unclassified reference constituent sounds, Beats are classified. The control unit 11 classifies the type related to the beat of each reference constituent sound into one of “front beat”, “back beat”, and “other than that”, and the evaluation table stored in the storage unit 22 with the result of the classification Described in beat classification information. In this way, the control unit 11 classifies each reference constituent sound into one of a plurality of predetermined beats according to the rules described above.

  When the process of step Sa4 is started, the control unit 11 reproduces the music based on the acquired musical tone data and the guide melody data, and supplies the reproduction signal to the audio output unit 16, thereby the audio output unit 16 Causes the musical accompaniment sound and the guide melody to be emitted from the speaker (step Sa5). And if a user starts singing using a microphone, the audio | voice input part 15 will memorize | store this in the memory | storage part 22 in the acquired order, if the audio | voice signal showing the user's audio | voice supplied from a microphone is acquired (step Sa6). ). Next, the control unit 11 analyzes the frequency of the acquired voice signal and specifies the pitch of the user's voice (step Sa7). Any known method may be used as the frequency analysis method. Next, the control part 11 specifies the user's singing timing with respect to each reference constituent sound, specifically, the singing start time by the user (step Sa8). Next, step Sa8 will be described with reference to the drawings.

  FIG. 8 is a schematic diagram for explaining detection of singing by the user. In FIG. 8, the horizontal axis represents time, and the time elapses toward the right in the figure. In FIG. 8, the vertical axis represents the pitch, and the pitch increases as it goes upward in the figure. The three rectangles shown represent reference constituent sounds RS0, RS1, and RS2 indicated by the guide melody data. A curve P indicates a pitch trajectory that is a pitch trajectory in a song. The pitch of the reference component sound RS0 is “F”, the pitch of the reference component sound RS1 is “G”, and the pitch of the reference component sound RS2 is “C”. Each reference component sound has a pitch detection range and detection period. For example, it is assumed here that the pitch detection range is one semitone up and down from the pitch of the reference constituent sound. That is, the pitch detection range of the reference component sound RS0 is “F #” to “E”, and the pitch detection range of the reference component sound RS1 is “G #” to “F #”. The pitch detection range of RS2 is “C #” to “B”. In FIG. 5, the upper and lower limits of the pitch detection range are represented by broken lines. Further, for example, here, the detection period is “two seconds before and after the sound generation start time of the reference constituent sound”. The contents of the pitch detection range and the detection period are stored in the storage unit 22, but the user may arbitrarily change the contents within an appropriate range using the UI unit 14.

  In step Sa8, the control unit 11 determines that the pitch of the user's singing is within the pitch detection range based on the pitch of the reference constituent sound within the detection period based on the pronunciation start time of a certain reference constituent sound. Is specified as a reference time that is a reference for the singing start time when the reference component sound is sung. For example, in the case of FIG. 8, the pitch of the user's singing with respect to the reference component sound RS1 is included in the pitch detection range of the reference component sound RS1 at time t1 included in the detection period. The standard time for the reference constituent sound RS1 is specified as “t1”. Then, the control unit 11 traces the pitch trajectory P from the reference time in the past for a predetermined period, and determines the timing at which the pitch trajectory P corresponds to one of the predetermined states as the singing of the reference constituent sound RS1. The start time is specified as “tx1”. The predetermined period is a period of “100 milliseconds”, for example, and is stored in the storage unit 12 in advance as a parameter.

  Next, the above-described predetermined state will be described. There are three patterns in the predetermined state, and when it is determined that the control unit 11 corresponds to any of these patterns, the timing is specified as the singing start time. The first pattern is when the trajectory is interrupted when the control unit 11 traces back the pitch trajectory P in the past. This state represents that the singing is started from the timing when the pitch trajectory P is interrupted. In this case, the control unit 11 specifies the timing at which the pitch trajectory P is interrupted as the singing start time of the reference constituent sound. In the second pattern, when the control unit 11 traces the pitch trajectory P in the past, the inclination angle formed by the curve made by the pitch trajectory P with respect to the time axis becomes gentler than a predetermined threshold value. Is the time. The predetermined threshold is, for example, “30 degrees”, and is stored in the storage unit 12 in advance as a parameter. In this case, it indicates that there is a certain difference between the pitches of the reference constituent sound specifying the standard time and the reference constituent sound immediately before. That is, in this case, since the user represents singing another reference constituent sound having a pitch different from that of the immediately preceding reference constituent sound, the control unit 11 uses this timing as the reference constituent sound that specifies the reference time. Is specified as the singing start time. In the third pattern, when the control unit 11 traces the pitch trajectory P in the past, the angle of inclination that the curve made by the pitch trajectory P makes with respect to the time axis becomes larger than a predetermined threshold value. Is the time. The predetermined threshold is, for example, “60 degrees”, and is stored in the storage unit 12 in advance as a parameter. In this case, it indicates that there is a large difference in pitch between the reference component sound specifying the standard time and the reference component sound immediately before. That is, also in this case, since the user represents singing another reference constituent sound having a pitch different from that of the immediately preceding reference constituent sound, this timing is used as the singing start time of the reference constituent sound specifying the reference time. As specified.

  As illustrated, “tx1” detected as the singing start time is later than the pronunciation start time of the reference constituent sound RS1. In this case, it represents a case where the user has delayed the singing or has intentionally made a mistake. Here, “Tame” is a state where the singing is delayed from the original timing, while “Hashiri” represents a state where the singing is performed earlier than the original timing. Similarly, the control unit 11 specifies the reference time of the reference constituent sound RS2 as “t2” and specifies the singing start time as “tx2”. The control unit 11 describes the specified singing start time in the evaluation table in association with the corresponding reference constituent sound.

  Returning to the description of FIG. Following step Sa8, the control unit 11 evaluates the user's song for each reference constituent sound (step Sa9). In accordance with the contents described in the evaluation table, the control unit 11 generates evaluation data that is higher as the absolute value of the difference between the singing start time by the user and the pronunciation start time of the reference constituent sound is smaller and lower as it is larger. Here, it is assumed that the evaluation data is a score represented by a numerical value. As a method of evaluation, a predetermined calculation formula may be used, but here, an evaluation criterion table that is a table in which the correspondence is described is used.

  FIG. 9 is a diagram illustrating an example of the evaluation criterion table. In the evaluation criteria table, an evaluation point is described in association with a time difference (milliseconds) that is a range in which the difference between the singing start time by the user and the pronunciation start time of the reference constituent sound can be taken. Here, the evaluation score is a six-level evaluation from “0” to “5”. Also, the evaluation points include “standard” used for normal evaluation, “tame” used for evaluation of singing songs, and “hashiri” used for evaluation of singing songs. There are three types. The evaluation points for “Tame” and “Hashiri” are used when it is determined that the song has “Tame” or “Hashiri” characteristics in the overall rhythmic evaluation after the evaluation points have been calculated once. It is done. First, evaluation is performed using “standard” evaluation points. As shown in the figure, the “standard” evaluation score is higher as the absolute value of the difference obtained by subtracting the sound generation start time of the reference constituent sound from the singing start time by the user is smaller. For example, when the difference obtained by subtracting the pronunciation start time of the target reference constituent sound from the singing start time by the user is calculated as “+100 milliseconds”, the control unit 11 refers to the evaluation standard table and calculates the reference constituent sound. The evaluation score is determined as “5”. The control part 11 describes the evaluation score in an evaluation table about each reference structure sound based on the determined content. This evaluation score is an example of constituent sound evaluation data according to the present invention.

  FIG. 10 is a diagram illustrating an example of the evaluation table after the evaluation is performed up to step Sa9. The “beat classification information”, which is blank except for the table beat when the evaluation table is created, describes the classification of the type related to the beat for each reference constituent sound by performing the classification of the type related to the beat in step Sa4. . Further, by detecting the user's song in step Sa8, a value is described in “song start time”. In step Sa9, the user's song is evaluated for each reference constituent sound, whereby a value is described in “evaluation point”. After step Sa9, the control unit 11 generates evaluation data for each type related to beats based on the evaluation table after the user's singing has been evaluated (step Sa10). Here, the evaluation data for each type related to the beat is indicated by a numerical value, and is an average score of the evaluation points for each type related to the beat, and is an example of the evaluation data for each beat type according to the present invention. Specifically, the control unit 11 divides a total value of evaluation scores of reference constituent sounds having the same “beat classification information” in the evaluation table by the number of reference constituent sounds having the “beat classification information”. By doing so, the average score for each type relating to the beat is calculated. That is, when step Sa10 is completed, the average score of the singing evaluation by the user is calculated for each of “top beat”, “back beat”, and “other than that”.

After step Sa10, the control unit 11 evaluates the overall sense of rhythm based on the average score of singing evaluations calculated for each beat classification (step Sa11). Evaluation of step Sa11 is performed based on the following formula (1), for example. In Expression (1), α is a coefficient that satisfies 0 <α <1.
Comprehensive Rhythm Evaluation = {(("Average score of" back beat "+" Average score of "other beat") / 2) / (Average score of "table beat")} x {(("Average score of" back beat ") + “Other average” score) / 2) × α + (Average score of “table beat”) × (1−α)} Expression (1)
The coefficient α is stored in the storage unit 12 and may be changed by the user using the UI unit 14. As the coefficient α is larger, the evaluation weight of “back beat” and “others” is increased, and as the coefficient α is smaller, the evaluation weight of “table beat” is increased. Since the sum of the coefficients “α” and “1−α” is 1, these coefficients represent the ratio of the coefficients to be multiplied for each type of beat. That is, the control unit 11 calculates a numerical value obtained by multiplying the numerical value indicated by the evaluation data for each type related to the beat by a coefficient of a ratio predetermined for each type and adding them. The control unit 11 is an example of a calculation unit according to the present invention. In addition, in the formula (1), for the calculation of the part of “{((“ average score of “back beat” + “average score of other beats”) / 2) / (average score of “table beat”)} ” The control unit 11 sets the maximum value to 1. That is, when a value exceeding 1 is calculated as the calculation result for this portion, the control unit 11 corrects the value to 1. In addition, for the part of “{((the average score of“ back beat ”+ the average score of“ other beat ”) / 2) / (the average score of“ table beat ”)}”, the average score of “table beat” is used as the denominator. The reason why the average score of “back beat” and “other than that” is used as the numerator is as follows. The “table beat” is generally easily recognized by the user, and the singing timing is less likely to be shifted. Therefore, the smaller the deviation of the singing timing in the reference constituent sounds of “back beat” and “other than that”, the higher the possibility that the user has a sense of rhythm. Based on this concept, in this embodiment, when the evaluation of “back beat” and “other than that” is high, “back beat” and “it The average score of “other than” is the numerator.

For example, in the formula (1), the coefficient α is “0.6”, the average score of “table beat” is “4”, and the average score of “back beat” and “other beats” is “3”. If there is, it can be expressed as the following formula (2).
Total rhythm evaluation = {((3 + 3) / 2) / 4} × {((3 + 3) / 2) × 0.6 + 4 × 0.4} = (3/4) × 3.4
= 2.55 ... Formula (2)
According to Equation (2), although the average score of “table beat” is high, the average score of “back beat” and “others” is lower than that of “table beat”, and the value of coefficient α is “back beat” "And other values" increase the weight of the evaluation. As a result, the overall rhythmic sense evaluation obtained by equation (2) is somewhat low. The user can change the weight of each beat type in the overall rhythm evaluation by changing the value of the coefficient α using the UI unit 14.

  The karaoke apparatus 10 causes the UI unit 14 to display a comment obtained by analyzing the evaluation result. For example, after step Sa11, the control unit 11 creates a frequency distribution table for each type related to the classified beat based on the evaluation table. FIG. 11 is a diagram illustrating an example of a frequency distribution table. In the figure, the horizontal axis represents the time difference between the pronunciation start time of the reference component sound and the singing start time. In FIG. 11, the singing timing is later than it should be as it proceeds in the right direction in the figure, and the singing timing is earlier than it should be as it proceeds in the left direction in the figure. The vertical axis in the figure represents the frequency, and the number of each singing timing is shown. In FIG. 11, the broken line represents the frequency distribution of “table beat”, the one-dot chain line represents the frequency distribution of “back beat”, and the two-dot chain line represents the frequency distribution of “others”. FIG. 11A shows a frequency distribution based on the evaluation result of user A's song, and FIG. 11B shows a frequency distribution based on the evaluation result of user B's song. As shown to Fig.11 (a), as for the song of the user A, the peak of frequency concentrates on the same singing timing about each of "front beat", "back beat", and "other than that". Therefore, it can be said that the singing timing of the user A is constant regardless of the type related to the beat, and the rhythm feeling is good. On the other hand, in the case of FIG. 11B, the singing timing of the frequency at which the peak of the frequency at which the user B's singing visits “top beat”, “back beat”, and “other than that” is distributed. Therefore, it can be said that the singing timing of the user B differs depending on the type related to the beat, and the rhythm feeling is not good. The control unit 11 may display the determination result based on such a frequency distribution table on the UI unit 14 as a comment, or may display the frequency distribution table itself on the UI unit 14.

  As another method for displaying a comment, for example, there is a method of using an evaluation comment table describing an evaluation comment. FIG. 12 is a diagram illustrating an example of the evaluation comment table. In the evaluation comment table of FIG. 12, evaluation comments are described for combinations of the average score of “table beat” and the average score of “back beat” and “other than that”. The average score is distinguished as high or low based on a threshold value of “3.5”, for example. The evaluation comment table is stored in the storage unit 12 together with the threshold value. This threshold value may be changeable by the user using the UI unit 14. For example, in step Sa9, when the average score of the table beat is calculated as “4” and the average score of the back beat and other points is calculated as “3”, the control unit 11 comments that “the sense of rhythm is bad”. It is displayed on the UI unit 14. By using such an evaluation comment table, it is possible to evaluate the overall sense of rhythm based on the result of combining the evaluation of “table beat” with the evaluation of “back beat” and “other than that”.

  In addition, here, when the average score of “table beat” is low and the average score of “back beat” and “other than that” is high, the control unit 11 comments that “there is a sword / hashiri in singing”. It is displayed on the UI unit 14. This is due to the following reason. In general, “back beat” and “other” are more difficult to match the rhythm than “table beat”, and “table beat” is easier to match the rhythm. Therefore, if the evaluation of “back beat” and “other” with high difficulty is high, and the evaluation of “table beat” with low difficulty is low, the result of intentionally using the tame and scissors for “table beat” It is considered that there is a difference between the pronunciation start time of the reference component sound and the singing start time. As a result of the evaluation using the evaluation comment table, when the average score of “table beat” is low and the average score of “back beat” and “other than that” is high, the control unit 11 evaluates the reference constituent sound of the table beat Recalculate points. If the difference is a positive value based on the evaluation criterion table of FIG. 9 and the difference obtained by subtracting the pronunciation start time of the target reference constituent sound from the singing start time by the user, the control unit 11 When the difference is a negative value using the evaluation score “”, the evaluation score “Hashiri” is used. For example, when the difference is “+400 milliseconds”, it means that the user sang 400 milliseconds later than the sound generation start time of the reference component sound. In this case, the control unit 11 in the evaluation criterion table Based on the evaluation score of “Tame”, the evaluation score is determined as “5”. In the case of normal evaluation, this is the evaluation score of “4”, but this is because the evaluation criterion table corresponding to the game is used. The control unit 11 updates the evaluation table with the determined evaluation points. In this way, it is possible to evaluate the intentionally sung wings and lashes.

  Thus, in this embodiment, each reference constituent sound in the guide melody data representing the guide melody of the music is classified by the type related to the beat. Then, the singing is evaluated for each reference constituent sound classified by the type related to the beat, and the evaluation for each type related to the beat is performed based on the evaluation result, thereby obtaining the rhythmic feeling evaluation for each type related to the beat. Thus, according to the present embodiment, it is possible to evaluate the rhythm feeling of the user's singing in consideration of the beat.

<Modification>
Although the embodiment of the present invention has been described above, the present invention can be modified as follows. Further, the following modifications may be implemented in combination as appropriate.
(Modification 1)
The guide melody data is not limited to “beat type”, and may include information on the following beats.
FIG. 13 is a diagram illustrating an example of guide melody data according to the first modification. In the guide melody data shown in FIG. 13, items “beat type 2” and “evaluation coefficient” are added as compared with those shown in FIG. 4. “Beat type 2” shown in FIG. 13 is one in which “table beat” is further classified into “strong beat” and “weak beat”. For example, if four quarter notes are arranged in a 4/4 time signature, the first quarter note is a strong beat and the remaining three quarter notes are weak beats. “Evaluation coefficient” is a coefficient by which the evaluation score is multiplied. Since the strong beats are easier to match the singing timing than the weak beats and stand out, the evaluation coefficient is set so that the evaluation points are weighted. The evaluation coefficient is stored in the storage unit 22 and may be set by the user using the UI unit 14. Note that items added to the guide melody data in Modification 1 as compared with the embodiment are also included in the created evaluation table. The same applies to the following modifications. In this way, by providing information for identifying strong beats and weak beats in advance in the guide melody data and making it possible to set the evaluation coefficient, the user can obtain a rhythmic evaluation that takes beats into account.

(Modification 2)
The guide melody data may include the following information on beats.
FIG. 14 is a diagram illustrating an example of guide melody data according to the second modification. In the guide melody data shown in FIG. 14, items “beat type 2” and “evaluation coefficient” are added as compared with those shown in FIG. 4. The “evaluation coefficient” is a coefficient to be multiplied by the evaluation score related to the reference constituent sound. “Beat type 2” shown in FIG. 14 is a type in which the types related to beats are classified by “syncopation”. In the example of FIG. 14, the syncopation is regarded as important, and an evaluation coefficient of “1.2” is set for the reference constituent sound corresponding to the beat of the syncopation. The control unit 11 generates an evaluation score obtained by multiplying the evaluation score determined based on the difference between the pronunciation start time of the reference constituent sound and the singing start time by the user by this evaluation coefficient. The classification of types related to beats is not limited to those described above, and for example, “triplet” or “shuffle” may be classified in “beat type 2”. In this case, when the control unit 11 performs evaluation for each type related to beats in step Sa10, not only “front beat”, “back beat”, and “others” but also types related to beats included in “beat type 2”. Also, each type of beat is evaluated. In this way, the user can obtain a more detailed rhythmic evaluation by performing a finer classification than the classification of “table beat”, “back beat”, and “other than that”.

(Modification 3)
Evaluation weighting may be arbitrarily enabled for each reference constituent sound.
FIG. 15 is a diagram illustrating an example of guide melody data according to the third modification. In the guide melody data in the third modification, an evaluation coefficient can be set for each reference constituent sound.
For example, it is difficult to sing at the beginning of singing, the part corresponding to rust, the part where the time signature changes due to bridge or break, etc., and the part where the time between beats changes with the tempo change, And since it is an impressive and conspicuous part throughout the music, an evaluation coefficient for positively weighting the evaluation point is described for the reference constituent sound corresponding to these parts. By setting such an evaluation coefficient, for example, even if the user's singing is rough, it is easy to get high evaluation as a rhythm feeling if there is a singing timing at a conspicuous place such as head out or rust Become.

(Modification 4)
The evaluation comment table is not limited to the example described above. FIG. 16 is a diagram illustrating an example of the evaluation comment table according to the fourth modification. In the evaluation comment table of FIG. 16, evaluation comments are described for combinations of the distribution of evaluation scores for “table beats” and the distribution of evaluation points for “back beats” and “others”. The distribution of evaluation scores for “table beats” and the distribution of evaluation scores for “back beats” and “other beats” are the standard deviation of evaluation points for each type of classified beats, and a predetermined threshold value. Based on whether it is narrow or wide. This threshold value is stored in advance in the storage unit 12 and may be changed by the user using the UI unit 14. For example, when the standard deviation of the evaluation score of “table beat” is equal to or greater than the threshold, the control unit 11 determines that the distribution of the evaluation points is “wide”, and sets the evaluation score of “back beat” and “other than that”. If the standard deviation is less than the threshold, the evaluation point distribution is determined to be “narrow”. In this case, the control unit 11 causes the UI unit 14 to display a comment “There is a sword / hashiri in the song”. If it does in this way, a user's song rhythm feeling can be evaluated by distribution of a user's song timing.

(Modification 5)
The information described in the “beat type” in the guide melody data in advance is not limited to “table beat”. For example, the beat type may describe “back beat” or “strong beat”. In short, as long as it can be used as a reference in classification of types related to beats, it is not limited to table beats. In the guide melody data, “beat type” may be described in advance for all reference constituent sounds. In this case, since “beat type” in the guide melody data is described as it is in “beat classification information” of the created evaluation table, the classification process in step Sa4 becomes unnecessary.

(Modification 6)
In the above-described embodiment, the classification of types related to beats is classified into three of “table beat”, “back beat”, and “other than that”, but is not limited to this. For example, classification of types related to beats may be performed based on only “front beats” and “back beats”, or may be classified in more detail.

(Modification 7)
The evaluation of the rhythm feeling and the overall rhythm feeling evaluation for each type relating to the beat may be displayed on the UI unit 14 together with the evaluation result of the melody or added together and notified to the user. In addition, the control unit 11 of the karaoke apparatus 10 uses the communication unit 13 to transmit these data to a printer (not shown), a web server apparatus managed by a karaoke store, or a mobile phone owned by the user. The user may be able to confirm the evaluation content using a mobile phone or on a web browser, or print the evaluation content. In this way, the user can know the skill of comprehensive singing in which a sense of rhythm considering the beat is reflected.

(Modification 8)
If the user's song is continuously highly evaluated, the subsequent songs may be weighted and evaluated. For example, when evaluating the user's song for each reference constituent sound in step Sa9, the control unit 11 has an evaluation score equal to or greater than a predetermined threshold (for example, “4” here) equal to or greater than a predetermined number. If it continues, the next evaluation point is multiplied by an evaluation coefficient such as “1.1”. The predetermined threshold and number and the evaluation coefficient are stored in the storage unit 22 and may be changed by the user using the UI unit 14. The control unit 11 uses the evaluation coefficient until the evaluation score again becomes less than a predetermined threshold value. In this way, the more the user's singing timing is correct, that is, the more the singing that matches the timing of the beat is performed correctly, the higher the rating can be obtained. An evaluation of feeling can be easily obtained.

(Modification 9)
Not only singing by the user but also performance of musical instruments by the user may be evaluated. In this case, the karaoke apparatus 10 includes an interface for acquiring a performance sound signal of a musical instrument. The data structure in the guide melody data is the same as that in the case of evaluating a song as in the embodiment, but the guide melody itself is not limited to the melody of the song, and may be a melody of a keyboard part, for example. The classification and evaluation processing relating to the beats is performed by the same procedure as in the embodiment. In this way, the user can obtain an evaluation of a sense of rhythm in consideration of beats not only for singing but also for playing musical instruments.

(Modification 10)
You may make it notify a user the classification regarding the beat with low evaluation in a user's song. For example, a comment such as “You seem not good at back beats” may be displayed on the UI unit 14 for a type related to a beat with an evaluation score lower than a predetermined threshold. In this way, the user can clearly understand the types of beats that he / she is not good at, so he can practice efficiently.

(Modification 11)
The guide melody data may include the following information in place of the “beat type”, so that the types of beats may be classified. In this case, the guide melody data includes the start time of each measure obtained by dividing the performance period from the start to the end of the music into a plurality of times, and the time information for specifying the time for each of the plurality of measures. Including. For example, when the time is expressed as “mm: ss: sss”, the measure whose start time is “00:00:00” is “4/4 time”, and the start time of the next measure is “00: 04:00 ”. In this case, the control unit 11 specifies the first measure as 4 seconds from the difference between the start times of the measures. In this case, since it is “4/4 time signature”, it is understood that it is 1 second per beat. In accordance with this, the control unit 11 specifies the timing of “00:00:00”, “00:01:00”, “00:02:00”, and “00:03:00” as a table beat in the first measure. To do. As described above, if the timing of the front beat is known, it is possible to classify the types related to beats based on the time difference as in the embodiment. Even if it does in this way, there can exist an effect similar to embodiment.

(Modification 12)
The server device 20 is not limited to the configuration in which the musical sound data representing the musical accompaniment sound and the guide melody data are stored, and the karaoke device 10 may store the data in the storage unit 12. With such a configuration, the karaoke device 10 does not need to communicate with the server device 20, and therefore processing time for communication can be omitted.

(Modification 13)
The evaluation of the tame and the hash based on the evaluation criterion table may be varied depending on the genre of music and the type of beat. In this case, the musical sound data transmitted from the server device 20 includes an identifier indicating the genre of the music. For example, when the genre of the music is “enka”, it is easy to use the song for singing. The karaoke apparatus 10 stores in the storage unit 12 in advance a correspondence table in which an identifier indicating the genre of music is associated with a flag indicating whether or not to perform evaluation for a target or a hash. The control unit 11 determines whether or not to perform evaluation for the target or the hash based on the identifier indicating the received music genre and the correspondence table. For example, if the genre of the song is “Enka”, generally, it is easy to use the song for singing. Therefore, in the correspondence table, “Enka” has the flag for performing evaluation for “Tame” turned on. Yes. In this case, when performing the evaluation in step Sa9, the control unit 11 performs the evaluation using the evaluation score “Tame” in the evaluation criterion table. Further, for example, the control unit 11 may determine in advance the evaluation method according to the type of beat, such as performing an evaluation on “tame” for “table beat”. At this time, which beat type is to be evaluated for the tame or hash is stored in advance in the storage unit 12 as a correspondence table. In this way, it is possible to evaluate the tame and the hash according to the characteristics of the music and the type of beat.

DESCRIPTION OF SYMBOLS 10 ... Karaoke apparatus, 11 ... Control part, 12 ... Memory | storage part, 13 ... Communication part, 14 ... UI part, 15 ... Voice input part, 16 ... Voice output part, 20 ... Server apparatus, 100 ... Singing evaluation system

Claims (6)

For each of a plurality of reference constituent sounds constituting a model sound to be referred to when a user plays a musical piece, model sound data representing the pitch of the reference constituent sound and the pronunciation start time of the reference constituent sound is acquired. Model sound data acquisition means;
Each of the plurality of reference constituent sounds whose pitches and pronunciation start times are represented by the model sound data acquired by the model sound data acquisition means is set to any one of a plurality of types related to beats determined according to a predetermined rule. A classification means for classifying;
A performance sound signal acquisition means for acquiring a performance sound signal representing a performance sound that is a sound generated by a user's performance;
Pitch specifying means for specifying the pitch of the user's performance sound from the performance sound signal acquired by the performance sound signal acquisition means;
For each of a plurality of reference constituent sounds whose pitches and pronunciation start times are represented by the model sound data, pronunciation start time specifying means for specifying the pronunciation start time at which the reference constituent sounds were played according to a predetermined rule;
For each of the plurality of reference component sounds represented by the model sound data, the sound generation start time of the reference component sound, and the sound generation start time of the performance specified by the sound generation start time specifying unit for the reference component sound, Component sound evaluation data generating means for generating component sound evaluation data indicating higher evaluation as the difference between
For each type related to beats, a beat indicating an evaluation for each type based on component sound evaluation data generated by the component sound evaluation data generation unit for one or more reference component sounds classified by the classification unit. A singing evaluation device comprising: evaluation data generating means for each beat type for generating evaluation data for each type. The sound generation start time specifying means relates to each of a plurality of reference constituent sounds whose pitches and sound generation start times are represented by the model sound data, within a period determined based on the sound generation start time of the reference constituent sounds. The time when the pitch of the user's performance sound specified by the pitch specifying means falls within the range of the pitch determined with reference to the pitch of the reference component sound, and the pronunciation of the performance of the reference component sound The singing evaluation device according to claim 1, wherein the singing evaluation device is specified as a start time. The exemplary sound data includes timing data indicating a timing at which the reference constituent sound is generated with respect to each of the at least one or more reference constituent sounds.
The singing evaluation apparatus according to claim 1, wherein the classification unit performs the classification based on the timing data. The exemplary sound data includes data for specifying the start time of each measure obtained by dividing the performance period of the music into a plurality of pieces, and data for specifying time signatures for each of the plurality of measures,
The said classification | category means performs the said classification based on the data which specify the start time of each of the said bar | burr, and the data which specify the time signature regarding each of these several bar | burrs. The classification | category of Claim 1 or 2 characterized by the above-mentioned. Singing evaluation device. The model sound data includes, for each of at least one or more of the reference component sounds, a coefficient to be multiplied by an evaluation score regarding the reference component sound,
The component sound evaluation data generating means indicates an evaluation point obtained by multiplying an evaluation point determined based on a difference between a pronunciation start time of the reference component sound and a pronunciation start time of a performance by the user by the coefficient. The singing evaluation apparatus according to any one of claims 1 to 4, wherein the constituent sound evaluation data is generated. The beat type evaluation data generated by the beat type evaluation data generating means is data indicating the numerical value of the evaluation of each type related to the beat,
6. A calculating means for calculating a numerical value obtained by multiplying a numerical value indicated by the evaluation data for each beat type by a coefficient of a ratio predetermined for each type and adding them. The singing evaluation apparatus in any one of.

Images