JP2017067998A - Singing evaluation system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a singing evaluation system that enables a karaoke device to display a result of accurate evaluation processing with a high processing load if the karaoke device displays a result of singing of a user, and enables the user to continue singing while watching the result of singing and correcting the singing.SOLUTION: A music reproduction device comprises: first display control means for having a result of detected pitch displayed for each section corresponding to prescribed timing when pitch is detected; result reception means for receiving a result of evaluated singing data from a control device; and second display control means for having the result of the singing data evaluated by the control device displayed instead of or in addition to the result of the detected pitch displayed by the first display control means.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a singing evaluation system that enables a user to advance singing while seeing and correcting the result of singing.

  2. Description of the Related Art Conventionally, a technique for evaluating a singing voice of a user who sings along with reproduction of karaoke music is known. Patent Document 1 discloses a scoring server device that acquires singing data from a karaoke device by a streaming method and scores the acquired singing data. The scoring server device divides the time axis of the acquired singing data into a plurality of scoring sections, and scores the user's singing voice in each divided scoring section. The scoring server device transmits the scored score result to the karaoke device.

JP 2010-237257 A

  If the scoring server device obtains the singing voice from the karaoke device and uses the scoring technology, the processing capability of the karaoke device is low and it is difficult for the karaoke device to perform precise scoring processing with a high processing load. In addition, it becomes possible to display a precise scoring result obtained by scoring the singing voice on the karaoke apparatus. However, on the other hand, since the karaoke device makes it necessary to transmit the singing data and receive the grading result with the scoring server device, the karaoke device collects the user's singing voice and generates the singing data. It takes time to transmit / receive to / from the scoring server device and display the singing voice scoring result, and the user's singing scoring result is displayed with a delay with respect to the user's singing progress.

  Displaying the evaluation result of the singing voice is implemented for the purpose of improving the singing ability of the user who sings using the karaoke device. When the evaluation result is displayed during singing, the user sings, The evaluation result displayed on the karaoke device can be viewed. However, as in Patent Document 1, if the karaoke device takes time to display the evaluation result by transmitting and receiving with the scoring server device, the evaluation result is displayed with a delay with respect to the progress of the singing, The user cannot proceed to sing while correcting the result of the evaluation.

  The present invention has been made to solve the above-described problems. When the karaoke apparatus displays the result of the user's singing, the result of precise evaluation processing with a high processing load is displayed. An object of the present invention is to provide a singing evaluation system that enables a user to proceed with singing while seeing and correcting the result of singing.

  The singing evaluation system according to claim 1 includes: a reproducing unit that reproduces music; a singing data obtaining unit that obtains song data sung in accordance with the music reproduced by the reproducing unit; and the singing data obtaining unit. Data transmission means for transmitting the acquired song data to a connected control device; and pitch detection means for detecting a pitch at a predetermined timing for the song data acquired by the song data acquisition means; Corresponding to the predetermined timing when the pitch is detected by the pitch detection means, a first display control means for displaying the pitch detected by the pitch detection means, and a first display control means A result receiving means for receiving a result of evaluation of the singing data from the control device when the result of detecting the pitch is displayed; and the control device And a second display control means for displaying the result of evaluating the song data in place of or in addition to the result of detecting the pitch displayed by the first display control means. And a data receiving means for receiving the singing data from the music reproducing device, and a singing evaluation for evaluating the singing data received by the data receiving means for at least a plurality of types of evaluation items including evaluation items other than the pitch. And a control device comprising a result transmitting means for transmitting the result evaluated by the singing evaluation means to the music reproducing device.

  The singing evaluation system according to claim 2, wherein the singing evaluation unit of the control device changes the pitch at intervals shorter than the pitch detection by the pitch detecting unit as the evaluation item from the singing data. The second display control unit of the music playback device includes a pitch transition evaluation to be evaluated, and the result receiving unit evaluates the song data including the pitch transition evaluation from the control device. At the timing corresponding to the received evaluation result, the result of evaluating the singing data is displayed instead of the result of detecting the pitch displayed by the first display control means. It is characterized by making it.

  5. The singing evaluation system according to claim 3, wherein the music reproducing device acquires singing evaluation reference data including at least a pitch and a length of a sound constituting the music reproduced by the reproducing means. And the pitch detection means detects a pitch at a single timing for each of the intervals, with the pitch and pitch lengths of the sound included in the evaluation reference data as intervals. The display control means displays the pitch detected by the pitch detection means so as to be comparable with the pitch of the evaluation reference data for each section, and the song evaluation means of the control device For each section, the pitches at a plurality of timings are evaluated, and the transition of the pitches at the timings at the plurality of points is evaluated as a pitch transition evaluation of the evaluation item.

  5. The singing evaluation system according to claim 4, wherein the music reproducing device is configured to determine, as the predetermined timing, a determination unit that determines a detection target timing at which a pitch is detected by the pitch detection unit, and the detection by the reproduction unit. Reproduction determination means for determining whether or not the music corresponding to the target timing has been reproduced, and the pitch detection means determines that the reproduction of the detection target timing has been performed by the reproduction determination means. When detected, the pitch of the detection target timing is detected, and the control device includes a feature acquisition unit that acquires a feature amount of a song including at least a pitch from the song data, and the song evaluation unit Is characterized in that the plurality of types of evaluation items are evaluated using the feature amounts acquired by the feature acquisition means.

  The singing evaluation system according to claim 5, wherein the music reproducing device has a performance acquisition unit that acquires processing performance by the control unit or device performance information indicating a resolution of the display unit, and at least one of the processing performance and the resolution. Correspondingly, a storage unit that stores setting interval information in which a setting interval for detecting a pitch by the pitch detection unit is determined according to the device performance information, and the device performance information acquired by the performance acquisition unit And determining means for determining a setting timing for detecting a pitch at the setting interval according to the apparatus performance information as the predetermined timing using the setting interval information, and the pitch detecting means Detects the pitch of the set timing determined by the determining means.

  The singing evaluation system according to claim 6, wherein the music reproducing device is configured to determine whether or not there are a plurality of phrases having overlapping sound transitions in the music reproduced by the reproducing means; and the phrase When it is determined by the determining means that there are phrases with overlapping sound transitions, a pitch difference between the pitches detected by the pitch detecting means for a plurality of phrases with overlapping sound transitions in the song data. Is further provided with a pitch difference determining means for determining whether or not the difference is within a predetermined range, wherein the first display control means is configured to detect a plurality of phrases having overlapping sound transitions by the pitch difference determining means. When it is determined that the pitch difference is within the predetermined range, a phrase with a fast playback time is detected as a result of detecting the pitch of a phrase with a slow playback time among the phrases having the same sound transition. The pitch is characterized in that to display the result detected.

  The program according to claim 7 is a program executed by a computer used in a singing evaluation system constituted by a music playback device and a control device, and includes a playback control step for playing back music and the playback control step. Singing data acquisition step of acquiring singing data sung in accordance with the music to be reproduced, the data transmission step of transmitting the singing data acquired by the singing data acquisition step to a connected control device, and For the singing data acquired by the singing data acquisition step, the pitch detection step for detecting the pitch at a predetermined timing, and the predetermined timing at which the pitch is detected by the pitch detection step, First display control step of displaying the pitch detected by the pitch detection means The result reception step of receiving the result of evaluating the singing data from the control device, and the result of detecting the pitch displayed by the first display control step as a result of detecting the singing data A second display control step for displaying instead of or in addition to is executed by a computer.

  In the singing evaluation system according to claim 1, in addition to displaying the result of detecting the pitch of the singing data by the pitch detecting means and the first display control means, the music reproducing device uses the result receiving means, The evaluation results of a plurality of types of evaluation items evaluated by the control device are received and displayed by the second display control means instead of or in addition to the result of detecting the pitch displayed on the first display control means. . Thereby, the user who uses the music reproducing device of the singing evaluation system can proceed while correcting the singing while seeing the result of detecting the pitch, and the music reproducing device shows the result evaluated by the control device. When received, the evaluation of a plurality of types of evaluation items evaluated by the control device can also be seen.

  In the singing evaluation system according to claim 2, the singing evaluation means of the control device evaluates the transition of the pitch while the pitch detecting means of the music reproducing device detects the pitch. The second display control means of the music reproducing device displays the pitch transition evaluation evaluated by the control device instead of the pitch display result displayed by the first display control means. Thereby, the user who uses the music reproducing device of the singing evaluation system can proceed while correcting the singing by looking at the result of detecting the pitch, and also the result of the evaluation of the music reproducing device by the control device. When received, the evaluation of a plurality of types of evaluation items including the transition of the pitch evaluated by the control device can be seen instead of the result of detecting the pitch.

  The singing evaluation system according to claim 3 detects a pitch at one timing for each section by the pitch detecting means of the music reproducing device, and a plurality of timings for each section by the singing evaluation means of the control device. Evaluate the transition of the pitch of. Thereby, even if it is a music reproduction apparatus with low processing capability, since a pitch can be specified and the result with many evaluation points evaluated by the song evaluation means of a control apparatus is also displayed by the 2nd display control means, a user is displayed. In addition to looking at the results of detecting the pitches displayed by the music playback device, multiple types of evaluation items including pitch transitions more detailed than the pitches evaluated and displayed by the music playback device. You can also see the evaluation.

  The singing display system according to claim 4, in the music playback device, when the playback determination unit determines that the music corresponding to the detection target timing has been played back, the pitch detection unit detects the pitch of the detection target timing. Is detected. In the control device, a plurality of types of evaluation items are evaluated from the transition of the pitch by the feature acquisition means. As a result, when reproduction of the detection target timing is performed, a result of detecting the pitch can be displayed, and a result evaluated from the control device for a plurality of types of evaluation items from the transition of the pitch Can be displayed instead of or in addition to the result of detecting the pitch.

  In the singing evaluation system according to the fifth aspect, the music reproducing device determines the setting timing set according to the setting interval information associated with the device performance information indicating the processing performance of the control unit or the resolution of the display unit. Thereby, in the music reproducing apparatus, the timing for detecting the pitch of the singing is determined by the determining means at set intervals according to the processing capacity or the display capacity. Therefore, the pitch detection to be displayed by the first display control means is detected. The result can be displayed without delay. While singing, the user can see the result of detecting the sung pitch without delay, and can proceed while correcting the singing immediately.

  In the singing evaluation system according to claim 6, when a difference in pitches of detected pitches is within a predetermined range for a plurality of phrases having overlapping sound transitions, a pitch of a phrase having a slow reproduction time is detected. As a result, the result of detecting the pitch of a phrase having a fast playback time is displayed. Thus, when a phrase with a slow playback time is played back, the pitch detection result can be easily displayed by the music playback device without detecting all the pitches of the phrase.

  According to the program of claim 7, in addition to being able to display the result of detecting the pitch of the singing data by the pitch detection step and the first display control step, the result receiving step allows the control device to Evaluation results of a plurality of types of evaluated evaluation items are received, and the second display control step displays instead of or in addition to the results of detecting the pitches displayed in the first display control step. Thereby, the user can proceed while correcting the singing while seeing the result of detecting the pitch, and is evaluated by the control device when the result of evaluation of the program to be executed is received by the control device. You can also see the evaluation of multiple types of evaluation items.

FIG. 1 is a diagram showing the overall configuration of the singing evaluation system 1. FIG. 2 is a diagram showing an outline of the electrical configuration of the music playback device 20 and the server 50 of the singing evaluation system 1. It is a main flowchart which shows the process sequence of a terminal song evaluation program. It is a sub-flowchart which shows the process sequence of the terminal pitch detection process of a terminal song evaluation program. It is a figure which shows the change transition of the display of the display of the music reproduction apparatus 20 along the process sequence of a terminal song evaluation program. It is the figure which showed the example of the detection timing of the pitch in the process of a terminal song evaluation program. 4 is a flowchart showing a processing procedure of processing executed by a server 50. It is a sub-flowchart which shows the process sequence of the terminal pitch detection process of the terminal song evaluation program in 2nd Embodiment. It is a sub-flowchart which shows the process sequence of the terminal pitch detection process of the terminal song evaluation program in 3rd Embodiment. It is a figure which shows the modification of the detection timing of the pitch detected from song data by the terminal song evaluation program of 2nd Embodiment.

  Hereinafter, embodiments of the singing evaluation system of the present invention will be described. In the singing evaluation system, a device that reproduces music and a server that scores singing are connected via a network such as the Internet. A device for reproducing music is a mobile communication terminal device such as a smartphone or a tablet terminal, a karaoke terminal device having a low processing capability, a music reproduction player, or the like. The processing capability is low, for example, the processing capability of the CPU of the karaoke device is lower than the processing capability of the CPU of the server that scores the song, or the processing capability of the CPU of the karaoke device is an evaluation process for a plurality of types of evaluation items. Is lower than the processing capacity that can be executed without delay. An apparatus for reproducing music includes a display and displays a result of evaluating a voice sung by the user in accordance with the reproduction of the music. The apparatus which reproduces music is connected so that communication with the server which scores a song is possible. The server that scores a song is, for example, a server provided by a karaoke service provider. Hereinafter, the present embodiment will be described with reference to the drawings.

[First Embodiment]
[Configuration of the embodiment]
The singing evaluation system 1 which is an example of the singing evaluation system of this invention is demonstrated. In the singing evaluation system 1 shown in FIG. 1, a music playback terminal 20 and a server 50 are connected via a network N. In FIG. 1, a mobile communication terminal 20 </ b> A and a karaoke terminal 20 </ b> B are shown as examples of the music playback terminal 20. In the first embodiment, there may be at least one music playback terminal 20. In the first embodiment, the mobile communication terminal 20 </ b> A and the karaoke apparatus 20 </ b> B are collectively referred to as the music playback terminal 20 as follows. The music playback terminal 20 includes a display 21. The music playback terminal 20 transmits and receives information to and from the server 50 via the network N. The music playback terminal 20 stores in advance the address of the server 50 that is the communication partner.

  As shown in FIG. 2, the music playback terminal 20 includes a CPU 22, a ROM 23, a RAM 24, a network connection unit 25, and an input / output interface 26. The ROM 23 includes a terminal song evaluation program storage area 231 and a terminal performance information storage area 232. The terminal song evaluation program storage area 231 stores a terminal song evaluation program executed by the CPU 22. Details of the terminal song evaluation program will be described later. In the terminal performance information storage area 232, the processing capability of the CPU 22 indicating the terminal performance and the display resolution of the display 21 are stored. The processing capability of the CPU 22 is, for example, a clock frequency. The RAM 24 includes a reproduction music storage area 241, a song data storage area 242, and a detection information storage area 243.

  The reproduction music storage area 241 of the RAM 24 stores performance data of music reproduced by the music reproduction terminal 20. The musical performance data is, for example, data including karaoke accompaniment data and melody data for a musical piece designated to be played. The melody data is, for example, MIDI data in which the pitch and length of the music are defined. In MIDI data, the pitch is determined by the note number, and the sounding start timing (note-on timing) and sounding end timing (note-off timing) corresponding to the sound length are determined in association with the note number. The sound generation start timing and the sound generation end timing are defined, for example, as the elapsed time from the start of the performance with the performance start time of the music set to 0 seconds. The sound generation end timing may be determined by an absolute time from the sound generation start timing. The melody data may be singing model data that can specify a pitch and a change in pitch. Further, lyrics telop data of music may be stored. The data stored in the playback music storage area 241 may be stored in advance in the music playback terminal 20, or may be acquired and stored from the server 50 when playback of music is instructed. The singing data storage area 242 stores the singing voice of the user who sang the music reproduced by the music reproducing terminal 20 and the result of evaluating the singing voice. The result of evaluating the singing voice includes a result evaluated by the music playback terminal 20 and a result evaluated by the server 50 described later. The detection information storage area 243 stores a detection timing indicating a timing for detecting a pitch of music played by the music playback terminal 20 and a pitch detected at the detection timing.

  The music playback terminal 20 is connected to a network N such as the Internet via the network connection unit 25, and exchanges information with the server 50. The music playback terminal 20 is connected to the network N by wire or wireless. The network connection unit 25 is specifically a network interface, for example. A display 21, a speaker 27, a microphone 28, and an input unit 29 are connected to the input / output interface 26. The display 21, the speaker 27, the microphone 28, and the input unit 29 may be configured integrally with the music playback terminal 20.

  The display 21 outputs the result of evaluating the singing voice. When the lyrics telop data is stored in the reproduction music storage area 241 of the RAM 24, the lyrics telop may be output to the display 21 when the music is being reproduced. From the speaker 27, the music sound is output according to the accompaniment data. The user's singing voice is input through the microphone 28. The input singing voice is stored in the singing data storage area 242 of the RAM 24 and transmitted to the server 50 via the network connection unit 25. The input unit 29 receives a designation of music to be played by the music playback terminal 20. The input unit 29 is, for example, a keyboard. Alternatively, when the display 21 is configured with a touch panel display, the display 21 may include the input unit 29.

  As shown in FIG. 2, the server 50 includes a CPU 52, a ROM 53, a RAM 54, an HDD 55, and a network connection unit 56. The ROM 53 includes a detailed song evaluation program storage area 531 and a detection timing determination reference storage area 532. The detailed song evaluation program storage area 531 stores a detailed song evaluation program executed by the CPU 52. The detection timing determination reference storage area 532 stores determination criteria for determining detection timing. Details of the detailed song evaluation program and the determination criteria will be described later.

  The RAM 54 includes a reception data storage area 541, a singing section data storage area 542, and an evaluation result storage area 543. The received data storage area 541 stores the singing voice received from the music playback terminal 20. The evaluation section data storage area 542 stores information indicating a part of the singing sections for evaluating the singing voice. The evaluation result storage area 543 stores a result of evaluating the singing voice for each singing section.

  The HDD 55 includes a song lyrics storage area 551 and a detailed evaluation data storage area 552. The music lyrics storage area 551 stores music accompaniment data, melody data, and lyrics telop. The detailed evaluation data storage area 552 stores singing evaluation data indicating evaluation criteria for a plurality of types of evaluation items for evaluating the user's singing voice transmitted from the music playback terminal 20. The plurality of types of evaluation items include, for example, the transition of the singing pitch and the singing technique.

  The detailed evaluation data includes at least model singing data indicating a singing model, singing technique information for identifying a singing technique, scoring information indicating a singing scoring standard, and relative evaluation data. The model song data is data in which the sound transition of the song is more detailed than the melody data of the music. The singing technique information includes waveform data indicating a criterion for singing techniques such as vibrato, fist, and so on, or information indicating a determination condition. The relative evaluation data is, for example, comprehensive evaluation data in which evaluation results obtained by evaluating singing voices of a plurality of other singers who have sung a song in the past are stored in association with ranks. By storing a plurality of comprehensive evaluation data for each singer as relative evaluation data in the HDD 55, a plurality of evaluation results of the singing voice input to the microphone 28 are compared with a plurality of comprehensive evaluation data. It is possible to derive a ranking that compares the skill of singing with other singers.

  The model song data includes at least a pitch (pitch), a sound length, and a sound pressure (volume) in the sound transition of the song. The degree of coincidence of the pitches of the singing can be evaluated by comparing the waveform in which the model singing data is reproduced with at least a time-series arrangement of pitches and the waveform of the pitch of the singing voice. The singing technique includes, for example, a plurality of types such as vibrato, and fist, and includes a waveform that represents an evaluation criterion for evaluating the transition of the sound that is characteristic of the user's singing voice for each singing technique. By comparing the waveform representing the evaluation criteria for each singing technique with the waveform of the singing voice, it can be determined whether or not the singing technique has been performed. The singing scoring standard indicates an evaluation standard for calculating an absolute score compared with a predetermined standard. The scoring standard may include an evaluation of the degree of coincidence of singing pitches and a determination as to whether or not the singing technique has been performed, which is scored according to the evaluation result and the determination result. Moreover, the relative evaluation criteria which determined the relative order | rank compared with the singing result of several other singer may be included.

  The server 50 receives the singing voice acquired by the music playback terminal 20 via the network connection unit 56, and evaluates the singing voice using the detailed evaluation data, and the result of the evaluation is obtained via the network connection unit 56. Transmit to the playback terminal 20.

[Operation and Action of Embodiment]
The music playback terminal 20 is activated by being connected to an external power source through an internal power source or an outlet. In the music playback terminal 20, when an instruction for evaluating song singing by a user is detected, the CPU 22 executes a terminal song evaluation program whose processing procedure is shown in FIGS. 3 and 4. Execution of the terminal song evaluation program is started when the user gives an instruction to reproduce a song to be sung and to execute a song evaluation function by a button (not shown) displayed on the input unit 29 or the display 21. FIG. 3 is a main flowchart showing the processing procedure of the terminal song evaluation program executed by the CPU 22 of the music playback terminal 20. FIG. 4 is a sub-flowchart showing the procedure of the terminal pitch detection process of the terminal song evaluation program executed by the CPU 22 of the music playback terminal 20.

(Terminal song evaluation program processing)
When the process of the terminal song evaluation program shown in FIG. 3 is executed, in step S2, the reproduction of the music specified by the user is started. In step S2, when the reproduction of the music is started, the server 50 is notified of information for identifying the music to be reproduced and starting the reproduction of the music. In step S2, playback of the karaoke accompaniment data read from the playback music storage area 241 is started. When reproduction of the accompaniment data is started in step S2, reproduction time measurement is started. When the process of step S2 is executed, the screen shown in FIG. When the playback of the music is started, as shown in FIG. 5A, a position bar 31 indicating the playback position is displayed in a thick solid line on the screen of the display 21, and the time of the music to be played back is in time series. A pitch bar 32 indicating the pitch is displayed as a rectangle. The position of the pitch bar 32 indicating the pitch is determined according to the pitch change indicated by the melody data. When the process of step S2 is performed, the position bar 31 is positioned at the head of the sound to be reproduced. When the reproduction of the music is started, the position of the position bar 31 on the screen of the display 21 moves as the music progresses. Alternatively, the pitch bar 32 may move as the music progresses, and the relative position between the pitch bar 32 and the position bar 31 may move.

  In step S4, it is determined whether or not an input to the microphone 28 has been acquired. In step S4, when an input sound from the microphone 28 is detected via the input / output interface 26, it is determined that an input to the microphone 28 has been acquired (step S4: YES). If it is determined that the input to the microphone 28 has been acquired, the process of step S6 is executed. When the input sound from the microphone 28 is not detected via the input / output interface 27, it is not determined that the input to the microphone 28 has been acquired (step S4: NO), and the process of step S4 is repeatedly executed.

  In step S6, the singing voice acquired in step S4 and stored in the singing data storage area 242 is transmitted to the server 50 as singing data. In step S6, the singing data obtained from the singing voice every time the process of step S6 is performed is transmitted to the server 50 by the streaming method. In step S <b> 6, the singing data to be transmitted to the server 50 may be transmitted to the server 50 as data obtained by storing singing voices for a predetermined time.

  In step S8, the terminal pitch detection process shown in FIG. 4 is executed. In the terminal pitch detection process, first, in step S81, the processing performance of the CPU 22 is read from the terminal performance information storage area 232 and acquired.

  In step S82, the detection timing for detecting the pitch from the singing voice is determined from the processing performance value of the CPU 22 acquired in step S81. In step S82, for example, the detection timing of a different detection interval is determined depending on whether or not the value is equal to or greater than a predetermined threshold according to the value of the clock frequency that is a value indicating the processing performance of the CPU 22. For example, a plurality of types of associations between clock frequency ranges and detection timings are stored as determination criteria in the detection timing determination criterion storage area 532, and the shorter the value of the clock frequency is, the shorter (narrower) the value is. The detection timing is determined at the detection interval, and the detection timing is determined at a longer (wider) detection interval as the value of the clock frequency is smaller. In the present embodiment, the detection timing is set at a predetermined detection interval for each pitch. For example, when the value of the clock frequency is smaller than the threshold value, as shown by a black inverted triangle (▼) in FIG. The detection timing is set at detection intervals every 2 seconds from the sound generation start timing. For example, when the value of the clock frequency is equal to or greater than the threshold value, as indicated by a white inverted triangle (▽) in FIG. As described above, the detection timing is set at detection intervals every second from the sound generation start timing. One pitch is a period from when a change in pitch is detected from the melody data until a sound generation end timing indicated by a pitch associated with the detected pitch after the change. Alternatively, one pitch is during a constant pitch from when the pitch changes until the next pitch is changed.

  In addition, the threshold value for determining the detection interval in step S82 may be determined as a plurality of stages of three or more stages. In step S82, three or more different detection intervals may be determined depending on whether or not a plurality of threshold values are exceeded. The detection intervals may be set at equal intervals from the start of music performance, regardless of the sound generation start timing of each sound. Further, the width of each detection interval may be determined to be a different interval according to the processing performance of the CPU 22. For example, when the value of the clock frequency is smaller than the threshold value, the detection timing is determined once every 3 seconds as shown by a black inverted triangle (▼) in FIG. When the value is equal to or greater than the threshold value, the detection timing may be determined once per second as indicated by a white inverted triangle (▽) in FIG. 6B. In this case, the interval from the sound generation start timing to the detection timing of each sound is not fixed.

  The detection timing determined in step S82 is stored in the reproduction music storage area 241 in association with the elapsed time from the start of music performance. Alternatively, the detection timing may be stored in the reproduction music storage area 241 in association with the melody data of the reproduction music or the accompaniment data of the reproduction music.

  In step S83, it is determined whether or not the music playback position has reached the detection timing determined in step S82. When it is determined that the playback position of the music has reached the detection timing (step S83: YES), the process of step S84 is executed. When it is not determined that the music playback position has reached the detection timing (step S83: NO), the process of step S83 is repeatedly executed.

  In step S84, the pitch of the singing voice at the reproduction position of the music determined to have reached the detection timing in step S83 is detected. When step S84 is executed, the process of step S10 shown in FIG. 3 is executed.

  In step S10, the pitch detected in step S84 is displayed on the display 21. Specifically, in step S10, drawing data for displaying the pitch detected in step S84 on the display 21 is created. On the display 21, for example, as shown in FIG. 5B, the pitch detected from the singing voice input to the microphone 28 along with the pitch bar 32 indicating the pitch of the melody data of the music is hatched. An audio bar 33 indicated by an attached rectangle is displayed. In the first embodiment, when the timing at which the playback time is the middle among the pitches of one pitch in the melody data is determined as the pitch detection timing in step S82, the melody data is determined in step S10. A pitch bar 32 indicating the pitch of one pitch at the time when the pitch at the middle timing of the pitch of one pitch is detected, even if the pitch of one pitch is not reproduced to the end In parallel, the audio bar 33 having the same length is displayed at the detected pitch.

  In step S12, it is determined whether or not there is a request to receive evaluation result data from the server 50. Details of the evaluation result data will be described later. In step S12, when unreceived evaluation result data is transmitted from the server 50, it is determined that there is a reception request (step S12: YES). If it is determined that there is a reception request, the process of step S14 is executed. When the evaluation result data not yet received is not transmitted from the server 50, it is not determined that there is a reception request (step S12: NO), and the process of step S16 is executed.

  In step S14, the evaluation result up to the reproduction time included in the evaluation result data received in step S12 is displayed on the display 21 in step S10. In step S14, the data included in the drawing data created in step S10 is taken over, and drawing data for displaying the evaluation result up to the reproduction time included in the evaluation result data received in step S12 on the display 21 is created. When the process of step S14 is performed when the reproduction of the music progresses, the singing voice until the evaluation completion time T, which is the reproduction time included in the evaluation result data, is displayed on the display 21 as shown in FIG. For example, as shown in FIG. 5C, the waveform 23 indicating the sound transition of the singing voice is displayed instead of the voice bar 33, and the technique name 35 indicating the singing technique is additionally displayed. Is done. At this time, the waveform 34 as the evaluation result up to the evaluation completion time T may be additionally displayed on the audio bar 33 displayed on the display 21. Further, as shown in FIG. 5 (C), the singing voice scoring results up to the evaluation completion time T and the singing voice evaluation results of the person who sang the same music using the singing evaluation system 1 were compared. The scoring order may be displayed on the display 21.

  In step S14, when the evaluation result data received in step S12 is the reproduction of the music, that is, the evaluation result up to the end of the music, the evaluation result data includes the result of comprehensive evaluation of the singing voice. It may be. FIG. 5D is a diagram illustrating an example of the evaluation result displayed on the display 21 when the evaluation result data is data including the evaluation result up to the end of the music. For example, as shown in FIG. 5D, in addition to the waveform 34 indicating the sound transition of the singing voice and the technique name 35 indicating the singing technique, the song 21 of the song showing the overall evaluation result of the singing voice is displayed on the display 21. The ranking which compared the evaluation result of the singing voice and the person who sang the same music using the scoring result and the singing evaluation system 1 is displayed.

  In step S16, it is determined whether or not a singing evaluation end instruction has been issued. It is determined that the instruction to end song evaluation has been given when the instruction to finish playing the music is given, when the music playing is completed, or when the execution of the song evaluation function is instructed to end. Is done. When it is determined that the singing evaluation end instruction has been given (step S16: YES), the server 50 is notified that the singing evaluation end instruction has been issued, and the processing of the terminal singing evaluation program is ended. When it is not determined that the singing evaluation end instruction has been given (step S16: NO), the process of step S4 is executed again.

  Next, processing executed by the server 50 will be described with reference to FIG. The server 50 is activated by being connected to an internal power source or an external power source via an outlet. When the server 50 receives from the music playback terminal 20 information for identifying the music to be played back and a notification to start playback of the music, the execution of the detailed song evaluation program whose processing procedure is shown in FIG. 7 is started. The FIG. 7 is a flowchart showing the processing procedure of the detailed song evaluation program executed by the CPU 52 of the server 50.

  In step S22, it is determined whether or not song data has been received from the music playback terminal 20. When it is determined that the song data has been received (step S22: YES), the song data is stored in the received data storage area 541, and the process of step S24 is executed. When it is not determined that the singing data has been received (step S22: NO), the process of step S22 is repeatedly executed.

  In step S24, a waveform indicating the transition of the pitch is created from the song data received in step 22. Specifically, in step S24, from the song data, the pitch of the singing voice is shorter than the pitch detection interval by the terminal pitch detection processing of the music playback terminal 20 (for example, 30 detection intervals per second). The result of connecting the pitches detected in) is created as a waveform indicating the pitch transition of the singing voice. In step S24, a waveform is created at predetermined intervals from song data transmitted from the music playback terminal 20 by the streaming method, and stored in the evaluation section data storage area 542 as evaluation section data to be evaluated. The waveform created in step S24 includes feature quantities including pitch change, sound length, sound pressure (volume), and the like.

  In step S26, the singing voice is evaluated from the waveform of the evaluation section data stored in the evaluation section data storage area 542 in step S24. Specifically, the evaluation process compares the pitch transition indicated by the waveform of the evaluation section data with the pitch transition indicated by the exemplary singing data of the detailed evaluation data stored in the detailed evaluation data storage area 552. The pitch difference with the model song data in the pitch transition of the section data is detected. The detected pitch difference is stored in the evaluation result storage area 543 in association with the singing timing.

  In step 28, feature amounts are detected from the waveform created in step S24. Then, the feature amount of the waveform of the evaluation section data is compared with the singing technique information of the detailed evaluation data stored in the detailed evaluation data storage area 552, and the predetermined singing technique indicated in the singing technique information is included. Determine whether or not. When it is determined that the singing technique is included, the singing technique type and the singing timing at which the singing technique was performed are stored in the singing result storage area 543 in association with the evaluation section data.

  In step S30, the pitch difference detected in step S26 and the singing timing determined in step S28 are checked against the scoring information of the detailed evaluation data, and the evaluation section data is scored. When evaluation section data of a plurality of singing sections is stored in the singing result storage area 543, a scoring result obtained by scoring a plurality of stored evaluation section data is stored in the singing result storage area 543. To do.

  In step S32, the pitch waveform created in step S24, the type of singing technique detected in step S28 and the singing timing at which the singing technique was performed, and the result scored in step S30 are the music playback terminal 20. Sent to. At this time, when the result of transmission of the music playback terminal 20 is displayed, an image shown in FIG. 5C is displayed on the display 21. When step S32 is performed and the scored result is displayed on the display 21, for example, as shown in FIG. 5C, the same is performed using the singing voice scoring result at the present time and the singing evaluation system 1. The scoring rank comparing the evaluation result of the singing voice with the person who sang the music is displayed on the display 21.

  In step S34, it is determined whether or not an instruction to end the song evaluation is given. When the singing evaluation end instruction is given to the music playback terminal 20, and the singing evaluation end instruction is notified from the music playback terminal 20, it is determined that the grading of the music has ended. The singing evaluation of the music is performed when the music playback terminal 20 is notified of the end or stop of the music playback, or the information for identifying the music to be played back notified by the music playback terminal 20 and the music When the reproduction completion time of the music can be detected from the notification of starting the reproduction, it may be determined that there is an end instruction when the reproduction completion time has elapsed from the reproduction start of the music. If it is determined that the reproduction of the music has ended (step S34: YES), the process of step S36 is executed. If it is not determined that the music has been played back (step S34: NO), the process of step S22 is repeated.

  In step S <b> 36, the overall evaluation result of the music is transmitted to the music playback terminal 20. The overall evaluation result of the music is relative to the plurality of singers determined by creating the comprehensive evaluation data from the grading results of the music scored in step S30 and comparing it with the comprehensive evaluation data of a plurality of other singers. It is a ranking. The determined order is transmitted to the music playback terminal 20 as comprehensive evaluation result data. Moreover, in step S30, about the singer of an evaluation object person, when a singing technique is performed in consideration of a singing technique, you may determine a relative order | rank from the comprehensive evaluation data which added it. .

  The music playback terminal 20 displays the comprehensive evaluation result transmitted from the server 50 on the display 28. At this time, for example, the image shown in FIG. 5D is displayed on the display 21 of the music playback terminal 20.

[Second Embodiment]
FIG. 8 is a diagram showing a second embodiment of terminal pitch detection processing executed in the music playback terminal 20 described above. In the singing evaluation system of the second embodiment, the configuration other than the terminal pitch detection processing of the music playback terminal 20 is the same as the configuration of the first embodiment described above, and thus the description thereof is omitted. In the terminal pitch detection process described above, the pitch detection timing is determined by a detection interval determined by comparing the value of the clock frequency of the CPU 22 of the music playback terminal 20 with a threshold value. However, the timing for detecting the pitch of a song is not limited to this. For example, the pitch detection timing may be determined from the length of each sound of the melody data.

  For example, in the terminal pitch detection process shown in FIG. 8, in step S91, the sound to be pronounced next is acquired from the melody data of the music that has been reproduced in step S2. The sound of the melody data acquired in step S91 is associated with a playback time from the start of music playback, which is the sound generation start timing. The sound to be sounded next is one sound whose sound generation start timing is determined, and one sound whose sound pitch is determined up to the sound whose next sound generation start timing is determined.

  In step S92, the sound generation start timing is read out from the sound acquired in step S91, and it is determined whether or not the reproduction time from the start of music reproduction has passed the sound generation start timing of the read sound. If it is determined that the playback time has passed the sound generation timing (step S92: YES), step S93 is executed. If it is not determined that the reproduction time has passed the sounding start timing (step S93: NO), step S92 is repeatedly executed.

  In the terminal pitch processing shown in FIG. 8, in addition to the reproduction time, the sound length of the sound may be acquired as the sound of the melody data acquired in step S91. When the sound length is acquired in step S91, in step S92, for example, as shown by a black inverted triangle (▼) in FIG. The timing at which the playback time is in the middle may be determined as the detection timing. Alternatively, as indicated by a white inverted triangle (▽) in FIG. 10, the pitch of the start point and end point of each one sound in the melody data may be detected. The start point and end point of a sound can be determined from the length of the sound.

  In step S93, the pitch of the microphone input sound acquired in step S4 is detected. When step S93 is performed, step S10 is executed.

[Third Embodiment]
FIG. 9 is a diagram showing a third embodiment of terminal pitch detection processing executed in the music playback terminal 20 described above. In the singing evaluation system of the third embodiment, the configuration other than the terminal pitch detection process of the music playback terminal 20 is the same as the configuration of the first embodiment described above, and thus the description thereof is omitted. In the terminal pitch detection process of the first embodiment, the pitch is detected at a detection interval determined by comparing the value of the clock frequency of the CPU 22 of the music playback terminal 20 with a threshold value. However, the method of detecting the pitch of a song is not limited to this. For example, when the same phrase as the already sung phrase appears later in one song, detection of the pitch of the same phrase appearing after the second time may be omitted.

  For example, in the terminal pitch detection process shown in FIG. 9, in step S100, the current reproduction time of the music that has been reproduced in step S2 is acquired. In step S101, the melody data of the music that has been played back in step S2 is read, and phrase breaks configured for a predetermined time or a predetermined number of beats are detected from the read melody data. In step S102, the phrase to be sung first among the phrases scheduled to be sung after the current playback time acquired in step S100 is detected from the phrase break detected in step S101. Identified as the target phrase. Note that the phrase may be configured to be divided for every predetermined time or a predetermined number of beats, or every display timing of the lyrics defined in the lyrics telop. Further, a break may be defined for each character displayed in a lump in the lyrics telop.

  In step S103, it is determined whether or not a phrase whose sound changes with the same pitch and length as the target phrase specified in step S102 exists in a phrase before the current reproduction time. If it is determined that the same phrase as the target phrase specified in step S102 previously exists (step S103: YES), the process of step S104 is executed. If it is not determined that the same phrase as the target phrase specified in step S102 previously exists (step S103: NO), the process of step S109 is executed.

  In step S104, it is determined whether or not the sound generation start timing of the head sound of the target phrase has passed. When the current playback time has passed the sound generation start timing of the first sound of the target phrase, it is determined that the sound generation start timing of the first sound of the target phrase has passed. If it is determined that the sound generation start timing of the first sound of the target phrase has passed (step S014: YES), the process of step S105 is executed. If it is not determined that the sound generation start timing of the first sound of the target phrase has passed (step S014: NO), the process of step S104 is repeatedly executed.

  In step S105, the pitch of the first sound of the target phrase is detected. The detected pitch is stored in the detection information storage area 243. In step S106, the pitch detected in the head sound of the phrase having the same sound transition as that of the target phrase determined to be before is read from the detection information storage area 243.

  In step S107, it is determined whether or not the difference between the pitch detected in step S105 and the pitch read in step S106 is within a predetermined range. If it is determined that it is within the predetermined range (step S107: YES), the process of step S108 is executed. If it is not determined that it is within the predetermined range (step S107: NO), the process of step S109 is executed.

  In step S108, the detected pitch of the target phrase is considered to be the same as the detected pitch of the phrase having the same sound transition as the target phrase determined to be present in step S103, and the detected pitch of the target phrase is the same as the detected phrase. The detected pitch is set. In step S109, the pitch of the target phrase is detected. The pitch detection interval in step S109 is not particularly mentioned.

  When the process of step S108 or step S109 is executed, the process of step S10 is executed.

(Modification)
In the singing evaluation system of the first embodiment, the pitch of the evaluation point indicated by the black inverted triangle (▼) in FIG. 6A is detected by the music playback terminal 20 at a detection timing once per pitch. It was detected. On the other hand, in the server 50, in step S24, the pitches of the singing voices are connected to the pitches detected at intervals shorter than the pitch detection intervals by the terminal pitch detection processing of the music playback terminal 20. The result was created as a waveform indicating the pitch transition of the singing voice. At this time, the server 50 may be a method that is detected more times than the number of times detected by the music reproduction terminal 20, and does not have to create a waveform. For example, as indicated by a white inverted triangle (▽) in FIG. 10, the pitch at the start point and end point of one pitch may be detected according to the melody data.

  In 1st Embodiment, in the process of step S82 of the server 50, the detection timing which detects a pitch from song voice was determined from the value of the processing performance of CPU22 acquired in step S81. The method is not limited to this. For example, in step S81, the display resolution of the display 21 of the music playback device 20 is acquired, and the detection timings of different detection intervals are determined depending on whether or not the display resolution is greater than or equal to a predetermined threshold value. It may be a thing.

[Correspondence between the present invention and the embodiment]
The music playback terminal 20 of the first embodiment is an example of a music playback device of the present invention. The server 50 of the first embodiment is an example of the control device of the present invention. The CPU 22 of the music playback terminal 20 of the present embodiment is an example of the playback means, pitch detection means, first display control means, result reception means, and second display control means of the present invention, and the server 50 of the first embodiment. The CPU 52 is an example of data receiving means, singing evaluation means, and result transmitting means.

  The processing capability of the CPU 22 indicating the terminal performance of the first embodiment and the display resolution of the display 21 are examples of the apparatus performance information of the present invention. The determination reference stored in the detection timing determination reference storage area 532 of the ROM 53 of the first embodiment is an example of the setting interval information of the present invention. The detection timing of 1st Embodiment is an example of the evaluation object timing and setting timing of this invention.

  The process of step S2 executed by the music playback terminal 20 of the first embodiment is an example of the playback control step of the present invention, the process of step S4 is an example of the song data acquisition step of the present invention, and step S8. Is an example of the pitch detection step of the present invention, the process of step S10 is an example of the first display control step of the present invention, and the process of step S12 is an example of the result receiving step of the present invention. Yes, the process of step S14 is an example of the second display control step of the present invention.

1 Singing Evaluation System 20 Music Playback Terminal 21 Display 22 CPU
23 ROM
24 RAM
25 Network Connection Unit 26 Input / Output Interface 27 Speaker 28 Microphone 29 Input Unit 50 Server 52 CPU
53 ROM
54 RAM
55 HDD
56 Network connection N Network

Claims (7)

Playback means for playing back music;
Singing data obtaining means for obtaining singing data sung in accordance with the music reproduced by the reproducing means;
Data transmitting means for transmitting the singing data acquired by the singing data acquiring means to a connected control device;
About the song data acquired by the song data acquisition unit, a pitch detection unit that detects a pitch at a predetermined timing;
First display control means for displaying the pitch detected by the pitch detection means in response to the predetermined timing when the pitch is detected by the pitch detection means;
Result receiving means for receiving a result of evaluation of the singing data from the control device;
Second display control means for displaying the result of evaluation of the singing data by the control device instead of or in addition to the result of detecting the pitch displayed by the first display control means;
A music playback device comprising:
Data receiving means for receiving the singing data from the music reproducing device;
Singing evaluation means for evaluating the singing data received by the data receiving means for a plurality of types of evaluation items including evaluation items other than at least pitches;
A result transmitting means for transmitting the result evaluated by the singing evaluation means to the music reproducing device;
A control device comprising:
A singing evaluation system. The singing evaluation means of the control device includes a pitch transition evaluation that evaluates a transition of pitches at intervals shorter than the pitch detection by the pitch detecting means as the evaluation item from the singing data. Yes,
The second display control means of the music reproducing device receives the evaluated result received when the result receiving means receives the evaluation result of the song data including the pitch transition evaluation from the control device. The result of evaluating the singing data is displayed in place of the result of detecting the pitch displayed by the first display control means at a timing corresponding to. Singing evaluation system. The music player is
Comprising evaluation reference data acquisition means for acquiring evaluation reference data of a song including at least a pitch and a tone length of a sound constituting the music reproduced by the reproduction means;
The pitch detection means detects the pitch at a single timing for each of the sections, with the pitch and pitch of the sound included in the evaluation reference data as sections.
The first display control means is configured to display the pitch detected by the pitch detection means so as to be comparable with the pitch of the evaluation reference data for each section.
The singing evaluation means of the control device evaluates pitches at a plurality of timings for each section, and evaluates transitions in pitches at the timings of the plurality of points as pitch transition evaluations of the evaluation items. The singing evaluation system according to claim 1, wherein the singing evaluation system is a singing evaluation system. The music player is
Determining means for determining a detection target timing for detecting a pitch by the pitch detecting means as the predetermined timing;
Replay determining means for determining whether or not the music corresponding to the detection target timing has been replayed by the replay means,
The pitch detection means detects the pitch of the detection target timing when the reproduction determination means determines that the detection target timing has been reproduced.
The controller is
From the song data, comprising a feature acquisition means for acquiring a feature quantity of a song including at least the pitch,
The singing evaluation unit according to any one of claims 1 to 3, wherein the singing evaluation unit evaluates the plurality of types of evaluation items by using the feature amount acquired by the feature acquisition unit. Evaluation system. The music player is
Performance acquisition means for acquiring processing performance by the control unit, or device performance information indicating the resolution of the display unit,
A storage unit that stores setting interval information in which a setting interval for detecting a pitch by the pitch detection unit is determined according to the device performance information in association with at least one of the processing performance and the resolution;
Using the apparatus performance information acquired by the performance acquisition unit and the setting interval information, a setting timing for detecting a pitch at the setting interval corresponding to the apparatus performance information is determined as the predetermined timing. A determination means;
With
The singing evaluation system according to claim 1, wherein the pitch detection unit detects a pitch at the set timing determined by the determination unit. The music player is
Phrase judging means for judging whether or not a plurality of phrases having overlapping sound transitions exist in the music reproduced by the reproducing means;
When it is determined by the phrase determination means that there are phrases with overlapping sound transitions, the pitch sounds detected by the pitch detection means for a plurality of phrases with overlapping sound transitions in the song data. Pitch difference judgment means for judging whether or not the height difference is within a predetermined range,
The first display control means includes
When it is determined by the pitch difference determining means that the pitch differences of a plurality of phrases with overlapping sound transitions are within a predetermined range, of the phrases with a slow playback time among the plurality of phrases with overlapping sound transitions The singing evaluation system according to any one of claims 1 to 5, wherein a result of detecting a pitch of a phrase having a quick playback time is displayed as a result of detecting a pitch. A program executed by a computer used in a singing evaluation system composed of a music playback device and a control device,
A playback control step for playing a song;
Singing data acquisition step of acquiring singing data sung according to the music played back by the playback control step;
A data transmission step of transmitting the singing data acquired by the singing data acquisition step to a connected control device;
About the song data acquired by the song data acquisition step, a pitch detection step of detecting a pitch at a predetermined timing;
A first display control step of displaying the pitch detected by the pitch detection means in response to the predetermined timing at which the pitch is detected by the pitch detection step;
From the control device, a result receiving step of receiving a result of evaluating the singing data;
A second display control step for displaying the result of detecting the song data instead of or in addition to the result of detecting the pitch displayed by the first display control step;
A program that causes a computer to execute.