JP6415341B2 - Karaoke system with pitch shift function for harmony singing - Google Patents

Description

  The present invention relates to a karaoke system having a pitch shift function for harmony singing. Specifically, in a harmony section of karaoke music composed of a plurality of singing parts, the singing voice input from a microphone is pitch-shifted appropriately. The present invention relates to a karaoke system capable of generating singing voices for harmony.

  A song (for example, a duet song) that a plurality of singers sing at the same time is composed of a section in which the singers alternately sing and a section in which the plurality of singers sing simultaneously while harming (harmony section). It is common to be. This harmony section is an exciting singing section for both the audience and the singer, and it is desired to sing the harmony well. In view of this, a technique for automatically generating a hammer sound by pitch shifting the singing voice input from the microphone is disclosed.

  For example, the technique described in Patent Document 1 determines which singing voice is close to the main melody among the singing voices input from a plurality of microphones, and uses the singing voice closer to the main melody as the main vocal. In this technique, the singing voice that is far from the main melody is used as sub-vocal, and the singing voice for harmony is output from the sub-vocal output.

  Moreover, the technique described in Patent Document 2 is a technique that can add a singing voice for harmony independently at an arbitrary timing to a singing voice inputted from a plurality of microphones.

  Further, the technique described in Patent Document 3 is a technique for generating a singing voice for harmony by extracting a main melody part by analyzing a plurality of parts of singing voice inputted from one microphone.

Japanese Patent Laid-Open No. 10-161672 JP-A-10-55191 JP-A-8-286689

  However, the technique described in Patent Document 1 is based on the premise that it is impossible to distinguish which of two microphones is a main vocal microphone or a sub vocal microphone. And since the singing voice for harmony is output using the output signal from the microphone for sub vocals, for example, in the section where two singers sing simultaneously, the main melody or the harmony melody If one of the singers is out of pitch, it cannot be forcibly pitch shifted to the correct pitch.

  Moreover, since the technique described in Patent Document 2 is a technique for independently adding a singing voice for harmony, for example, in a section where two singers sing simultaneously, the pitch of either one of the singing voices is When it is shifted, the singing voice whose pitch is shifted cannot be corrected.

  In addition, since the technique described in Patent Document 3 is a technique for generating a harmony sound by extracting a main melody part from a singing voice input from a single microphone, correcting a pitch shift is not possible. Can not.

  Furthermore, when the pitches of singing voices input from multiple microphones are deviated from the correct pitches, the pitch deviations are forcibly corrected for all singing voices without setting a pitch shift reference. If this happens, the pitch shift will be confused and the pitch shift may not be converged.

  The present invention has been proposed in view of the circumstances described above, and generates a suitable harmony singing voice by pitch-shifting the singing voice inputted from the microphone in the harmony section of the karaoke music composed of a plurality of singing parts. An object of the present invention is to provide a karaoke system.

  The karaoke system having a pitch shift function for harmony singing according to the present invention (hereinafter sometimes abbreviated as karaoke system) has the following characteristics in order to achieve the above-described object. In other words, the karaoke system according to the present invention is capable of generating an appropriate harmony singing voice by pitch shifting the singing voice inputted from the microphone in the harmony section of the karaoke music composed of a plurality of singing parts. A system comprising: a harmony section detecting means, a plurality of microphones, a pitch shift executing means, a singing scoring means, a singing scoring value comparing means, and a pitch shift control means. .

  The harmony section detecting means is means for detecting a harmony section of the karaoke music piece. The plurality of microphones are associated with each singing part, and when a singing voice of a singer in charge of the singing part is input, a singing voice signal is output. The pitch shift execution means is means for performing a pitch shift on the singing voice signal output from each microphone.

  The singing scoring means is a means for comparing the singing voice signal output from each microphone with the reference data of the singing part associated with each microphone and calculating a singing scoring value for each microphone. The singing score comparison means is a means for comparing the singing score values for each microphone. The pitch shift control means is based on the comparison of the singing score values, and the harmony detected by the harmony section detecting means when the difference between the singing score value having the highest score and the singing score value having the lowest score is not less than a predetermined threshold. In the section, the pitch shift execution means is operated to cause the pitch shift to be performed so that the difference between the singing score values is less than a predetermined threshold value.

  Moreover, in the karaoke system of the structure mentioned above, a singing score value comparison means can compare a singing score value based on the singing score value calculated in the singing part with few pitch fluctuations of a singing voice signal.

  In the karaoke system having the above-described configuration, the singing score comparison means can comprehensively evaluate the singing score values calculated in a predetermined singing section and compare the singing score values.

  Moreover, in the karaoke system of the structure mentioned above, a pitch shift control means can perform a pitch shift with respect to the singing voice with the lowest singing score value.

  The karaoke system having the above-described configuration may further include note section distinguishing means for distinguishing between a long note section in which the notes are equal to or greater than a predetermined threshold and a short note section that is less than the predetermined threshold in the harmony section. . In such a configuration, the pitch shift control means causes the singing voice signal having the lowest singing score value in the long note section to be pitch-shifted, and does not cause the pitch shift to be performed in the short note section. To work.

  The karaoke system provided with the pitch shift function for the harmony singing according to the present invention adds the singing voice for harmony to the singing voice determined to be the harmony singing as in the prior art, or a plurality Rather than adding the singing voice for harmony to the singing voice signal output from the microphone, the pitch shift is corrected when the pitch is shifted by a predetermined threshold or more in the harmony section. Accordingly, it is possible to generate a harmony singing voice that does not give a sense of incongruity to the audience and the singer without cheating the deviation of harmony due to the effect or the like.

The block diagram which shows schematic structure of the karaoke system which concerns on embodiment of this invention. FIG. 3 is an explanatory diagram illustrating a pitch shift mode according to the first embodiment. Explanatory drawing which shows the aspect of the pitch shift which concerns on Example 2. FIG.

  Hereinafter, an embodiment of a karaoke system having a pitch shift function for harmony singing will be described with reference to the drawings. 1 to 3 illustrate a karaoke system according to an embodiment of the present invention. FIG. 1 is a block diagram showing a schematic configuration of the karaoke system, and FIGS. 2 and 3 are explanatory diagrams showing modes of pitch shift. .

  In the following description, a program is stored in a RAM or the like and executed by hardware such as a CPU, so that not only software that exhibits its function but also a logic circuit that can exhibit an equivalent function. It is a concept that includes.

<Outline of Karaoke System>
The karaoke system 100 according to the embodiment of the present invention generates appropriate singing voices for harmony by pitch-shifting the singing voices input from the plurality of microphones 13 in the harmony section of the karaoke music composed of a plurality of singing parts. A harmony section detecting means 33, a plurality of microphones 13, a pitch shift executing means 14, a singing scoring means 27, a singing scoring value comparing means 28, and a pitch shift control means 29. And note section distinguishing means 34 may be further provided. Each means is realized as a function means of the karaoke performance apparatus 10. In the embodiment described below, for ease of explanation, an embodiment in which two microphones 13 are provided is shown, but the number of microphones 13 is not limited to two, and the number of singing parts of karaoke music There may be three or more corresponding to the above.

<Karaoke performance device>
A karaoke performance device 10 to which a karaoke system 100 according to an embodiment of the present invention is applied includes a karaoke main body 11, a karaoke remote control device 12, a microphone 13, a pitch shift execution means 14, a mixing amplifier 15, and a speaker 16, as shown in FIG. The display device 17 is provided.

<Karaoke remote control device>
The karaoke remote control device 12 has a user interface function, and transmits / receives data to / from the karaoke main body 11 via the router 40 and the data communication line 50. The karaoke remote control device 12 includes a program functioning as a music search means 12a, a music index database 12b, a data storage unit 12c for storing various data, and an input / output display unit 12d for inputting / outputting data. Yes. By operating switches attached to the karaoke remote control device 12 and various icons displayed on the input / output display section 12d, various operations such as music selection operation, volume adjustment, pitch adjustment, tempo adjustment, performance stop, etc. And settings can be made.

<Music search means / music index database>
The music search means 12a includes a program for searching for music by referring to the music index database 12b based on a user instruction. The song index database 12b is a database that stores attribute information of karaoke songs used for performance by the karaoke performance device 10, and includes, for example, song numbers, song names, artist names, lyrics of singing parts, popular times, and music. Various attribute information such as whether or not it is a genre duet song is included in this.

<Microphone / Speaker>
The microphone 13 is a device for inputting a singing voice, and includes two microphones 13 in the present embodiment. The singing voice inputted from the microphone 13 is outputted as a singing voice signal, inputted to the mixing amplifier 15 through the pitch shift execution means 14 described later, mixed with the karaoke performance signal sent from the music reproduction control means 31, and amplified. And output to the speaker 16. The singing voice signal output from the microphone 13 is digitally converted by the A / D converter 30 and used for calculating the singing score value in the singing scoring means 27.

  In addition, when adjusting the pitch of the harmony singing voice when playing a karaoke piece (for example, a duet piece) composed of a plurality of singing parts, the adjustment is made among the singing voice signals output from the microphone 13. The necessary singing voice signal is pitch-shifted by the pitch-shift execution means 14 and input to the mixing amplifier 15.

  Further, when playing karaoke music composed of a plurality of singing parts, each microphone 13 is associated with each singing part in advance, and the relationship between the microphone 13 and the singing part is presented to the user. For example, when performing a duet song, the microphone A is used for inputting a singing voice of a male part (singing part a), and the microphone B is used for inputting a singing voice of a female part (singing part b). It is displayed on the display screen of the device 17.

<Pitch shift execution means>
The pitch shift execution means 14 includes electronic devices for performing a pitch shift on the singing voice signal output from each microphone 13. As will be described later in detail, the pitch shift control means 29 determines which microphone 13 receives the singing voice signal from which the pitch shift is performed.

<Display device>
The display device 17 is a device for displaying a background video, lyrics telop, and the like related to karaoke music, and is configured by, for example, a liquid crystal display.

<Karaoke body>
The karaoke main body 11 includes a central control means 21, ROM 22, RAM 23, HDD 24, communication control means 25, reserved music management means 26, singing scoring means 27, singing scoring value comparison means 28, pitch shift control means 29, A / D converter 30. , Music playback control means 31, video playback control means 32, harmony section detection means 33, and note section discrimination means 34.

<Central control means>
The central control means 21 is an electronic device for comprehensively controlling the karaoke main body 11, and is composed of, for example, a CPU and its peripheral devices. When the CPU or the like operates according to a program stored in the ROM 22 or the like, The control function can be demonstrated.

<ROM / RAM>
The ROM 22 is an electronic device for storing program data and numerical data for controlling each device constituting the karaoke main body 11, and is constituted by a semiconductor memory, for example. The RAM 23 functions as a temporary storage area for temporarily storing programs and various data, and is composed of, for example, a semiconductor memory. In the present embodiment, the reserved music management table 23 a is temporarily stored in the RAM 23.

<HDD>
The HDD 24 functions as a mass storage device, and stores a music database 24a and a background video database 24b. Note that a large-capacity storage unit such as a DVD that can rewrite data may be used instead of the HDD 24 or together with the HDD 24.

<Music database / background video database>
The music database 24a is a database configured by associating music control data (MIDI standard data) and lyrics rendering data in synchronism with reference data used for singing, in association with music IDs. . The performance control data is digital data for controlling the performance of each musical piece, and the lyric rendering data includes display timing data and color change data of lyric characters synchronized with the performance. Reference data is data for calculating a singing score value in comparison with the singing voice signal of each singing part, and is prepared for each singing part. The background video database 24b is a database that stores a predetermined number of background videos corresponding to karaoke songs to be played as background video files corresponding to the song IDs of the karaoke songs.

<Communication control means>
The communication control unit 25 includes an electronic device and a program for controlling transmission and reception of data and signals with the karaoke remote control device 12. The function of the communication control means 25 performs synchronization and error control of data and signals transmitted and received between the karaoke main body 11 and the karaoke remote control device 12.

<Reserved music management means / reserved music management table>
When an arbitrary user reserves a karaoke song, the reserved song management unit 26 associates the user ID of the user with the song ID of the reserved song and manages the reserved song management table 23a. For the program. That is, the reserved music management means 26 creates a reserved music management table 23 a and stores this reserved music management table 23 a in the RAM 23 for management. The reserved music management table 23a is a data table created by arranging music IDs selected by the user using the function of the music search means 12a in the order of performance. Note that the user ID of the user who selects the music can be acquired when the user logs in to the karaoke system 100. System login can be performed using the karaoke remote control device 12, a portable information terminal (not shown) possessed by the user, or the like.

<Singing method>
The singing scoring means 27 compares the singing voice signal output from each microphone 13 with the reference data of the singing part associated with each microphone 13 and calculates a singing scoring value for each microphone 13. Become. The singing scoring means 27 is not only for karaoke music composed of a plurality of singing parts, but also for karaoke music singed by a single singer, the singing voice signal output from the microphone 13 and reference data of the karaoke music. Singing can be performed by comparing.

<Singing score comparison means>
The singing score comparison means 28 includes a program for comparing the singing score values for each microphone 13. That is, the singing score comparison means 28 compares the singing score for each microphone 13 corresponding to each singing part when playing karaoke music composed of a plurality of singing parts. The timing which compares a singing score value can be set suitably. For example, as shown in FIG. 2 and FIG. 3, the singing score values may be compared based on the singing score values calculated in the singing portion where the pitch variation of the singing voice is small. The singing score values calculated in the section may be comprehensively evaluated to compare the singing score values.

  The singing part with a small pitch fluctuation of the singing voice is, for example, a part where the singing pitch is detected horizontally for 200 msec, for example, a part where the fluctuation of the singing pitch is within plus or minus 30 cents. The reason why the singing is performed using the horizontal portion in this way is that accurate singing can be performed by performing the singing scoring at the portion where the fluctuation of the singing pitch is small. In order to detect the horizontal portion, the technique described in Japanese Patent Application Laid-Open No. 2008-225117 can be applied.

  In addition, although it is not illustrated that the singing score value calculated in the predetermined singing section is comprehensively evaluated, for example, the comparison result of the singing score value of the first chorus is determined as the pitch shift target in the harmony section of the second chorus. It means to reflect the above, to use the latest singing score value, or to use the accumulated value or average value of the singing score values so far. Note that the predetermined singing section may be set in any way, and singing scoring may be performed in a singing section where the superiority or inferiority of the singing ability can be appropriately determined according to the characteristics of the karaoke music. In addition, as described above, the comparison of the singing score value is not limited to one time. As the singing progresses, the singing score value is appropriately calculated, and the singing voice signal to be pitch-shifted for each harmony section. It may be different.

  In the present invention, instead of determining the singing voice to be pitch-shifted using the singing grading history, the singing ability of the singer is determined based on the singing voice actually being sung. Harmony without a sense of incongruity can be generated by accurately responding to various factors such as physical condition and singing environment.

<Pitch shift control means>
The pitch shift control means 29 operates the pitch shift execution means 14 when the difference between the singing score value with the highest score and the singing score value with the lowest score is equal to or greater than a predetermined threshold in the comparison of the singing score values. And a program for causing the pitch shift so that the difference between the singing score values is less than a predetermined threshold value. The singing voice signal to be pitch-shifted will be described in detail later.

<Harmony section detection means>
The harmony section detecting means 33 is composed of a program for detecting a singing section (harmonic section) to be pitch-shifted in karaoke music composed of a plurality of singing parts such as duet music, and section information and reference of music data. By detecting a section in which a plurality of data exists in the data, it is possible to specify a harmony section that is a target of pitch shift.

<Note section distinction means>
The note section distinguishing means 34 uses the reference data prepared for each singing part to distinguish between a long note section in which the notes are equal to or greater than a predetermined threshold and a short note section that is less than the predetermined threshold in the harmony section. It consists of the program. In other words, the singing section includes a section with a long note and a section with a short note. The note length is, for example, 400 msec as a reference value, a singing section of 400 msec or longer as a long note section, and a singing section of less than 400 msec as a short note section.

<Example 1 of pitch shift>
In Example 1, as shown in FIG. 2, the pitch shift is performed on the singing voice signal having the lowest singing score value. That is, when the difference between the singing score values is equal to or greater than a predetermined threshold (for example, 10 points or more), the singing voice signal with the lowest singing score value is likely because both the audience and the singer will feel uncomfortable with the harmony. A pitch shift is performed on (the singing voice signal output from the microphone B), and correction is performed so that neither the audience nor the singer will feel uncomfortable with the harmony.

  When the pitch shift is performed, the pitch shift control unit 29 instructs the pitch shift execution unit 14 about the target singing part (singing voice signal) and the target pitch of the pitch shift. As the target pitch, the pitch of the reference data corresponding to the target singing part (singing part b in this embodiment) can be indicated. Or you may instruct | indicate a target pitch so that it may become a shift | offset | difference comparable as the singing part a in consideration of the singing scoring value of the singing part (single part a in this embodiment) which does not pitch shift.

<Example 2 of pitch shift>
In Example 2, as shown in FIG. 3, even in the harmony period, only in the long note period, the singing voice signal output from the microphone B having the lowest singing score value is pitch-shifted, and the short note No pitch shift is performed in the section. In other words, the long note section is a section that allows the audience to listen to the singing carefully, and therefore the pitch shift is performed as a section in which the audience can enjoy the harmony comfortably. On the other hand, the short note section gives priority to the singer's ease of singing rather than making the audience enjoy harmony, and does not shift the pitch.

<Music playback control means>
The music playback control means 31 drives a sound source (synthesizer) based on performance control data extracted from the music data based on the music ID, generates a musical sound signal that constitutes a karaoke performance signal, and converts the analog sound into a mixing amplifier. 15 is an electronic device for outputting to the electronic device. The mixing amplifier 15 is a device for mixing the singing voice signal output from the microphone 13 and the karaoke performance signal transmitted from the music reproduction control means 31, amplifying it by an amplifier function, and outputting it from the speaker 16. It is. In the present embodiment, the singing voice signal output from the microphone 13 may be pitch-shifted in the harmony section, and the pitch-shifted singing voice signal may be input to the mixing amplifier 15 in some cases.

<Video playback control means>
The video reproduction control means 32 uses the background video data extracted from the background video database 24b and the lyric characters created based on the lyric rendering data included in the music data during the performance of the karaoke music, as the music of the karaoke music. This is an electronic device for outputting to the display device 17 in synchronization with data.

<Specific pitch shift>
Hereinafter, a specific example of pitch shift will be described by taking a duet song in which two singers sing. In a duet song, the microphone A is made to correspond to the singing part a, the microphone B is made to correspond to the singing part b, and the fact is displayed on the display screen of the display device 17 so that the singer is recognized. For example, when the singing part a is a main melody, the singing part b is a singing part for harmony, and when the singing part a is a male voice part, the singing part b is a female part.

  For the singing voice signal output from the microphone A, singing is performed using the reference data for the main melody or the reference data for the male part, and for the singing voice signal output from the microphone B, Singing is performed using harmony reference data or female part reference data.

  When the singing scoring value is calculated, the singing scoring value is compared, and when the difference between the singing scoring value with the highest score and the singing scoring value with the lowest score is equal to or higher than a predetermined threshold (for example, 10 points), In the harmony section, a pitch shift is performed on the singing voice signal to generate a harmony voice without a sense of incongruity. For example, as shown in FIG. 2, the singing score of the singing voice of the singing part a input from the microphone A is 94 points, and the singing part b input from the microphone B is sung with a substantially correct pitch. The singing score of the singing voice is 82 points, and the pitch is shifted. In this case, a pitch shift is performed on the singing voice signal output from the microphone B in the harmony period, and the pitch difference between the singing voices input to the microphone A and the microphone B is eliminated. To do.

  Also, as shown in FIG. 3, when deciding whether or not to shift the pitch according to the length of the note, the pitch shift is performed on the singing voice signal output from the microphone B in the long note section. Do. On the other hand, in the short note section, pitch shift is not performed even for the singing voice signal output from the microphone B.

DESCRIPTION OF SYMBOLS 100 Karaoke system 10 Karaoke performance apparatus 11 Karaoke main body 12 Karaoke remote control apparatus 12a Music search means 12b Music index database 12c Data storage part 12d Input / output display part 13 Microphone 14 Pitch shift execution means 15 Mixing amplifier 16 Speaker 17 Display apparatus 21 Central control means 22 ROM
23 RAM
23a Reserved music management table 24 HDD
24a Music database 24b Background video database 25 Communication control means 26 Reserved music management means 27 Singing scoring means 28 Singing scoring value comparison means 29 Pitch shift control means 30 A / D converter 31 Music playback control means 32 Video playback control means 33 Harmony section detection Means 34 Note section distinction means 40 Router 50 Data communication line

Claims (5)

In a harmony section of karaoke music composed of a plurality of singing parts, a karaoke system capable of generating a singing voice for an appropriate harmony by pitch-shifting a singing voice inputted from a microphone,
Harmony section detecting means for detecting a harmony section of the karaoke music piece;
A plurality of microphones that are associated with each of the singing parts and that input the singing voice of each singing part;
Pitch shift execution means for performing a pitch shift on the singing voice signal output from each microphone;
Compare the singing voice signal output from each microphone and the reference data of the singing part associated with each microphone, singing scoring means for calculating a singing scoring value for each microphone,
Singing score comparison means for comparing the singing score for each microphone;
Based on the comparison of the singing score values, when the difference between the singing score value having the highest score and the singing score value having the lowest score is equal to or greater than a predetermined threshold, the pitch shift execution means in the detected harmony section A pitch shift control means for operating and performing a pitch shift so that the difference between the singing score values is less than a predetermined threshold;
A karaoke system with a pitch shift function for harmony singing.   The singing scoring value comparison means compares the singing scoring value based on the singing scoring value calculated in a singing portion where the pitch variation of the singing voice is small. Karaoke system with a pitch shift function.   2. The pitch for harmony singing according to claim 1, wherein the singing scoring value comparison means comprehensively evaluates the singing scoring values calculated in a predetermined singing section and compares the singing scoring values. Karaoke system with shift function.   The pitch shift function for harmony singing according to any one of claims 1 to 3, wherein the pitch shift control means performs pitch shifting on a singing voice signal having the lowest singing score value. Karaoke system with In the harmony section, the notebook further comprises a note section distinguishing means for distinguishing between a long note section having a predetermined threshold or more and a short note section having a predetermined threshold or less,
The pitch shift control means performs pitch shift on the singing voice signal having the lowest singing score value in the long note interval, and does not cause pitch shift in the short note interval. The karaoke system provided with the pitch shift function for harmony singing of any one of claim | item 1-3.

Images