JP2007140548A - Portrait output device and karaoke device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a karaoke device with a readily amusement function. <P>SOLUTION: Portrait data and voice formant data of a plurality of singers are managed in a database. When a singing voice of a singing person is input, formants of the singing voice are analyzed to calculate similarities to the formant data in the database. Portraits of two or three singers having high similarities are read out and put together to generate and display a portrait of the singing person. Alternatively, a portrait of a singer having the highest similarity of formants is read out and a message of "You look like this singer." is displayed. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a caricature output device and a karaoke apparatus for inputting a voice signal such as a singing voice and selecting and synthesizing a caricature from the voice.

  In addition to playing a karaoke song, a karaoke device having a game function and a service function using a voice for singing the karaoke song has been put into practical use. For example, a scoring game in which the score is calculated and displayed according to how much the pitch and volume of the singing voice match the pitch and volume indicated by the guide melody of the song is one of its functions.

  However, this scoring function, as a result, determines the skill of the singer's singing, so anyone could not participate easily. Moreover, even if the singing was scored and the score was displayed, the singer did not know how to use the score as a reference for future singing.

  An object of the present invention is to provide a portrait output device that anyone can use easily, and a karaoke device that can be easily used and can reference the output portraits for music selection.

  According to the first aspect of the present invention, a voice input means for inputting a voice, a formant analysis means for analyzing a formant of the inputted voice, a plurality of sample formants and a facial caricature having resonance characteristics of the sample formants are associated with each other. The storage means is stored, and the storage means is searched with the analyzed formant, and the portrait selection means for selecting and outputting the portrait corresponding to the sample formant that is most approximated.

  According to the second aspect of the present invention, the voice input means for inputting voice, the formant analysis means for analyzing the formant of the inputted voice, the material storage means for storing the portrait material for creating the portrait, and the formant analysis means analyze And caricature output means for synthesizing and outputting a caricature of the face shape having the resonance characteristics of the formant using the caricature material based on the formant.

  The invention of claim 3 is a voice input means for inputting a singing voice, a formant analysis means for analyzing a formant of the inputted voice, a singing analysis means for analyzing a singing mode of the inputted voice, and a portrait for creating a portrait Based on the material storage means for storing the caricature material, the formant analyzed by the formant analysis means and the singing mode analyzed by the singing analysis means, singing in the singing mode with the facial shape having the resonance characteristics of the formant using the caricature material And caricature composition means for synthesizing and outputting the caricatures to be output.

  In the present invention, formant data is extracted from the input speech (vowel). The formant data is a dominant frequency component on the spectrum of the uttered vowel, and is called first formant, second formant,. Of these, up to the third formant is said to contribute to phonological properties. This formant depends on the shape of the resonator, such as the vocal cords and face of the speaker, and it is said that the voice is similar if the face is similar.

  Therefore, in the present invention, a plurality of formants and a portrait of a person having the formants are stored, and a portrait similar to the input speech and formant is selected and output. The output portrait may be displayed on a monitor, printed on a printer, and presented to the user. Thereby, it is possible to inform the user what kind of face is imagined by his / her voice and to provide an amusement to the user. The formants and caricatures may be those collected from real people or may be created by simulation from a resonator model of the head.

  In addition, the singer's portrait and the singer's formant are stored in the karaoke device, and when the user inputs a voice, the singer's voice is displayed (printed). You may make it inform by. Since the user can sing with a voice that matches the song if the song is selected, the display of the portrait can be used as a reference for the song selection.

  In the present invention, the caricature material is stored in the storage means, and the caricature material is synthesized using the caricature material based on the formants extracted from the speech. As a material, a plurality of components such as facial contours, eyebrows, eyes, nose, mouth, etc. may be stored for each part, and each may be selected based on a formant. A plurality of components may be stored, and component parts such as a face outline, eyebrows, eyes, nose, and mouth may be extracted from each portrait and combined. Alternatively, the sample caricature that is the material may be transformed as it is to synthesize the caricature of the user. In any case, the user's face can be well estimated by creating a portrait that has the extracted formant as the resonance frequency.

  As described above, according to the present invention, it is possible to extract a voice formant and display a caricature of a user (singer) or a caricature of a singer with a similar voice. If a portrait of a singer with a similar voice is displayed, the song can be selected according to the voice quality by selecting the singer's song. It can be used as a reference for karaoke singing.

Embodiments of the present invention will be described with reference to the drawings. 1 to 3 are functional block diagrams of a karaoke apparatus according to an embodiment of the present invention. In this karaoke apparatus, a plurality of singer portrait data and singing voice formant data are stored in the sample database 4 as sample data. An example of this sample database 4 is shown in FIG. A singer whose voice resembles the voice of the singing voice of the karaoke song, and the formant data (extracted formant data) extracted from this singing voice is compared with the formant data (sample formant data) in the sample database. Is selected to display the portrait, or based on the portrait, the singer's portrait is synthesized and displayed.
In FIG. 1, the singing voice input unit 1 includes a microphone for karaoke singing. The singing voice converted into an electric signal by the singing voice input unit 1 is input to the formant extraction unit 2 and the karaoke performance unit 6. The formant extraction unit 2 cuts out vowels from the input singing voice and extracts formants for each vowel. A vowel is a periodic signal, which lasts for several tens of milliseconds to several seconds in karaoke singing, so that it can be distinguished from a consonant that is a short-period aperiodic signal by cutting out a waveform section of the same period. Moreover, it is possible to identify which vowel is a, i, u, d, or o based on the shape of the periodic waveform. A formant is a dominant frequency component on the frequency spectrum of a vowel, and is called first, second, third,. The formant extraction unit 2 extracts first to third formants of the cut vowel. This formant extraction may be performed by FFT (fast Fourier analysis) or the like. The formant extraction unit 2 extracts the formants of the singing voices of all the karaoke songs, accumulates and stores them for each vowel, and outputs the averaged value as extracted formant data after the performance of the karaoke songs.

  This extracted formant data is input to the formant comparison unit 3. The formant comparison unit 3 compares the input extracted formant data with the sample formant data in the sample database 4. As shown in FIG. 4, the sample database 4 includes a portrait database 4b storing portrait data of a plurality of singers and a formant database 4a storing formant data of each singer's singing voice corresponding thereto. The comparison unit 3 correlates and compares the extracted formant data and each sample formant data, and calculates the degree of similarity between the extracted formant data, that is, the voice of the singer, and the sample formant data, that is, the voice of each singer. Then, this similarity is output to the portrait synthesis unit 5.

  The portrait selection unit 5 selects a sample portrait showing the highest similarity among the input similarities, and reads the portrait data from the portrait database 4b. The caricature data selected and read by the caricature selection unit 5 is output and displayed on the display unit 9 where the display of the lyrics has been completed, and is printed by the printing unit 10. At the same time, a phrase such as “Your voice is similar to this singer XX” is displayed and printed so that it can be used as a reference for subsequent song selection. At this time, the singer's karaoke song may be searched for music selection support.

  In the example of FIG. 1, one sample (singer) having the most similar formants is selected and the portraits of the singer are displayed. However, a plurality of portraits having a high formant similarity are selected. A single caricature may be synthesized by combining them.

  FIG. 2 shows a functional block diagram of a karaoke apparatus provided with this portrait synthesis function. 1 is different from the functional block diagram of FIG. 1 in that the portrait synthesis unit 15 does not select the portrait with the highest similarity input from the formant comparison unit 3, but a plurality of samples with high formant similarity. Is selected, and one portrait is synthesized based on the portrait data of this sample.

  The caricature composition unit 15 selects caricature data of two or three singers whose voices are very similar from the caricature database 4b based on the input similarity and selects the caricatures of the singer who sang based on these caricatures. Synthesize. As this synthesis method, a method such as a component combination method or a morphing method may be used.

  The component combination method is a method in which facial component parts such as a face outline, eyebrows, eyes, nose, and mouth are appropriately selected from the selected plurality of sample portrait data and combined to synthesize a portrait. For example, a face outline is selected from a portrait of a singer whose first formant is most similar, a mouth is selected from a portrait of a singer whose second formant is most similar, and a singer whose third formant is most similar Each part may be selected by a method such as selecting the nose from the caricature.

  In this example, face components such as face outline, eyebrows, eyes, nose, mouth, etc. are selected from a plurality of caricatures, but face outline, eyebrows, eyes, nose, mouth, etc. A database may be provided for each component, and a caricature may be constructed by picking up an appropriate component for each component based on the extracted formants.

  The morphing method is a method of synthesizing intermediate figures of the plurality of selected portrait data. In the synthesis of intermediate figures, the intermediate shape of multiple portrait data is determined for each facial component such as the face outline, eyebrows, eyes, nose, mouth, etc., and placed at an intermediate position between the multiple portrait data. Is synthesized by The synthesis is performed so as to be closer to the sample portrait data having a high formant similarity.

  The caricature synthesized in this way is displayed on the display unit 9 where the karaoke performance has been completed and printed by the printing unit 10. The amusement effect can be further enhanced by displaying the name of the selected sample singer and its composition ratio.

  In addition to the method of synthesizing caricatures by synthesizing components, the head that stores the model of the resonance body of the head and forms the sound (formant) when singing voice is input A resonator may be simulated, and a caricature may be rendered based on this.

  In the example of FIG. 2, the portraits are synthesized according to the formant similarity, but in addition to this, the shape of the portrait may be adjusted according to the singing mode of the singer. The singing mode includes singing such as legato and accent, dynamic volume, and the like.

  FIG. 3 is a functional block diagram of a karaoke apparatus having the above functions. The functional block diagram of FIG. 2 differs from that of FIG. 2 in that it includes the singing mode detection unit 11 and the caricature composition unit 16 based on the singing mode data detected and output by the singing mode detection unit 11. This is the point at which the composition operation is controlled. The singing voice signal converted into an electric signal by the singing voice input unit 1 is input to the singing mode detection unit 11 in addition to the formant extraction unit 2 and the performance unit 6. The singing mode detection unit 11 inputs the song data being played from the performance unit 6 and detects the singing mode of what the singer is singing according to the song data. As described above, the singing mode includes singing such as legato and accent, dynamic volume, and the like. The detected singing mode information is input to the portrait synthesizer 5 '. The caricature composition unit 16 synthesizes a caricature according to the formant similarity as in the example of FIG. 2, and deforms the synthesized caricature according to the singing mode information. For example, in the case of a singer who is singing with a strong accent and a large dynamic, the eyebrows may be deformed thickly, and in the case of a singer singing in legato, the bottom of the eyes may be lowered. Thus, by creating a portrait in accordance with the singing mode in addition to the formant, a more accurate or amusement portrait synthesis function can be realized.

FIG. 5 is a block diagram of hardware of a karaoke apparatus that realizes the above function. The functions shown in FIGS. 1 to 3 are realized by executing a program as shown in FIG. 6 on this hardware.
This karaoke device is composed of a karaoke device main body 21, a control amplifier 22, an audio signal processing device 23, a CD-ROM changer 24, a speaker 25, a monitor 26, a microphone 27, an infrared remote control device 28 and a printer 29. The karaoke device main body 21 controls the operation of the entire karaoke device. The CPU 30, which is a control device of the karaoke device main body 21, has a ROM 31, RAM 32, a hard disk storage device 37, a communication control unit 36, a remote control reception unit 33, a display panel 34, a panel switch 35, a sound source device 38, and the like via an internal bus. The audio data processing unit 39, the pattern development unit 40, and the display control unit 41 are connected, and the control amplifier 22, the audio signal processing device 23, and the CD-ROM changer 24, which are the external devices, are connected via an interface. .

  The ROM 31 stores an activation program necessary for activating this apparatus. A system program for controlling the operation of the apparatus, a karaoke performance execution program, and the like are stored in the hard disk storage device 37. When the power of the karaoke apparatus is turned on, a system program and a karaoke performance program are read into the RAM 32 by the above-described startup program.

  The hard disk storage device 37 stores a program database of the above program and a music database composed of a large number of music data, and also stores a sample database 37a. The sample database 37a has a formant database that stores formant data of original singers of karaoke songs, and a portrait database that stores portraits of each original singer, and each original singer is identified by a singer number.

  The RAM 32 is set with a program storage area for reading a program from the hard disk storage device 37 when the apparatus is started up, an execution song data storage area for reading song data of a karaoke song to be played, and the like, and is detected during karaoke performance. A formant storage area 32a for storing and storing formants is also set.

  The communication control unit 36 is connected to the distribution center 19 via an ISDN line. The distribution center 19 periodically calls the karaoke device to download music data of new songs, upgraded control programs, and the like. The sample database 37a is also downloaded from the distribution center 19.

  The remote control device 28 includes key switches such as a numeric keypad. When a user operates these switches, a code signal such as a song number is output in infrared according to the operation. The remote control receiving unit 33 receives the infrared signal sent from the remote control device 18, restores the code signal, and inputs it to the CPU 30.

  Here, the song number of each karaoke song (music data) is composed of 6 digits of a 4-digit singer number + a 2-digit singer-specific song number. Therefore, by paying attention to the upper 4 digits of the 6 digits, it is possible to easily determine which singer is singing the karaoke song. The singer's portrait and formant stored in the sample database 37a are also identified by this 4-digit singer number.

  The display panel 34 is provided on the front surface of the karaoke apparatus main body 21 and includes a matrix display for displaying the number of the currently played song and the number of reserved songs, and a group of LEDs for displaying the currently set key and tempo. Contains. The panel switch 35 includes a numeric keypad for inputting a music number similar to the general-purpose remote control device 28.

  The sound source device 38 forms a musical sound signal based on the music data. The music data includes performance data of a plurality of tracks, and the tone generator 38 simultaneously forms a plurality of parts of tone signals based on this data. The audio data processing unit 39 forms an audio signal having a specified length and a specified pitch based on the audio data included in the music data. The audio data is obtained by storing a signal waveform that is difficult to form electronically, such as a human voice such as a back chorus, as it is as a PCM signal. The musical tone signal formed by the sound source device 38 and the audio signal reproduced by the audio data processing unit 39 are input to the control amplifier 22.

  In addition, two microphones 27a and 27b are connected to the control amplifier 22, and the singing voice of the karaoke singer is input. The control amplifier 22 gives a predetermined effect such as echo to these audio signals, amplifies them, and outputs them to the speaker 25. The audio signal processor 23 converts the singing voice signal (any one microphone signal) input from the control amplifier 22 into digital data, cuts out a periodic signal (vowel), and performs FFT analysis on the periodic signal. To extract the formants. Further, based on the shape of the periodic waveform, the vowel of A, B, U, D, or A is identified, and vowel information indicating this is generated. The extracted formant data and vowel information are input to the CPU 30. Moreover, the audio | voice signal processing apparatus 23 is equipped with the function which corrects the shift | offset | difference of the pitch of a song voice, or creates the harmony song of another part. The corrected singing voice and the harmony singing voice of other parts are input to the control amplifier 22 again. This correction function may be applied to the signals of both microphones.

  The pattern development unit 40 includes a VRAM and develops pattern data input from the CPU 30 into a matrix corresponding to the display area of the monitor 26. The pattern data includes lyrics (character pattern) data during performance of a karaoke song and caricature data after completion of the karaoke performance. The developed matrix data is sequentially scanned and input to the display control unit 41 as a video signal. During the karaoke performance, the CD-ROM changer 24 reproduces the background video, and this video signal is also input to the display control unit 41. The display control unit 41 synthesizes the character pattern of the lyrics with the background image in a superimposition and displays it on the monitor 26. After the karaoke performance is completed, since the background video is not input, the display control unit 41 makes a blue background, develops the portrait data input from the CPU 30 and displays it on the monitor 26.

  When a karaoke performance is executed by the karaoke apparatus having the above configuration, the singing voice input from the microphone 27 is input to the audio signal processing apparatus 23 via the control amplifier 22. The audio signal processing device 23 converts this signal into digital data, determines the period of the periodic signal, and cuts it out. Determine which vowel is a, i, u, d, or o in this section. This can be determined by matching with sample data of a, i, c, e and o. Then, the formant of the vowel is extracted by FFT analysis. The formant data and the vowel information are input to the CPU 30.

  The CPU 30 accumulates and stores formant data for each vowel. When the karaoke performance is completed, the formant data stored and stored in the formant storage area 32a of the RAM 32 is averaged for each vowel to calculate an extracted formant data value. Then, this value is compared with the sample formant data read from the sample database 37a, and the similarity to each sample formant data is determined. Then, the caricature database is accessed based on the similarity, and one caricature with the most similar formant is selected and displayed. Alternatively, a plurality of portraits with similar formants are read out, and parts are combined or morphed to synthesize one portrait. At this time, the caricature may be transformed according to the singing mode.

  FIG. 6 is a flowchart showing the operation of the karaoke apparatus. This operation indicates a portrait selection / combination operation when a karaoke song is played. When a karaoke song is played and the singer inputs a singing voice to the microphone 7, the singing voice is taken in and formant data of a vowel is extracted (s1). Then, this extracted formant is compared with the sample formants in the formant database 4a to obtain the similarity (s2). The formant data is extracted continuously from the beginning to the end of the karaoke song, and the accumulated formant data is averaged to obtain the extracted formant data. However, the formant data extracted in a part of the karaoke song is extracted. It may be used as data.

  A portrait is synthesized or selected with the degree of similarity determined by the correlation comparison, and it is determined which mode is currently in effect (s3). This mode may be selectable by the user, or may be automatically set by an attendant or online. In the selection mode, the sample portrait with the highest similarity in the comparison is selected and read from the portrait database 4b (s4), displayed on the monitor 26 and printed out (s5).

  On the other hand, in the synthesis mode, the portrait data of 2 to 3 samples having high similarity in the above comparison are read from the portrait database 4b (s6), and based on this, the singer's portrait is synthesized (s7). The synthesis method may be performed by the component combination method or the morphing method as described above. The caricature synthesized in this way is displayed on the monitor 26 and printed out (s5).

  In this embodiment, the first, second, and third formants are used as the formants. However, only the first and second formants may be used, and higher-order formants that are equal to or higher than the fourth formants may be used. Further, the formant database shown in FIG. 4 stores formant frequencies and formant levels, but only formant frequencies may be stored. Further, a continuous spectrum waveform may be used as the formant.

  FIG. 7 is a flowchart showing the music selection support operation of the karaoke apparatus. This music selection support operation is an operation of extracting a voice formant of a user (singer), selecting a song of a singer having a formant voice similar to the extracted formant, and presenting it to the user.

  First, a formant is extracted from the input sound of the microphone 27 (s11). This input voice may be a singing voice of a karaoke song, or may be a voice input for music selection. At this time, the formant music selection mode is set in advance in the karaoke apparatus. The extracted formant of the user is compared with the sample formant stored in the sample database 37a (s12), and the most similar sample data is selected (s13). The singer number of the sample data is read (s14), the song database is searched with this singer number, and the karaoke song of this singer number is extracted (s15). The list of karaoke songs is displayed on the monitor (s16). When the user inputs the number of the displayed karaoke song from the remote controller (s17), the karaoke song can be selected and played (s18).

  Further, the portraits stored in the so-called portrait database 4b are not limited to the so-called line drawings as shown in FIG. 4, and photographic data may be used.

  In this embodiment, in addition to the invention described in the claims, the sample formant is associated with the music data of the karaoke song, and the sample formant similar to the voice formant input by the user is searched. The invention of selecting music data corresponding to this is described. The sample formant and music data may be associated with each other by, for example, associating the singer's formant (sample formant) with the karaoke song sung by the singer. Also, the voice quality may be associated with the music atmosphere.

  Thus, the most suitable karaoke song can be automatically selected by inputting a voice. The music selection means includes not only selecting one song but also extracting a plurality of songs as candidate songs and allowing the user to select one song from them. In any case, the user can easily select a song and can select a karaoke song suitable for his / her voice quality.

Functional block diagram of a karaoke apparatus as an embodiment of the present invention Functional block diagram of a karaoke apparatus as an embodiment of the present invention Functional block diagram of a karaoke apparatus as an embodiment of the present invention Diagram showing sample database Hardware block diagram of the karaoke device Flow chart showing operation of the karaoke apparatus Flow chart showing operation of the karaoke apparatus

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Singing voice input part 2 ... Formant extraction part 3 ... Formant comparison part 4 ... Sample database 5 ... Caricature selection part 9 ... Display part 10 ... Printing part,
DESCRIPTION OF SYMBOLS 11 ... Singing mode detection part 15, 16 ... Caricature composition part 23 ... Audio | voice signal processing apparatus 27 ... Singing microphone 29 ... Printer 30 ... CPU
32 ... RAM
32a ... Formant storage area 37 ... Hard disk storage device 37a ... Sample database 40 ... Pattern development unit

Claims (3)

Voice input means for inputting voice;
Formant analysis means for analyzing the formant of the input speech,
A storage means for storing a plurality of sample formants and facial caricatures having resonance characteristics of the sample formants in association with each other;
Carrying out the storage means with the analyzed formant, selecting a portrait corresponding to the closest sample formant and outputting the portrait
Caricature output device with Voice input means for inputting voice;
Formant analysis means for analyzing the formant of the input speech,
Material storage means for storing caricature material for creating a caricature,
Based on the formant analyzed by the formant analysis means, a portrait output means for synthesizing and outputting a portrait of the face shape having the resonance characteristics of the formant using the portrait material;
Caricature output device with Voice input means for inputting singing voice;
Formant analysis means for analyzing the formant of the input speech,
Singing analysis means for analyzing the singing mode of the input voice;
Material storage means for storing caricature material for creating a caricature,
Based on the formant analyzed by the formant analysis means and the singing form analyzed by the singing analysis means, the portrait synthesis is performed by synthesizing and outputting the portraits singing in the singing form with the face shape having the resonance characteristics of the formants using the portrait material. Means,
Karaoke device equipped with.

Images