JP2009265541A - Singer image synthesizing display system according to singing score result - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enhance the entertainment of scoring singing for a musical piece including a plurality of singing parts. <P>SOLUTION: The system comprises: a singer image management means 33 for managing a singer image 36 which is related to a user ID; a log-in user management means 53 which specifies and manages a user who actually logs in a karaoke performance terminal 11 on the basis of the user ID; a singing scoring means 54 which specifies the user who is a singer for each singing section corresponding to each singing part for a karaoke musical piece including the plurality of singing parts on the basis of the user ID, and scores the singing capability of the singer in the prescribed number of singing sections; a singer image synthesizing means 55 which creates a synthesized singer image 45b by synthesizing each singer image of each singer specified for each singing section according to each score result; and a synthesizing singer image display means 56 which displays the synthesized singer image 45b on a display section. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a singer image synthesis display system according to a singing scoring result that synthesizes and displays a singer image according to each singer's scoring result for a karaoke piece having a plurality of singing parts.

  Many karaoke apparatuses that are currently popular have a function for scoring the singing ability of a singer (see Patent Document 1). The technique described in Patent Document 1 sequentially detects the degree of coincidence of pitch transitions between the model melody data of the singing part attached to the accompaniment music data and the singing voice signal of the singer performing the singing along with the accompaniment music. The degree of coincidence is accumulated and scoring data is output. In addition, there is a karaoke apparatus that can score each singer's singing ability when two male and female singers sing each singing part in turn, such as duet music.

  In addition, the user's human video shot with the video camera is sampled appropriately to generate still picture data of the face photograph, and the modification process based on the singing scoring result is added to the still picture data of the face photograph There is also a karaoke device that can be displayed (see Patent Document 2). In the technique described in Patent Document 2, the higher the singing score is, the smaller the degree of deformation of the face photograph is, and the lower the singing point is, the larger the degree of deformation of the face photograph is. It is intended to be displayed.

JP 2007-121917 A Japanese Patent Application Laid-Open No. 11-231881

  However, in the conventional technique represented by the above-described Patent Document 1, when the performance of the karaoke music is completed, the singing ability of the singer is displayed in a numerical value or the winner of the scoring battle is displayed. In general, such a scoring result display method allows the singer and viewers not to judge the superiority or inferiority of the singing ability in the middle of singing, and to fully enjoy the fun of singing using a karaoke device. In some cases, it was not possible. Furthermore, even if the scoring results in the middle of singing are sequentially displayed, the scoring battle lacks excitement and is less entertaining.

  Moreover, the technique described in the patent document 2 mentioned above presupposes displaying the singing score result of a single singer. Therefore, when performing a scoring battle for a duet song, etc., even if each singer's face photo is individually transformed and displayed by applying the technology, the singing ability of each singer can be judged, It cannot be intuitively judged which singer's singing ability is excellent.

  The present invention has been proposed in view of the above-described circumstances, and according to a singing scoring result that can enhance the entertainment of singing scoring using the singing scoring function of the karaoke apparatus for karaoke music having a plurality of singing parts. An object of the present invention is to provide a singer image synthesis display system.

  The singer image composition display system according to the singing scoring result of the present invention has the following characteristic points in order to achieve the above-described object. That is, the singer image composition display system according to the singing scoring result of the present invention has a function of displaying a singer image and a function of scoring a singer's singing power when playing karaoke music. A system for synthesizing and displaying a singer image according to the singer's scoring result for a karaoke piece having a plurality of singing parts, the singer image management means, a login user management means, and a singing scoring means And a singer image synthesis means and a synthesized singer image display means.

  The singer image management means is means for managing an image representing the user in association with the user ID as a singer image in the customer database. The login user management means is means for specifying and managing a user who is currently logged in to the karaoke performance terminal based on the user ID. The singing scoring means specifies a user who is a singer for each singing section corresponding to each singing part based on the user ID for a karaoke piece having a plurality of singing parts, and determines the singing ability of the singer in a predetermined number. It is a means for scoring in the singing section. The singer image synthesis means is a means for creating a synthesized singer image by synthesizing each singer image of each singer identified for each singing section at a ratio corresponding to each scoring result in the singing scoring means. It is. The synthesized singer image display means is a means for causing the display unit to display the synthesized singer image at a predetermined timing.

  In the singer image composition display system according to the singing scoring result having the above-described configuration, for example, a karaoke piece in which two men and women sing each singing part, such as a duet piece having a male voice part, a female voice part, and a mixed voice part. The singing grading means scores the singing ability of a singer who sings each singing part in a predetermined number of singing sections. Here, when the predetermined number is set to “1”, scoring is performed for each singing section, and when the predetermined number is set to “2” or more, scoring is performed over a plurality of singing sections.

And, by the singer image synthesis means, a singer image of a singer who sings a male voice part and a singer image of a singer who sings a female voice part are synthesized to create a composite singer image. Image synthesis is performed at a ratio according to the scoring results. Specifically, when the score of a singer who sings a male voice part is higher, the ratio of the singer image of the male voice part is higher, and when the score of a singer who sings the female voice part is higher, Image synthesis is performed so that the ratio of the part singer images increases.
The synthesized singer image combined with the image is displayed on the display unit of the main monitor that projects the background image by the synthesized singer image display means.

  According to the singer image synthesis display system according to the singing scoring result of the present invention, the singer images can be synthesized and sequentially displayed according to the scoring result of each singer for karaoke music pieces having a plurality of singing parts. Therefore, the entertainment of singing using the singing scoring function of the karaoke device can be enhanced.

Hereinafter, an embodiment of a singer image composition display system according to the singing scoring result of the present invention will be described with reference to the drawings.
The singer image composition display system according to the singing scoring result according to the embodiment of the present invention has a function of displaying a singer image and a function of scoring a singer's singing ability when playing karaoke music. And it is a system which synthesize | combines and displays a singer image about the karaoke music which has several singing parts according to each singer's scoring result.

<Outline of Karaoke System>
FIG. 1 is an explanatory diagram showing a configuration of a karaoke system including a singer image synthesis display system according to a singing scoring result according to an embodiment of the present invention, FIG. 2 is an explanatory diagram showing a configuration of a karaoke performance terminal, and FIG. These are explanatory drawings of a customer database.

  As shown in FIG. 1, a karaoke system including a singer image synthesis display system according to a singing scoring result according to an embodiment of the present invention is connected to a host device 13 and a plurality of karaokes via a communication network 12. And a performance terminal 11. The communication network 12 can use a public telephone line, a dedicated telephone line, an optical communication line, a LAN, etc., but it is difficult for a third party to infiltrate the network, to listen to and alter data, and to not monopolize the bandwidth. It is preferable to use a VPN constituted by the Internet because it is an inexpensive communication network.

<Host device>
As shown in FIG. 1, the host device 13 is a customer database storing user information including control means 31, data processing means 32 having functions as a singer image management means 33, transmission / reception means 34, and singer image 36. 35. Among these constituent elements, the singer image management means 33 and the customer database 35 are the login user management means 53, the singing scoring means 54, the singer image synthesizing means 55, and the synthetic singer who are constituent elements of the karaoke performance terminal 11. Together with the image display means 56, it functions as a singer image composition display system according to the singing score result.

  The transmission / reception means 34 is means for transmitting / receiving data to / from each karaoke performance terminal 11, and is composed of a communication circuit and software for maintaining the consistency of the communication system in the communication network 12. The control unit 31 is a unit for controlling the host device 13 in an integrated manner, and includes a CPU and its peripheral devices. The control unit 31 operates according to an application program stored in a ROM or the like and has a control function. It comes to show.

<Data processing means / customer database>
The data processing means 32 is a means that also functions as the singer image management means 33. Specifically, the data processing means 32 includes a program for accessing the customer database 35 and processing data extraction, change, and addition. In particular, the singer image management means 33 includes a program for managing an image representing the user as a singer image 36 in association with the user ID in the customer database 35.
The customer database 35 is a database in which user information is registered for each user, and has a singer image 36 as one of the user information. As shown in FIG. 3, the customer database 35 has a user ID, name, second name (for example, nickname), address, telephone number, date of birth, gender, occupation, and singer image associated with each user. It is configured. The singer image 36 may be a photograph of the user's face or an avatar selected by the user. The information registered in the customer database 35 is not limited to the information described above, and may be only necessary information as appropriate, or other information may be added.

<Karaoke performance terminal>
The karaoke performance terminal 11 to which the singer image composition display system according to the singing scoring result according to the embodiment of the present invention is applied, as shown in FIG. 2, the karaoke main body 21, the karaoke remote control device 22, the mixing amplifier 23, and the speaker 24. , A microphone 25 and a display means 26 are provided. The karaoke main body 21 and the display means 26 are connected by a wired system or a wireless system.

<Karaoke remote control device>
The karaoke remote control device 22 has a user interface function, and is configured to transmit / receive data to / from the local transmission / reception means 47 of the karaoke main body 21 by a wired method or a wireless method, and a circuit for transmitting / receiving data. A board and a program, a program for functioning as the music search means 22a, a music index database 22b, and a flash memory (not shown) as a temporary storage area are provided.

<Music index database>
The music index database 22b referred to by the music search means 22a is a database in which attribute information about the music used for performance at the karaoke performance terminal 11 is described. For example, the song number, the song name, the singer name, and the lyrics of the singing part -Various attribute information such as whether it is popular, music genre classification, duet or not is included. The contents of the music index database 22b are updated to the latest one at an appropriate timing when the karaoke performance terminal 11 communicates with the host device 13.

<Karaoke body>
The karaoke main body 21 includes a bus 41, network transmission / reception means 42, central control means 43, ROM 44, RAM 45, HDD 46, local transmission / reception means 47, reservation management means 48, music reproduction control means 49, A / D converter 50, video reproduction control means. 51, a user ID acquisition unit 52, a login user management unit 53, a singing scoring unit 54, a singer image synthesis unit 55, and a synthetic singer image display unit 56.

<Central control means>
The central control means 43 is a means for comprehensively controlling the karaoke performance terminal 11, and is composed of, for example, a CPU and its peripheral devices, and the CPU or the like operates according to an application program stored in the ROM 44 or the like. The control function can be demonstrated.

<ROM / RAM>
The ROM 44 is a device for storing application program data and numerical data for controlling each device constituting the karaoke performance terminal 11, and is composed of, for example, a semiconductor memory. The RAM 45 functions as a temporary storage area for temporarily storing data such as an application program and the synthesized singer image 45b, and is constituted by, for example, a semiconductor memory. Note that the RAM 45 may be configured using a hard disk storage device or the like instead of the physical semiconductor memory. In the present embodiment, a reservation queue 45 a and a synthesized singer image 45 b are stored in the RAM 45.

<HDD>
The HDD 46 stores a music database 46a and a video database 46b. The HDD 46 also functions as a singer image temporary storage unit 46 c for temporarily storing a singer image acquired from the customer database 35 of the host device 13. As will be described in detail later, the singer image temporarily stored in the singer image temporary storage unit 46 c is used when the singer image synthesizing means 55 synthesizes the singer image. Note that a mass storage device such as a DVD capable of rewriting data may be used instead of the HDD 46 or together with the HDD 46.

<Music database / video database>
The music database 46a is a database that stores music data configured by synchronizing performance data (MIDI (registered trademark) data) and lyrics telop data in association with music IDs. The performance data is digitized performance data of each piece of music, and the lyrics telop data is the lyrics character data of the music synchronized with the performance data.
The video database 46b is a database in which a predetermined number of background videos corresponding to the music to be played are stored as video files corresponding to the music ID of the music.

<User ID acquisition means>
The user ID acquisition means 52 includes an electronic circuit and a program for acquiring a user ID from an ID medium 61 possessed by the user when the user uses the karaoke performance terminal 11. The user ID acquired by the user ID acquisition unit 52 is temporarily stored in the RAM 45 and managed by the login user management unit 53. The ID medium 61 includes, for example, a non-contact type IC card, a mobile phone, a portable information terminal, etc., can store a user ID, and can exchange data with the user ID acquisition means 52. It is like that.

<Login user management means>
The login user management means 53 includes a program for specifying and managing a user who is currently logged in to the karaoke performance terminal 11 based on the user ID. That is, when the user ID is acquired by the user ID acquisition unit 52, the login user management unit 53 identifies and manages the user who has logged in to the karaoke performance terminal 11 based on the user ID. To do.

<Singing method for singing>
The singing scoring means 54 specifies the user who becomes a singer for each singing section corresponding to each singing part based on the user ID, and determines the singing ability of the singer for a karaoke piece having a plurality of singing parts. It consists of a program for scoring in a number of song sections. Karaoke music includes a non-singing section in which a singer does not sing, such as a prelude section, an interlude section, and a follower section, and a singing section in which a singer actually sings. In the singing scoring means 54, for example, for a karaoke piece having a plurality of singing parts such as a male voice part, a female voice part, and a mixed voice part, a user who becomes a singer for each singing section corresponding to each singing part is assigned a user ID. Based on And, in a predetermined number of singing sections, scoring is performed for each classified singing part.

  In singing scoring, singing evaluation data such as pitch, volume and tempo are extracted from the singer's voice signal input from the microphone 25, and the singing evaluation data is compared with reference information such as the main melody of the performance music. Scoring. Such a singing scoring method is described in detail in Japanese Patent Application Laid-Open No. 2007-121917.

<Singer image synthesis means>
The singer image synthesizing unit 55 synthesizes the singer images of each singer specified for each singing section at a ratio according to the respective scoring results in the singing grading unit 54 to create a synthesized singer image 45b. For the program. The created composite singer image 45 b is temporarily stored in the RAM 45. The singer image composition means 55 extracts and synthesizes parts such as eyes, eyebrows, ears, nose, mouth, chin, and hairstyle, respectively, or synthesizes images using a morphing technique. When synthesizing, an optimal synthesized singer image 45b can be created according to the scoring result by weighting each part of the face, such as by setting a high point of the part that affects the facial expression. .
The image composition technique in the singer image composition means 55 may be any method, and for example, an image composition technique described in Japanese Patent Application Laid-Open No. 2004-5265 may be used.

<Synthetic singer image display means>
The synthesized singer image display means 56 is composed of a program for causing the display part of the display means 26 to display the synthesized singer image 45 b synthesized by the singer image synthesis means 55 and temporarily stored in the RAM 45. The composite singer image display means 56 may display the composite singer image 45b in any way. For example, the composite singer image 45b that has already been displayed is faded out and a new composite singer image 45b is displayed. The image 45b can be faded in, or the synthesized singer image 45b that has been displayed can be continuously changed by using a morphing technique to display a new synthesized singer image 45b.

<Music playback control means>
The music reproduction control means 49 is an electronic circuit for digitally reproducing the performance data extracted from the music database 46 a based on the music ID and converting it to analog and outputting it to the mixing amplifier 23. The mixing amplifier 23 mixes the singer's singing voice signal input from the microphone 25 and the performance voice signal sent from the music reproduction control means 49, and amplifies the signal by the amplifier function to output from the speaker 24. Device. The number of microphones 25 is not limited to one and may be two or more.
The microphone 25 is connected to the bus 41 via the A / D converter 50, and the audio signal input from the microphone 25 is used for singing scoring by the singing scoring means 54.

<Video playback control means / display means>
The video reproduction control means 51 synchronizes the video data extracted from the video database 46b and the lyrics telop data (lyric character data) of the music data extracted from the music database 46a with the performance data of the music during the karaoke performance. This is an electronic circuit for outputting to the display means 26. Further, the video reproduction control means 51 has a function of causing the display means 26 to display a reserved music list display in response to an operation instruction from the user via the karaoke remote control device 22.
The display means 26 is a device for displaying a video at the time of karaoke performance, a lyrics telop, a synthesized singer image 45b, and the like, and includes, for example, a CRT display, a liquid crystal display, a plasma display, and the like.

<Reservation management means>
The reservation management means 48 uses the function of the music search means 22a in the karaoke remote control device 22 by the user to select music and transmits the user information (for example, user ID) of the music selecter transmitted via the local transmission / reception means 47, and This is a program for associating the selected music ID and storing it in the RAM 45 as a reservation queue 45a for management. Note that the music selection display and the reservation queue 45a can be displayed on either or both of the input / output display unit and the display means 26 of the karaoke remote control device 22.

<Transmitting / receiving means>
The local transmission / reception means 47 includes an electronic circuit and a program for transmitting / receiving data to / from the karaoke remote control device 22. The network transmission / reception means 42 includes an electronic circuit and a program for maintaining the consistency of the communication method in the communication network 12 when transmitting / receiving data to / from the host device 13 via the communication network 12.

<Comparison of singing score and creation of composite image>
With reference to FIG. 4, the comparison of the singing score value in the singer image synthetic | combination display system according to the singing score result of this invention and preparation of a synthesized image are demonstrated. FIG. 4 is an explanatory diagram showing procedures for comparing singing score values and creating a composite image. 5-7 is explanatory drawing which shows an example of the synthetic singer image displayed with the performance progress of karaoke music.

  In the singer image composition display system according to the singing scoring result of the present invention, the music played on the karaoke performance terminal 11 is distinguished into a non-singing section and a singing section. The example shown in FIG. 4 is a case where a duet music is played, and the singing section is distinguished into a male voice part, a female voice part, and a mixed voice part. In the case of a duo song sung by two men or a duo song sung by two women, it is only necessary to distinguish between each singer's part and mixed voice part. Furthermore, what is necessary is just to distinguish a singing part similarly about the music which three or more singers sing like a trio music or a quartet music.

  When performing music selection using the karaoke remote control device 22, a singer who sings each singing part (singing section) is specified by inputting the user ID of the music selector. In addition, since the user information stored in the customer database 35 includes gender data, in the case of a duet song in which two men and women sing, each singing is performed based on the acquired gender information of the user. A singer who sings a part may be specified. At this time, two or more microphones 25 may be prepared, and each singing part (singing section) may be associated with the microphone 25.

  When the performance of karaoke music is started, first, a start image as shown in FIG. 5 is displayed. Singing is performed in a predetermined number of singing sections. In the example shown in FIG. 4, the singing scoring means 54 first performs singing scoring for the male voice part (M1) and the female voice part (F1), and the singer image compositing means 55 compares the singer's scoring results. The singer image of the male voice part and the singer image of the female voice part are synthesized at a ratio according to the comparison result, and temporarily stored in the RAM 45 as a synthesized singer image 45b. Here, for example, if the singing score result of the male voice part is 60 points and the singing score result of the female voice part is 80 points, the ratio of the singer image of the male voice part and the singer image of the female voice part is 60:80. Is synthesized. In the synthesis of a singer image, image synthesis is performed by extracting portions such as eyes, eyebrows, ears, nose, mouth, chin, and hairstyle, respectively, in accordance with the singing score result. Then, in the mixed voice part (MF1) following the male voice part (M1) and female voice part (F1) and the subsequent interlude section, the synthesized singer image display means 56 displays the synthesized singer image 45b. The synthetic singer image 45b displayed at this time is as shown in FIG.

  Similarly, the singing score for the male voice part (M2) and the female voice part (F2) is performed, and the singer image of the male voice part and the singer image of the female voice part are synthesized at a ratio according to the comparison result of the singing scores. The mixed singer image 45b is displayed in the mixed voice part (MF2) following the male voice part (M2) and female voice part (F2) and the subsequent interlude section. Here, for example, if the singing score of the male voice part is 75 points and the singing scoring result of the female voice part is 68 points, the ratio of the singer image of the male voice part and the singer image of the female voice part is 75:68. Is synthesized. The synthetic singer image 45b displayed at this time is as shown in FIG.

  In the same way, singing is performed for the male voice part (M3) and female voice part (F3), respectively, and the singer image of the male voice part and the singer image of the female voice part are synthesized at a ratio according to the comparison result of the singing scores. Then, the mixed singer image 45b is displayed in the mixed voice part (MF3) following the male voice part (M3) and female voice part (F3), and the subsequent interlude section.

  Then, when the follow-up section ends, the singer image of the male voice part and the singer image of the female voice part are synthesized at a ratio according to the overall evaluation that combines the scoring results in each singing part, and the synthesized singer image 45b is obtained. The composite singer image 45b is displayed for a predetermined period until it is created and the next musical piece starts to be played.

<Other embodiments>
The apparatus and means which comprise the singer image composition display system according to the singing scoring result of the present invention and its peripheral devices are not limited to those described above, depending on the purpose of use of the singer image composition display system and the karaoke system. It is possible to configure only the necessary devices and means, or to add other devices and means as appropriate. Further, each unit may be configured as a unit in which a plurality of functions are integrated, instead of being configured separately.

  Further, the karaoke performance terminal 11 may be operated stand-alone without being connected to the network. In this case, the singer image management means 33 and the customer database 35 which are components of the host device 13 are components of the local karaoke device.

Explanatory drawing which shows the structure of the karaoke system containing the singer image composition display system according to the singing scoring result which concerns on embodiment of this invention. Explanatory drawing which shows the structure of a karaoke performance terminal. Explanatory drawing of a customer database. Explanatory drawing which shows the procedure of comparison of a singing score value, and preparation of a synthesized image. Explanatory drawing which shows an example of the synthetic | combination singer image displayed with the performance progress of karaoke music. Explanatory drawing which shows an example of the synthetic | combination singer image displayed with the performance progress of karaoke music. Explanatory drawing which shows an example of the synthetic | combination singer image displayed with the performance progress of karaoke music.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Karaoke performance terminal 12 Communication network 13 Host apparatus 21 Karaoke main body 22 Karaoke remote control apparatus 22a Music search means 22b Music index database 23 Mixing amplifier 24 Speaker 25 Microphone 26 Display means 31 Control means 32 Data processing means 33 Singer image management means 34 Transmission / reception Means 35 Customer database 41 Bus 42 Network transmission / reception means 43 Central control means 44 ROM
45 RAM
45a reservation queue 45b composite singer image 46 HDD
46a Music database 46b Video database 46c Singer image temporary storage section 47 Local transmission / reception means 48 Reservation management means 49 Music reproduction control means 50 A / D converter 51 Video reproduction control means 52 User ID acquisition means 53 Login user management means 54 Singing Scoring means 55 Singer image synthesizing means 56 Synthetic singer image displaying means 61 ID medium

Claims (1)

When playing karaoke music, it has a function of displaying a singer image and a function of scoring the singing ability of the singer, and for karaoke music having a plurality of singing parts, according to the scoring result of each singer. A singer image synthesis display system according to a singing scoring result to synthesize and display a singer image,
A singer image management means, a login user management means, a singing scoring means, a singer image synthesis means, and a synthesized singer image display means,
The singer image management means manages an image representing the user in association with a user ID as a singer image in the customer database,
The login user management means identifies and manages the user who is currently logged in to the karaoke performance terminal based on the user ID,
The singing scoring means specifies a user who is a singer for each singing section corresponding to each singing part for a karaoke piece having a plurality of singing parts based on the user ID, and determines the singing ability of the singer. Scoring in a number of song segments,
The singer image synthesis means creates a synthesized singer image by synthesizing each singer image of each singer identified for each singing section at a ratio corresponding to each scoring result in the singing scoring means,
The synthetic singer image display means displays the synthetic singer image on a display unit at a predetermined timing.
The singer image composition display system according to the singing scoring result characterized by this.

Images