JP2015210344A - Communication karaoke system and server - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow a server to communicate with both of portable information terminals of a singing person and photographer at the same time, and increase variations of karaoke use modes.SOLUTION: A communication karaoke system comprises: first storage means 13 for storing karaoke music data 13a; distribution means 14 for reading the karaoke music data 13a stored in the first storage means 13 and distributing the karaoke music data 13a by streaming distribution to a portable information terminal 30 which a singing person has, and a portable information terminal 30 which a photographer who images the singing person has; voice data reception means 15 for receiving singing voice data from the portable information terminal 30 which the singing person has, after completion of the streaming distribution; and video data reception means 16 for receiving singing video data from the portable information terminal 30 which the photographer has, after completion of the streaming distribution.

Description

  The present invention relates to a communication karaoke system and a server, and more specifically, streaming karaoke music data to a portable information terminal possessed by a user and receiving singer's singing voice data and singer's singing video data. The present invention relates to a communication karaoke system and a server capable of performing various data processing.

  A technique related to a communication karaoke system using a portable information terminal is known (see, for example, Patent Documents 1 to 5). In particular, portable information terminals have been widely used in recent years, and if such a technique is used, anyone can easily sing karaoke songs of their choice anywhere.

  Patent Document 1 discloses a mobile phone comprising a server that distributes stored karaoke music data to a singer's mobile phone, and a mobile phone that reproduces the karaoke music and scores the singer's singing and transmits the scoring results to the server. A telephone system is disclosed. If this technique is used, in addition to being able to sing karaoke easily, the scoring results are managed by the server, so that the effect of helping to improve the singing ability can be obtained.

  In addition, there is disclosed a technique for proposing various ways to enjoy karaoke using a singing video of a singer photographed with a portable information terminal by a singer's accompanying person in a karaoke room. For example, the technique described in Patent Document 2 combines a singing video shot with a digital video camera installed in a karaoke room and a video shot using a mobile phone in the karaoke room as a background video. It is displayed on the display in the karaoke room or posted to the public server as video data.

  Also, a technology related to a service for displaying an avatar selected by a user on a display for displaying background video and lyrics in a karaoke room is disclosed. Such services include virtual reality and online RPG (role-playing game) services that allow an avatar to sing in a karaoke room provided in a virtual space.

  For example, the technique described in Patent Document 3 displays an animation in which characters corresponding to two singers fight in a karaoke battle game where two singers enjoy singing while competing for the grading process of the singing. The superiority or inferiority of both characters changes according to the scoring results for each singing section.

  In addition, a technique is known in which a singer's movement (choreography) is extracted by a motion capturing technique and the movement of an avatar in a virtual space is controlled and displayed. Some home game machines can simultaneously capture the actions of six game participants.

  For example, the technique described in Patent Document 4 estimates the posture of a photographed subject person and displays an avatar having the same posture in a virtual space on a monitor.

  In addition, in a karaoke room, in addition to a karaoke device, a photographing unit, a recording and recording unit, and the like are provided, which shoots a song video of a singer, uploads it to a server, and makes it available for viewing. For example, the technique described in Patent Document 5 appropriately extracts an index (thumbnail) image of a singing moving image when the photographed singing moving image is recorded and released.

JP 2001-197168 A JP 2013-210464 A Japanese Patent Laid-Open No. 10-268876 JP 2001-195595 A JP 2014-17570 A

  As described above, in a communication karaoke system using a portable information terminal, a technique in which a server communicates with a singer's portable information terminal or a photographer's portable information terminal is known. However, the server communicates with both the singer and the photographer's portable information terminal at the same time, the singer enjoys karaoke singing using the portable information terminal, the photographer shoots the song video using the portable information terminal, There is no technology to display in the virtual space, and its development has been desired.

  Moreover, in order to shoot and post a karaoke song video, the user has to go to the karaoke room, which is troublesome for the user. For example, if you can shoot karaoke songs and singing videos using your personal digital assistant at home without going to a karaoke room and post (open) karaoke song videos, you can further expand your enjoyment of karaoke. be able to.

  The present invention has been proposed in view of the above-described circumstances, and a communication karaoke system and server capable of increasing the number of variations of karaoke usage modes by allowing the server to communicate with both the singer and the photographer's portable information terminal at the same time. The purpose is to provide.

  The communication karaoke system and server of the present invention have been proposed in view of the above-described circumstances, and have the following features. That is, the server of the online karaoke system of the present invention reads the karaoke song data stored in the first storage means, the first storage means for storing karaoke song data, and the portable information terminal possessed by the singer and the song Distribution means for streaming distribution to a portable information terminal possessed by the photographer who photographs the person, voice data receiving means for receiving singing voice data from the portable information terminal possessed by the singer after the end of streaming distribution, and after the end of streaming distribution And video data receiving means for receiving singing video data from a portable information terminal possessed by the photographer.

  In addition to the above-described configuration, the second storage means for storing background video data for configuring the virtual space and avatar data for displaying a plurality of different avatars, and a portable information terminal possessed by the singer An accepting means for accepting designation of one of a plurality of different avatars as a singer avatar based on the input; an extracting means for extracting the movement of the singer from the singing video data received by the video data receiving means; The movement of the singer avatar is controlled in accordance with the movement of the singer extracted by the extraction means, and a virtual space is configured by the singer avatar and the background video based on the background video data stored in the second storage means, Raw data for generating a virtual karaoke room video synchronized with the singing voice data and the karaoke music data received by the voice data receiving means It is possible and means.

  In addition to the configuration described above, it is possible to provide scoring means for evaluating the voice data received by the voice data receiving means and scoring the song of the singer. In such a configuration, the second storage unit stores a plurality of background video data including at least video data for representing the karaoke room, and the generation unit further includes a plurality of background video data according to the scoring result by the scoring unit. It is possible to select one of the background video data and generate a virtual karaoke room video.

  In addition to the above-described configuration, the accepting unit further accepts designation of one of a plurality of different avatars as the audience avatar who is the audience of the singer avatar from the portable information terminal possessed by the photographer. The generating means can further determine the positional relationship between the singer avatar and the audience avatar based on the song video data, and generate a virtual karaoke room image including the audience avatar.

  Further, in addition to the above-described configuration, the video data receiving means further receives the voice data of the photographer collected simultaneously with the shooting of the song video together with the song video data from the portable information terminal owned by the photographer, and generates The means further generates a virtual karaoke room video by mixing the photographer's voice data received by the video data receiving means in synchronization with the singing voice data and the karaoke music data received by the voice data receiving means. Is possible.

  Further, in addition to the above-described configuration, it is possible to provide a publishing unit that publishes the virtual karaoke room video so that it can be viewed.

  The communication karaoke system of the present invention includes a server and a portable information terminal capable of data communication with the server. And a server reads the karaoke music data memorize | stored in the 1st memory | storage means and the 1st memory | storage means which memorize | store karaoke music data, and the photographer who image | photographs the portable information terminal which a singer possesses, or the said singer A distribution means for streaming distribution to only one of the portable information terminals possessed, a voice data reception means for receiving singing voice data from the portable information terminal possessed by the singer after the end of streaming distribution, and a photograph after the end of streaming distribution Video data receiving means for receiving singing video data from the portable information terminal possessed by the person.

  Also, the portable information terminal transfers the karaoke song data from one portable information terminal that has received streaming distribution of karaoke song data to the other portable information terminal (a portable information terminal that has not received streaming distribution of karaoke song data). The transfer means is provided.

  In the communication karaoke system and server having such a configuration, the karaoke music data is streamed to a singer's portable information terminal (for example, a smartphone), the singer sings along with the karaoke performance, and the recorded singing voice data is recorded. Send to server. At the same time, the server simultaneously distributes karaoke music data to the photographer's portable information terminal, the photographer shoots a song video, and transmits the recorded song video data to the server. The streaming distribution of karaoke music data is performed only on one of the portable information terminal of the singer or the portable information terminal of the photographer, and from the portable information terminal receiving the streaming distribution of the karaoke music data, The received karaoke song data may be transferred to the information terminal.

  Then, the server reflects the movement of the singer extracted from the singing video data on the avatar specified by the singer, synthesizes it with the background of the karaoke room or stage, and synchronizes it with the karaoke performance sound or singing voice. Generate virtual karaoke room video.

  The server also performs singing scoring based on the singing voice data, and determines the specification (grade) of the virtual karaoke room according to the scoring result.

  In addition, the server generates a virtual karaoke room video in which the avatar designated by the photographer is arranged as a spectator. At that time, the positional relationship between the singer avatar and the audience avatar is determined based on the singing video data. For example, when the singer's face is reflected from the front, the photographer is shooting in the front of the singer, and when the singer's right profile is reflected, the photographer The positional relationship between the singer's avatar and the audience's avatar can be determined based on the singing video data, such as shooting on the left side (left side toward the stage).

  In addition, the voice of the photographer collected at the same time as shooting can be mixed with the karaoke performance sound or singing voice to generate a virtual karaoke room video.

  In the communication karaoke system and server of the present invention, it is easy to enjoy karaoke singing using a portable information terminal in a friend's room, for example, in a friend's room, and to record and record singing voice and singing video. Can be sent to the server for sharing. Furthermore, based on these singing voices and singing images, a singing image in a virtual karaoke room in which the avatars of singers and photographers are arranged can be generated and released to a third party for enjoyment. In addition, it is possible to respond to a request to publish a song with a movie, although it does not want to disclose a private shooting place or a singer's face.

  Thus, according to the communication karaoke system and the server of the present invention, since the server can communicate with both the singer and the photographer at the same time, the variation of the karaoke usage mode can be increased, and the karaoke fun can be further enhanced. This can be further enhanced.

The block diagram which shows schematic structure of the server of the communication karaoke system which concerns on embodiment of this invention. Explanatory drawing which shows the production | generation procedure of a virtual karaoke room image | video. The schematic diagram which shows the produced | generated virtual karaoke room image | video. The block diagram which shows schematic structure of the portable information terminal of the communication karaoke system which concerns on other embodiment of this invention. Explanatory drawing which shows the other example of the production | generation procedure of a virtual karaoke room image | video.

  With reference to the drawings, an embodiment of the communication karaoke system and server of the present invention will be described. 1 to 5 illustrate a communication karaoke system and a server according to an embodiment of the present invention, FIG. 1 is a block diagram showing a schematic configuration of the server, and FIG. 2 is a description showing a procedure for generating a virtual karaoke room video. FIGS. 3A and 3B are schematic diagrams showing generated virtual karaoke room images, and FIG. 4 is a block diagram showing a schematic configuration of a portable information terminal of a communication karaoke system according to another embodiment. FIG. 5 is an explanatory diagram showing another example of the procedure for generating the virtual karaoke room video.

  In the following description, a program is stored in a RAM or the like and executed by hardware such as a CPU, so that not only software that exhibits its function but also a logic circuit that can exhibit an equivalent function. It is a concept that includes.

<Outline of online karaoke system and server>
As shown in FIG. 1, the communication karaoke system and server according to the embodiment of the present invention do not use a karaoke performance device 70 installed in a karaoke room or the like, and a portable information terminal 30 (for example, a smartphone) possessed by a user. ) And a server 10 for the communication karaoke system and the server 10 for realizing the function.

<Mobile information terminal>
In the communication karaoke system according to the embodiment of the present invention, karaoke application software is downloaded and installed in the portable information terminal 30 owned by the users (singer and photographer). The downloading method of karaoke application software should just use a well-known technique, for example, by accessing an application delivery server (not shown) using the data communication function of the portable information terminal 30, and downloading karaoke application software, The karaoke application software can be installed in the portable information terminal 30.

  Moreover, in the communication karaoke system which concerns on embodiment of this invention, the system login to the server 10 can be carried out using the portable information terminal 30 which a user (singer and photographer) owns. A known login method may be used as a method for system login to the server 10. For example, it is possible to log in to the server 10 by accessing the server 10 using the data communication function of the portable information terminal 30 and performing authentication based on the device ID or the like of the portable information terminal 30. In addition, download and installation of karaoke application software using the portable information terminal 30, system login to the server 10 of the user (singer and photographer), and data registration may be in any order.

<Mobile information terminal>
As shown in FIG. 4, the portable information terminal 30 can include a transfer unit 31. When only one of the portable information terminal 30 possessed by the singer or the portable information terminal 30 possessed by the photographer who photographs the singer receives the streaming distribution of the karaoke song data, The mobile information terminal 30 includes an electronic device and a program for transferring the karaoke music data. For this transfer means 31, for example, near field communication can be used. The transfer means 31 performs wireless data communication between the portable information terminals 30, and transfers the karaoke music data that has been stream-distributed by one portable information terminal 30 to the other portable information terminal 30. A karaoke performance can be performed simultaneously on the information terminal 30. There may be one portable information terminal 30 that receives streaming distribution of karaoke song data, and there may be a plurality of portable information terminals 30 that receive transfer of karaoke song data from this portable information terminal 30.

  When both the portable information terminals 30 owned by the users (singer and photographer) receive streaming distribution of karaoke music data from the server 10, the apparatus configuration and the stream distribution of karaoke music data are simplified. There is. However, in such a configuration, if the carrier (communication service provider) of the portable information terminal 30 owned by each user is different, there may be a time lag in the distribution of karaoke song data. In this case, if the mobile information terminal 30 receives streaming distribution of karaoke music data on the one hand and transfers the karaoke music data to the other mobile information terminal 30, the time lag hardly occurs. Therefore, it is possible to perform karaoke performance almost simultaneously on both the portable information terminals 30 owned by the users (singer and photographer).

<Server>
The server 10 includes first storage means 13, distribution means 14, audio data receiving means 15, and video data receiving means 16 as basic means for realizing the above-described functions. Further, in addition to these means, the second storage means 17, the receiving means 18, the extracting means 19, and the generating means 20 can be provided, and further, a scoring means 21 may be provided. The disclosure means 22 may be provided. Further, not all the functional means described above are realized by one server 10, but the functional means described above may be divided and realized by a plurality of servers 10. In this case, one unit is realized by virtualization technology. The server 10 may be operated as a plurality of servers 10.

  The server 10 is a device for comprehensively managing a plurality of karaoke performance devices 70 and transmitting / receiving data to / from the portable information terminal 30. The server 10 is an internet, a dedicated telephone line, a general telephone line, a wireless Data communication can be performed with the portable information terminal 30 possessed by the user via the data communication line 50 such as a communication line and the communication base station 40. The server 10 is connected to a plurality of karaoke performance apparatuses 70 via the data communication line 50 and the router 60.

  As shown in FIG. 1, the server 10 includes central control means 11 including a CPU and communication control means 12 as main devices for functioning as the server 10. Furthermore, as described above, as means for realizing the communication karaoke system according to the present invention, the first storage means 13, the distribution means 14, the audio data reception means 15, the video data reception means 16, the second storage means 17, A receiving unit 18, an extracting unit 19, a generating unit 20, a scoring unit 21, and a disclosure unit 22 are provided.

<First storage means>
The 1st memory | storage means 13 consists of mass storage apparatuses, such as HDD, and has memorize | stored the some karaoke music data 13a. The karaoke music data 13a is distributed to the portable information terminal 30 owned by the user (singer and photographer), but may be distributed to the karaoke performance device 70 connected to the server 10 via a network. In addition, the 1st memory | storage means 13 of this embodiment has memorize | stored the lyric display data 13b with the karaoke music data 13a.

<Distribution means>
The distribution unit 14 reads out the karaoke music data 13a stored in the first storage unit 13 and performs streaming distribution to the portable information terminal 30 possessed by the singer and the portable information terminal 30 possessed by the photographer who photographs the singer. It consists of a program to do. In this embodiment, the distribution unit 14 is a unit for distributing the karaoke song data 13a to the portable information terminal 30 possessed by the user (singer and photographer). As described above, the server 10 And the network-connected karaoke performance device 70 can function to distribute karaoke music.

  When the lyric display data 13b is stored in the first storage unit 13, the distribution unit 14 transmits the lyric display data 13b to the portable information terminal 30 owned by the singer in advance, and then streams the karaoke song data 13a. By distributing, the singer can sing while watching the lyric display on the display of the portable information terminal 30. At this time, the singer can sing with a natural posture without wearing an unreasonable posture by wearing a wireless headset and monitoring the karaoke performance while inputting the singing voice. Further, the photographer can easily perform the photographing operation by monitoring the karaoke performance with the headphones. As will be described later, since the video of the singer is replaced with an avatar in the virtual space, the final video does not remain even when singing or wearing a large headphone while watching the lyrics display. Therefore, it is possible to concentrate on the singing without worrying about the singing appearance.

<Audio data receiving means>
The voice data receiving means 15 is composed of a program for receiving singing voice data from the portable information terminal 30 possessed by the singer after the end of streaming distribution of the karaoke song data 13a. That is, the voice data receiving means 15 requests the portable information terminal 30 to transmit the singing voice data when the singer performs the singing by reproducing the karaoke music data 13a distributed in a streaming manner, and receives it. The received singing voice data is stored in a predetermined data storage area in a storage device (not shown) composed of an HDD or the like.

<Video data receiving means>
The video data receiving means 16 comprises a program for receiving singing video data from the portable information terminal 30 owned by the photographer after the end of streaming distribution of the karaoke song data 13a. That is, the video data receiving means 16 requests the portable information terminal 30 to transmit the singing video data when the photographer reproduces the karaoke music data 13a streamed and the singing state of the singer is photographed. Receive. The received singing video data is stored in a predetermined data storage area in a storage device (not shown) composed of an HDD or the like.

  Further, as a function of the video data receiving means 16, it is possible to have a function of receiving the voice data of the photographer collected simultaneously with the shooting of the song video data from the portable information terminal 30 possessed by the photographer. It is. In the case of such a configuration, the generation unit 20 can generate a virtual karaoke room image including audio data.

<Second storage means>
The 2nd memory | storage means 17 consists of mass storage apparatuses, such as HDD, and memorize | stores the background image data 17a for comprising virtual space, and the avatar data 17b for displaying several different avatars. That is, the background video data 17a and the avatar data 17b stored in the second storage means 17 are used to perform video and audio synthesis based on the received singing voice data and singing video data, and a karaoke room (virtual space consisting of a virtual space)・ Karaoke room).

  Further, the background video data 17a stored in the second storage means 17 can be a plurality of data including at least video data for expressing a karaoke room. In the case of such a configuration, the aspect of the virtual karaoke room video generated by the generating means 20 can be further enhanced.

<Reception means>
The accepting means 18 is composed of a program for accepting designation of one of a plurality of different avatars as a singer avatar based on an input from the portable information terminal 30 possessed by the singer. That is, the accepting means 18 accepts the designation of the singer avatar desired by the singer based on the singer's instruction input using the portable information terminal 30. Thereby, a virtual space can be comprised using the singer avatar which a singer desires.

  Further, as a function of the accepting unit 18, a function of accepting designation of one of a plurality of different avatars as a spectator avatar who is an audience of the singer avatar from the portable information terminal 30 possessed by the photographer is provided. Is possible. In such a configuration, the generation unit 20 can generate an image including the audience avatar as the virtual karaoke room image.

<Extraction means>
The extracting means 19 is composed of a program for extracting the movement of the singer from the singing video data received by the video data receiving means 16. A known motion capture technique may be used to extract the movement of the singer. In particular, in the present embodiment, it is preferable to use a technique for analyzing the song video data and capturing it as a 3D motion file in order to extract the movement of the singer from the received song video data.

<Generation means>
The generation unit 20 controls the movement of the singer avatar according to the movement of the singer extracted by the extraction unit 19, and the background video based on the singer avatar and the background video data 17 a stored in the second storage unit 17. Is composed of a program for generating a virtual karaoke room video in synchronization with the singing voice data received by the voice data receiving means 15 and the karaoke song data 13a.

  That is, the generation means 20 reflects the movement of the singer extracted from the singing video data in the movement of the singer avatar, and synthesizes the singer avatar and the background video data 17a to generate virtual space data. Furthermore, the received singing voice data and karaoke music are synchronized to generate karaoke singing voice data, and the virtual space data and the voice data are synchronized to generate a virtual karaoke room video.

  Thus, by performing video synthesis and voice synthesis based on the singing voice data and the singing video data acquired using the portable information terminal 30 without the user (singer and photographer) going to the actual karaoke room, Feel free to enjoy karaoke.

  When the second storage unit 17 stores a plurality of background video data 17a including video data for representing a karaoke room, the generation unit 20 generates a plurality of background video data 17a according to the scoring result by the scoring unit 21. One of the background video data 17a is selected to generate a virtual karaoke room video. That is, the aspect of the virtual karaoke room is changed according to the singing score value in the scoring means 21. For example, the singing score value is divided into a plurality of stages, and an image is generated in which the luxury of the virtual karaoke room increases as the lowest value is changed to the highest value.

  Further, when the accepting means 18 accepts from the portable information terminal 30 possessed by the photographer that one of a plurality of different avatars is designated as the audience avatar that is the audience of the singer avatar, the creating means 20 Determines the positional relationship between the singer avatar and the audience avatar based on the song video data, and generates a virtual karaoke room image including the audience avatar. Here, as a method of determining the positional relationship between the singer avatar and the audience avatar based on the singing video data, for example, when the singer's face is reflected from the front, the photographer is positioned in front of the singer. When the singer's right profile is shown, the photographer is positioned on the left side (left side toward the stage) of the singer. It is determined that shooting is being performed (see FIG. 3B). And the virtual karaoke room image | video which arrange | positioned the singer avatar according to a singer's position and the audience avatar according to the photographer's position based on both positional relationship is produced | generated.

  Further, when the video data receiving means 16 has a function of receiving the voice data of the photographer collected simultaneously with the shooting of the singing video together with the singing video data from the portable information terminal 30 possessed by the photographer, the generating means 20 The voice data of the photographer received by the video data receiving means 16 is mixed in synchronization with the singing voice data received by the voice data receiving means 15 and the karaoke song data 13a to generate a virtual karaoke room video. In other words, the virtual karaoke room video generated by the generating means 20 is obtained by synthesizing the voice data of the photographer in addition to the singing voice data and the karaoke music data 13a.

<Scoring method>
The scoring means 21 consists of a program for evaluating the voice data received by the voice data receiving means 15 and scoring the singer's singing. Singing scoring is a technique for comparing a singer's voice data with reference data and calculating a scoring value based on the degree of approximation, and a known singing scoring technique can be used. In singing scoring, not only scoring on basic matters such as rhythm and pitch, but also scoring values may be calculated by taking into account other techniques such as vibrato, shackles, and fall.

<Public means>
The publishing means 22 comprises a program for publishing the virtual karaoke room video so that it can be browsed. That is, various virtual karaoke room videos are generated by the function of the generation means 20, and the generated virtual karaoke room videos are stored in a storage device (not shown) such as a video release server (not shown). When there is an instruction to publish either or both of the singer and the photographer, the function of the publishing means 22 publishes the virtual karaoke room video stored in the storage device so that it can be viewed. Note that it is preferable that the virtual karaoke room video by the publishing means 22 is performed not only when the singers and the photographers are instructed to publish but also when the administrator of the publishing server 10 has permission to publish.

<Procedure for generating virtual karaoke room video>
Next, a procedure for generating a virtual karaoke room image in the embodiment of the present invention will be described with reference to FIGS. FIG. 2 shows a case where the karaoke music data 13a is streamed and distributed to both the portable information terminal (a) 30 owned by the singer and the portable information terminal (b) 30 owned by the photographer. The case where the karaoke music data 13a is stream-distributed only to one of the portable information terminal (a) 30 and the portable information terminal (a) 30 owned by the singer or the photographer is shown. In addition, the same step number is attached | subjected about the procedure common to both.

  First, as a premise for generating a virtual karaoke room video, the singer installs karaoke application software on the mobile information terminal (a) 30 owned by himself, and the photographer installs on the mobile information terminal (b) 30 owned by himself. Install karaoke application software. And a singer and a photographer log in to the server 10 using the data communication function of each portable information terminal 30 (A1, B1), and pair both portable information terminals 30 in the server 10 (S1). In order to pair the portable information terminal 30, for example, the device ID of each portable information terminal 30 may be associated and managed.

  When music selection is performed at the singer or photographer's portable information terminal 30 (a music selection signal is transmitted) (A2), the server 10 accepts the music selection (receives the music selection signal) (S2). The music selection operation can be configured to cause the user to search for and specify an arbitrary piece of music from the karaoke piece data 13a of the first storage unit 13 of the server 10 according to an instruction from the server 10, for example. When an operation (transmission of a performance start instruction signal) for starting the performance of the selected music is performed (A3), the server 10 receives the performance start instruction signal (S3), and a stream of karaoke music Distribution is started (S4). Thereby, karaoke performance (stream reproduction | regeneration of a karaoke music piece) starts simultaneously with the portable information terminal 30 of a singer and a photographer (A4, B2). Note that music may be selected and played one by one, or a plurality of music may be selected and reserved, registered in a reservation queue, and played sequentially.

  Further, as shown in FIG. 5, the server 10 may perform stream distribution of karaoke music only to either the singers or the photographers' portable information terminals 30. In this case, when an operation for starting the performance of the selected music piece (transmission of performance start instruction signal) is performed (A3), the server 10 receives the performance start instruction signal (S3), and one mobile phone Stream distribution of karaoke music is started only to the information terminal (a) 30 (S4). Then, the mobile information terminal (a) 30 that has received the stream distribution of the karaoke music stream reproduces the karaoke music stream and sends the karaoke music music to the other mobile information terminal (b) 30 by the function of the transfer means 31. Transfer (A4-2). Thereby, in both portable information terminals (a) 30 and portable information terminals (b) 30, karaoke performance can be performed at the same time with almost no lime lag.

  When the singer carries the portable information terminal (a) 30 and receives the stream distribution of the karaoke music, the singing voice of the singer is mixed with the karaoke music while the singing voice is mixed with the photographer's portable information terminal (b) 30. May be transferred.

  The singer sings with the portable information terminal (a) 30, and the portable information terminal (a) 30 collects the singing voice of the singer and stores it as singing voice data (A5). On the other hand, the photographer shoots the singer's singing video with the portable information terminal (b) 30, and the portable information terminal (b) 30 collects the voice of the photographer's hand, cheering, chorus sound, etc. Store as video data (B3). The photographer's voice can be collected arbitrarily.

  When the karaoke performance is completed, the server 10 requests the singer's portable information terminal (a) 30 to transmit the singing voice data and requests the photographer's portable information terminal (b) 30 to transmit the singing video data ( S5). When each portable information terminal 30 receives a transmission request for song voice data or song video data (A6, B4), each portable information terminal 30 transmits song voice data or song video data from each (A7, B5). The server 10 receives the singing voice data and the singing video data (S6).

  The server 10 performs analysis such as pitch detection based on the received singing voice data, and calculates the singing score value by comparing with the singing score reference data included in the karaoke song data 13a (S7). Then, the server 10 determines one background video data 17a according to the scoring result from the plurality of stored background video data 17a (S8). At this time, the grade of the background video can be changed according to the singing score value. For example, when the singing score value is less than 50 points, the fixtures are also set as a plain background image in a small room of about 5 people. When the singing score value is 50 points or more and less than 80 points, a room with a seat of about 10 people and a small stage is set as a background image. In this way, the grade of the background video is sequentially raised according to the singing score value, and when the singing score value is 98 points or more, the room with the illuminated stage is set as the background video in the luxurious VIP room.

  Moreover, the singer can select one of a plurality of avatars stored in the server 10 using the portable information terminal (a) 30 and designate it as a singer avatar (A8). The person can use the portable information terminal (b) 30 to select one from a plurality of avatars stored in the server 10 and designate it as the audience avatar (B6). In addition, the singer avatar selected by the singer and the audience avatar selected by the photographer are stored in association with the device ID of the portable information terminal 30 of the singer and the photographer, respectively. The procedure for selection may be omitted.

  In this case, the server 10 estimates the positional relationship between the singer and the photographer based on the received singing video data, determines the positional relationship between the singer avatar and the audience avatar based on the estimated positional relationship, and sings. The person avatar is placed on the singer seat or stage, and the audience avatar is placed on the audience seat (S9).

  And the server 10 analyzes the received singing video data, extracts the movement of the singer synchronized with the karaoke music, and the movement of the singer avatar designated by the singer by the extracted movement data of the singer. And controlling (S10), synthesizing the singer avatar, the audience avatar and the selected background video, and mixing the received singing voice data, the karaoke song data 13a and the photographer's voice in synchronism, A virtual karaoke room image is generated (S11). Note that, as described above, the virtual karaoke room video generated by the generation unit 20 can have various aspects.

  When the virtual karaoke room video is generated, the server 10 may distribute the virtual karaoke room video to the mobile information terminal 30 of the singer and the photographer (not shown). In addition, when the virtual karaoke room video is disclosed, the server 10 publishes the virtual karaoke room video (S12).

<Other embodiments>
The devices and means constituting the system of the present invention and its peripheral devices are not limited to those described above, and only the necessary devices and means are configured according to the purpose of use, or other devices and means are appropriately added. Can be. Further, each unit may be configured as a unit in which a plurality of functions are integrated, instead of being configured separately.

  In the above-described embodiment, a single singer and a photographer have been described as examples. However, a plurality of photographers may exist for a single singer, and a plurality of singers may be present. There may be one photographer. Furthermore, there may be a plurality of singers and a plurality of photographers. In this case, there may be a plurality of sets with the photographer shooting the singer and the singers as one set, and each set does not necessarily have to be gathered in one place. You may be in a separate place every time.

  In addition, the server 10 does not guide the operation of the singer and the photographer, but the song selection reservation operation, the performance start instruction operation of the singer and the photographer by the function of the karaoke application software installed in the portable information terminal 30 in advance, It is good also as a structure which induces avatar selection operation | movement and transmits singing voice data and singing video data to the server 10 automatically.

  Further, the singer avatar and the audience avatar may be automatically determined by the function of the server 10 instead of being determined by receiving an instruction input from the portable information terminal 30. For example, the singer avatar and the audience avatar may be determined based on the member information (birth date, sex, etc.) of the singer and the photographer, or the singer avatar and the audience avatar may be determined at random. Further, the singer avatar and the audience avatar may be determined according to the result of the singing score. For example, if the singing score is high, the avatar of the visual artist is selected, and if there is a lot of fist, the avatar of the Japanese enka singer is selected.

DESCRIPTION OF SYMBOLS 10 Server 11 Central control means 12 Communication control means 13 1st memory | storage means 13a Karaoke music data 13b Lyrics display data 14 Distribution means 15 Audio | voice data receiving means 16 Image | video data receiving means 17 2nd memory | storage means 17a Background image data 17b Avatar data 18 Reception Means 19 Extraction means 20 Generation means 21 Scoring means 22 Disclosure means 30 Portable information terminal 40 Communication base station 50 Data communication line 60 Router 70 Karaoke performance device

Claims (7)

A server for an online karaoke system,
First storage means for storing karaoke song data;
Distribution means for reading out the karaoke music data stored in the first storage means and streaming the data to a portable information terminal possessed by a singer and a portable information terminal possessed by a photographer who photographs the singer,
Voice data receiving means for receiving singing voice data from the portable information terminal possessed by the singer after the end of the streaming distribution;
Video data receiving means for receiving singing video data from the portable information terminal possessed by the photographer after the end of the streaming distribution;
The server of the communication karaoke system characterized by comprising. Second storage means for storing background video data for configuring a virtual space and avatar data for displaying a plurality of different avatars;
Accepting means for accepting designation of one of the plurality of different avatars as a singer avatar based on an input from the portable information terminal possessed by the singer;
Extraction means for extracting the movement of the singer from the singing video data received by the video data receiving means;
The virtual space is controlled by controlling the movement of the singer avatar according to the movement of the singer extracted by the extraction means, and the background video based on the background video data stored in the second storage means. Generating means for generating a virtual karaoke room video in synchronization with the singing voice data and the karaoke song data received by the voice data receiving means,
The server of the communication karaoke system of Claim 1 characterized by the above-mentioned. The scoring means for scoring the song of the singer by evaluating the voice data received by the voice data receiving means,
The second storage means stores at least a plurality of background video data including video data for expressing a karaoke room,
The generating means further generates the virtual karaoke room video by selecting one of the plurality of background video data according to the scoring result by the scoring means.
The server of the communication karaoke system according to claim 2. The accepting means further accepts from the portable information terminal possessed by the photographer that one of the plurality of different avatars is designated as an audience avatar that is an audience of the singer avatar,
The generation means further determines a positional relationship between the singer avatar and the audience avatar based on the song video data, and generates a virtual karaoke room video including the audience avatar.
The server of the communication karaoke system according to claim 2 or 3. The video data receiving means further receives voice data of the photographer collected simultaneously with the shooting of the song video together with the song video data from the portable information terminal possessed by the photographer,
The generating means further mixes the photographer's voice data received by the video data receiving means in synchronization with the singing voice data and the karaoke song data received by the voice data receiving means, Generate karaoke room video,
The server of the communication karaoke system according to claim 4. Comprising a publishing means for publishing the virtual karaoke room video so that it can be viewed;
The server of the communication karaoke system of any one of Claims 2-5 characterized by the above-mentioned. A communication karaoke system comprising a server and a portable information terminal capable of data communication with the server,
The server
First storage means for storing karaoke song data;
Distribution which reads the karaoke music data memorize | stored in the said 1st memory | storage means, and carries out streaming delivery only to either one of the portable information terminal which the singer possesses, or the portable information terminal which the photographer who photographs the said singer possesses Means,
Voice data receiving means for receiving singing voice data from the portable information terminal possessed by the singer after the end of the streaming distribution;
Video data receiving means for receiving singing video data from the portable information terminal possessed by the photographer after the end of the streaming distribution,
The portable information terminal
A transfer means for transferring the karaoke song data from one portable information terminal receiving the streaming distribution of the karaoke song data to the other portable information terminal;
A communication karaoke system characterized by this.

Images