JP5275184B2 - Karaoke system, central apparatus and content data generation method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a Karaoke system which increases amusement by creating and disclosing a medley relay content by combining a plurality of video and audio data on singing of one or the other person. <P>SOLUTION: The Karaoke system includes a central device for transmitting a Karaoke music data, and a Karaoke terminal device for receiving and reproducing the Karaoke music data. The Karaoke terminal device transmits a video and audio data of singing performed during reproduction of the Karaoke music data, and an evaluation result for singing, to the central device. The central device receives the video and audio data, and the evaluation result, stores each data in association with each other to form a plurality of sets, and creates content data of a medley relay by using multiple sets of data selected by the user, from the data stored in a storage means. Motion picture data, audio data having the highest score, and music data are reproduced in parallel, and disclosed. <P>COPYRIGHT: (C)2011,JPO&amp;INPIT

Description

  The present invention relates to a karaoke system, a central device, and a content data generation method including a central device that transmits karaoke song data and a karaoke terminal device that receives and reproduces karaoke song data transmitted from the central device.

Video data and audio data obtained by imaging and collecting a user during karaoke singing can be uploaded from a karaoke terminal device installed in a karaoke store to a central device and can be viewed on a video browsing site as video content related to singing A karaoke system that has been made public has been put to practical use.
On the other hand, Patent Literature 1 and Patent Literature 2 disclose a karaoke device that stores a scoring result of a karaoke user's singing and creates a medley musical piece using music that each karaoke user is good at. .

JP 2007-298690 A JP 2000-148167 A

  However, the conventional video browsing site only publishes the user's singing appearance as video content so that it can be browsed, and higher entertainment is required.

The present invention has been made in view of such circumstances, and enables creation and viewing release of medley relay content combining a plurality of video data and audio data related to singing by itself or others, and improving entertainment. An object of the present invention is to provide a karaoke system, a central apparatus, and a content data generation method.
According to Patent Documents 1 and 2, a medley relay music can be created, but the amusement at a video browsing site cannot be improved.

The karaoke system according to the present invention is a karaoke system comprising a central device that transmits karaoke song data, and a karaoke terminal device that receives and reproduces karaoke song data transmitted from the central device. singing performed while playing a video input means for singing image is input made during playing a karaoke data, the karaoke data in the reproducing means by said reproducing means Voice input means for inputting the voice, evaluation means for evaluating the voice data related to the voice input by the voice input means at a plurality of different playback times, and input by the video input means and the voice input means. video data and audio data according to the video and audio have, as well as a transmitting means for transmitting the evaluation result by the evaluation unit to the central unit, said central instrumentation Means for receiving video data and audio data transmitted from the karaoke terminal device, and evaluation results obtained by evaluating the voice of the singing at a plurality of different playback points; and a plurality of received video data, audio data and evaluation results Storage means for storing the data in association with each other , and a plurality of sets of video data and audio data among the video data and audio data of the singing performed while reproducing one karaoke song data stored in the storage means And selecting means for selecting audio data; generating means for generating content data combining a plurality of sets of video data and audio data selected by the selecting means; and viewing the content data generated by the generating means and a public means to expose to the public means, image according to the image data of the singing performed while playing the one karaoke data Juxtaposed, in accordance with the evaluation results for each audio data of the singing performed while playing the karaoke data of the one lies so as to selectively reproduce one audio data according to the video data It is characterized by that.

  The karaoke system according to the present invention is characterized in that the publishing means selectively reproduces audio data having the highest evaluation result at each reproduction time point.

The central unit according to the present invention, in a central unit that transmits karaoke data, video data and audio data according to the singing transmitted from a terminal device, and the evaluation result of evaluating the voice data in a plurality of different playback point A means for receiving, a storage means for storing a plurality of sets of received video data, audio data, and evaluation results in association with each other , and while playing back one piece of karaoke song data stored in the storage means A selection unit that selects a plurality of sets of video data and audio data from the video data and audio data of the singing song , and content data that combines the plurality of sets of video data and audio data selected by the selection unit is generated. generation means for, and a public means viewable to publish content data generated by said generating means, said public means, the one karaoke de It juxtaposed the image according to the image data of the singing made during playing a motor, according to the evaluation results for each audio data of the singing performed while playing the karaoke data of the one One audio data related to each video data is selectively reproduced.

The content data generation method according to the present invention uses a central device that transmits karaoke song data, and a karaoke terminal device that receives and reproduces karaoke song data transmitted from the central device, to generate content data related to singing. In the content data generation method to be generated, the karaoke terminal device receives video and audio of singing performed while reproducing karaoke song data, and the audio data related to the input audio is different from a plurality of reproduction time points. The video data and audio data related to the input video and audio and the evaluation result are transmitted to the central device, and the central device transmits the video data and audio data and the evaluation result transmitted from the karaoke terminal device. It receives the received audio data, video data and the evaluation results a plurality of sets, and association with each other, one karaoke Of singing of the video data and audio data that occurred while playing over data, a plurality of sets of selected video data and audio data, content data that is a combination of video data and audio data of plural sets selected While the video data relating to each video data of the song performed while reproducing the one karaoke song data is juxtaposed and the generated karaoke song data is being reproduced. According to the evaluation result for each audio data of the singing performed, the audio data is disclosed so as to be viewable so that one audio data related to each video data is selectively reproduced.

In this invention, a karaoke terminal device transmits the video data and audio | voice data which concern on a song to a central apparatus.
The central device receives the video data and audio data transmitted from the karaoke terminal device, and the evaluation result transmitted from the karaoke terminal device or an external communication device, and the storage means stores the data in association with each other. Note that the evaluation of the singing may be performed by the user who has listened to the singing based on the subjectivity, or may be performed mechanically based on the voice data of the singing. The selection unit selects a plurality of sets of video data and audio data stored in the storage unit, and the generation unit generates content data that combines the selected sets of video data and audio data. Then, the content data is made available for viewing so that one piece of audio data is selectively reproduced according to the evaluation result for each piece of audio data.

  In the present invention, the karaoke terminal apparatus evaluates the audio data related to the input voice of the singing at a plurality of different playback time points, and evaluates the video data and audio data related to the singing and the evaluation result for the singing to the central device. Send to. The central device receives the video data and audio data transmitted from the karaoke terminal device, and the evaluation result, and the storage means stores the data in association with each other. The following processing is the same as described above. Therefore, the evaluation of the singing is automatically performed, and the content data is made available for viewing so that one piece of audio data is selectively reproduced according to the evaluation result.

  In the present invention, video data relating to two or more songs is juxtaposed, and one audio data selected according to the evaluation result is reproduced.

  In the present invention, since the audio data having the highest scoring result is selected and reproduced, the quality or entertainment of the content data is improved.

  In the present invention, it is possible to create and view a medley relay content combining a plurality of video data and audio data related to the singing of oneself or another person, and improve entertainment.

It is explanatory drawing which shows typically the example of 1 structure of the karaoke system which concerns on Embodiment 1 of this invention. It is a block diagram which shows one structural example of a central apparatus. It is explanatory drawing which shows notionally an example of the record layout of user information DB. It is explanatory drawing which shows notionally an example of the record layout of music DB. It is explanatory drawing which shows an example of the record layout of animation DB notionally. It is explanatory drawing which shows notionally an example of the record layout of hybrid moving image DB. It is a block diagram which shows the example of 1 structure of a karaoke terminal device. It is explanatory drawing which showed notionally the production | generation and publication method of a hybrid moving image content. It is a flowchart which shows the process sequence of the control part which concerns on the production | generation and registration of the moving image content which concerns on a song. It is a flowchart which shows the process sequence of the control part which concerns on the production | generation and registration of the moving image content which concerns on a song. It is a flowchart which shows the process sequence of the control part which concerns on the production | generation and release of hybrid moving image content. 10 is an explanatory diagram conceptually showing an example of a record layout of a user information DB in Embodiment 2. FIG. FIG. 10 is an explanatory diagram conceptually illustrating an example of a record layout of a moving image DB in the second embodiment. 12 is an explanatory diagram conceptually illustrating an example of a record layout of a hybrid video DB in Embodiment 2. FIG. FIG. 10 is an explanatory diagram conceptually showing a method for generating and publishing mixed moving image content in a second embodiment. It is the schematic diagram which showed an example of the user screen which displayed the moving image content as the thumbnail. It is the schematic diagram which showed an example of the user screen containing the setting screen for performing various settings which concern on moving image content. It is the schematic diagram which showed an example of the user screen which displayed as a thumbnail the moving image content by which public setting and karaoke delivery setting were carried out. It is the schematic diagram which showed an example of the exhibition screen where the moving image content was exhibited. It is the schematic diagram which showed an example of the user screen which displayed the thumbnail image of the moving image content registered by the hybrid moving image material. It is the schematic diagram which showed an example of the user screen which displayed the hybrid moving image content as the thumbnail. It is the schematic diagram which showed an example of the user screen containing the setting screen for performing the various setting which concerns on a hybrid moving image content. It is the schematic diagram which showed an example of the public screen which displayed as a thumbnail the hybrid moving image content set publicly. It is the schematic diagram which showed an example of the public screen on which the expansion reproduction | regeneration screen of the hybrid moving image content was displayed. It is the schematic diagram which showed an example of the public screen on which the expansion reproduction | regeneration screen of the hybrid moving image content was displayed. It is the schematic diagram which showed an example of the public screen on which the partial expansion reproduction screen was displayed. It is the schematic diagram which showed an example of the public screen on which the partial expansion reproduction screen was displayed. It is a flowchart which shows the process sequence of the control part which concerns on the public setting of a moving image content, and a karaoke delivery setting. It is a flowchart which shows the process sequence of the control part which concerns on disclosure of a moving image content. It is a flowchart which shows the process sequence of the control part which concerns on the production | generation and registration of hybrid moving image content. It is a flowchart which shows the process sequence of the control part which concerns on the production | generation and registration of a hybrid moving image content. It is a flowchart which shows the process sequence of the control part which concerns on the public setting of a hybrid moving image content. It is a flowchart which shows the process sequence of the control part which concerns on publicity of a hybrid moving image content.

Hereinafter, the present invention will be described in detail with reference to the drawings illustrating embodiments thereof.
The karaoke system according to the present invention provides new entertainment on a video browsing site using video content including video data and audio data obtained by capturing and collecting the appearance of a user singing at a karaoke store Specifically, it enables creation and viewing / release of medley relay content combining a plurality of video data and audio data related to the singing of the person or others, and improves entertainment.

Hereinafter, the present invention will be described in detail with reference to the drawings illustrating embodiments thereof.
(Embodiment 1)
FIG. 1 is an explanatory diagram schematically showing a configuration example of a karaoke system according to Embodiment 1 of the present invention. The karaoke system according to the first embodiment is installed in each of the central device 1 that distributes karaoke information and generates, manages, and publishes the hybrid video content according to the present invention, and each karaoke store 2. A karaoke terminal device 4 connected to the central device 1 via a communication network N1 and one or a plurality of user terminal devices 6 connected to the central device 1 via a communication network N2 are provided.

  The communication network N1 is a dedicated communication line for transmitting and receiving karaoke information and information related to the hybrid video content between the central device 1 and the karaoke terminal device 4, and is an optical fiber communication network, an ADSL line, a public telephone line. Etc., and a hard disk built-in type router 31 is connected. The type of communication line is not limited to this as long as information can be transmitted and received between the central apparatus 1 and the karaoke terminal apparatus 4, and may be a non-dedicated line.

  The router 31 is mainly installed in each karaoke store 2, and the hard disk built in the router 31 stores the karaoke information distributed from the central device 1 and the moving image data and sound that are the material of the mixed moving image content. Temporarily store data. The router 31 has a function of sharing or backing up karaoke information by relaying karaoke information, moving image data and audio data between each store and the central device 1.

  One karaoke terminal device 4 is normally installed in each private room 21 of the karaoke store 2, and is connected to a router 31 installed in the karaoke store 2 via a wired LAN (Local Area Network) cable 32. Each private room 21 is provided with one or a plurality of remote operation devices 5 for remotely operating the karaoke terminal device 4. The remote control device 5 is used for user login to the karaoke system, karaoke song selection, and operations related to hybrid video content.

The remote control device 5 is a board type provided with a liquid crystal panel having a pen touch sensor, and the karaoke store 2 in the same private room 21 is linked by infrared communication. Note that a dedicated application may be executed on the mobile phone to function as the remote operation device 5. In addition, the karaoke store 2 is provided with an access point 33 connected to the wired LAN cable 32, and the remote control device 5 provided in each private room 21 provides information on various operations via the access point 33. Are transmitted / received to / from the karaoke terminal device 4 in the same private room 21 that is associated.
In addition, although wired LAN cable 32 was illustrated as a communication line which connects between the karaoke terminal apparatuses 4, you may comprise so that it may mutually connect by wireless LAN, another wired or wireless communication. Moreover, although the example which installs the access point 33 was demonstrated, you may comprise so that an operation signal may be directly transmitted / received between the karaoke terminal device 4 and the remote control device 5. FIG.

  The user terminal device 6 is a communication device capable of transmitting / receiving information to / from the central device 1, for example, a personal computer 6a, a mobile phone terminal 6b, a PDA (Personal Digital Assistant) 6c, and the like, and uses the karaoke store 2. This is for the user to access the central apparatus 1 and use the video browsing site related to the hybrid video content.

  FIG. 2 is a block diagram illustrating a configuration example of the central apparatus 1. The central device 1 includes a control unit 10 that controls the operation of each component of the central device 1, for example, a CPU (Central Processing Unit). The control unit 10 includes a ROM (Read Only Memory) 11 via a bus. A RAM (Random Access Memory) 12, an internal storage device 13, an external storage device 14, a first communication unit 15, a second communication unit 16, and a clock unit 17 are connected.

  The control unit 10 reads out the computer program recorded in the RAM 12 and the internal storage device 13 to the RAM 12 and executes it, thereby executing the content data generation method according to the first embodiment, that is, generation of hybrid moving image content data described later. And a process related to disclosure. Moreover, the control part 10 performs the process which concerns on the web server which displays a moving image browsing site on a user terminal device.

  The ROM 11 is a non-volatile memory such as a mask ROM or EEPROM that stores a control program necessary for the operation of the computer.

  The RAM 12 is a volatile memory such as a DRAM or SRAM that temporarily stores various data generated when the arithmetic processing of the control unit 10 is executed.

  The external storage device 14 is a DVD (Digital Versatile Disc) -ROM, BD (Blu-ray Disc), or CD (Compact Disc) in which a computer program for causing a computer to function as a central device according to the present invention is recorded in a computer-readable manner. An optical disk drive that reads a computer program and other various data from a recording medium such as a ROM. The optical disk drive is an example of the external storage device 14, and the external storage device 14 may be configured by a device that reads and writes data from a magnetic disk such as a hard disk and a flexible disk, a magneto-optical disk, and a semiconductor memory.

The internal storage device 13 is a hard disk, a nonvolatile semiconductor memory, or the like that stores a computer program for causing a computer to function as the karaoke device 1.
The internal storage device 13 includes a user information DB (Data Base) 13a for storing information related to the user of the karaoke system, a music DB 13b for storing music data (karaoke music data), etc., and moving image content data of a song created by the user. For example, a moving image DB 13c for storing singing mixed moving image content data created by the user, and the like. Details of the various databases will be described later.
Furthermore, the internal storage device 13 stores image components for creating a user screen, a public screen, and the like of the moving image posting site.

  The first communication unit 15 is an interface for transmitting and receiving karaoke information, singing video data and audio data, and other data relating to various services to and from the karaoke terminal device 4 connected via the communication network N1. The transmission / reception of various information by the first communication unit 15 is controlled by the control unit 10.

  The second communication unit 16 is an interface that transmits and receives moving image data and audio data related to the mixed moving image content and other data related to various services to and from the user terminal device 6 connected via the communication network N2. Transmission and reception of various information by the second communication unit 16 is controlled by the control unit 10. When the physical layers of the communication networks N1 and N2 are common, the first and second communication units 15 and 16 may be configured with the same hardware.

  The clock unit 17 is a circuit that counts the date and time and gives a timing result to the control unit 10 in response to a request from the control unit 10. The clock unit 17 manages the date and time when the user singing at the karaoke store 2 is imaged, the storage period of moving image data obtained by imaging, and the like.

  In addition, although the example which implement | achieves the central apparatus 1 using one computer was demonstrated, it cannot be overemphasized that you may carry out a distributed process using a some computer.

FIG. 3 is an explanatory diagram conceptually showing an example of the record layout of the user information DB 13a. The table (file) of the user information DB 13a includes a plurality of columns (fields), for example, a “user ID” column, a “password” column, a “nickname” column, a “user information” column, a “my video” column, and the like. Each row (record) stores information corresponding to each column.
The “user ID” column stores a user ID that identifies each of a plurality of users who use the karaoke system according to the first embodiment, for example, an e-mail address of the user. The user ID is information required when the user logs in to the karaoke system and uses various services related to the hybrid video.
The “password” column stores password information necessary for logging in to the karaoke system. The password information consists of a plurality of alphanumeric character strings.
The “nickname” column stores the nickname of the user. The nickname is a temporary name of the user on the video posting site, and is used by the users of the karaoke system to identify each other. Further, it is used to display the creator of the hybrid video content that is open to each user.
“User information” stores information such as a user's gender and birthplace.
The “My Movie” column stores a movie ID for specifying the movie content created by the user, as will be described later.
The “my hybrid video” column stores a hybrid video ID for specifying the hybrid video created by the user.

FIG. 4 is an explanatory diagram conceptually showing an example of the record layout of the music DB 13b. The table of the song DB 13b includes a plurality of columns, for example, a “song ID” column that stores a song ID for identifying a song, a “song name” column that stores a song name of the song, and a “singer” that stores a singer name of the song. The “name” column, a “music data” column for storing music data including accompaniment data of the music, and the like. The song data stores, for example, accompaniment data (karaoke song data), lyrics data, and main melody data. In the music data, karaoke music is divided into specific playback sections, for example, “Prelude”, “Melody Part A”, “Melody Part B”, “Chibi”, “Interlude”, etc. Is included. The structure of the music data is not particularly limited, and the accompaniment data, the lyrics data, and the main melody data may be configured as a single music file, or may be configured as separate files and managed in association with each file. May be.
The accompaniment data is, for example, MIDI (Musical Instrument Digital Interface) data, which specifies data on / off of sound, pitch data specifying pitch, volume data (velocity) specifying volume, instrument control data, sound Is composed of timbre data for designating a musical instrument that outputs a sound, time data (delta time) for designating a time interval of each sound reproduced in time series.
The main melody data indicates the melody of the sound to be sung in accordance with the accompaniment, that is, the data indicating the on / off of the sound that constitutes the main melody, the pitch data, the volume data, and the time interval of each sound of the main melody that is played back in time series Time data.

FIG. 5 is an explanatory diagram conceptually illustrating an example of a record layout of the moving image DB 13c. The table of the moving image DB 13c includes a plurality of columns, for example, a “moving image ID” column, a “singer” column, a “song ID” column, a “moving image content” column, a “shooting date / time” column, a “play count” column, ”Column,“ storage period ”column,“ comment ”column,“ evaluation ”column, and the like, and information corresponding to each column is stored.
The “moving image ID” column stores a moving image ID for identifying moving image content.
The “singer” column stores the user ID of the singer who created the video content associated with the video ID.
The “music ID” column stores a music ID for identifying the music used when creating the moving image content associated with the moving image ID.
The “video content” column includes video data obtained by capturing the image of a user singing at the karaoke store 2, voice data obtained by collecting singing voice, and evaluation of the singing related to the voice data. The moving image content data independently including the indicated scoring result and subjective evaluation (hereinafter referred to as user evaluation result) by the user who listened to the song is stored. The moving image content data is, for example, a flv format file. The user evaluation result is obtained by, for example, quantifying evaluations for singing such as “delicious”, “astringent”, “cool”, “similar”. Needless to say, it is sufficient that at least a pair of video data, audio data, and scoring results are linked, and video content data is scored with a plurality of files, for example, video data files, audio data files, and so on. You may comprise with a result file. Also, the file format of the moving image content data is not particularly limited.
The “shooting date and time” column stores the date and time when the moving image related to the moving image content was captured.
The “number of playback times” column stores the number of playback times of the video content on the video posting site.
The “open / disallowed” column stores data indicating whether or not to release the moving image content on the moving image posting site.
The “storage period” column stores data indicating a period during which the central apparatus 1 stores and discloses moving image content.
The “comment” column stores comments on the moving image content.
The “evaluation” column stores the evaluation for the moving image content. The evaluation is, for example, a third party comment on the moving image content. In addition, the evaluation includes voting results for items such as “cute”, “cool”, “similar”, “delicious”, “astringent”, “interesting”, and “can cry”.

FIG. 6 is an explanatory diagram conceptually showing an example of a record layout of the hybrid video DB 13d. The table of the hybrid video DB 13d includes a plurality of columns, for example, a “mixed video ID” column, a “music ID” column, a “first singer” column, a “second singer” column,..., A “video content 1” column, “Video content 2” column,..., “Shooting date / time” column, “Number of playbacks” column, “Public availability” column, “Retention period” column, “Comment” column, “Evaluation” column, etc. Stores information corresponding to columns.
The “mixed moving image ID” column stores a mixed moving image ID for identifying mixed moving image content formed by combining moving image contents related to a plurality of songs.
The “music ID” column stores a music ID for identifying a music used for creating the hybrid video content.
The “first singer” column stores data for identifying the user who created the first moving image content constituting the mixed moving image content.
The “second singer” column stores data for identifying a user who has played the second moving image content constituting the mixed moving image content. Here, hybrid video content composed mainly of two video content will be mainly described. However, three or more “singer” columns are provided so that hybrid video content including three or more video content can be created. You can also.
The “moving image content 1” column stores one moving image content data for composing the mixed moving image content.
The “moving image content 2” column stores other moving image content data for composing the mixed moving image content.
The “shooting date and time” column stores data indicating the date and time when the moving image content that is the main of the mixed moving image content, for example, the moving image related to the most recently created moving image content is captured.
The “number of times of reproduction” column stores the number of times of reproduction of the mixed moving image content data obtained by combining the one moving image content and other content on the moving image posting site.
The “open / disallowed” column stores data indicating whether or not to release the mixed moving image content on the moving image posting site.
The “storage period” column stores data indicating a period during which the central apparatus 1 stores and discloses the hybrid video content.
The “comment” column stores comments on the hybrid video content.
The “evaluation” column stores an evaluation for the hybrid moving image content.

  FIG. 7 is a block diagram illustrating a configuration example of the karaoke terminal device 4. The karaoke terminal device 4 includes a control unit 40 that controls the operation of each component of the karaoke terminal device 4. The control unit 40 includes a ROM 41, a RAM 42, a storage device 43, and a communication unit 44 via a bus. The receiver 45, the video encoder 46a, the video control circuit 46b, the video decoder 46c, the scoring unit 47, the audio encoder 48a, the sound source 48b, and the audio decoder 48c are connected. A video mixer 46d is connected to the video control circuit 46b and the video decoder 46c, and an audio mixer 48d is connected to the sound source 48b and the audio decoder 48c. Furthermore, the karaoke terminal device 4 includes a video input terminal, a video output terminal, an audio input terminal, and an audio output terminal (not shown), and each terminal includes an imaging device 46e, a video display device 46f, a microphone 48e, and a speaker. 48f is connected. The video display device 46f is, for example, a CRT (Cathode-ray Tube), a liquid crystal display, a plasma display, an organic EL display, or the like.

  The control unit 40 reads out the computer program recorded in the ROM 41 and the storage device 43 to the RAM 42 and executes the computer program, thereby executing processing related to generation, management, and release of mixed moving image content data, which will be described later, according to the present invention. To function as the karaoke terminal device 4.

  The ROM 41 is a non-volatile memory such as a mask ROM or EEPROM that stores a control program necessary for the operation of the computer.

  The RAM 42 is a volatile memory such as a DRAM or SRAM that temporarily stores various data generated when the arithmetic processing of the control unit 40 is executed.

  The storage device 43 is a hard disk, a nonvolatile semiconductor memory, or the like that stores a computer program for causing a computer to function as the karaoke terminal device 4. The storage device 43 stores a karaoke terminal music DB 43a. The karaoke terminal music DB 43a stores music data and the like distributed from the central device 1, and the basic configuration is the same as that of the music DB 13b.

  The communication unit 44 is an interface that transmits and receives karaoke information, information related to the hybrid video content, and other data related to various services with the karaoke terminal device 4 connected via the communication network N1. Transmission / reception of various types of information is controlled by the control unit 40.

  The receiving unit 45 is a circuit that performs infrared communication with the remote operation device 5 so as to link the karaoke terminal device 4 and the remote operation device 5.

  The video encoder 46a is a circuit that compresses and encodes video data output from the imaging device 46e and input through the video input terminal. The compressed and encoded video data is temporarily written in the RAM 42 or the storage device 43. . The compression encoding format is, for example, MPEG (Moving Picture Experts Group), but is not particularly limited thereto.

  The video control circuit 46b reads various information related to the video from the RAM 42 or the storage device 43 in accordance with a control command from the control unit 40, generates video data representing various videos, and supplies the video data to the video mixer 46d. Execute the process. For example, when reproducing karaoke song data, the video control circuit 46b supplies video data representing the lyrics to the video mixer 46d based on the lyrics data included in the song data provided from the control unit 40. In addition, the video control circuit 46b gives video data representing a menu screen, scoring result display screen, etc. relating to karaoke entertainment to the video mixer 46d.

  The video decoder 46c reads and decodes video data, for example, predetermined background video data displayed at the time of karaoke singing, from the RAM 42 or the storage device 43 in accordance with the control command of the control unit 40, and the decoded video data 46d. To give. The background video data is, for example, MPEG format data.

  The video mixer 46d synthesizes all or any two of the videos given from the video control circuit 46b, the video decoder 46c, and the imaging device 46e, and outputs the video data obtained by the synthesis to the video display device 46f. To do. The video display device 46f displays a video related to the video data output from the video mixer 46d.

  The scoring unit 47 acquires audio data output from the microphone 48e and input through the audio input terminal, reads out music data from the storage device 43, and sings based on the audio data and main melody data included in the music data. By comparing the pitch, volume, rhythm, etc. in the voice and the main melody, the singing is scored sequentially in real time in real time, and the scored result is written in the RAM 42.

  The audio encoder 48a acquires audio data output from the microphone 48e and input through an audio input terminal, and compresses and encodes the acquired audio data. The compression encoding format is, for example, MP3, AAC (Advanced Audio Coding), AC3 (Audio Code number 3), etc., but is not particularly limited thereto.

  The sound source 48b is a MIDI sound source that creates sound data that reproduces musical instrument sounds based on accompaniment data included in music data read from the RAM 42 or the storage device 43, for example, MIDI data, in accordance with a control command of the control unit 40. The audio data thus obtained is given to the audio mixer 48d.

  The audio decoder 48c decodes the audio data read from the RAM 42 or the storage device 43 according to the control command of the control unit 40, and gives the audio data obtained by decoding to the audio mixer 48d.

  The audio mixer 48d synthesizes audio related to the audio data provided from the sound source 48b and the audio decoder 48c, and outputs the audio data obtained by the synthesis to the speaker 48f through the audio output terminal. The speaker 48f emits sound related to the sound data output from the sound mixer 48d.

  In addition, although the example which comprises the karaoke terminal device 4 as an exclusive machine was demonstrated, you may comprise the karaoke terminal device 4 using a general purpose computer and the computer of a home video game. In this case, a part of the hardware circuit constituting the karaoke terminal device 4, for example, the scoring unit 47 may be processed by software.

  Next, an overall flow of generation and release of the hybrid video content will be described, and then specific processing procedures of the control units 10 and 40 of the central device 1 and the karaoke terminal device 4 will be described.

  FIG. 8 is an explanatory diagram conceptually showing a method for generating and publishing mixed moving image content. Note that a user who uses a service related to a hybrid video needs to access the karaoke system in advance and perform user registration before using the service. When a user registration request is received from the user terminal device 6 or the remote operation device 5 via the communication network N2, the control unit 10 of the central device 1 causes the user terminal device 6 or the remote operation device 5 to display a user registration screen. The e-mail address, password, nickname, gender, birthplace, etc. corresponding to the login ID are accepted, and the received various information is registered in the user information DB 13a as described above. Hereinafter, it is assumed that the user logs in to the karaoke system and performs various operations unless otherwise specified.

<1. Singing / Imaging / Sound Collection / Scoring>
First, one user logs in to the karaoke system using the remote control device 5 of the karaoke terminal device 4 installed in the private room 21 of the karaoke store 2 and selects a desired karaoke song. The karaoke terminal device 4 reproduces the selected music data, collects the user's singing performed while the music data is being reproduced, with the microphone 48e, and captures the image of the user singing. Image is taken at 46e. Moreover, the karaoke terminal device 4 scores a user's song based on the audio | voice data obtained by collecting and the main melody data contained in music data.

<2. Post>
After singing, the user selects whether to upload, that is, upload, moving image data and audio data related to the singing obtained by imaging and sound collection to the central device 1. When the karaoke terminal device 4 is instructed to upload, the karaoke terminal device 4 transmits the moving image data, the sound data, and the scoring result relating to the song to the central device 1 together with the user ID and music ID of the logged-in user. The central device 1 receives the user ID, the song ID, the moving image data, the audio data, and the scoring result transmitted from the karaoke terminal device 4, and the moving image content data that independently includes the received moving image data, the audio data, and the scoring result. It is generated and registered in the moving image DB 13c in association with the user ID and the music ID. Note that generating moving image content data includes not only processing for creating new data including moving image data, audio data, and scoring results, but also processing for associating moving image data, audio data, and scoring results. Also, the moving image ID unique to the moving image content data, the received user ID, and the like are associated and registered in the user information DB 13a.

<3. Movie release / Karaoke distribution settings>
Next, the one user who has finished karaoke logs in to the video posting site of the karaoke system using the user terminal device 6 or the remote control device 5. Hereinafter, the login to the video posting site is described as being performed using the user terminal device 6. When the login is successful, a user screen dedicated to the user is displayed on the display unit of the user terminal device 6. In addition, when a transition is made to a public screen by performing a predetermined operation, a thumbnail list of moving image contents that are uploaded to the central device 1 and generated is displayed on the display unit of the user terminal device 6. In addition, although it is normal to log in to a movie posting site using the user terminal device 6 outside the karaoke store 2, the login may be performed using the remote control device 5 in the karaoke store 2.

  The user selects a plurality of moving image contents as materials of the mixed moving image content from the moving image contents configured on the public screen, and instructs the creation of the mixed moving image content. The central device 1 generates and publishes hybrid moving image content based on a plurality of moving image contents selected by the user.

When the central device 1 is instructed to reproduce the mixed video content, the central device 1 juxtaposes the videos related to the plurality of video content constituting the mixed video content and reproduces them in parallel. For example, as shown in FIG. 8, three moving images are reproduced in parallel in the horizontal direction.
In addition, the central device 2 compares the scoring results included in each moving image content for each specific scoring division, for example, for each break of “melody part A”, “melody part B”, and “rust”. Then, the audio data included in the moving image content having the highest scoring result and the music data are reproduced in parallel. That is, voice data having a high scoring result is sequentially reproduced for each karaoke song reproduction category. In the example shown in FIG. 8, the voice data of the first singer is played in “melody part A”, the voice data of the third singer is played in “melody part B”, and the second singer is played in “sabi”. The audio data is sequentially reproduced. Therefore, the mixed moving image content having a high degree of completeness is reproduced as if each singer was in charge of the part he was good at and sang a medley relay.

  Next, a processing procedure of the control units 10 and 40 related to generation and release of the hybrid video content will be described. In the following processing, the main part of the processing content will be described on the assumption that the user is logged in to the karaoke system.

9 and 10 are flowcharts showing the processing procedure of the control units 10 and 40 related to the generation and registration of moving image content related to singing. The user can select a song that he / she wants to sing at the karaoke store 2 by using the remote operation device 5, and the control unit 40 of the karaoke terminal device 4 accepts the song by the remote operation device 5 (step S <b> 11). The music request data including the music ID related to the music is transmitted to the central apparatus 1 via the communication unit 44 (step S12). The control unit 40 temporarily stores the music ID accepted in step S11.
Note that if the music data is previously transmitted from the central device 1 to the karaoke terminal device 4 and already registered in the karaoke terminal music DB 43a, the music request data is not transmitted to the central device, and the music data is read from the storage device 43. good. The same applies to the case where the hard disk of the router 31 is stored.

  The control unit 10 of the central device 1 receives the music request data transmitted from the karaoke terminal device 4 via the first communication unit 15 (step S13), and the music corresponding to the music ID included in the received music data. Data is read from the music DB 13b of the internal storage device 13, and the read music data is transmitted to the karaoke terminal device 4 that is the music request source via the second communication unit 16 (step S14).

The control unit 40 of the karaoke terminal device 4 receives the music data transmitted from the central device 1 via the communication unit 44 (step S15), and reproduces the received music data (step S16). Next, the control unit 40 inputs a moving image of the song performed while reproducing the music data (step S17), and inputs the voice data of the song (step S18).
Note that the moving image data output from the imaging device 46e and input to the karaoke terminal device 4 is compression encoded by the video encoder 46a and written to the RAM 42. The moving image data is input to the video mixer 46d. The video mixer 46d superimposes the text image of the lyrics given from the video control circuit 46b on the video related to the video data, and outputs the video data related to the video on which the lyrics are superimposed to the video display device 46f. Is done. Therefore, the user's own singing appearance is displayed on the video display device 46f. For this reason, the user can sing while confirming the content of the captured moving image.
The voice data output from the microphone 48e and input to the karaoke terminal device 4 is compression-coded by the voice encoder 48a and written in the RAM 42. The audio data is input to the audio mixer 48d. The sound mixer 48d synthesizes the sound related to the sound data input from the microphone 48e and the sound of the music given from the sound source, and outputs the sound data obtained by the synthesis to the microphone 48e.

  Next, the control unit 40 gives the input voice data to the scoring unit 47, and performs scoring on the singing related to the voice data (step S19).

  Next, the control unit 40 receives the user evaluation result at the remote operation device 5 (step S20). That is, the user can listen to another user's song at the karaoke store 2, subjectively evaluate the song, and input the user evaluation result with the remote operation device 5. The control unit 40 can In 5, the input user evaluation result is acquired. The user evaluation result input method may be configured such that, for example, items such as “delicious”, “astringent”, and “cool” are displayed on the display of the remote control device 5 and points can be input for each item. . Also, instead of the score, characters that express the evaluation corresponding to the score, such as “very good”, “somewhat good”, “normal”, “slightly bad”, “poor”, etc. are displayed, and one of the evaluations is selected You may comprise so that it can. Needless to say, the user evaluation result may be an evaluation by one user or an evaluation by a plurality of users.

  Next, the control unit 40 determines whether or not to end the reproduction of the music data (step S21). Note that the reproduction is ended when the music data has been reproduced to the end, when the remote operation device 5 is instructed to stop the reproduction, or the like. When it determines with not complete | finishing reproduction | regeneration (step S21: NO), the control part 40 returns a process to step S16. When it determines with complete | finishing reproduction | regeneration (step S21: YES), the control part 40 monitors the operation state of the remote control apparatus 5, and determines whether the input moving image and the moving image data and audio | voice data of a video are uploaded. (Step S22). When it determines with not uploading (step S22: NO), the control part 40 finishes a process.

  When it determines with uploading (step S22: YES), the control part 40 makes the user ID, music ID, moving image data, audio | voice data, scoring result, and user evaluation result which RAM42 memorize | stores temporarily via the communication part 44. It transmits to the central apparatus 1 (step S23). The user ID temporarily stored in the RAM 42 is input when the user logs in to the karaoke system.

  The control unit 10 of the central device 1 receives the user ID, music ID, moving image data, audio data, scoring result, and user evaluation result transmitted from the karaoke terminal device 4 via the first communication unit 15 (step S24). ). And the control part 10 produces | generates the moving image content data which independently contains the received moving image data, audio | voice data, a scoring result, and a user evaluation result, and produced | generated moving image content data with received user ID, music ID, etc. The video ID registered in the video DB 13c and the video ID issued at the time of registration in the video DB 13c is registered in the user information DB 13a (step S25), and the process is terminated.

  FIG. 11 is a flowchart illustrating a processing procedure of the control unit 10 related to generation and release of the hybrid video content. The control unit 10 of the central device 1 displays a public screen on the display unit of the user terminal device 6 (step S31). Then, the control unit 10 makes a list of thumbnail images of moving images uploaded from the user and information on the moving images.

  Next, the control unit 10 selects a plurality of moving image contents for configuring the mixed moving image content from the released moving image contents (step S33). The user can use the user terminal device 6 to specify a plurality of desired moving image contents as materials of the mixed moving image content, and the user terminal device 6 transmits the mixed material specifying information indicating the specified moving image content to the central device. Send to 1. The control unit 10 receives the hybrid material designation information via the second communication unit 16, and selects moving image content based on the hybrid material designation information.

  And the control part 10 produces | generates the composite video content data which contains the video data and audio | voice data which comprise each of the selected some video content, and a scoring result and a user evaluation result, The hybrid video ID registered in the hybrid video DB 13d and issued at the time of registration in the hybrid video DB 13d is registered in the user information DB 13a (step S34).

  Next, the control unit 10 displays a list of thumbnail images of the mixed moving image content and information on the moving images (step S35).

  Then, the control unit 10 determines whether or not there is an instruction to reproduce the hybrid video content (step S36). The user can use the user terminal device 6 to instruct the reproduction of the hybrid video content. For example, playback is instructed by clicking a thumbnail image. The user terminal device 6 transmits to the central device 1 information instructing the reproduction of the specific hybrid video content. The control unit 10 receives the information via the second communication unit 16 and determines whether or not there is a reproduction instruction. Note that the video content that can be selected here is only the video content associated with the same music data.

  When it determines with there being no reproduction | regeneration operation (step S36: NO), the control part 10 finishes a process. When it is determined that there is a reproduction operation (step S36: YES), the control unit 10 transmits a plurality of pieces of moving image data constituting the mixed moving image content instructed to be reproduced to the user terminal device 6, and Two sets of moving image data included in the mixed moving image content are reproduced in parallel (step S37).

  In addition, the control unit 10 compares the scoring results and the user evaluation results for a plurality of video content constituting the composite video content for each specific reproduction category, and the video content having the highest overall evaluation result of the scoring results and the user evaluation results The audio data and the music data constituting the data are transmitted to the user terminal device 6, and the audio data and the music data are reproduced in parallel with the moving image data (step S38). The comprehensive evaluation is, for example, the sum, maximum value, average value, etc. of the scoring results and user evaluation results, but the calculation method is not particularly limited. Moreover, although the example which determines a reproduction | regeneration division | segmentation based on the data which music data memorize | stores was demonstrated, you may determine a reproduction | regeneration division | segmentation by other methods, such as a silent period of a song. Furthermore, all audio data and music data constituting the moving image content data are transmitted to the user terminal device 6, and then information specifying the audio data having the highest comprehensive evaluation result is transmitted to the user terminal device 6, and the user terminal device On the 6th side, audio data to be output as audio may be selected.

  In the karaoke system, the central device 1 and the content generation method according to the first embodiment configured as described above, the medley relay mixed moving image content combining a plurality of video data and audio data related to the singing of the user or another person Can be created and viewed and the entertainment can be improved.

  In addition, the singing is evaluated by combining the scoring results obtained by comparing the audio data and the main melody data with the subjective scoring results by the user who listened to the singing. Since it is the structure to reproduce | regenerate, the hybrid moving image content with a higher completeness can be produced | generated.

In the flowchart shown in FIG. 10, an example in which the user evaluation result is received by the karaoke terminal device 4 by the remote operation device 5 has been described, but the user evaluation result is configured to be received through a mobile phone owned by the user or other communication device. You may do it.
Moreover, although the example which the karaoke terminal device 4 receives a user evaluation result and transmits to the central apparatus 1 was demonstrated, it comprised so that a user evaluation result may be directly transmitted to the central apparatus 1 through the communication apparatus etc. which a user owns. Also good.
Furthermore, in Embodiment 1, the audio data is selectively reproduced using the scoring result by the karaoke terminal device 4 and the user evaluation result. However, based on the scoring result alone or only the user evaluation result, The audio data may be selectively played back. In the embodiment described later, an example using only the scoring results will be described.

(Embodiment 2)
The karaoke system according to Embodiment 2 mainly differs in the details of the selection method of the moving image content constituting the mixed moving image content, the configuration of the user information DB 13a, the moving image DB 13c, the mixed moving image DB 13d, and the processing procedures of the control units 10 and 40. Hereinafter, the above differences will be mainly described.

  FIG. 12 is an explanatory diagram conceptually showing an example of a record layout of the user information DB 13a in the second embodiment. The table of the user information DB 13a includes a plurality of columns, for example, a “user ID” column, a “password” column, a “nickname” column, a “user information” column, a “my video” column, a “my hybrid video material” column, and the like. The information corresponding to each column is stored in each row (record). In particular, the “my mixed moving image material” column stores a moving image ID for identifying the moving image content registered as a material for the user to create the mixed moving image content.

  FIG. 13 is an explanatory diagram conceptually illustrating an example of a record layout of the moving image DB 13c in the second embodiment. The table of the moving image DB 13c includes a plurality of columns, for example, a “moving image ID” column, a “singer” column, a “song ID” column, a “moving image content” column, a “shooting date / time” column, a “play count” column, ”Column,“ storage period ”column,“ karaoke distribution ”column,“ comment ”column,“ partner recruitment ”column,“ evaluation ”column, and the like, and information corresponding to each column is stored. In particular, the “karaoke distribution” column stores data indicating whether or not the moving image content can be distributed to the karaoke terminal device 4 as a mixed moving image content material. The “Partner Recruitment” column stores data indicating whether or not to publish video content in the “Partner Recruitment” column described later on the video posting site in order to encourage generation of hybrid video content using video content. .

  FIG. 14 is an explanatory diagram conceptually showing an example of the record layout of the hybrid video DB 13d in the second embodiment. The table of the hybrid video DB 13d includes a plurality of columns, for example, a “mixed video ID” column, a “singer” column, a “partner” column, a “music ID” column, a “mixed video content” column, a “shooting date / time” column, “ The column is composed of a “number of times of reproduction” column, a “disclosure availability” column, a “storage period” column, a “comment” column, an “evaluation” column, and the like, and stores information corresponding to each column. In particular, the “mixed moving image content” column stores two sets of moving image content data. That is, the moving image data, audio data, and scoring results constituting each of the two sets of moving image content data are stored.

  First, the overall flow of generating and publishing mixed moving image content will be described, and then specific processing procedures of the control units 10 and 40 of the central device 1 and the karaoke terminal device 4 will be described.

  FIG. 15 is an explanatory diagram conceptually showing a method for generating and publishing mixed moving image content in the second embodiment. Hereinafter, it is assumed that the user logs in to the karaoke system and performs various operations unless otherwise specified. Since the singing / imaging / sound collection and posting processes are the same as those in the first embodiment, detailed description thereof is omitted, and description after the release of the moving image is performed.

<3. Movie release / Karaoke distribution settings>
Next, the one user who has finished karaoke logs in to the video posting site of the karaoke system using the user terminal device 6 or the remote control device 5. Hereinafter, the login to the video posting site is described as being performed using the user terminal device 6. When the login is successful, a user screen dedicated to the user is displayed on the display unit of the user terminal device 6. In addition, although it is normal to log in to a movie posting site using the user terminal device 6 outside the karaoke store 2, the login may be performed using the remote control device 5 in the karaoke store 2.

  FIG. 16 is a schematic diagram illustrating an example of a user screen 7 on which moving image content is displayed as a thumbnail. The user screen 7 includes a user information display unit 71 displayed on the left side of the screen. The user information display unit 71 displays information such as the nickname of the logged-in user, for example, “Mr. XXX”, the profile image of the user, the national ranking of the karaoke scoring results, and the highest scoring score. Further, on the upper side of the user information display unit 71, a “To Public Screen” button 70 is arranged. The “To Public Screen” button 70 is a button for transitioning to the public screen 8 on which the moving image content or the mixed moving image content uploaded or generated by each user is disclosed. Further, a hyperlink character string 72 for switching display contents on the user screen 7 is arranged on the upper right side of the user information display unit 71. Specifically, the hyperlink character string “my video” that transitions the video content uploaded by the logged-in user to the screen displaying thumbnails, and the screen that displays the mixed video content created by the logged-in user themselves as thumbnails The hyperlink character string “My hybrid video material”, the hyperlink character string “My hybrid video material” that transitions to the thumbnail display screen of the video content registered as material for creating the hybrid video content is displayed in a horizontal row Has been. In FIG. 16, the hyperlink character string “my video” is selected. Further, a moving image content display unit 73 is arranged below the hyperlink character string. The video content display unit 73 displays a thumbnail image 73a showing the content of the video content uploaded by the user, the song title and singer name 73b of the song used when singing the video content, and various settings relating to the video content. A “setting” button 73c for performing, a “deleting” button 73d for deleting moving image content, a setting content display portion 73e, and the like are included. The setting content display unit 73e displays whether or not the moving image content can be disclosed, whether or not the moving image content can be distributed to the karaoke terminal device 4 as a mixed moving image material, and the like.

  Here, when the user operates the “setting” button 73 c, a setting screen for performing various settings related to the moving image content is displayed on the display unit of the user terminal device 6.

  FIG. 17 is a schematic diagram illustrating an example of a user screen 7 including a setting screen for performing various settings related to moving image content. The setting screen includes a user information display unit 71 and a moving image content information display unit 74 that shows details of moving image content to be subjected to various settings, and a setting content input unit 75. The setting content input unit 75 includes a radio button 75a for selecting whether or not to release moving image content, a radio button 75b for setting a storage period for moving image content, a radio button 75c for selecting whether or not karaoke distribution is possible, and a moving image. A text input field 75d for inputting the user's own comment on the content, and selection of whether or not to publish the video content in the “partner recruitment” column to be described later in order to encourage the video content to be used as the material of the hybrid video content Including a radio button 75e.

  The user inputs various setting contents to the setting contents input unit 75. The user terminal device 6 transmits data indicating various settings for the moving image content to the central device 1. The central device 1 receives the data transmitted from the user terminal device 6, and registers various setting contents in the “open / disallowed” column, the “karaoke distribution” column, and the like of the moving image DB 13c.

  FIG. 18 is a schematic diagram showing an example of a user screen 7 that displays thumbnails of moving image content that has been set to be public and karaoke distributed. In particular, when karaoke distribution is enabled, a mixed material registration icon 73 f is displayed on the thumbnail image 73 a displayed on the moving image content display unit 73. The mixed material registration icon 73f is a button for selecting the moving image content as a mixed moving image content material.

<4. Mixed material registration>
Here, as shown in FIG. 15, another user whose nickname is “Mr. XXX” logs in to the video posting site of the karaoke system using the user terminal device 6 and operates the “go to public screen” button 70. In this case, the central device 1 displays the public screen 8 on the display unit of the karaoke terminal device 4.

FIG. 19 is a schematic diagram illustrating an example of the disclosure screen 8 on which the moving image content is disclosed. The public screen 8 includes a “go to user screen” button 80, a “partner recruitment” field 81, a “recommended video” field 82, a “recommended mixed video” field 83, and a video playback screen 84. The “to user screen” button 80 is a button for transitioning to the user screen 7.
In the “partner recruitment” column 81, a thumbnail image 81a of a video content that is set to be posted in the “partner recruitment” column is displayed. A composite material registration icon 81b is displayed on the thumbnail image 81a, for example, in the lower left corner of the thumbnail image 81a. On the right side of the thumbnail image 81a, a song name and a singer name 81c relating to the moving image content are displayed. In the “partner recruitment” column 81, the thumbnail image 81a of the moving image content is displayed regardless of the evaluation level, so that the possibility of being used for the mixed moving image content is increased. Therefore, a user who desires to use the content as a hybrid video content should set the video content of his / her own so that it is posted in the “partner recruitment” column.
In the “recommended moving image” column 82, thumbnail images 82a of moving image content highly evaluated by each user are displayed. In addition, a mixed material registration icon 82b is displayed on the thumbnail image 82a of the moving image content permitted to be delivered by karaoke. Further, below the thumbnail image 82a, the title of the song, the name of the singer, and the nickname of the user who created the moving image content are displayed.
The “recommended hybrid video” column 83 will be described later.
The video playback screen 84 has a screen 84a that displays a video obtained by playing back video data included in the video content, and a song title display that displays a song name and a singer name related to the video content under the screen 84a. An operation unit 84c including an operation button for instructing reproduction, pause, cueing, a time code slider, a volume adjustment slider, and the like is provided.

The user can instruct playback of the moving image content by operating the thumbnail images 81a and 82a of the moving image content desired to be viewed, for example, by clicking with a mouse. The music data associated with the content is transmitted to the user terminal device 6, and the user terminal device 6 reproduces the video data and audio data included in the video content transmitted from the central device 1 and the music data in parallel. And displayed on the moving image playback screen 84. By reproducing the moving image content and the music data in parallel, it is possible to reproduce the state of the user singing at the karaoke store.
Further, the user selects the mixed material registration icons 81b and 82b displayed on the thumbnail images 81a and 82a of the desired moving image content as the mixed moving image content material, thereby registering the moving image content as the mixed moving image content material. be able to. When the hybrid material registration icon is selected, the central device 1 associates the video ID of the video content selected as the composite material with “My hybrid video material” in the user information DB 13a in association with the user ID of the logged-in user. In the "" column.

  FIG. 20 is a schematic diagram showing an example of the user screen 7 displaying the thumbnail image 76 of the moving image content registered with the mixed moving image material. When the hyperlink character string “my hybrid material video” is operated on the user screen 7, a thumbnail image 76 of the video content registered as the hybrid material is displayed as shown in FIG.

<5. Karaoke distribution>
Then, the other user who has completed the mixed material registration, that is, the user whose nickname is “Mr. XX”, uses the remote control device 5 of the karaoke terminal device 4 installed in the private room 21 of the karaoke store 2 to use the karaoke system. Log in to select music related to the video content of the mixed material. The central device 1 transmits music data and audio data included in the moving image content of the hybrid material to the karaoke terminal device 4.

<6. Singing / Imaging / Sound Collection / Scoring>
The karaoke terminal device 4 receives the music data and audio data transmitted from the central device 1 and reproduces the received music data and audio data in parallel. Since the audio data is audio related to the mixed video material, when the user sings here, the song is in a state of being double-sung, and the user enjoys double-singing with the user who is not on the spot. And enjoy the creation of mixed video content. Then, the karaoke terminal device 4 collects the user's singing performed while the music data and audio data are being reproduced with the microphone 48e, and images the appearance of the user singing with the imaging device 46e. To do.

<7. Post>
After singing, the singer selects whether to upload, that is, upload, video data and audio data related to the singing obtained by imaging and collecting sound to the central device 1. When the karaoke terminal device 4 is instructed to upload, the karaoke terminal device 4 transmits the moving image data and audio data related to the song to the central device 1 together with the user ID and the music ID. The central device 1 generates mixed moving image content data including moving image data and audio data, and moving image data and audio data selected as a mixed material, and receives the generated mixed moving image content data as a user ID and a song ID. And the like are registered in the hybrid video DB 13d. Also, the hybrid video ID unique to the hybrid video content data, the received user ID, and the like are associated and registered in the user information DB 13a.

<8. Mixed video release>
Next, the other user who has finished karaoke accesses the central device 1 using the user terminal device 6 and logs in to the video posting site. When the login is successful, a user screen 7 dedicated to one user is displayed on the display unit of the user terminal device 6. When the hyperlink character string “my hybrid video” is operated on the user screen 7, a thumbnail image of the hybrid video content created by the other user is displayed.

  FIG. 21 is a schematic diagram showing an example of a user screen 7 on which the hybrid video content is displayed as a thumbnail. As shown in FIG. 21, a hybrid video content display unit 77 is displayed on the user screen 7. The hybrid video content display unit 77 includes a thumbnail image 77a showing the content of the hybrid video content uploaded and created by the user, and the song title and singer name 77b used when singing the hybrid video content. A “setting” button 77c for performing various settings relating to the hybrid video content, a “delete” button 77d for deleting the hybrid video content, a setting content display unit 77e, and a user who created the video content that is the material of the hybrid video content The profile image 77f is included. The setting content display section 77e displays whether or not the hybrid video content can be released.

  Here, when the user operates the “setting” button 77 c, a setting screen for performing various settings related to the hybrid video content is displayed on the display unit of the user terminal device 6.

  FIG. 22 is a schematic diagram illustrating an example of a user screen 7 including a setting screen for performing various settings related to the hybrid video content. The setting screen includes, together with the user information display unit 71, a mixed moving image content information display unit 78 indicating the content of the mixed moving image content to be subjected to various settings, and a setting content input unit 79. The setting content input unit 79 is a radio button 79a for selecting whether or not to release the hybrid video content, a radio button 79b for setting the storage period of the hybrid video content, and a text input for inputting its own comment on the hybrid video content. The field 79c and the like are included.

  FIG. 23 is a schematic diagram illustrating an example of a public screen 8 that displays thumbnails of the hybrid video content that has been set to public. In the “recommended mixed movie” column 83, a thumbnail image 83a of the mixed movie content that has been configured is displayed. Here, the user can instruct playback of the mixed video content by operating the thumbnail image 83a of the mixed video content desired to be viewed, for example, by clicking with a mouse. The music data and the scoring result are transmitted to the user terminal device 6, and the user terminal device 6 includes the two sets of moving image data included in the hybrid moving image content transmitted from the central device 1, the audio data having the highest scoring result, and the music Play data in parallel. That is, the moving image and the sound are reproduced as if the two users are singing the parts that they are good at. In addition, the user can enlarge and play the hybrid video content by displaying an enlargement playback button (not shown), and display more detailed information.

  24 and 25 are schematic diagrams illustrating an example of a public screen 8 on which an enlarged playback screen of hybrid video content is displayed. The enlarged playback screen 86 is composed of a screen portion 86a on which one video content constituting the mixed video content is played back and a screen portion 86a on which the other video content is played side by side. A search condition input unit 85 for searching for video content or mixed video content is arranged above, and an operation for instructing a song title display unit, playback, pause, cueing, etc. below the screen portion 86a. An operation unit 86c including a button, a time code slider, a volume adjustment slider, and the like is disposed. In addition, a tone adjustment target selection unit 86d for selecting a volume adjustment target is provided in order to selectively adjust the volume of the two moving image contents. The public screen 8 also includes a music-related video content display unit 86e that displays video content created using the same music as the hybrid video content, an evaluation input unit 86f for evaluating the hybrid video content, and a hybrid video. A comment display section 86g for displaying a comment on the content, a text field 86h for inputting a comment, a user information display section 86i for displaying a user who generated each moving picture content constituting the mixed moving picture content, and a tag attached to the mixed moving picture content A tag display unit 86j for displaying information, a music information display unit 86k for displaying information on music used to create the hybrid video content, and the like are displayed.

  In addition, the user operates the screen portion 86a of the enlarged playback screen 86, for example, by clicking with the mouse, and thereby enlarges and plays back only one moving image content displayed on the screen portion 86a. Can be displayed.

  26 and 27 are schematic diagrams illustrating an example of the public screen 8 on which the partial enlarged playback screen 87 is displayed. The partial enlargement playback screen 87 has a screen on which one moving image content is played, and at the bottom of the screen is a song name display section, operation buttons for instructing playback, pause, cueing, etc., a time code slider An operation unit 87b including a volume adjustment slider and the like is disposed. The public screen 8 also includes a music related video content display unit 87c that displays video content created using the same music as one video content, an evaluation input unit 87d for evaluating one video content, A music-related mixed video content display unit 87e that displays a hybrid video content created using the same music as one video content, a comment display unit 87f that displays a comment on one video content, and a text field for inputting a comment 87g, a user information display portion 87h that displays the user who generated each moving image content constituting one moving image content, and the like are displayed.

  Next, a processing procedure of the control units 10 and 40 related to generation and release of the hybrid video content will be described. In the following processing, the main part of the processing content will be described on the assumption that the user is logged in to the karaoke system. In addition, since the process procedure of the control part which concerns on the production | generation and registration of the moving image content which concerns on a song is the same as the process procedure shown in FIG.9 and FIG.10, detailed description is abbreviate | omitted.

  FIG. 28 is a flowchart illustrating a processing procedure of the control unit 10 related to the moving image content disclosure setting and the karaoke distribution setting. The control part 10 of the central apparatus 1 displays the user screen 7 on the display part of the user terminal device 6 as shown in FIG. 16 (step S51). More specifically, the control unit 10 transmits image data for displaying the user screen 7 and user information of the logged-in user to the user terminal device 6. The user screen 7 is constructed based on the screen data and the user information transmitted from, and is displayed on the display unit. Hereinafter, the contents of the process of displaying various screens on the display unit of the user terminal device 6 from the central device 1 are the same, and hence the description thereof is omitted.

  Next, the control unit 10 specifies a moving image uploaded to the central apparatus 1 based on my moving image, that is, the logged-in user ID (step S52). Specifically, my moving image is specified by reading out the moving image ID registered in the “my moving image” column of the user information DB 13a in association with the user ID of the logged-in user.

  Then, the control unit 10 displays a list of thumbnail images 73a of my moving images, a “setting” button 73c for performing settings relating to the moving images, information on the moving images, and the like on the user screen 7 (step S53).

  Next, the control unit 10 determines whether or not the “setting” button 73c has been operated (step S54). Specifically, when data indicating that the “setting” button 73c corresponding to a specific video is operated is transmitted from the user terminal device 6 to the central device 1, the control unit 10 of the central device 1 The data transmitted from the terminal device 6 is received, and based on the data, it is determined whether or not the “setting” button 73c has been operated.

  When it determines with the "setting" button 73c having been operated (step S54: YES), the control part 10 displays a setting screen on the display part of the user terminal device 6 (step S55). And the control part 10 receives a public setting and a karaoke delivery setting (step S56). Specifically, the control unit 10 receives disclosure permission / inhibition information, karaoke distribution permission information, and the like transmitted from the user terminal device 6 to the central device 1 via the second communication unit 16. Then, the control unit 10 registers the disclosure availability information, karaoke delivery availability information, and the like received in step S56 in the moving image DB 13c (step S57).

  After the process of step S57 is completed, or when it is determined in step S54 that the “set” button 73c has not been operated (step S54: NO), the control unit 10 identifies moving image data for which karaoke distribution is permitted. Then, the mixed material registration icon 73f is displayed on the thumbnail image 73a of the moving image permitted to be delivered by karaoke (step S58).

  Next, the control unit 10 determines whether or not the hybrid material registration icon 73f has been operated (step S59). Specifically, the composite material designation information indicating that the composite material registration icon 73f corresponding to the specific video is operated is transmitted from the user terminal device 6 to the central device 1, and the control unit 10 of the central device 1 The hybrid material designation information transmitted from the user terminal device 6 is received via the second communication unit 16.

  When it is determined that the hybrid material registration icon 73f has not been operated (step S59: NO), the control unit 10 ends the process. When it is determined that the mixed material registration icon 73f has been operated (step S59: YES), the control unit 10 registers the moving image selected as the mixed material in the user information DB 13a, that is, the moving image ID of the moving image is stored in the user information DB 13a. It is registered in the “My hybrid video material” column (step S60), and the process is completed.

  FIG. 29 is a flowchart illustrating a processing procedure of the control unit 10 relating to the release of moving image content. As shown in FIG. 19, the control unit 10 of the central apparatus 1 displays the public screen 8 on the display unit of the user terminal device 6 (step S71). Then, the control unit 10 identifies a moving image permitted to be released from among the moving images uploaded from all users (step S72). Then, the control unit 10 discloses the thumbnail images 81a and 82a of the moving images permitted to be released and the information of the moving images (step S73).

  Next, the control unit 10 identifies moving image data for which karaoke distribution is permitted, and displays the mixed material registration icons 81b and 82b on the thumbnail images 81a and 82a of the moving image for which karaoke distribution is permitted (step S74). .

  Next, the control unit 10 determines whether or not the mixed material registration icons 81b and 82b have been operated (step S75). When it is determined that the mixed material registration icons 81b and 82b have been operated (step S75: YES), the control unit 10 registers the moving image selected as the mixed material in the user information DB 13a, that is, the moving image ID of the moving image is registered in the user information DB 13a. Are registered in the “my mixed video material” column (step S76).

  After the process of step S76 is completed, or when it is determined in step S75 that the mixed material registration icons 81b and 82b have not been operated (step S75: NO), the control unit 10 determines whether or not there has been a playback operation for moving image content data. Is determined (step S77). When it determines with there being no reproduction | regeneration operation (step S77: NO), the control part 10 complete | finishes a process.

  If it is determined that there has been a playback operation (step S77: YES), the control unit 10 reads the video content data to be played back from the video DB 13c, and reads the music data associated with the video content from the music DB 13b, The moving image data, audio data, and music data included in the read moving image content data are reproduced in parallel (step S78), and the process ends.

  30 and 31 are flowcharts showing the processing procedure of the control units 10 and 40 related to the generation and registration of the hybrid video content. The control unit 40 of the karaoke terminal device 4 receives the music by the remote operation device 5 (step S91), and receives the music request data including the user ID and the music ID related to the received music via the communication unit 44. 1 (step S92).

The control unit 10 of the central device 1 receives the music request data transmitted from the karaoke terminal device 4 via the first communication unit 15 (step S93), and the user ID and the music ID included in the received music request data. The audio data of the hybrid video material corresponding to is selected (step S94). That is, the control unit 10 identifies the moving image content registered as the mixed moving image material from the moving image ID registered in the “my mixed moving image material” column in association with the user ID, and the registered moving image content. Are identified with the same music ID as the received music ID.
And the control part 10 transmits the specified music data and audio | voice data to the karaoke terminal device 4 via the 1st communication part 15 (step S95). In step S95, the example in which only the audio data constituting the moving image content data is transmitted has been described. However, the moving image content data including the moving image data and the audio data may be transmitted.

  The control unit 40 of the karaoke terminal device 4 receives the music data and audio data transmitted from the central device 1 via the communication unit 44 (step S96). And the control part 40 reproduces | regenerates the received music data and audio | voice data in parallel (step S97). Specifically, the control unit 40 provides music data to the sound source 48b and provides audio data to the audio decoder. The sound source 48b gives the accompaniment sound related to the music data to the audio mixer 48d, and the audio decoder 48c decodes the audio data, and gives the decoded audio data to the audio mixer 48d. The audio mixer 48d synthesizes the accompaniment sound and the sound given from the sound source 48b and the audio decoder 48c, respectively, and outputs the audio data obtained by the synthesis to the speaker 48f. Therefore, the sound related to the mixed moving image material is reproduced from the speaker 48f in accordance with the accompaniment of karaoke.

Next, the control unit 40 inputs a moving image of the song performed while reproducing the music data (step S98), and inputs the voice of the song (step S99).
The audio data output from the microphone 48e is input to the audio mixer 48d, and the audio mixer 48d combines and synthesizes the karaoke accompaniment sound, the audio related to the mixed material moving image, and the audio input to the microphone 48e. The audio data obtained in this way is output to the speaker. Therefore, in accordance with the accompaniment of karaoke, two users are in a state of singing duet.

  Next, the control unit 40 gives the input voice data to the scoring unit 47, and performs scoring sequentially for the song related to the voice data (step S100).

  Next, the control unit 40 determines whether or not to end the reproduction of the music data (step S101). When it determines with not complete | finishing reproduction | regeneration (step S101: NO), the control part 40 returns a process to step S97. When it is determined that the reproduction is to be ended (step S101: YES), the control unit 40 monitors the operation state of the remote operation device 5 and determines whether or not the input moving image data and audio data are to be uploaded (step S102). ). When it determines with not uploading (step S102: NO), the control part 40 finishes a process.

  When it determines with uploading (step S102: YES), the control part 40 transmits a user ID, music ID, a scoring result, moving image data, and audio | voice data to the central apparatus 1 via the communication part 44 (step S103).

  The control unit 10 of the central device 1 receives the user ID, music ID, scoring result, moving image data, and audio data transmitted from the karaoke terminal device 4 by the first communication unit 15 (step S104). Then, the control unit 10 generates hybrid video content data including the received video data and audio data, the scoring result of each audio data, and the video data and audio data selected as the hybrid material, and the generated hybrid The moving image content data is registered in the mixed moving image DB 13d together with the received user ID, music ID, etc., and the mixed moving image ID issued at the time of registration in the mixed moving image DB 13d is registered in the user information DB 13a (step S105), and the processing is completed.

  FIG. 32 is a flowchart illustrating a processing procedure of the control unit 10 related to the setting for releasing the mixed video content. As shown in FIG. 21, the control unit 10 of the central device 1 displays the user screen 7 on the display unit of the user terminal device 6 (step S111).

  Next, the control unit 10 identifies my hybrid video, that is, a hybrid video created by the logged-in user (step S112). Specifically, my moving image is specified by reading out the mixed moving image ID registered in the “my mixed moving image” column of the user information DB 13a in association with the user ID of the logged-in user.

  Then, the control unit 10 displays a list of my mixed moving image content thumbnail image 77a, a “setting” button 77c for performing settings related to the mixed moving image, the mixed moving image content information, and the like on the user screen 7 (step S113). .

  Next, the control unit 10 determines whether or not the “setting” button 77c has been operated (step S114). Specifically, data indicating that the “setting” button 77c corresponding to the specific video is operated is transmitted from the user terminal device 6 to the central device 1, and the control unit 10 of the central device 1 6 is received, and based on the data, it is determined whether or not the “set” button 77c has been operated.

  If it is determined that the “setting” button 77c is not operated (step S114: NO), the control unit 10 ends the process. When it is determined that the “setting” button 77c has been operated (step S114: YES), the control unit 10 displays the mixed moving image content information display unit 78 and the setting content input unit 79 on the setting screen as shown in FIG. The information is displayed on the display unit of the device 6 (step S115). Then, the control unit 10 accepts the public setting (Step S116). Specifically, the disclosure permission / prohibition information input to the user terminal device 6 and transmitted to the central device 1 is received via the second communication unit 16. Then, the control unit 10 registers the disclosure availability information, karaoke distribution availability information, and the like received in step S116 in the moving image DB 13c (step S117), and ends the process.

  FIG. 33 is a flowchart illustrating a processing procedure of the control unit 10 related to the release of the hybrid video content. The control unit 10 of the central device 1 displays the public screen 8 on the display unit of the user terminal device 6 (step S131). Then, the control unit 10 identifies a hybrid video that is permitted to be released from the hybrid video content generated by all users (step S132). Then, the control unit 10 displays a list of the thumbnail images 83a of the mixed moving image content permitted to be released and information on the moving images (step S133).

  Next, the control unit 10 determines whether or not there has been a reproduction operation of the hybrid moving image content (step S134). When it determines with having performed reproduction operation (step S134: YES), the control part 10 transmits a composite moving image content and music data to the user terminal device 4, and two sets of moving images contained in a mixed moving image content in this user terminal device. Data is reproduced in parallel (step S135).

  Further, the control unit 10 compares a plurality of scoring results included in the hybrid video content for each specific reproduction category, and reproduces the audio data having the highest scoring result and the music data in parallel (step S136). ) Finish the process.

  In the karaoke system, the central device 1 and the content data generation method according to the second embodiment configured as described above, the content of the medley relay that combines a plurality of video data and audio data related to the singing of the user or another person. Creation and release are possible, and entertainment can be improved.

  In addition, since a single moving image content can be selected as a material of the mixed moving image content from a plurality of moving image contents registered in the moving image DB 13c of the central apparatus 1, the moving image content can be created. There is a high degree of freedom to create, and entertainment is improved.

  Furthermore, the video content created by one user is published on the video browsing site, and other users can select video content as the material of the hybrid video content from the published video browsing site. Compared to the case where hybrid video content is created using only the video content created in step 1, the degree of freedom in creating video content is high, and the entertainment is improved.

  Furthermore, since it is possible to select whether or not to publish video content to a video posting site and whether or not to distribute karaoke video content, it is desirable to publish video content but avoid karaoke distribution. Can respond to various needs for

  Furthermore, the volume of the audio data of each moving image content constituting the mixed moving image content can be changed individually. Therefore, the entertainment when viewing the content data is improved.

  In the second embodiment, the example in which the hybrid video content is generated on the central device side has been described. However, the hybrid video content may be generated on the karaoke terminal device side and transmitted to the central device.

  Moreover, although the example which produces | generates the hybrid video content which combines two video content was demonstrated, you may produce | generate a hybrid video content combining 3 or more video content. Further, the method for arranging moving images related to a plurality of moving image contents is not particularly limited. Also, audio data included in a plurality of moving image contents may be reproduced in parallel, but may be selectively reproduced.

  Furthermore, although the moving image has been described as the material of the mixed content formed by combining the video and audio of the singing, the video may be a video or a still image that is switched in chronological order in a slide show format.

  Furthermore, as the video content that is the material of the hybrid video content, the video content that the user is singing is used, but the video content prepared in advance is stored in the internal storage device of the central device, You may make it produce | generate a hybrid moving image content using this moving image content.

  Furthermore, although the configuration in which the user selects the video content that is the basis of the hybrid video content has been illustrated, the central device may randomly select and distribute to the karaoke terminal device.

  The embodiment disclosed this time is to be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the meanings described above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

DESCRIPTION OF SYMBOLS 1 Central apparatus 2 Karaoke store 4 Karaoke terminal apparatus 5 Remote operation apparatus 6 User terminal apparatus 10, 40 Control part 12, 42 RAM
13 Internal storage device 13a User information DB
13b Music DB
13c Movie DB
13d hybrid video DB
14 external storage device 15 first communication unit 16 second communication unit 43 storage device 44 communication unit 46a video encoder 46b video control circuit 46c video decoder 46d video mixer 46e imaging device 46f display device 47 scoring unit 48a audio encoder 48b sound source 48c audio decoder 48d audio mixer 48e microphone 48f speaker N1, N2 communication network

Claims (4)

In a karaoke system comprising a central device for transmitting karaoke song data, and a karaoke terminal device for receiving and reproducing karaoke song data transmitted from the central device,
The karaoke terminal device
Video input means for inputting a video of a song performed while the karaoke song data is being reproduced by the reproduction means;
An audio input means for singing voice performed while reproducing the karaoke data in the reproducing means is input,
Evaluation means for evaluating voice data related to the voice input by the voice input means at a plurality of different playback points;
Transmission means for transmitting video data and audio data related to video and audio input by the video input means and audio input means, and an evaluation result by the evaluation means to the central device,
The central device is
Means for receiving evaluation results obtained by evaluating video data and audio data transmitted from the karaoke terminal device, and voices of the singing at different reproduction points;
Storage means for storing a plurality of sets of received video data, audio data and evaluation results in association with each other;
Selection means for selecting a plurality of sets of video data and audio data from video data and audio data of a song performed while reproducing one karaoke song data stored in the storage means;
Generating means for generating content data combining a plurality of sets of video data and audio data selected by the selecting means;
Publishing means for making the content data generated by the generating means publicly viewable, and
The public means is:
Videos related to each video data of a song performed while reproducing the one karaoke song data are juxtaposed, and for each voice data of the song performed while reproducing the one karaoke song data A karaoke system, wherein one audio data related to each video data is selectively reproduced according to an evaluation result. The public means is:
2. The karaoke system according to claim 1, wherein audio data having the highest evaluation result is selectively reproduced at each reproduction time point. In the central device that transmits karaoke song data,
It means for receiving video data and audio data according to the singing transmitted from a terminal device, and the evaluation result of evaluating the voice data in a plurality of different playback time,
Storage means for storing a plurality of sets of received video data, audio data and evaluation results in association with each other;
Selection means for selecting a plurality of sets of video data and audio data from video data and audio data of a song performed while reproducing one karaoke song data stored in the storage means;
Generating means for generating content data combining a plurality of sets of video data and audio data selected by the selecting means;
Publishing means for making the content data generated by the generating means publicly viewable, and
The public means is:
Videos related to each video data of a song performed while reproducing the one karaoke song data are juxtaposed, and for each voice data of the song performed while reproducing the one karaoke song data A central apparatus, wherein one audio data related to each video data is selectively reproduced according to an evaluation result. In a content data generation method for generating content data related to singing using a central device that transmits karaoke song data and a karaoke terminal device that receives and reproduces karaoke song data transmitted from the central device,
The karaoke terminal device
Video and audio of singing performed while playing karaoke song data are input,
Evaluate the audio data related to the input audio at multiple different playback times,
Sending video data and audio data related to the input video and audio, and evaluation results to the central device,
The central device is
Receive video data and audio data and evaluation results transmitted from the karaoke terminal device,
Multiple sets of received audio data, video data and evaluation results are stored in association with each other,
A plurality of sets of video data and audio data are selected from video data and audio data of a song performed while playing one karaoke song data ;
Generate content data combining a plurality of selected video data and audio data,
The generated content data was performed while the video related to each video data of the singing performed while the one karaoke song data was being reproduced, and while the one karaoke song data was being reproduced. A content data generation method characterized in that, in accordance with an evaluation result for each piece of audio data of singing, the audio data is disclosed so that it can be viewed so that one piece of audio data related to each video data is selectively reproduced.

Images