JP2004501576A - Karaoke system - Google Patents

Abstract

The concept of singing karaoke is based solely on audio-based techniques. The present invention expands the concept of falling into video, allowing users to insert and view their images in video clips or movies instead of their favorite singers or players, and thus clip / song on videotape. Can be played and replaced with any famous person. More specifically, the invention relates to a means for capturing a user's image and sound, a means for analyzing and processing the resulting signal, mixing the audio / video signal thus analyzed and pre-recorded material. Means and a karaoke system having successively means for displaying the resulting combined signal.

Description

[0001]
The present invention relates to a karaoke system that sings during a sequence such as a video clip or movie.
[0002]
For example, in a karaoke system as disclosed in European Patent Application EP0782338, music, lyrics, or any type of audio data is transmitted from a transmitting station to a distribution station. The music control means of the main module of the system plays the music through the built-in speaker of the monitor and the sound from the invisible microphone through the speaker. The image control means displays a background image (for example, a video image or a still image extracted from the background image storage means) on a monitor, and the lyrics control means displays lyrics superimposed on the background image. An image capture device, such as a CCD camera, captures the image of the singer and superimposes that image on the screen of the monitor through the video image control means to produce a superimposed image. Such a system is defined as "video mixing" in the karaoke concept.
[0003]
It is an object of the present invention to provide another type of karaoke system with further functionality.
[0004]
To this end, the system comprises a capture device for singing in a sequence, such as a video clip or movie, capturing the image and sound of the user, an analysis and processing device for separating at least a part of the user from the background, and the analysis and processing device. The present invention relates to a karaoke system having a mixing and rendering device for combining the output signal of the above with pre-recorded material and a display device for displaying the combined signal.
[0005]
To date, the concept of singing karaoke has been based solely on audio-based techniques, with limited functionality, and it has never been possible for a user to actually get into the imaginary world of video. The proposed solution, which introduced the idea of video mixing, evolved this karaoke concept into the concept of video, and more specifically, completely into audio-video, and according to the above concept, in the video clip of a song, The voice and face may be replaced by the voice and face of a casual singer (hereafter referred to as a user as he / she may be a singer, player, dancer, etc.). The same proposed technique can be applied in other contexts as well, for example in the field of e-commerce, or in video editing of pre-recorded content.
[0006]
The invention will now be described by way of example with reference to the accompanying drawings.
[0007]
As shown in FIG. 1, the different subsystems required to implement a karaoke system according to the invention are mainly an analysis and processing system 11 and a mixing and rendering device 12.
[0008]
The analysis and processing device 11 receives the user's image and sound (the person shown in black) captured by the capture device 10 and, for example, segments the user's face from the background to define an alpha plane. (Such circuits may be based on chrome key techniques, for example, if the user steps up to a stage before the blue screen). The mixing and rendering device 12 uses the shape information analyzed in the device 11 to generate a pre-recorded video or audio-video of the background transmitted from the medium 13 (the pre-recorded material is shown on the left side of the medium). This is a circuit for combining users. This configuration completes the audio portion that is provided to mix the user's voice with the music with pre-recorded song background. Using the alpha plane defined by the device 11, the following type of relationship can be easily obtained: [(Video 1 × Alpha) + (Video 2 × (255-Alpha))] / 255 = Two sources according to the final video. Can be combined. A display device 14, such as a monitor, is ultimately used to display the final result (i.e., a combination of pre-recorded material and especially belonging to the user).
[0009]
Obviously, in order to improve the quality, the analysis performed in the device 11 can produce an 8-bit alpha plane, which allows to mix better at the limits of the object covered by the surface . Note that the system can replace only the user's head or his / her entire body.
[0010]
Other cases for audio / video types are possible:
a) Two audio / video sources are uncompressed: This option is used when the entire singer's body is covered in a clip / movie (pre-recorded data is stored on tape and some singer's Video can be analyzed and transmitted directly to a video mixer), for example, used in a karaoke restaurant;
b) One or two sources are compressed: one adapted configuration in such a case is the newly expanded MPEG-4 standard, which is the object's shape and alpha plane, here the user's Enables encoding faces (MPEG-4 defines an overall system configuration that allows for the organization of audio and video objects).
[0011]
Applications for other cases of the invention are also conceivable:
a) The user wants to record the result of the mixing operation: this is a system similar to the implementation of FIG. 1 but including a further recording device 25, as shown in FIG.
[0012]
b) In some cases, the karaoke system runs online: the pre-recorded clips are stored in a database (eg, the Internet) and the user records his / her performance at home and generates a synthetic karaoke clip (Using compression techniques is particularly useful in such applications, and more generally, in all applications performed in a bandwidth-limited environment). .),
c) In some further cases, the user wants to replace only his / her head with the original singer's head, and the user's head position must match the original singer's body orientation and posture. Therefore, further processing is required in the mixing and rendering device 12.
[Brief description of the drawings]
FIG. 1 is a block diagram of a karaoke system according to the present invention.
FIG. 2 is a diagram showing another execution method of the karaoke system according to the present invention.

Claims (1)

A karaoke system for singing during a sequence such as a video clip or movie,
A capture device for capturing a user's image and sound; an analysis and processing device for separating at least a portion of the user from the background; and a mixing and rendering device for combining the output signal of the analysis and processing device with pre-recorded material. Karaoke system having a display device for displaying the combination signal continuously.