WO2008111746A1

WO2008111746A1 - System and method for realizing virtual stuio through network

Info

Publication number: WO2008111746A1
Application number: PCT/KR2008/001170
Authority: WO
Inventors: Chulwoong Jahng
Original assignee: Ebroadcast Technologies Co., Ltd.
Priority date: 2007-03-09
Filing date: 2008-02-28
Publication date: 2008-09-18
Also published as: KR20080082759A; KR100889367B1

Abstract

Provided are a system and a method for realizing a virtual studio through a network in which a two-dimensional (2D) or three-dimensional (3D) virtual studio video image can be generated by using discrete processing of a client terminal and a virtual studio server through an Internet.

Description

SYSTEM AND METHOD FOR REALIZING VIRTUAL STUDIO THROUGH NETWORK

Technical Field

The present invention relates to a system and a method for realizing a virtual studio through a network in which a two-dimensional (2D) or three-dimensional (3D) virtual studio video image can be generated without provision of a high-performance computer terminal by using discrete processing of a client terminal and a virtual studio server through an Internet.

Background Art

Conventionally, a two-dimensional or three-dimensional virtual studio for generating a 2D or 3D virtual video image has been used as a high-priced dedicated solution that can be used by a television (TV) broadcasting station or manufacturers of a broadcast program before a multimedia technology is developed to be used popularly.

In such 2D or 3D virtual studio technology, a high central processing unit (CPU) power is required in rendering for designing a two-dimensional or three-dimensional set, and a high-priced graphic card must be used. Furthermore, a work requiring a professional processing technology for using a high-priced solution such as a chroma key is necessary so that it is not easy for a user to realize the 2D or 3D virtual studio technology using a computer haying the resource of a conventional system.

In other words, in the virtual studio technology, a 2D or 3D set background image that is generated by using both a technology for generating a 2D or 3D set background image and a technology for synthesizing the set background image with an image photographed by a camera, is naturally synthesized with a moving picture of an actor photographed by a camera, and the movement of a camera is sensed by a sensor, and a background corresponding to the position of the camera is drawn and is synthesized with the synthesized image so that the effect like that the actor is actually positioned in a set background realized in graphic can be realized. However, it is difficult to realize the effect in a general computer.

Recently, as a moving picture processing and 2D or 3D design processing solution becomes popularized with high-performance of a personal computer (PC), a user can design and generate a virtual video moving picture arbitrarily by using his/her own PC, and a user created contents (UCC) service that distributes a generated moving picture through the Internet has been briskly provided.

However, a technology for processing a 2D or 3D moving picture using a PC can be used to contents requiring a simple manufacturing technique such as an UCC etc. without any difficulty. However, this technology requires precise rendering regarding a 2D or 3D background design, or it is still difficult to use this technology as a platform of a virtual studio technology using a chroma key technology.

Disclosure of the Invention

The present invention provides a system and a method for realizing a virtual studio through a network in which a two-dimensional (2D) or three-dimensional (3D) virtual studio video image can be generated by using a discrete processing technique between a client computer and a virtual studio server through an Internet.

According to an embodiment of the present invention, there is provided a system for realizing a virtual studio through a network, the system including: a client terminal transmitting an image including an actor photographed by a camera to a virtual studio server through a network, receiving, reproducing and storing a 2D (two-dimensional) or a 3D (three-dimensional) virtual studio video image that is finally generated from the virtual studio server; a virtual studio server extracting the actor image from the image photographed by the camera from the client terminal received through the network by using a chroma key and transmitting a 2D or 3D virtual studio image that is finally generated by synthesizing the extracted actor image with a 2D or 3D virtual set background, to the client terminal; and a virtual studio database storing a plurality of types of 2D or 3D set background images, storing the image photographed by the camera provided from the client terminal, storing the image of the actor extracted by suing the chroma key, and storing finally-generated 2D or 3D virtual studio video image data.

According to another aspect of the present invention, there is provided a method for realizing a virtual studio through a network, the method including: (a) accessing a virtual studio server through a network and transmitting an image including an actor photographed by a camera to the virtual studio server by using a client terminal; (b) extracting an actor image from the image photographed by the camera by using a chroma key by using the virtual studio server; (c) synthesizing the extracted actor image with a 2D (two-dimensional) or 3D (three-dimensional) virtual set background; and (d) transmitting a final 2D or 3D virtual studio video image in which the actor image is synthesized with a 2D or 3D virtual set background, to the client terminal, receiving a 2D or 3D virtual studio video image from the client terminal and reproducing and storing the received 2D or 3D virtual studio video image by using the virtual studio server.

Brief Description of the Drawings

FIG. 1 illustrates a system for realizing a virtual studio through an Internet according to an embodiment of the present invention;

FIG. 2 is a detailed diagram illustrating a client terminal of FIG. 1 ; FIG. 3 is a detailed diagram illustrating a virtual studio server of

FIG. 1 ; FIG. 4 illustrates an example in which a virtual video is manufactured by using a two-dimensional (2D) or three-dimensional (3D) virtual studio technique using discrete processing between a client terminal and a virtual studio server according to an embodiment of the present invention;

FIG. 5 illustrates an example of a screen regarding a chroma key technique according to an embodiment of the present invention; and

FIG. 6 is a flowchart illustrating a method for realizing a virtual studio through a network according to an embodiment of the present invention.

Best mode for carrying out the Invention

Exemplary embodiments of the present invention will now be described with reference to the attached drawings. FIG. 1 illustrates a system for realizing a virtual studio through an

Internet according to an embodiment of the present invention. Referring to FIG. 1 , the system for realizing a virtual studio through an Internet according to an embodiment of the present invention comprises a plurality of client terminals A1 through to An, a virtual studio server 100, and a virtual studio database 200.

A network adaptor such as a modem and a LAN card that accesses the virtual studio server 100 through an Internet so that multimedia information can be exchanged, an Internet web browser program, and a camera for photographing a peripheral image are mounted on the plurality of client terminals A1 through to An. The client terminals A1 through to An select a desired set background from a 2D or 3D set background presented by the virtual studio server 100, photographs an image including an actor using the camera to provide the photographed image to the virtual studio server 100, and receives and reproduces a virtual studio video image that is finally generated by synthesizing set background rendering from the virtual studio server 100 with the actor extracted by using a chroma key.

Here, each of the client terminals A1 through to An comprises a key inputting unit 10, a camera 12, a hard disk drive (HDD) 14, a microprocessor 16, a video processor 18, a monitor 20, an audio processor 22, a speaker 24, a microphone 26, and a data communication module 28, as illustrated in FIG. 2.

The key inputting unit 10 comprises a coordinate inputting device such as a keyboard and a mouse. The key inputting unit 10 selects a predetermined set background from a 2D or 3D set background presented by the virtual studio server 100 and performs key inputting for transmitting the image photographed by the camera 12 to the virtual studio server 100 through the Internet.

The camera 12 photographs the peripheral image including the actor to be inserted in the studio image. An operating system (OS) program such as Windows for operating a system of a corresponding client terminal, an Internet web browser program, and an application program in which a transmission protocol for providing the image photographed by the camera 12 to the virtual studio server 100 is defined are stored in the hard disk drive (HDD) 14. The HDD 14 receives 2D or 3D virtual studio video image data that is finally generated by the virtual studio server 100.

The microprocessor 16 enables a predetermined set background to be selected from a 2D or 3D set background presented by the virtual studio server 100 using key inputting performed by the key inputting unit 10 while being connected to the virtual studio server 00 using the Internet web browser program, performs processing for transmitting the image photographed by the camera 12 to the virtual studio server 100 through the Internet, and controls video/audio processing for reproducing the final 2D or 3D virtual studio video image from the virtual studio server 100. Here, the microprocessor 16 uses ActiveX Control so as to perform virtual studio processing using an interlocking function with the virtual studio server 100 through the Internet. Since ActiveX Control is not used in the microprocessor 16 when Window Vista is used as an OS program of a corresponding client terminal, ActiveX Control is not used in the microprocessor 16, other method may also be used in the microprocessor 16.

The video processor 18 performs video processing of an image signal for displaying the 2D or 3D set background represented through an Internet web browser, the image photographed by the camera 12, or the final 2D or 3D virtual studio video image on the monitor 20. The audio processor 22 performs audio processing for outputting an audio signal included in the image photographed by the camera 12 and an audio signal included in the final 2D or 3D virtual studio video image to a head set comprising the speaker 24 and the microphone 26 as sound, and audio signal processing for synthesizing sound inputted from the microphone 26 with the image photographed by the camera 12.

The client terminals A1 through to An select a desired set background from the 2D or 3D virtual set background presented by the virtual studio server 100. However, the present invention is not limited to this, and the client terminals A1 through to An may directly render and draw a 2D or 3D set background of a desired design by using a multimedia design dedicated software program such as "3D MAX" or "MAYA" in a corresponding client terminal, may upload the directly-drawn 2D or 3D set background on the virtual studio server 100 and select the 2D or 3D set background as a set background of a virtual studio image. In FIG. 1 , the virtual studio server 100 accesses each of the client terminals A1 through to An through the Internet, provides 2D or 3D set background information so that a desired set background can be selected from each of the client terminals A1 through to An, extracts the actor from moving picture data photographed by a camera provided by each of the client terminals A1 through to An by using a chroma key, and transmits a 2D or 3D virtual studio vide image that is finally generated by synthesizing the extracted actor image with a set background image.

The virtual studio data base 200 stores a variety of types of 2D or

3D set background images presented by the client terminals A1 through to An, stores the image photographed by the camera provided from each of the client terminals A1 through to An, stores an image of the actor extracted using the chroma key, and stores the extracted 2D or 3D virtual studio video image data.

The 2D virtual set background may be comprised of image files,

2D moving picture files, or flash files. The 3D virtual set background may be formed in a format (i.e., multimedia files + XML files + Java Script files) in which image files, 2D moving picture files, or flash files are combined with XML files and Java Script files so as to set a dynamic position of a background.

Here, the virtual studio server 100 comprises a network interface 30, an operating system 32, a decoder 34, a live video manager 36, a chroma key engine 38, a set manager 40, a 2D mixer 42, a 3D renderer engine 44, a user interface 46, and an encoder 48, as illustrated in FIG.

3.

The network interface 30 accesses the client terminals A1 through to An through the Internet to receive an image photographed by the camera and transmits a finally-synthesized 2D or 3D virtual studio video image. The operating system 32 connects a function between hardware and software in a corresponding server and manages the operation of the corresponding server. If the image photographed by the camera received from the client terminals A1 through to An is compressed by using an image compression algorithm such as "MPEG4", the decoder 34 performs decoding processing for decompressing the image.

The live video manager 36 provides the image photographed by the camera that is decoded by the decoder 34 to the chroma key engine

38 so that the actor image can be extracted and stores the image photographed by the camera and the actor image extracted by using the chroma key in the virtual studio database 200 and manages them.

The chroma key engine 38 extracts the actor image to be synthesized with the set background from the image photographed by the camera by using the chroma key.

The set manager 40 provides a variety of types of 2D or 3D set background data stored in the virtual studio database 200 to the client terminals A1 through to An through the Internet and manages so that the 2D or 3D background image selected by a user of the client terminals A1 through to An can be synthesized with the actor image by using the 2D mixer 42 or the 3D renderer engine 44.

The 2D mixer 42 performs image mixing processing for synthesizing the actor image extracted by the chroma key engine 38 with the 2D set background selected by the user of the client terminals A1 through to An according to manager's manipulation via the user interface 46.

The 3D renderer engine 44 performs 3D graphic rendering processing for synthesizing the actor image extracted by the chroma key engine 38 with the 3D set background selected by the user of the client terminals A1 through to An according to manager's manipulation via the user interface 46.

The encoder 48 performs encoding processing for compressing the finally-generated 2D or 3D virtual studio video image by using a high-efficient compression algorithm such as "MPEG4". Next, FIG. 4 illustrates an example in which a virtual video is manufactured by using a two-dimensional (2D) or three-dimensional (3D) virtual studio technique using discrete processing between a client terminal and a virtual studio server according to an embodiment of the present invention. As illustrated in FIG. 4, if a variety of types of 2D or 3D set background information are presented by the virtual studio server 100 in the state where a predetermined client terminal accesses the virtual studio server 100 through the Internet, the user selects a predetermined 2D or 3D set background in the predetermined client terminal.

In that state, a peripheral image including an actor is photographed by the camera and is transmitted to the virtual studio server 100 through the Internet by using the client terminal, and the virtual studio server 100 extracts the actor image by using a chroma key technique from the image photographed by the camera, synthesizes the extracted actor image with a 2D or 3D set background image selected by the user, and then transmits the finally-synthesized 2D or 3D virtual studio video image to the client terminal.

Accordingly, the client terminal displays the final 2D or 3D virtual studio video image provided from the virtual studio server 100 on a monitor and reproduces and outputs the final 2D or 3D virtual studio video image as sound, and stores the final 2D or 3D virtual studio image in the HDD 14.

In the chroma key technique performed in the virtual studio server 100, processing (i.e., an operation of removing pixels of color within ellipses at a right menu) for removing an unnecessary peripheral image pixels excluding an image of the actor from each image of an image photographed by the camera including the actor in a blue or green background is performed so that all backgrounds excluding the actor can be processed to be transparent and the processed image of the actor can be synthesized with a 2D set background or a 3D set background to be synthesized, as illustrated in FIG. 5.

Subsequently, the operation of the present invention will now be described with reference to the flowchart of FIG. 6.

Firstly, in the state where a predetermined client terminal A1 among a plurality of client terminals A1 through to An accesses a virtual studio server 100 through an Internet (Operation S10), the virtual studio server 100 presents a variety of types of 2D or 3D set background information through an Internet web browser according to the request of the client terminal A1 (Operation S11 ), and the client terminal A1 selects a desired 2D or 3D set background from the variety of types of set backgrounds presented (Operation S12). In that state, the client terminal A1 photographs a peripheral image including an actor to be included in a virtual studio video image by using a camera 12 (Operation S13), compresses the photographed moving picture data by using a moving picture compression algorithm such as "MPEG4" and transmits the compressed moving picture data to the virtual studio server 100 through the Internet (Operation S14).

A decoder 34 of the virtual studio server 100 performs decoding processing for decompressing the moving picture of the image photographed by the camera 12 received from the network interface 30, and the chroma key engine 38 extracts only the actor image from the image photographed by the camera 12 under management of the live video manager 36 (Operation S15), and performs image synthesis by using a 2D mixer 42 so as to synthesize the extracted actor image with the 2D set background selected by the user or performs image synthesis by using a 3D renderer engine 44 so as to synthesize the extracted actor image with the 3D set background selected by the user (Operation S16).

If the 2D or 3D set background image is synthesized with the actor image, an encoder 48 of the virtual studio server 100 performs encoding processing for compressing a final 2D or 3D virtual studio video image by using a moving picture compression algorithm such as "MPEG4" and transmits the encoded 2D or 3D virtual studio video image to the client terminal A1 through the network interface 30 (Operation S 17).

The client terminal A1 stores the final 2D or 3D virtual studio video image data transmitted from the virtual studio server 100 in a hard disk drive (HDD) 14, and displays the final virtual studio video image on a monitor and outputs the final virtual studio video image to a speaker 24 as sound after video processing performed by a video processor 18 and audio processing performed by an audio processor 22 (Operation S18).

The virtual studio server 100 transmits the final virtual studio video image in which the 2D or 3D set background image is synthesized with the actor image, as a video stream to the client terminal to reproduce or store the virtual studio video image, and the corresponding virtual studio server 100 directly stores the stream and uses the stream as a VOD file according to the request of a user of the client terminal or directly provides a real-time service of live video through the Internet.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present invention as defined by the following claims.

Industrial Applicability

According to the present invention, discrete processing is performed by using information exchange between a client-side computer terminal and a virtual studio server through an Internet such that an image photographed by a camera including an actor is provided from the client-side computer terminal, and a desired image of the actor is extracted from the image photographed by the camera in the virtual studio server by using a chroma key and is synthesized with a 2D or 3D virtual set background such that a finally-generated 2D or 3D virtual studio video image is provided to the client-side computer terminal. Thus, a desired virtual studio video image can be obtained even by using a computer terminal having a conventional system specification without provision of a large computing power and a virtual studio equipment such as a high-priced video card etc.

Claims

What is claimed is:

1. A system for realizing a virtual studio through a network, the system comprising: a client terminal transmitting an image including an actor photographed by a camera to a virtual studio server through a network, receiving, reproducing and storing a 2D (two-dimensional) or a 3D (three-dimensional) virtual studio video image that is finally generated from the virtual studio server; a virtual studio server extracting the actor image from the image photographed by the camera from the client terminal received through the network by using a chroma key and transmitting a 2D or 3D virtual studio image that is finally generated by synthesizing the extracted actor image with a 2D or 3D virtual set background, to the client terminal; and a virtual studio database storing a plurality of types of 2D or 3D set background images, storing the image photographed by the camera provided from the client terminal, storing the image of the actor extracted by suing the chroma key, and storing finally-generated 2D or 3D virtual studio video image data.

2. The system of claim 1 , wherein the virtual studio server presents the plurality of types of 2D or 3D virtual set background data so that a 2D or 3D virtual set background selected by the client terminal is synthesized with the actor image.

3. The system of claim 2, wherein the client terminal comprises: a key inputting unit selecting a predetermined set background from a 2D or 3D set background presented by the virtual studio server and performing key inputting for the image photographed by the camera to the virtual studio server through the network; a camera photographing and providing a peripheral image including an actor to be inserted in a virtual studio image; a HDD (hard disk drive) in which an operating program operating a system of a corresponding client terminal, an Internet web browser program, and an application program for providing the image photographed by the camera to the virtual studio server are stored and which receives and stores final 2D or 3D virtual studio video image data generated from the virtual studio server; a microprocessor controlling selecting of a 2D or 3D set background presented by the virtual studio server, transmission processing of the image photographed by the camera, and video/audio processing for reproducing the final 2D or 3D virtual studio video image; a video processor performing video processing for displaying the 2D or 3D set background, the image photographed by the camera, and the final 2D or 3D virtual studio video image on a monitor; and an audio processor performing audio processing for outputting an audio signal included in the image photographed by the camera and an audio signal included in the final 2D or 3D virtual studio video image as sound and performing audio processing for synthesizing sound inputted from a microphone with the image photographed by the camera.

4. The system of claim 1 , wherein the virtual studio server comprises: a network interface accessing the client terminal through the network to receive the image photographed by the camera and transmitting a finally-synthesized 2D or 3D virtual studio video image; an operating system connecting a function between hardware and software in a corresponding server and managing an operation of the corresponding server; a decoder performing decoding processing for decompressing the image photographed by the camera received from the client terminal; a live video manager managing image data so that the actor image is extracted from the image photographed by the camera decoded by the decoder; a chroma key engine extracting the actor image to be synthesized with the set background from the image photographed by the camera by using the chroma key; a set manager managing so that a 2D or 3D set background image is synthesized with the actor image by using a 2D mixer or a 3D Tenderer engine; a 2D mixer performing image mixing processing for synthesizing the actor image extracted by the chroma key engine with the 2D set background; a 3D renderer engine performing 3D graphic rendering processing for synthesizing the actor image extracted by the chroma key engine with the 3D set background; and an encoder performing encoding processing for compressing the finally-generated 2D or 3D virtual studio video image by using a compression algorithm.

5. The system of claim 1 , wherein the 2D or 3D virtual set background is directly designed and uploaded using a multimedia design dedicated software program by using the client terminal.

6. A method for realizing a virtual studio through a network, the method comprising:

(a) accessing a virtual studio server through a network and transmitting an image including an actor photographed by a camera to the virtual studio server by using a client terminal;

(b) extracting an actor image from the image photographed by the camera by using a chroma key by using the virtual studio server;

(c) synthesizing the extracted actor image with a 2D (two-dimensional) or 3D (three-dimensional) virtual set background; and

(d) transmitting a final 2D or 3D virtual studio video image in which the actor image is synthesized with a 2D or 3D virtual set background, to the client terminal, receiving a 2D or 3D virtual studio video image from the client terminal and reproducing and storing the received 2D or 3D virtual studio video image by using the virtual studio server.

7. The method of claim 6, wherein (a) comprises: presenting a plurality of types of 2D or 3D virtual set background data to the client terminal by using the virtual studio server; and selecting a predetermined set background from the presented 2D or 3D virtual set background data from the client terminal.

8. The method of claim 6, wherein (a) comprises: directly designing a 2D or 3D virtual set background using a multimedia design dedicated software program and uploading the designed 2D or 3D virtual set background on the virtual studio server by using the client terminal; and selecting the 2D or 3D virtual set background uploaded on the virtual studio server as a virtual studio background by using the client terminal.

9. The method of claim 6, wherein in (a), the client terminal compresses the image photographed by the camera as an image according to an image compression algorithm and transmitting the compressed image, and the virtual studio server decompresses the compressed image photographed by the camera, and in (d), the virtual studio server compresses the finally-generated 2D or 3D virtual studio video image as an image according to an image compression algorithm and transmitting the compressed image, and the client terminal decompresses the compressed 2D or 3D virtual studio video image.