KR101879166B1 - A real-world studio system capable of producing contents using the control of a virtual studio and its operating method - Google Patents

Abstract

The present invention relates to a real-world studio system capable of producing content by using control of a virtual studio and an operating method thereof. More specifically, a three dimensional virtual studio corresponding to a real-work studio is constructed. When using a simulation of the virtual studio to record presentation information of content production and editing information of content production to be applied to content to be produced in the real-work studio on a timeline of a content production project, the content is photographed in the real-work studio in accordance with the presentation information of content production recorded on the content production project by re-play of the corresponding content production project, and the editing information of content production recorded on the content production project is applied to a photographed real-world image in real time to produce the content.

Description

Technical Field [0001] The present invention relates to a real-time studio system and a method for operating a virtual studio,

The present invention relates to a real-world studio system and a method for operating a virtual studio, and more particularly, to a 3D virtual studio corresponding to a real-world studio, If the editing information of the content production to be applied to the content to be produced and the editing information of the content production are recorded on the timeline of the content production project, the content production is performed through the re- The real-world video is photographed in the real-world studio, and the content can be produced using the control of the virtual studio, which allows the content to be produced in real time by applying the editing information of the content production recorded in the content production project to the photographed real- Room World studio system and method of operation.

Due to the development of the Internet, which is spread to the general public, contents including various information are also provided to users in various forms. In particular, since the late 2000s, the paradigm shift of the IT industry into smart and 3D has provided new opportunities for the content industry. More intelligent smart devices and services such as smart phones, smart pads, smart cars and smart health services have been highlighted, and 3D contents such as augmented reality and 3D movies are rapidly spreading.

With the proliferation of smart devices, users want to consume more and more content, but content produced by professional producers or professional producers has not met the content requirements of each user. As a result, the user has created user-generated content (UGC, User Generation Contents), which is a direct video content, and has started to mutually share with the users. At this time, the YouTube website (www.youtube.com) The number of productions has increased explosively.

One of the reasons that the user-generated content has increased dramatically is the improvement of the camera performance provided in the smartphone and the increase of the user penetration rate due to the popularization of the DSLR camera capable of video shooting. Users can shoot video regardless of time and place with their own smartphone, and can produce high quality video contents using popular DSLR camera as needed.

Also, one of the other reasons is the improved accessibility of editing software that can edit user-generated content. Adobe's Premiere, Adobe's After Effects, and Sony's begas, which allow you to edit your video content in the past, It has been recognized as the best software and it is difficult to use the general user. However, Microsoft's Windows Movies Maker, Vapmix, etc. have a simple function for video editing and provide an interface for ordinary users to edit video directly.

The editing program provided in the past provides an interface for performing editing of one captured moving image and has a problem that it is not easy to edit multi-moving image contents obtained by simultaneously photographing a single object with a plurality of cameras.

US 6,618,058 (September 9, 2003) entitled " Editing device and editing method "of Sony discloses a method in which a plurality of source images photographed by a plurality of cameras are output to a screen based on time codes of image data An edited image can be generated from a plurality of source images by selecting a desired image clip after classifying the selected plurality of image data into image clip groups.

However, the editing application of the prior art 1 merely allows editing of multi-image clips generated after the shooting ends, and can not provide the shooting information for shooting, There is a problem in that it is not possible to perform real-time editing in practice.

Japanese Patent Registration No. 5044503 (Jul. 20, 2012) of Kawai Musical of the prior art 2, "directed image reproducing apparatus, directed image reproducing method, directed image reproducing program and recording medium" The edited content set in the middle of the content can be reflected on the content and output. In other words, it is possible to set in advance various presentations for each page area of the electronic picture book in which a plurality of directors are outputted. When the user selects the directors and executes the electronic picture book, the directing effect set by the selected directors is outputted .

In the electronic picture book capable of directing the conventional art 2, a sound describing the electronic picture book in an electronic picture book composed of a certain story can be set as male or female, bass or treble, and a screen change on the timeline or an effect sound There is also an effect that the presentation can be set in advance.

However, according to the conventional art 2, it is possible to newly create data of a presentation part that can constitute a whole story of an electronic picture book, only an effect can be inserted into a configuration of a part of a basic story of a preset electronic picture book, There is no problem.

Korean Patent Registration No. 934471 (December 21, 2009) entitled " Apparatus and method for producing an image using chroma key ", which provides an image producing apparatus using chroma key, In the process of synthesizing new image contents by compositing a subject of a captured image with a preset background image or image, if a sensed signal is sensed at the time of the chroma key photography, a background image or a background sound in the content control information is synthesized To provide a directing effect.

However, in the prior art 3, the quality of the content can be confirmed through the video content finally generated after the directing or editing effect is synthesized with the moving image to be photographed. That is, the ratio of the photographed subject to the background image to be synthesized, or the illumination type on the background to be combined with the photographed subject, has a problem that the optimized photographic environment can be derived after performing a plurality of photographed images.

Eventually, the moving picture contents can be photographed by the user (producer) using the camera application of the camera or the smart phone. However, most of the shooting by the user (manufacturer) is manually set by the user (manufacturer) manually, such as the shooting composition of the camera, the shooting magnification, the illumination, etc., In order to secure a substantially smooth shooting environment, it is possible to find an optimized shooting environment by using a resultant value obtained by conducting a plurality of test shots and photographing one video content Time and cost resources are consumed inefficiently.

U.S. Patent No. 6,618,058 (September 9, 2003) Japanese Patent Registration No. JP 4044503 (July 20, 2012) Korean Patent Publication No. 0934471 (2009.12.21)

A real-world studio system and method for producing contents using the control of a virtual studio according to the present invention is characterized in that a content production project capable of controlling real-world photographing equipments provided in a content production studio includes a photographing camera, And the editing information that can edit the photographed real-world image is recorded, so that the content production studio performs movie shooting in accordance with the presentation information of the content production project, And the editing of the real-time image in accordance with the editing information of the real-time content production project is performed so that the moving image contents can be automatically produced.

A real-world studio system and a method for operating a virtual studio according to an exemplary embodiment of the present invention include a 3D virtual studio and a 3D main object synchronized with a performer (a subject) located in a real- Generates a virtual image in which a 3D main object is synthesized in a 3D virtual stage based on presentation information recorded in a content production project, applies editing information recorded in the content production project to the generated virtual image, It is possible to preliminarily judge trial and error and anticipated editing contents to be generated in the photographing step in the real-world studio in the real-world studio by simulating the moving picture contents which can be created by photographing and editing the contents, The purpose.

A real-world studio system and a method of operating a virtual studio capable of producing content using the control of a virtual studio according to an embodiment of the present invention include: The content production studio controller automatically controls real-world photographic equipment of a real-world studio and real-time automatic editing of a real-world image photographed in a controlled real-world studio to produce a video content, The present invention aims to create a virtual image through a content production project in which presentation information and edit information of a content is recorded and predict video contents to be produced in advance and to produce video contents through the content production project.

A real-world studio system and a method of operating a real-studio system that can produce contents using the control of a virtual studio according to an embodiment of the present invention include a real-world studio suitable for a kind or genre of contents to be produced from a plurality of real- So that a user can select a real-world studio suitable for contents to be produced from real-world studios provided in various forms.

A real-world studio system and a method of operating a virtual studio capable of producing contents using the control of a virtual studio according to an embodiment of the present invention include a location of a photographing equipment, a musical instrument, a performer, and the like that can be configured on a stage of a real- Setting and the like in a virtual stage created by rendering a virtual studio corresponding to a real-world studio.

A method of operating a real-world studio system capable of producing contents using the control of a virtual studio according to the present invention is characterized in that the content production project generation apparatus includes a plurality of real-world systems including a real- A project for creating a content production project capable of presetting production information of production scheduled contents including control commands of real-world photographic equipment and editing information for post production of production scheduled contents in a real-world studio of a studio A project creation step of receiving a creation command from a user terminal and creating a content creation project of a file type and a content creation project creation step of creating a content creation project of a file format by using the selection information of a real- A studio selection step of receiving the input from the user terminal and recording the selected studio information of the real-world studio in the content production project, and rendering the 3D modeling data of the real-world studio selected by the content production project generation device to the real- A virtual stage corresponding to a virtual camera taken by the virtual camera is displayed on the screen of the user terminal, and the corresponding virtual studio is created, and a virtual stage of the virtual studio, including a virtual camera, a virtual camera, a virtual microphone, And the content creation project generation device receives selection information of a 3D main object to be included in the virtual stage among a plurality of 3D main objects from the user terminal and outputs the selected 3D main object to the virtual A step of constructing a stage for outputting a first virtual image synthesized in a scene to a screen of the user terminal, and a step of, when a camera control command for changing the motion of the camera is inputted from the user terminal, Wherein the time information is recorded in at least one time line among a plurality of time lines that can be generated in the content production project, and when a re-play is performed on the time line, A project presentation step of outputting a second virtual image generated by adjusting the output values of the virtual stage and the 3D main object according to the presentation information recorded on the line on the screen of the user terminal, The real world image photographed from the camera When an effect command for applying an effect to a user is input from the user terminal, editing information including the effect command is recorded in one of the plurality of time lines that can be generated in the content production project Synthesizing an effect selected corresponding to the editing information recorded in the time line on the second virtual image, and outputting the edited third virtual image to a screen of the user terminal when a replay is performed on the time line And the content production studio controller controls the real-world photographic equipment provided in the real-world studio using the production information including the camera control command recorded in the time line of the content production project, The image is photographed, And producing a edited moving picture content by applying an effect corresponding to the effect command included in the editing information recorded in the timeline of the project to the real world image.

The real-world studio system and method for producing contents using the control of the virtual studio according to the embodiment of the present invention may include a step of, before the content creation step, And a studio reservation step of making a reservation for the reservation.

A method of operating a real-world studio system capable of producing contents using a control of a virtual studio according to an embodiment of the present invention is characterized in that the project directing step includes a step of classifying at least one of a scene or a cut A time line segmentation step of receiving time line segmentation information for dividing at least one of the plurality of time lines and generating the time line segmentation information in the content production project; Recording the production information including a control command for controlling at least one of the camera, the light or the microphone in correspondence with at least one of a scene or a cut in a timeline of the content production project A control command generation step of generating a control command of the real- .

A method of operating a real-world studio system capable of producing contents using a control of a virtual studio according to an embodiment of the present invention is characterized in that, in the studio selecting step, the selected real- .

A method of operating a real-world studio system capable of producing contents using the control of a virtual studio according to an embodiment of the present invention is characterized in that, in the studio setting step, the virtual studio is configured to perform the rendering in proportion to the size of the real- .

A method of operating a real-world studio system capable of producing contents using control of a virtual studio according to an embodiment of the present invention is characterized in that the control command for controlling the real-world photographic equipment provided in the real- And the control command is compatible with the virtual photographing equipment.

The operation method of the real-world studio system capable of producing contents using the control of the virtual studio according to the embodiment of the present invention is characterized in that when the replay of the timeline is performed in the project directing step, And the output values of the virtual stage and the 3D main object corresponding to the expected shooting change are adjusted within the control range.

A method of operating a real-world studio system capable of producing content using control of a virtual studio according to an embodiment of the present invention is characterized in that, in the stage configuration step, the content production project generation apparatus includes: Synthesizes the 3D main object with the background image, and outputs the synthesized background image to the screen of the user terminal.

A method of operating a real-world studio system capable of producing content using a control of a virtual studio according to an embodiment of the present invention is characterized in that the background image is in the form of either 2D or 3D.

A method of operating a real-world studio system capable of producing content using the control of a virtual studio according to an embodiment of the present invention is characterized in that, in the stage configuration step, the content production project generation apparatus generates, from the user terminal, When the replay of the time line is performed, the background sound is reproduced and output to the user terminal.

A method of operating a real-world studio system capable of producing contents using a control of a virtual studio according to an embodiment of the present invention is characterized in that the stage configuration step comprises: And adjusting the output value of the 3D main object such that at least one of the position, size, and direction of the 3D main object is controlled in the virtual stage in the studio.

The method of operating a real-world studio system capable of producing contents using the control of a virtual studio according to an embodiment of the present invention is characterized in that in the control command generation step, A lighting control command for controlling the virtual lighting of the virtual studio corresponding to one or more of the scene or the cut is recorded in at least one of the plurality of timelines And the illumination control command is capable of controlling at least one of on / off, position, direction, illuminance, and hue of the virtual illumination.

A method of operating a real-world studio system capable of producing contents using a control of a virtual studio according to an embodiment of the present invention is characterized in that in the project editing step, the effects include an image effect and an acoustic effect, dissolve, overlap, wipe, fade, special effect (SFX), visual effect (VFX), digital effect (DFX) Wherein the acoustic effect comprises at least one of a compressor, a gate, an equalizer, a filter, a reverb, a delay, a de-esser, a chorus, And includes at least one of a flanger, a harmonizer, and a pitch shift.

The method of operating a real-world studio system capable of producing content using the control of a virtual studio according to an embodiment of the present invention is characterized in that, in the studio reservation step, the content production project generation apparatus, when the reservation of the real- And transmits the content production project to the content production studio controller for controlling a content production studio device that generates a control signal for controlling the real-world image capture devices included in the reserved real-world studio.

A method of operating a real-world studio system capable of producing contents using control of a virtual studio according to an embodiment of the present invention is characterized in that the content production studio device receives the control command from the content production studio controller, And transmits the generated control signal to the real-world photographic equipment.

A real-world studio system capable of producing contents using the control of a virtual studio according to the present invention creates a content production project that allows video contents to be produced according to a plurality of timelines that can be generated, Recording the production information of the to-be-produced contents and the editing information of the to-be-produced contents including the control commands of the real-world photographing apparatuses (including the camera, microphone, illumination and prompt monitor) in the content production project based on the time line, A virtual camera corresponding to the real-world studio is created by rendering 3D modeling data of the real-world studio, and a virtual camera of the virtual studio includes a virtual camera, a virtual camera, a virtual microphone, Virtual studio shot And outputting a virtual image including a virtual stage, which is an output of the content generation project, and a replay operation of the content production project transmitted from the content production project generation apparatus, The method according to claim 1, further comprising: providing the production information recorded in a content production project to a content production studio device, receiving a real-world image photographed by the real-world photographic equipment using the provided production information, A content production studio controller for performing editing according to information and the control command recorded in the content production project from the content production studio controller and generating a control signal corresponding to the control command to provide the control command to the real- Ha And a content recorder for receiving each frame of the real world image to which the editing information is applied by the content production studio controller and completing the production of the moving image content.

The real-world studio system capable of producing contents using the control of the virtual studio according to the embodiment of the present invention is characterized in that the content production project generation apparatus includes a virtual studio which is generated in proportion to the size of the real- The virtual image is generated by using the virtual photographing equipments by synthesizing the 3D main object and adjusting the output form and size of the 3D main object in proportion to the output form and size of the virtual stage .

In the real-world studio system capable of producing contents using the control of the virtual studio according to the embodiment of the present invention, when the background image to replace the virtual stage is input from the user terminal, The rendering of the virtual studio displayed on the screen of the user terminal is stopped, the 3D main object is synthesized with the background image, and the synthesized background image is output to the screen of the user terminal.

A real-world studio system capable of producing contents using the control of a virtual studio according to an embodiment of the present invention is characterized in that the content production project generation apparatus includes: A time line scene editor module for generating a time line segmentation command for dividing at least one of the time lines corresponding to a scene or a cut of the real time photographing device, A camera control module for generating a camera control command for controlling the operation of the 3D modeling data, a plurality of 3D main data corresponding to a plurality of real-world studios, and a plurality of 3D main data to be synthesized in a virtual stage in a virtual studio, Each 3D modeling data of an object A 3D modeling data storage module for providing each 3D modeling data for a virtual studio and a 3D main object corresponding to a real-world studio selected by the user terminal to a virtual image rendering module, An effect editing module for providing at least one of an image effect or an acoustic effect to the real-world image photographed by the photographing equipment; and an effect editing module for providing at least any one of a plurality of time lines that can be generated in the content production project under the control of the user terminal A preset timeline division command in which a division of a timeline is predefined corresponding to a scene or a cut to be recorded in one or more timeline and a preset timeline division command in which, Insert into the timeline And a preset camera control command in which a control command for controlling the operation of the camera is defined in advance, and a memory control module for storing an image included in the real-world image according to the control of the user terminal, An online binding module for searching and downloading at least one of resource data of at least one of video, image effect, and sound effect, and an image binding module for receiving image, video, image effect, A resource loader module for searching and providing at least one resource data of at least one of the virtual studio and the 3D effect object and the sound effect, and a 3D loader module for performing 3D modeling of the virtual studio and the 3D main object according to the camera control command generated by the camera control moduleA virtual image rendering module for adjusting each output value of the data and a prompt information generating module for generating prompt information to be outputted to the prompt module included in the content production studio device under the control of the user terminal .

The real-time studio system capable of producing contents using the control of the virtual studio according to the embodiment of the present invention is characterized in that the prompt module outputs a virtual image generated using the content production project.

In the real-world studio system capable of producing contents using the control of the virtual studio according to the embodiment of the present invention, the control command generated in the camera control module is transmitted to the real-world photographing equipment in the real- And is set to the same control range so as to control the photographing equipments.

In a real-world studio system capable of producing contents using the control of a virtual studio according to an embodiment of the present invention, the resource storage module may store an image, a moving picture, an audio, An effect, and an acoustic effect, and further stores at least one of the resource data.

According to the above configuration, a real-world studio system and a method of operating a real-time studio system capable of producing contents using the control of a virtual studio according to the present invention include a control instruction for controlling the operation of an image- And real-time editing is performed on the photographed real-world image, so that the moving image content can be automatically produced. [0031] According to an aspect of the present invention, Effect.

A real-world studio system and a method of operating a virtual studio capable of producing contents using the virtual studio control according to the present invention include generating a 3D virtual studio and a 3D main object synchronized with a performer (subject) located in a real- By simulating the production of the content in the real-world studio through the virtual image including the 3D main object in the 3D virtual stage based on the presentation information recorded in the content production project, the real- And anticipated editing contents can be determined in advance.

A real-world studio system and method for producing contents in a real-world content production studio through the control of a virtual studio according to the present invention includes editing information for controlling the virtual studio and editing virtual images generated through the virtual studio A content production project in which information is recorded automatically controls real-world photographic equipment in a real-world content production studio and real-time automatic editing of a real-world image photographed in a controlled real-world studio to produce a video content, It is possible to generate a virtual image through a content production project in which editing information is recorded, anticipate the video content to be produced in advance, and produce the video content through the content production project.

A real-world studio system and a method of operating a real-studio system that can produce contents using the control of a virtual studio according to an embodiment of the present invention include a real-world studio suitable for a kind or genre of contents to be produced from a plurality of real- So that it is possible to select a real-world studio suitable for contents to be produced by a user from real-world studios provided in various forms.

A real-world studio system and a method of operating a virtual studio capable of producing contents using the control of a virtual studio according to an embodiment of the present invention include a location of a photographing equipment, a musical instrument, a performer, and the like that can be configured on a stage of a real- Setting and the like can be set in advance by using a virtual studio corresponding to a real-world studio.

FIG. 1 is a block diagram illustrating a real-world studio system according to an embodiment of the present invention.
2 is a block diagram illustrating a real-world studio system according to another embodiment of the present invention.
3 is a block diagram illustrating a content production project generation apparatus according to an embodiment of the present invention.
4 is a diagram illustrating a content production studio controller according to an embodiment of the present invention.
FIG. 5 is a diagram illustrating a content production studio apparatus according to an embodiment of the present invention. FIG.
FIG. 6 is a flowchart illustrating a content production method using a real-world studio system according to an embodiment of the present invention; FIG.
FIG. 7 is a flowchart illustrating a method of selecting a virtual studio using a content production project generation apparatus according to an embodiment of the present invention; FIG.
FIG. 8 is a flowchart for illustrating a method for recording presentation information in a content production project using a content production project generation apparatus according to an embodiment of the present invention; FIG.
FIG. 9 is a flowchart illustrating a method for recording edit information in a content production project using a content production project generation apparatus according to an embodiment of the present invention; FIG.
FIG. 10 is a flowchart illustrating a method of reserving a real-world studio using a content production project generation apparatus according to an embodiment of the present invention; FIG.
11 is a flowchart illustrating a content production method using a content production studio controller and a content production studio apparatus according to an embodiment of the present invention.
FIG. 12 is an exemplary view for showing a content production project creation screen according to an embodiment of the present invention; FIG.
FIG. 13 is an exemplary view illustrating an input screen of content production project information according to an embodiment of the present invention; FIG.
FIG. 14 is an exemplary view illustrating an information input screen of a real-world studio according to an embodiment of the present invention; FIG.
15 is an exemplary view for showing an existing template selection screen according to an embodiment of the present invention;
FIG. 16 is an exemplary view illustrating an output screen of a virtual studio according to an embodiment of the present invention; FIG.
17 is an exemplary view illustrating a 3D main object output screen according to an embodiment of the present invention;
18 is an exemplary view for selecting a background image to be included in a content production project according to an exemplary embodiment of the present invention;
FIG. 19 is an exemplary view illustrating a screen for setting a background image on a timeline of a content production project according to an exemplary embodiment of the present invention; FIG.
20 is an exemplary view illustrating a screen for setting a scene on a timeline of a content production project according to an embodiment of the present invention;
FIG. 21 is an exemplary view illustrating a screen for setting camera movement on a timeline of a content production project according to an embodiment of the present invention; FIG.
FIG. 22 is an exemplary view illustrating a screen for setting an effect (image) on a timeline of a content production project according to an embodiment of the present invention; FIG.
23 is an exemplary view showing a screen for setting an effect (subtitle) on a timeline of a content production project according to an embodiment of the present invention,
24 is an exemplary view showing a screen for setting a sound effect on a timeline of a content production project according to an embodiment of the present invention;
FIG. 25 is an exemplary view showing a screen for reserving a real-world studio according to an embodiment of the present invention; FIG.
26 is an exemplary diagram illustrating a content creation screen using a content creation project according to an embodiment of the present invention.

The terms and words used in the present specification and claims should not be construed in a conventional or dictionary sense and should be construed in accordance with the principles by which the inventor can define terms and concepts to best describe his invention And should be construed in accordance with the meaning and concept consistent with the technical idea of the present invention.

Therefore, the embodiments described in the present specification and the configurations shown in the drawings correspond to the preferred embodiments of the present invention and do not represent all the technical ideas of the present invention. Therefore, There may be equivalents and variations.

The specification that " comprises " any element in any specification throughout the specification does not exclude other elements, but may also include other elements, unless the context clearly indicates otherwise. The terms " module, " " part, " " system, " and the like, which are described in the specification, mean a unit for processing at least one function or operation, And may be included in one device or in another device.

Hereinafter, a real-world studio system and a method of operating a virtual studio capable of producing contents using the virtual studio according to the present invention will be described with reference to the drawings.

As shown in FIGS. 1 to 11, the real-world studio system 1000 generates a content production project for producing video content according to a plurality of timelines that can be generated, The production information of the to-be-produced contents and the editing information of the to-be-produced contents including the control command of the real-world photographing equipment - including the camera, the microphone, the illumination and the prompt monitor - are recorded in the contents production project based on the timeline, Virtual camera, virtual microphone, and virtual lighting of a virtual studio by rendering a 3D modeling data of a virtual studio by creating a virtual studio corresponding to a real-world studio The virtual image including the virtual stage And a content production project generation apparatus 100 for outputting the content production project.

The real-world studio system 1000 performs a re-play of the content production project transmitted from the content production project generation apparatus 100 and transmits the presentation information recorded in the content production project to the content production studio apparatus 300 And a content production studio controller 200 for receiving the real-world image photographed by the real-world photographic equipment using the provided presentation information and performing editing according to the editing information in the real-world image according to the time line, .

The real-world studio system 1000 includes a content production studio device 200 for receiving a control command recorded in the content production project from the content production studio controller 200, generating a control signal corresponding to the control command, and providing the control signal to the real- 300).

The real-world studio system 1000 further includes a content recorder 400 that receives each frame of the real-world image to which editing information is applied by the content production studio controller 200 and completes the production of the moving image content.

6, a method of operating the real-world studio system 1000 includes a content production project creation apparatus 100, a real-world photographic apparatuses-cameras 311, 312 and 313, microphones 321 and 322, Editing of production schedule contents including control instructions of real-world photographing equipment in any real-world studio of a plurality of real-world studios including the lighting 331, 332 and the prompt monitor 370 and a project creation step (S100) of receiving a project creation command for creating a content creation project that can set edit information for post production, from a user terminal, and creating a content creation project in the form of a file.

The operation method of the real-world studio system 1000 is such that the content production project generation apparatus 100 receives the selection information of the real-world studio for selecting the real-world studio controlled according to the content production project from the user terminal 500, And a studio selection step (S200) of recording studio information of a real-world studio to a content production project.

The method of operating the real-world studio system 1000 includes the steps of rendering the 3D modeling data of the real-world studio selected by the content production project generation apparatus 100 to create a virtual studio corresponding to the real-world studio, A studio setting step 300 for outputting, to the screen of the user terminal 500, a virtual stage, which is an output form of a virtual camera in which a virtual camera is photographed, including a virtual camera, a virtual microphone, and virtual lighting .

The operation method of the real-world studio system 1000 is such that the content production project generation apparatus 100 receives selection information of a 3D main object to be included in a virtual stage among a plurality of 3D main objects from the user terminal 500, And a stage (S400) of outputting a first virtual image obtained by synthesizing the object to the virtual stage on the screen of the user terminal.

The operation method of the real-world studio system 1000 is such that when the content production project creation apparatus 100 inputs a camera control command for changing the camera movement from the user terminal 500, When a replay is performed on at least one of the plurality of timelines that can be generated in the content production project, and replay is performed on the timeline, the virtual stage and the 3D And outputting a second virtual image generated by adjusting an output value of the main object to a screen of the user terminal (S500).

The operation method of the real-world studio system 1000 is such that when the effect production command for generating the effect production result is input from the user terminal 500, Is recorded in one of the time lines that can be generated in a plurality of times in the content production project, and when the replay is performed for the time line, the selected effect corresponding to the edit information recorded in the time line And a project editing step (S600) of outputting the edited third virtual image to the screen of the user terminal (500).

The operation method of the real-world studio system 1000 is a method in which the content production studio controller 200 controls the real-world imaging apparatuses provided in the real-world studio using the presentation information including the camera control commands recorded in the time line of the content production project (S800) for creating edited movie contents by applying the effect corresponding to the effect command included in the edit information recorded in the time line of the content production project to the real world image .

The method of operating the real-world studio system 1000 may be such that the content production project creation apparatus 100 saves a project input from the user terminal 500 in step S600 after the project editing step and in step S800 And a studio reservation step (S700) of storing the content production project according to the command and reserving the real-world studio according to the studio information recorded in the content production project.

Hereinafter, FIG. 1 through FIG. 11 will be described with reference to FIGS. 12 through 26.

The content production project generation apparatus 100 may be included in the real-world studio system 1000 as shown in FIG. In addition, if necessary, it may be provided outside the real-world studio system 1000, as shown in FIG.

When the content production project generation apparatus 100 is configured inside the real-world studio system 1000, a user who intends to produce the content transmits a communication network such as the Internet to the content production project generation apparatus 100 through the user terminal 500 It is possible to create a content production project by accessing it.

In the case where the content production project generation apparatus 100 is provided outside the real-world studio system 1000, a program for creating a content production project is installed in the user terminal 500 of the user who intends to produce the content, The user terminal 500 may be used as the content production project generation apparatus 100, as described above.

The content production project generation apparatus 100 described below is provided in the real-world studio system 1000 and includes a content generation system 100 shown in FIG. 1 as an embodiment in which the user terminal 500 accesses the content production project generation apparatus 100 using a communication network, .

The content production project control module 105 of the content production project generation apparatus 100 provides the content production project creation screen 160 of FIG. 12 to the user terminal 500 in step S100, A content production project is created using the content production project generation information.

In the content creation project creation screen 160 of FIG. 12, a new content creation project can be created, and a file of a previously produced content creation project can be executed to modify a previously produced content creation project. 15, the content production project control module 105 provides an existing template selection screen 163, and when a template to be used is selected by the user The content creation project corresponding to the selected template is executed and output to the user terminal 500 is controlled.

13, when the content creation project control module 105 receives a request to create a new content creation project from the user terminal 500, the content creation project control module 105 creates a content creation request And controls the project information input screen 161 to be output to the user terminal 500. The content production project information input screen 161 receives information such as 'project title', 'project type', and 'project sharing', which are information of a content production project.

The content production project information input screen 161 displays summary information of contents to be produced, such as 'description', 'main theme', character no. And the like.

The content production project information input screen 161 receives information on the background sound of the content to be produced. The background sound is the sound to be output from the produced content, and the sound effect except for the background sound is set in the editing process in the future.

The background sound is located in the BGM area of the time line area 180 and is output to the screen of the user terminal 500, as shown in FIGS. 18 to 24. When the virtual image rendered by the virtual image rendering module 145 is outputted to the user terminal 500, the BGI / BGM processing module 140 sets the background sound to the content production project To the control module 105 and controls the output to the user terminal 500 together with the virtual image.

When the creation of the content production project is completed in step S100, the content production project control module 105 generates a content production project information input screen 162 for receiving the real-world studio selection and provides it to the user terminal 500, The studio selection information input to the content production project information input screen 162 of the terminal 500 is received (S210), and the studio information of the selected real-world studio is recorded in the content production project according to the received studio selection information (S220) .

Since the real-world studios are formed so that a plurality of real-world studios can capture contents of different sizes and different genres, the user can select a real-world studio suitable for contents to be produced. For example, dedicated studios for singing and dedicated studios for dancing may have differences in acoustics and there may be differences in stage size. Real-world studios for singing should have a high-quality sound system even if the stage size is small. Real-world studios for dancing do not require high-quality sound facilities, but it is desirable that the size of the stage should be of a size such that a dance for content production can be performed.

In operation S220, a virtual studio corresponding to a real-world studio for creating a content is selected using the studio information on the real-world studio selected in operation S230.

In step S230, the virtual studio corresponding to the real-world studio is selected by rendering the 3D modeling data corresponding to the real-world studio selected in step S220 among the 3D modeling data corresponding to the real-world studio.

Each of the 3D modeling data corresponding to the real-world studio is produced in the same form as that of the real-world studio, and the virtual studio, in which the 3D modeling data is rendered and output to the user terminal 500, The virtual studio is generated by rendering 3D modeling data corresponding to a real-world studio, and the virtual image rendering module 145 of the content production project apparatus 100 renders 3D modeling data of a real- And controls the output to the user terminal 500 to show the same effect as that taken by using the photographing equipment.

 The virtual shooting devices are composed of a virtual camera, a virtual microphone, virtual lighting, and the like, and are set to have the same control range so that they are output in the same manner as those shot in real-world photography equipment provided in a real-world studio corresponding to a virtual studio.

14, the user inputs the studio information of the real-world studio selected on the real-world studio information input screen 162 transmitted from the content production project control module 105 of the content production project generation apparatus 100 to the user terminal (500), the virtual studio corresponding to the real-world studio is controlled to be output to the user terminal (500) by using the inputted studio information.

Information such as 'city', 'studio type', and 'studio name', which are real-world studio information, can be input to the real-world studio information input screen 162 output to the user terminal 500.

The virtual image rendering module 145 of the content production project generation apparatus 100 receives the studio information of the real-world studio input on the real-world studio information input screen 162 of the user terminal 500, do.

In operation S300, the 3D modeling data of the real-world studio according to the studio information recorded in the content production project is rendered to have the same effect as that shot using the virtual photography equipment, and the rendered virtual studio is output to the user terminal 500 .

The 3D modeling data of a real-world studio is stored in a 3D modeling data storage module (not shown). The 3D modeling data is produced in proportion to the size of a real-world studio. do.

When a real-world studio is newly created, 3D modeling data for a virtual studio corresponding to a newly created real-world studio may also be newly produced and stored in a 3D modeling data storage module (not shown).

As shown in FIG. 16, the output screen 164 of the virtual studio rendered corresponding to the real studio is output to the user terminal 500. In the real world studio, the floor and back background of the virtual stage 164-1 are set to green or blue, which is mainly used as the chromakey key value, It is possible to form each surface with a different design from the real-world studio.

In operation S400, a 3D main object to be synthesized to the virtual stage 164-1 output to the user terminal 500 is set to configure the virtual stage 164-1. The 3D main object is also composed of 3D modeling data in the same manner as the virtual stage 164-1. The 3D main object is synthesized in the virtual stage 164-1 instead of the photographed object (performer) . A plurality of 3D modeling data is stored in a 3D modeling data storage module (not shown) so that the 3D main object corresponds to a plurality of shooting objects (performers) for the representation of various shooting objects (performers).

17, the 3D main object output screen 165 is provided with a 3D main object selection area 165-1 that can be selected by the user, and the virtual image rendering module 145 selects the 3D main object selection area 165-1, The 3D main object 165-2 is synthesized with the virtual stage 164-1 rendered in step S300 and output to the user terminal 500. [

The content production project control module 105 of the content production project generation apparatus 100 generates a plurality of 3D main objects according to the number of characters input in the 'character no.' Of the content production project information input screen 161 of FIG. 13 And requests the virtual image rendering module 145 to render the virtual image. That is, the 3D main object is generated according to the number of shooting objects (performers) participating in the content production in the real-world studio, and is synthesized in the virtual stage 164-1.

The virtual image rendering module 145 of the content production project generation apparatus 100 may determine whether the 3D main object 165-2 is located at a specific position in the bottom surface of the virtual stage 164-1, ), Adjusts the size of the 3D main object 165-2 in accordance with the size of the 3D main object 165-2, and outputs the synthesized result to the virtual stage 164-1.

For example, when the 3D main object 165-2 is positioned on the front side of the virtual stage 164-1 and when the 3D main object 165-2 is positioned on the rear side of the virtual stage 164-1, The production project creation apparatus 100 determines the position of each 3D main object 165-2 in the virtual stage 164-1 and adjusts the size of the 3D main object 165-2 according to the determined position And controls the rendering to be output to the user terminal 500.

In step S500 of recording the production information in the content production project using the content production project generation apparatus 100, the timeline scene editor module 110 generates a content production project in accordance with the timeline division information received from the user terminal 500 And dividing the included timeline into a content production project (S510).

In step S500, the camera PTZM control module 115 records the production information including the control command of the real-world photographing equipment received from the user terminal 500 in the content production project corresponding to at least one timeline of the divided timelines (S520).

In operation S500, the virtual image rendering module 145 generates a 3D main object 165-2 based on the virtual stage 164-1 and the 3D main object 165-2 output to the user terminal 500 according to the control command of the real- (Step S530).

 Step S500 further includes a step S540 of outputting the virtual stage 164-1 and the 3D main object 165-2 to the user terminal 500 according to the adjusted output value by the content production project control module 105 .

In step S510, a timeline is divided through the timeline scene editor module 110 of the content production project generation apparatus 100. The timeline, which is formed according to the sequential order in which content is produced, is divided into a scene of shooting, And may be divided into one or more scenes or cuts in correspondence with each other. The command for dividing the timeline is received by the timeline scene editor module 110 of the content production project generation apparatus 100, and the timeline scene editor module 110 receives the timeline division command transmitted from the user terminal 500, Performs the time line division using the received time line division command.

When a scene is switched through camera switching in shooting for producing contents, the scene is classified into one cut or scene, and one selected from images captured simultaneously using a plurality of cameras according to the classification Is set to select an image corresponding to the selected scene or cut so as to be included in the produced content.

20, in the scene setting screen 168 output to the user terminal 500, the user can select a scene from among the various scene types outputted in the scene selection area 168-1, Selects a scene to be used in the timeline, and sets the selected scene in the timeline area 180.

The scene selected by the user is input to the timeline through operations such as drag and drop of the input device and the length of time of the selected scene is determined by dragging the input device, Or by increasing or decreasing the area of the scene 168-2 recorded on the timeline.

Step S520 is a process of recording the production information including the control command of the real-world photographing equipment in the content production project through the camera PTZM control module 115 of the content production project generation apparatus 100. When the user creates the content in the real- Such as cameras 311, 312, and 313, microphones 321 and 322, lights 331 and 332, and a prompt monitor 370, Record.

In step S510, the lighting setting, the microphone setting, the camera setting, and the like of the photographing room (real-world studio) are set in step S520 according to the type of the scene set on the divided timeline.

The lighting setting can set on / off, position, direction, illuminance, and color of each lighting according to the number of the form lighting of the illumination of the real-world studio, and the set illumination is set in step S510 It is preferable to set it so as to correspond to a scene.

The microphone setting sets the microphone on / off of each shooting object (performer), the microphone sound input size, and the like when a plurality of shooting objects (performers) exist in a real-world studio. The microphone setting is preferably set to correspond to the scene set in step S510, but may be set according to the microphone usage time of each shooting object (performer).

The camera settings are set to control PTZM (pan, tilt, zoom, move) of one or more cameras provided in the real-world studio for shooting objects (performers). The camera provided for photographing in the real-world studio is provided with a camera control command for pan in left and right rotation, tilt in up and down rotation, zoom in image size control, and movement in up and down and left and right movement It is possible to perform an operation corresponding to each scene.

The setting of each of the real-world image capturing devices recorded in the time line is controlled in accordance with the sequential order of the time line of the content production project while the image capturing is being performed in the real-world studio.

21, the camera control setting screen 169 output to the user terminal 500 by the content production project creating apparatus 100 includes a camera control selection area 169-1 and a selected camera control selection area 169-1 And camera control 169-2 recorded in the time line in which the camera control selected in the time line area 180 is recorded in the time line area 169-1.

The camera control selection region 169-1 outputs information for selecting a pan, tilt, zoom, and move of the camera, and allows the user to input The camera PTZM control module 115 of the content production project creation apparatus 100 records the selected camera control command in the time line area 180. [

The virtual image rendering module 145 of the content production project generation apparatus 100 adjusts the output values of the virtual studio, the 3D main object, and the background image according to the camera control command 169-2 recorded in the time line, And the content production project control module 105 controls the rendered virtual image to be output to the user terminal 500 using the camera control screen 169-3.

The camera control command generated by the camera PTZM control module 115 is set to a control range for controlling the cameras 311, 312, and 313 among the real-world imaging devices, and the cameras 311, 312, and 313 can not perform operations out of the control range.

Among the real-world photographing equipments provided in a real-world studio, the camera is coupled to a movable module implemented to enable PTZM movement, and a pan and a tilt of the camera are operated by a rotation module, And is operated by a movable module including an up, down, left, right or diagonal line. The zoom can be operated according to the movement of the lens included in the camera.

Among the camera control commands at this time, the control commands for pan and tilt drive the rotation module, and the control command for move drives the movement module so that the operation of the camera for shooting can be performed have.

The rotation module for the operation of the camera is preferably a hinge type module capable of up-and-down rotation and left-right rotation, but it is not limited thereto. The movement module may be a rail type But it is not limited thereto.

The scene 168-2 recorded in the time line in FIG. 21, the background image 167-2 recorded in the time line, and the camera control command 169-2 recorded in the time line are recorded in the time line area 180 The virtual image rendering module 145 of the content production project generation apparatus 100 applies the 3D main object, the background image, and the camera control command according to the information of each timeline to the selected camera control screen 169-3 The output of the virtual image can be adjusted, and the output virtual image can be output to the user terminal 500.

In step S520, the presentation information recorded in the content production project includes a setting of a background image to be used for the content.

In the actual scene captured in the real-world studio, a shooting object is synthesized with a background image through a chroma key shooting, and a background image to be synthesized with the shooting object can be set in the presentation information of the S520 process. The background image may be set to at least one of a still image and a moving image, and is preferably set corresponding to a scene set in operation S510.

The background image may be an actual photographed real image, a 2D graphic image, and a 3D graphic image.

18, the background image selection screen 166 output to the user terminal 500 by the BGI / BGM processing module 140 of the content production project generation apparatus 100 includes a background image selection area 166- 2 and the background image 166-1 selected in the background image selection region 166-2 are synthesized with the virtual stage 164-1 and the 3D main object 165-2.

As shown in FIG. 19, the background image 166-1 selected in FIG. 18 is recorded in the timeline in which the background image can be input in the time line area 180 by the BGI / BGM processing module 140 of FIG. , And the background image 167-2 to be recorded on the time line is set to correspond to the scene set in step S510.

The background image to be recorded in the timeline may be recorded in more than one scene according to the scene information recorded in the content production project in step S510. The recorded background image may include a 3D main object rendered and a BGI / BGM processing module 140 And is output to the user terminal 500 (S540).

That is, in step S530, when the production information is recorded in the content production project, the virtual image rendering module 145 and the BGI / BGM process are performed according to the recorded production information (including control commands of the real- Module 140 adjusts the output values of the virtual stage 164-1 and the 3D main object 165-2.

Thereafter, in step S540, the content production project control module 105 of the content production project generation apparatus 100 controls the virtual terminal to reflect the adjusted output value to the user terminal 500. [

In step S600 of recording edit information in the content production project using the effect processing module 120 of the content production project generation apparatus 100, the user terminal 500 And a step (S610) of recording edit information including the received effect command in the content production project.

The step S600 further includes a step (S620) of synthesizing the effect on the virtual image including the virtual stage 164-1 and the 3D main object 165-2 in accordance with the effect command included in the editing information.

The step S600 further includes a step S630 of outputting the synthesized virtual image to the user terminal 500.

In step S610, editing information including an effect command for editing a real-world image photographed in a real-world studio through the effect processing module 120 of the content production project generation apparatus 100 is recorded in a content production project. That is, an effect command to be edited according to each timeline is transmitted to the content production project generation apparatus 100 through the user terminal 500 so that the user can edit the real-world video shot in the real-world studio in real time, And the content production project control module 105 of the project generation apparatus 100 records it in the timeline of the content production project.

The effect command recorded on the timeline can be pre-applied to the virtual image through the replay of the timeline, so that the position and size of the effect to be applied can be simulated in advance, and the position and size of the effect determined by the user through the simulation And finally recorded. Thereafter, when a real-world image is photographed, the corresponding effect recorded on the time line is applied to the real-world image according to the decided position and size.

The effect commands transmitted from the user terminal 500 and recorded in the timeline of the content production project include video effects and sound effects.

Video effects include dissolve, overlap, wipe, fade, special effect (SFX), visual effect (VFX), digital effect (DFX) ≪ / RTI >

The acoustic effects include the effects of a compressor, a gate, an equalizer, a filter, a reverb, a delay, a de-esser, a chorus, a flanger, Harmonizers, pitch shifts, and the like.

In step S620, the effect processing module 120 of the content production project generation apparatus 100 generates a virtual stage 164-C that is output to the user terminal 500 according to the effect command included in the edit information recorded in the time line of the content production project, 1) and the 3D main object 165-2 are synthesized with the effect according to the effect command.

Thereafter, the content production project control module 105 of the content production project generation apparatus 100 transmits the virtual image synthesized with the effect to the user terminal 500, and outputs the virtual image synthesized with the effect to the user terminal 500 .

22, an effect setting screen 170 output to the user terminal 500 includes an effect selection screen 170-1 and an effect (image) outputting a selected effect (image) An output screen 170-3 and an effect 170-2 recorded in a time line in which an effect is recorded in the time line area 180. [

The effect selection region 170-1 outputs one or more effect (video) commands that can be composited into a real-world image on the selected timeline through the user terminal 500. [ When a user selects an effect command for synthesizing a real-world image in the effect selection region 170-1 and records it in the time line region 180, the effect processing of the content production project generation apparatus 100 The module 120 outputs the effect (video) corresponding to the recorded effect (video) command on the effect (video) output screen 170-3.

The image output to the effect (video) output screen 170-3 includes a scene 168-2 recorded on the corresponding time line, a background image 167-2 recorded on the time line, a camera control An effect (image) selected in the virtual image including the virtual stage 164-1 and the 3D main object 165-2 together with the instruction 169-2 is synthesized and output.

23, an effect (caption) setting screen 171 to be output to the user terminal 500 includes an effect (caption) selection area 171-1 and an effect (caption) for outputting a selected effect (caption) And an effect (subtitle) 171-2 recorded on the time line in which the effect (subtitle) is recorded in the output screen 171-3 and the time line area 180. [

The effect (subtitle) selection area 171-1 outputs one or more effect (subtitle) commands that can be combined with a real-world image photographed on the selected time line through the user terminal 500. [ When a user selects an effect (caption) command for compositing a real-world image in the effect (caption) selection area 171-1 and records the effect in the time line area 180, the effect processing The module 120 outputs the effect (caption) corresponding to the recorded effect (caption) command to the effect (caption) output screen 171-3.

The image output to the effect (caption) output screen 171-3 is divided into a scene 168-2 recorded on the corresponding time line, a background image 167-2 recorded on the time line, a camera control (Subtitle) selected in the virtual image including the virtual stage 164-1 and the 3D main object 165-2 together with the command 169-2 and the effect recorded in the time line are synthesized and output .

24, the sound effect setting screen 172 output to the user terminal 500 includes a sound effect selection area 172-1, a virtual image output screen 172-3 in which a virtual image is output, And a sound effect 172-2 recorded in the time line in which the sound effect is recorded in the line area 180. [

The sound effect selection region 172-1 outputs one or more sound effect commands that can be combined with a real-world image photographed on the selected timeline through the user terminal 500. [ When the user selects a sound effect command for synthesizing the real-world image in the sound effect selection region 172-1 and records the selected sound effect command in the timeline region 180, the effect processing module 120 of the content production project generation apparatus 100 Outputs a sound effect corresponding to the recorded sound effect command using an audio device (not shown).

When a re-paly of the timeline is requested from the user terminal 500 in steps S500 and S600 of recording the presentation information and the editing information in the timeline area 180 of the content production project, The project control module 105 provides the virtual image to the user terminal 500 in accordance with the sequence of the requested timeline in which the presentation information and the editing information recorded in the timeline of the content production project are respectively applied. At this time, if at least one of a background image, a background sound, a virtual studio, and a 3D main object is set in the content production project, the virtual image includes a set item.

The resource loader module 130 of the content production project generation apparatus 100 may request the 3D modeling data of the virtual studio stored in the 3D modeling data storage module (not shown) 3D modeling data, and the like to the virtual image rendering module 145.

When there is a request from the user terminal 500, the resource loader module 130 loads resource data such as images, moving pictures, audio, video effects, and audio effects stored in the resource storage module 150, The BGM processing module 140, and the like.

The online binding module 135 downloads the resource data such as images, moving pictures, audio, video effects, etc., which are not stored in the resource storage module 150, The user terminal 500 or the content production project generation module 100 and an online resource storage module (not shown) provided on-line to interconnect resource data such as a requested image, video, audio, To the respective modules such as the BGI / BGM processing module 140 and the like.

The resource storage module 150 stores preset data in which at least one of a time line division command, a presentation information, and editing information is preset. Preset data is preset for timeline division, camera control command, effect command, etc. for specific shooting techniques when contents are produced in a real-world studio. Accordingly, the user does not newly generate a timeline and a camera control command in the timeline of the content production project for the content to which the specific shooting technique is applied, loads the preset data stored in the resource storage module 150 Can be recorded in a content production project.

The prompt information generation module 125 of the content production project generation apparatus 100 generates prompt information to be provided to the photographed objects (performers) when the content is produced in the real-world studio when a request is made from the user terminal 500 do. The prompt information includes text information such as lyrics, lyrics, and the like, and includes image information such as photographs and moving pictures necessary for photographing objects (performers) while photographing for content production.

Although not shown in the drawing, the prompt information generated by the prompt information generation module 125 is also recorded in the timeline area 180. When a re-play of the content production project is performed, the prompt information generated by the user terminal 500 / RTI >

In addition, the prompt UI providing module 230 of the content production studio controller 200 provides a prompt UI so that a prompt UI is output to the prompt monitor 370 of the real-world studio.

Thereafter, the content production project generation apparatus 100 stores the content production project using the content production project control module 105, and performs step S700 of reserving the real-world studio.

In step S700, the content creation project control module 105 stores the content creation project in the content creation project creation apparatus 100 according to the content creation project save command transmitted from the user terminal 500 by the content creation project control module 105 (S710).

In operation S700, the content production project control module 105 reserves the real-world studio according to the studio information recorded in the content production project (S720).

The step S700 further includes a step (S730) of uploading (transmitting) the stored content production project to the content production studio controller 200 capable of controlling the real-world photographic equipment of the real-world studio reserved by the content production project control module 105 .

As shown in FIG. 25, a real-world studio reservation screen 173 for reserving a real-world studio corresponding to a virtual studio includes information of a shooting object (performer) participating in the production of contents. The information of the shooting object (performer) includes basic information such as an ID, a password, a name, a nickname of a shooting object (performer), an e-mail, and a telephone number.

The real-world studio reservation screen 173 includes studio information for specifying a real-world studio to reserve a real-world studio. The studio information includes information such as the local information of the real-world studio, the type information of the studio, and the name of the studio.

The real-world studio reservation screen 173 includes reservation information such as a reservation date and a reservation time for reserving a real-world studio.

The real-world studio reservation screen 173 includes studio upload file information for uploading a content production project necessary for producing contents in a real-world studio. And information for attaching a content production project necessary for content production in a real-world studio.

Meanwhile, the step S720 of reserving the real-world studio may be performed after the creation of the content production project is completed, but may be performed at any time before or during the creation of the content production project.

That is, the real-time studio can be reserved at any time after the step S220 of inputting the real-world studio information to create the content production project. If the real-time studio is reserved before the content production project is created, Uploaded content production project.

The step S800 of producing the content using the content production project uploaded to the content production studio controller 200 is performed in the following detailed process.

In step S800, the content production project execution module 210 executes a re-play of the content production project, and the content production studio device control module 220 of the content production studio controller 200 re- When the presentation information including the camera control command is sequentially transmitted to the content production studio device 300 in accordance with the time line of the content production project, the content production studio device 300 controls the real-world photographing devices provided in the real- , And a process () of photographing a real-world studio and generating a real-world image.

The step S800 further includes a step (S820) of synthesizing a background image according to the background image information to a real-world image photographed in a real-world studio when the content production studio controller 200 records background image information in a content production project .

In step S800, the content production studio controller 200 edits the real-world image synthesized with the background image using the effect command included in the edit information at the time in accordance with the time line of the content production project to be re- (S830).

In operation S800, the content production studio controller 200 generates a content by encoding a real-world image to be edited (S840).

The step S800 further includes a step S850 in which the content production studio controller 200 publishes the produced content on-line or transmits the content to the user terminal 500. [

At this time, each control command included in the presentation information of the content production project transferred to the content production studio apparatus 300 is transmitted to the control command signal generation module 360, and the control command signal generation module 360 receives the control commands Command to a control command signal suitable for real-world imaging equipment in a real-world studio, and transmits the command to each real-world imaging equipment.

26, the content production screen 174 using the content production project re-reproduced by the content production project execution module 210 has a real-life photographing area 174-1 and a real- A screen area 174-2 to which presentation information and editing information are applied, and a time line area 180. [

The real-world shooting region 174-1 includes a scene 618-2 recorded in the time line of the content production project and a scene 618-2 recorded in the time line by the content production studio device control module 220 of the content production studio controller 200. [ Command 169-2, and the like to the content production studio device 300, the content production studio device 300 controls the real-world image capture devices including the camera, the microphone, the illumination, and the prompt monitor A real-world image to be photographed is output.

The real-time AV composition module 240 of the content production studio controller 200 outputs only the photographed object in which the background color is removed by applying the chroma key technique in the real-world image photographed and transmitted in the real world studio, to the real-world photographed area 174-1 . Accordingly, a real-world studio that produces contents is configured with a color corresponding to a key value in the background of the studio so as to be able to capture a chroma key.

Referring to FIGS. 1 and 3 to 5, when a plurality of photographing cameras 311, 312, and 313 are provided in a real-world studio, a plurality of cameras 311, 312, and 313 Corresponding camera control timelines are formed. In each camera control timeline, camera control commands are recorded so that each camera can be controlled according to each scene recorded in the scene timeline.

The content production project execution module 210 of the content production studio controller 200 executes a re-play of the content production project and the content production studio device control module 220 receives the content production project execution module 210 from the content production project execution module 210 In the re-paly content production project, the presentation information including the camera control commands of the plurality of camera control timelines is transmitted to the content production studio device 300, And controls the cameras 311, 312, and 313 with the camera control commands recorded on the time line to transmit a plurality of real-world images according to the number of cameras to the content production studio controller 200 through the video signal switch 340.

At this time, the microphone control command, the illumination control command, and the prompt control command included in the presentation information are also used to control the respective real-world imaging devices.

Even when a plurality of microphones 321 and 322 and lights 331 and 332 provided in a real-world studio are provided, a plurality of time lines are formed in the contents production project, and control commands for controlling the timelines can be recorded. The content production studio apparatus 300 transmits control commands recorded in the time lines corresponding to the microphones 321 and 322 and the lights 331 and 332 to the microphones 1 and 321, Microphone 2 322, illumination 1 331, and illumination 2 332, respectively.

The content production studio device 300 receives only a real-world image obtained by capturing a scene corresponding to a scene in response to a control command of a scene timeline, and the content production studio controller 200 receives other scenes A real-time image of another photographed camera is controlled by a scene timeline so that a plurality of real-world images photographed by a plurality of cameras 311, 312, and 313 can be selectively transmitted so that the content recorder 400 can receive the real- Command to the video signal switch 340.

For example, if a scene control command sequentially recorded in a scene timeline is shot in the camera 1 311, the camera 3 313, the camera 2 312, and the camera 1 311 , The content production studio device 300 generates each of the real-world images as content production so that the real-world images of the cameras 311, 312, and 313 corresponding to the scene control command of the received time line can be selectively used. Can be selectively transmitted to the studio controller (200) and the content recorder (400).

Also, the content production studio apparatus 300 transmits a control command of a scene timeline or a control command of a plurality of mic timelines to the audio signal mixer / switch 350.

The audio signal mixer / switch 350 transmits the real-time sound generated by each of the microphones 321 and 322 to the content production studio controller (not shown) using the control command of the received scene timeline or the control command of the plurality of microphone timelines 200 and the content recorder 400. The content-

Also, the content production studio apparatus 300 can transmit the control command of each illumination time line to the illumination 1 331 and the illumination 2 332, so that the illumination necessary for the real-world image photographed in the real-world studio can be controlled have.

The video signal switch 340 receives real-world images photographed by the camera 1 311 through the camera 3 313 provided in a real-world studio from each camera and outputs a scene received from the content production studio apparatus 300, The real-world image photographed by the camera corresponding to the scene to be included in the content produced according to the control command of the timeline is transmitted to the content production studio controller 200. Each of the remaining real- (400).

For example, when a plurality of scenes recorded at time-sequentially on the timeline are images photographed by the camera 1 311, the camera 3 313, and the camera 2 312, the video signal switch 340 The real-time images photographed by the camera 1 311 are transmitted to the content production studio controller 200 in accordance with the order of the scene timelines and the respective real-world images of the remaining cameras 2 312 and 3 To the content recorder (400).

Thereafter, the video signal switch 340 transmits the real-world image photographed by the camera 3 313 to the content production studio controller 200 according to the order of the scene timeline, 2 312 are transferred to the content recorder 400.

In addition, the video signal switch 340 transmits the real-world image photographed by the camera 2 312 to the content production studio controller 200 in accordance with the order of the corresponding scene timeline, 3 (313) are transmitted to the content recorder (400).

The real-world image transmitted from the video signal switch 340 to the content production studio controller 200 is output to the real-world shooting area 174-1 as shown in Fig.

The real-world image transmitted by the video signal switch 340 to the content production studio controller 200 in accordance with the order of the scene timeline is subjected to chroma key processing in the content production studio controller 200 to generate real- Only the shooting object excluding the background color of the shooting object is extracted.

Thereafter, the real-time AV composition module 240 of the content production studio controller 200 extracts the background of the real-world studio, and compares the real-world image in which only the photographed object exists and the background image included in the presentation information of the content production project, . In step S820, the real-time AV synthesis module 240 synthesizes the sound signal transmitted from the audio signal mixer / switch 350 together with the real-world image and the background image.

The prompt UI providing module 230 of the content production studio controller 200 transmits the virtual image generated using the content production project to the prompt monitor 370 provided in the content production studio apparatus 300 and the prompt Provides a prompt UI to display information.

The prompt UI provided to the prompt monitor 370 is controlled by the content production studio device control module 220 and displayed in a prompt created in the prompt information generation module 125 of the content production project generation device 100, Output information. The prompt UI outputs a virtual image according to the control commands recorded in the content production project through the control of the content production studio device control module 220.

The virtual image output to the prompt UI is generated by applying the presentation information and the editing information including the respective control commands recorded in the time line of the content production project to the virtual image in which the virtual studio and the 3D main object are rendered, 18 to 24 corresponds to a virtual image.

The effect application module 250 of the content production studio controller 200 applies (combines) the effect corresponding to the control command recorded in each effect time line of the content production project in real time to the real-world image synthesized with the background image in step S820, S830 < / RTI >

Applying effects is a process of performing additional editing on real-world images and sound such as video effects, sound effects, subtitles, etc., and is performed by using real time shooting equipment of real-world studios in accordance with each frame unit of the photographed real- (Synthesize) the effect control command recorded on the line.

Accordingly, each frame of the real-world image to which the effect is applied according to the timeline in the effect application module 250 is transmitted to the content recorder 400 by the video frame providing module 260.

The effect application module 250 applies the effects recorded in the respective time lines of the content production project, and the effects of applying the effects to the individual frames of the real-world image are different according to the types of applied effects and the number of applied effects . That is, when an individual frame of the real-world image to which the effect is applied is delayed by the effect application time and transmitted to the content recorder 400, when the final content is streamed on-line in real time in the content recorder 400, The video frame providing module 260 acts as a buffer when the individual frames to which the effects are applied in the effect applying module 250 to the content recorder 400 so that the individual frames are not disconnected To be continuously transmitted to the content recorder (400).

The content recorder 400 receives each frame of the real-world image transmitted from the video frame providing module 260, real-encodes it, and performs step S840 of generating final contents.

In step S840 of performing real-time encoding on each frame of the real-world image, final contents are generated by determining compression of the real-world image, bit rate, frame rate, aspect ratio of the image, frame size, pixel aspect ratio,

Thereafter, the content recorder 400 performs step S850 of posting the generated final content on-line or providing the final content to the user terminal 500. The S850 process can be performed in real time automatically and can be performed only when a user's request occurs.

The online posting space where the final contents are posted is preferably a website that can post videos such as a social networking service (SNS), a video sharing site, etc., and is preferably posted with the user's consent.

The description of the invention in this specification is for illustrative purposes only, and the invention is not limited to these embodiments. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. .

100: Content production project generation device
105: Content production project control module
110: Timeline scene editor module 115: Virtual camera PTZM control module
120: effect processing module 125: prompt information generating module
130: resource loader module 135: online binding module
140: BGI / BGM processing module 145: Virtual projection rendering module
150: resource storage module 200: content production studio controller
210: Content production project execution module
220: Content production studio device control module
230: Prompt UI providing module 240: Real time AV compositing module
250: Effect applying module 260: Video frame providing module
300: Content production studio device 311: Camera 1
312: camera 2 313: camera 3
321: Microphone 1 322: Microphone 2
331: Lighting 1 332: Lighting 2
340: Video signal switch 350: Audio signal mixer / switch
360: Control command signal generation module 370: Prompt monitor
400: content recorder 500: user terminal

Claims (22)

The contents production project generation apparatus may be provided with the production information of the to-be-produced contents including the control command of the real-world photographing equipment in the real-world studio of any one of a plurality of real-world studios including real- A project generation step of receiving a project generation command for generating a content production project capable of presetting edit information for post production of content to be produced from a user terminal and generating a content production project in a file form;
Wherein the content production project generation device receives selection information of a real-world studio for selecting a real-life studio controlled according to the content production project from the user terminal, and records studio information of the selected real- Selection step;
The virtual studio, the virtual camera, the virtual microphone, and the virtual lighting of the virtual studio by rendering the 3D modeling data of the real-world studio selected by the content production project generation apparatus, A studio setting step of outputting, to a screen of the user terminal, a virtual stage, which is an output of a virtual camera taken by the virtual camera;
Wherein the content production project generation apparatus receives a selection information of a 3D main object to be included in the virtual stage among a plurality of 3D main objects from the user terminal and displays a first virtual image synthesized with the selected 3D main object on the virtual stage, A stage constructing step for outputting to a screen of the user terminal;
Wherein the content production project generation apparatus generates a plurality of production information including the camera control command in the content production project when a camera control command for changing the camera movement is input from the user terminal, And when the replay of the time line is performed, the output values of the virtual stage and the 3D main object are adjusted according to the presentation information recorded on the time line, And outputting the generated second virtual image to the screen of the user terminal;
Wherein when the effect production command for applying an effect to a real-world image photographed by the camera is input from the user terminal, the content production project generation apparatus generates edit information including the effect command, Wherein the second virtual image is recorded on one of the plurality of time lines, and when the replay is performed on the time line, the selected effect corresponding to the editing information recorded on the time line is synthesized with the second virtual image And outputting the edited third virtual image to a screen of the user terminal; And
A real-world image is captured as the content production studio controller controls the real-world photographic equipment provided in the real-world studio using the production information including the camera control command recorded in the time line of the content production project, And a content production step of producing an edited moving picture content by applying an effect corresponding to the effect command contained in the editing information recorded in the time line of the content production project to the real world video, A method of operating a real-world studio system capable of producing contents using the control of a studio.
2. The method of claim 1, wherein the real-
And a studio reservation step of registering the real-world studio capable of producing the moving image content using a content production project before the content production step. A method of operating a real-world studio system.
The method according to claim 1,
From the user terminal, time line division information for dividing at least one or more time lines of the plurality of time lines according to at least one of a scene or a cut, A time line segmentation step of recording segmentation information in the content production project; And
And a control command for controlling at least one of the camera, the illumination, and the microphone corresponding to at least one of the scene or the cut, And generating a control command of the real-world photographic equipment to be recorded on the line. The method of claim 1, further comprising:
The method according to claim 1, wherein in the studio selection step, the selected real-
And a screen for shooting a chroma key is provided on the screen of the virtual studio.
The method according to claim 1, wherein in the studio setting step,
Wherein the rendering is performed in proportion to the size of the real-world studio, and the contents can be produced using the control of the virtual studio.
6. The apparatus according to claim 5, wherein the control command for controlling the real-
Wherein the control command is set to a control range for controlling the real-world imaging devices, and the control command is compatible with the virtual imaging devices.
7. The apparatus as claimed in claim 6,
Wherein when the replay of the time line is performed in the project directing step, the output values of the virtual stage and the 3D main object corresponding to the expected shooting change are adjusted within the control range. A method of operating a real-world studio system capable of producing contents using the method of the present invention.
2. The method according to claim 1, wherein, in the stage construction step,
Wherein when the background image is selected from the user terminal, the 3D main object is synthesized with the background image, and the synthesized background image is output to the screen of the user terminal. How to operate a real-world studio system that can be produced.
9. The method according to claim 8,
2D, or 3D. A method of operating a real-world studio system capable of producing contents using a control of a virtual studio.
The content production method according to claim 1, wherein, in the stage construction step,
Wherein when the replay of the time line is performed, the background sound is reproduced and output to the user terminal when the user terminal further selects a background sound to be used for the content production, A method of operating a real-world studio system capable of producing contents using the method of the present invention.
2. The method according to claim 1,
Adjusting an output value of the 3D main object so that at least one of a position, a size, and a direction of the 3D main object is controlled in the virtual stage in the virtual studio according to a control command of the 3D main object transmitted from the user terminal The method of claim 1, further comprising the steps of: generating a content by using the control of the virtual studio.
4. The method according to claim 3, wherein in the control command generation step, the control command of the virtual illumination is transmitted to at least one of the plurality of timelines that can be generated in accordance with the illumination control command transmitted from the user terminal, A virtual lighting control command for controlling the virtual lighting of the virtual studio corresponding to at least one of the virtual lighting and the virtual lighting is recorded, and the lighting control command includes on / off, position, Wherein the virtual studio is capable of controlling at least one of the colors of the virtual studio and the color of the real studio.
The method of claim 1, wherein in the project editing step, the effect includes an image effect and an acoustic effect, and the image effect includes a dissolve, an overlap, a wipe, a fade, wherein the sound effect includes at least one of a compressor effect, a special effect, a SFX effect, a VFX effect, and a digital effect, and the sound effect includes at least one of a compressor, a gate, At least one of a filter, a reverb, a delay, a de-esser, a chorus, a flanger, a harmonizer, and a pitch shift The method of claim 1, wherein the virtual studio is a virtual studio.
The method according to claim 2, wherein, in the studio reservation step,
When the reservation of the real-world studio is completed, the content production studio controller for controlling a content production studio device that generates a control signal for controlling the real-world photography devices included in the real- And transferring the project to the virtual studio, wherein the virtual studio transmits the project to the virtual studio.
15. The content production studio apparatus according to claim 14,
Wherein the control unit receives the control command from the content production studio controller and generates a control signal corresponding to the control command and transmits the control signal to the real-world imaging equipment. How the system operates.
And a control command of a real-world photographing equipment (including a camera, a microphone, a lighting, and a prompt monitor) of a real-world studio transmitted from a user terminal, and generates a content creation project for creating a video content according to a plurality of timelines And the editing information of the to-be-produced contents is recorded in the content production project based on the time line, and the 3D modeling data of the real-world studio is rendered, and the virtual studio corresponding to the real- And outputting a virtual image including a virtual stage, which is an output of a virtual camera in which the virtual camera photographs the virtual studio, of the virtual photography equipment - including a virtual camera, a virtual microphone, and virtual lighting Project creation chapter Tooth;
Reproducing the content production project transmitted from the content production project generation apparatus and providing the presentation information recorded in the content production project to the content production studio apparatus according to the replay, A content production studio controller for receiving a real-world image photographed by the real-world photographic equipment using information and performing editing according to the editing information in the real-world image according to the time line;
A content production studio device for receiving the control command recorded in the content production project from the content production studio controller and generating a control signal corresponding to the control command and providing the control signal to the real world photographic equipment; And
And a content recorder for receiving each frame of the real-world image to which the editing information is applied by the content production studio controller and completing the production of the moving image content. A real-world studio system.
17. The content production apparatus according to claim 16,
Synthesizing a 3D main object in the virtual stage of the virtual studio rendered in proportion to the size of the real-world studio, and adjusting an output form and size of the 3D main object in proportion to an output form and size of the virtual stage, And generating the virtual image in the form of being photographed using virtual photographing equipment, wherein the virtual studio is capable of producing contents using the control of the virtual studio.
18. The content production apparatus according to claim 17,
Wherein when the background image to be substituted for the virtual stage is input from the user terminal, rendering of the virtual studio output on the screen of the user terminal is stopped, the 3D main object is synthesized with the background image, And outputting the video to the screen of the user terminal.
17. The content production apparatus according to claim 16,
A time for generating a timeline division command for dividing at least one of the timelines corresponding to a scene or a cut among a plurality of timelines that can be created in the content production project under the control of the user terminal Line scene editor module;
A camera control module for generating a camera control command for controlling an operation of the camera among the real-world imaging devices provided in the real-world studio;
Storing 3D modeling data corresponding to a plurality of real-world studios and respective 3D modeling data of a plurality of 3D main objects to be synthesized in a virtual stage in a virtual studio generated through rendering of the 3D modeling data, A 3D modeling data storage module for providing each 3D modeling data for a virtual studio and a 3D main object corresponding to a real-world studio to a virtual image rendering module;
An effect editing module for providing at least one of an image effect or an acoustic effect to the real world image photographed by the real world photographic equipment under the control of the user terminal;
A division of a timeline is predefined corresponding to a scene or a cut in order to be recorded in at least one of the plurality of timelines that can be created in the content production project under the control of the user terminal A preset camera control command inserted in the timeline of the content production project according to a preset timeline division command and control of the user terminal and having a predefined control command for controlling the operation of the camera, A resource storage module for storing at least one of the resource blocks;
An on-line binding module for searching and downloading resource data of at least one of image, moving picture, image effect, and sound effect included in the real-world image under the control of the user terminal;
A resource loader module for searching for and providing at least one or more resource data of an image, a moving picture, an image effect, and an audio effect included in the real world image under the control of the user terminal;
A virtual image rendering module that adjusts each output value of each 3D modeling data of the virtual studio and the 3D main object according to the camera control command generated by the camera control module; And
And a prompt information generating module for generating prompt information to be output to a prompt module included in the content production studio device under the control of the user terminal. A real-world studio system.
20. The system of claim 19,
And outputting a virtual image generated using the content production project, wherein the virtual studio generates a virtual image using the virtual studio.
20. The apparatus of claim 19, wherein the control command generated by the camera control module comprises:
Wherein the virtual studio is set in the same control range so as to control real-world imaging equipment in the real-world studio and virtual imaging equipment in the virtual studio.
20. The apparatus of claim 19,
Wherein the control unit further includes at least one of resource, data, image, video, audio, image effects, and sound effects included in the real-world image under the control of the user terminal. A real-world studio system that can produce contents.

Images