KR102412595B1 - Method and device for providing special film production service using 3d character - Google Patents

Abstract

The special recording production service providing server may provide an actor virtual reality (VR) terminal with a virtual space and a virtual character image of a counterpart moving in the virtual space. The server may acquire a single image of an actor wearing the actor VR terminal from the camera terminal. The server may use the virtual character image and the single image to produce a special filmed image in which both the actor and the virtual character appear.

Description

Method and apparatus for providing special film production service using 3D characters

Embodiments of the present invention relate to a special recording production service providing server and an operating method thereof.

Special effects (　SFX, 　SPFX, 　FX) refer to effects used in the film, television, and entertainment industries to simulate a video event in a story. In modern cinema, special effects are often used to change elements of a previous film by removing, adding, or enhancing objects in a scene. Special effects are commonly used in high-budget films, but animation and compositing software have made it possible for amateur filmmakers to create and manipulate professional-quality effects.

Special effects are traditionally divided into scene effects and mechanical effects. In recent years, the difference between special effects and visual effects (VFX) has become more pronounced. Visual effects refer to optical effects and post-production, while special effects refer to mechanical effects.

Optical effect is a technology that creates and processes video or movie frames. Optical effects are made to look like photos, and actors can be inserted behind different backgrounds, or animals can talk as if they were talking.

Mechanical effects are usually done while shooting live action. You can create effects such as making the car drive itself, making the actor look like a monster, and so on.

Since the 1990s, computer-generated imagery (CGI) has been introduced as a special effect technology in earnest. CGI gives filmmakers greater control and allows them to create safer and more reliable effects at lower cost. As a result, many optical and mechanical effects techniques have become overpowered by CGI.

When a character is created with computer graphics, the actor acts while imagining the opponent in the empty space or takes chroma key shooting, which can be a burden for the actor to perform, and it can cause awkwardness in the finished work.

Embodiments of the present invention may provide a special recording production service providing server and an operating method thereof.

Technical problems to be achieved in the embodiments are not limited to those mentioned above, and other technical problems not mentioned may be considered by those of ordinary skill in the art from various embodiments to be described below. can

According to embodiments, there is provided a method for providing a special recording production service performed by at least one server, the method comprising: providing a virtual space to an actor virtual reality (VR) terminal and a virtual character image of a counterpart moving in the virtual space; ; obtaining a single image of an actor wearing the actor VR terminal from a camera terminal; and producing a special filmed image in which both the actor and the virtual character appear by using the virtual character image and the single image.

Here, the virtual space may be a virtual space implemented based on a real space in which the actor plays.

Here, the single image may include an image including the movement and dialogue of the actor in the real space.

Here, the server may further include the step of providing the special filmed image to the actor VR terminal.

Here, the server, the step of obtaining the analysis information related to the movement of the actor from the VR terminal; and providing information related to a part requiring correction of the actor's movement to the actor VR terminal based on the analysis information.

An embodiment of the present invention provides a server for providing a special recording production service, comprising: a processor; transceiver; and memory; comprising: providing an actor virtual reality (VR) terminal with a virtual space and a virtual character image of a counterpart moving in the virtual space; acquiring a single image of an actor wearing the actor VR terminal from a camera terminal; And it may be set to produce a special filmed image in which both the actor and the virtual character appear by using the virtual character image and the single image.

Here, the virtual space may be a virtual space implemented based on a real space in which the actor plays.

Here, the single image may include an image including the movement and dialogue of the actor in the real space.

Here, the server may be further set to provide the special filmed image to the actor VR terminal.

Here, the server obtains analysis information related to the movement of the actor from the actor VR terminal; And based on the analysis information, it may be further configured to provide information related to a part that needs to be corrected among the movement of the actor to the actor VR terminal.

According to embodiments, an actor can wear a VR terminal and act with a virtual character seen in the VR terminal, so that natural acting is possible. In the subsequent editing process, it is possible to synthesize the image of the actor based on the 3D virtual character information provided to the VR terminal, so that a natural special filmed image can be produced.

Effects obtainable from the embodiments are not limited to the effects mentioned above, and other effects not mentioned are clearly derived and understood by those of ordinary skill in the art based on the detailed description below. can be

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are included as part of the detailed description to aid understanding of the embodiments, provide various embodiments and, together with the detailed description, explain technical features of the various embodiments.
1 is a system diagram including a special photographed material production service providing server according to an embodiment of the present invention.
2 is a block diagram showing the configuration of a server that provides a special recording production service according to an embodiment of the present invention.
3 is a diagram illustrating an embodiment of a method of operating a server for providing a special photographed material production service.
4 is a diagram illustrating an embodiment of a screen shown in an actor VR terminal.
5 is a view showing an embodiment of the appearance of a special filming site.
6 is a diagram illustrating an embodiment of a screen in which a virtual character is synthesized in a special photographed material.

The following embodiments combine elements and features of the embodiments in a predetermined form. Each component or feature may be considered optional unless explicitly stated otherwise. Each component or feature may be implemented in a form that is not combined with other components or features. In addition, various embodiments may be configured by combining some components and/or features. The order of operations described in various embodiments may be changed. Some features or features of one embodiment may be included in another embodiment, or may be replaced with corresponding features or features of another embodiment.

In the description of the drawings, procedures or steps that may obscure the gist of various embodiments are not described, and procedures or steps that can be understood at the level of those of ordinary skill in the art are also not described. did

Throughout the specification, when a part is said to "comprising or including" a certain component, it does not exclude other components unless otherwise stated, meaning that other components may be further included. do. In addition, terms such as "...unit", "...group", and "module" described in the specification mean a unit that processes at least one function or operation, which is hardware or software or a combination of hardware and software. can be implemented as Also, "a or an," "one," "the," and like related terms are used herein in the context of describing various embodiments (especially in the context of the claims that follow). Unless indicated otherwise or clearly contradicted by context, it may be used in a sense including both the singular and the plural.

Hereinafter, embodiments according to various embodiments will be described in detail with reference to the accompanying drawings. DETAILED DESCRIPTION The detailed description set forth below in conjunction with the appended drawings is intended to describe exemplary embodiments of various embodiments, and is not intended to represent the only embodiments.

In addition, specific terms used in various embodiments are provided to help the understanding of various embodiments, and the use of these specific terms may be changed to other forms without departing from the technical spirit of various embodiments. .

Referring to FIG. 1 , a method for providing a special recording production service according to an embodiment of the present invention is performed on a computing device that has a storage space such as a personal computer (PC), can be connected to the Internet, and is not easy to carry, or is performed on a smart phone, etc. The same may be performed in a portable terminal. In this case, the 　 special filmed material production service provision method may be executed after an application implementing the special photographed material production service provision method is downloaded from an app store or the like and installed in the portable terminal.

And the method of providing a special recording production service is inserted into a computing device such as a PC in a state recorded on a recording medium such as a CD (Compact Disc) or USB (Universal Serial Bus) memory, etc., and is performed through the access operation of the computing device or After being stored in the storage space of the computing device from the recording medium, it may be performed through an access operation of the computing device.

On the other hand, when the computing device or the portable terminal can access a server connected to the Internet, the method of providing a special recording production service may also be executed in the server according to the request of the computing device or the portable terminal.

Hereinafter, a computing device, a portable terminal, or a server on which the method of providing a special filmed material production service is executed may be collectively referred to as a special photographed material production service providing apparatus.

The above 　 special filmed material production service providing apparatus may have the same configuration as the 　 special photographed material production service provision apparatus illustrated in FIG. can

The system according to an embodiment may include an actor VR terminal 100 , a camera terminal 110 , and a special recording production service providing server 200 (hereinafter, server 200 ). The network may include an Internet portal site server, an SNS server, a server operating a blog, and the like.

Actor VR terminal 100, camera terminal 110, a smartphone, a tablet PC, a PC, a smart TV, a mobile phone, a personal digital assistant (PDA), a laptop, a media player, a micro server, a global positioning system (GPS) device, It may be an e-book terminal, a digital broadcasting terminal, a navigation system, a kiosk, an MP3 player, a digital camera, a home appliance, a device equipped with a camera, and other mobile or non-mobile computing devices, but is not limited thereto. In addition, the actor VR terminal 100 and the camera terminal 110 may be a wearable device having a communication function and a data processing function. However, it is not limited thereto.

The server 200 may be implemented as a computer device or a plurality of computer devices that communicate with the actor VR terminal 100 and the camera terminal 110 through a network to provide commands, codes, files, contents, services, etc. .

For example, the server 200 may provide a file for installing an application to the actor VR terminal 100 and the camera terminal 110 connected through a network. In this case, the actor VR terminal 100 and the camera terminal 110 may install an application using a file provided from the server 200 .

In addition, the actor VR terminal 100, the camera terminal 110 is connected to the server 200 under the control of an operating system (Operating System, OS) and at least one program (eg, a browser or an installed application), and the server 200 ) can be provided with services or contents provided by

As another example, the server 200 may establish a communication session for data transmission/reception, and route data transmission/reception between the actor VR terminal 100 and the camera terminal 110 through the established communication session.

The actor VR terminal 100 , the camera terminal 110 , and the special recording production service providing server 200 may perform communication using a network. For example, a network includes a local area network (LAN), a wide area network (WAN), a value added network (VAN), a mobile radio communication network, a satellite communication network, and their It is a data communication network in a comprehensive sense that includes a mutual combination and enables each network constituent entity shown in FIG. 1 to communicate with each other smoothly, and may include a wired Internet, a wireless Internet, and a mobile wireless communication network. In addition, wireless communication is, for example, wireless LAN (Wi-Fi), Bluetooth, Bluetooth low energy, Zigbee, WFD (Wi-Fi Direct), UWB (ultra wideband), infrared communication (IrDA, infrared Data Association), NFC (Near Field Communication), etc. may be there, but is not limited thereto.

2 is a block diagram showing the configuration of a server that provides a special recording production service according to an embodiment of the present invention.

Referring to FIG. 2 , a server 200 (hereinafter, referred to as server 200 ) that provides a special recording production service may include a communication unit 210 , a processor 220 , and a DB 230 . In the server 200 of FIG. 2, only the components related to the embodiment are shown. Accordingly, it can be understood by those skilled in the art that other general-purpose components may be further included in addition to the components shown in FIG. 2 .

The communication unit 210 may include one or more components that enable wired/wireless communication between terminals. For example, the communication unit 210 may include at least one of a short-distance communication unit (not shown), a mobile communication unit (not shown), and a broadcast receiving unit (not shown).

For example, a request generated according to a program code stored in a recording device such as the DB 230 may be transmitted to the terminal through a network under the control of the communication unit 210 . Conversely, a control signal, command, content, file, etc. provided under the control of the processor of the terminal may be received by the server 200 through the communication unit 210 through the network. For example, a control signal, command, content, file, etc. of the server 200 received through the communication unit 210 may be transmitted to the processor 220 or transmitted to the DB 230 and stored.

The DB 230 is hardware for storing various data processed in the server 200 , and may store a program for processing and controlling the processor 220 .

DB 230 is a random access memory (RAM), such as dynamic random access memory (DRAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), CD- It may include ROM, Blu-ray or other optical disk storage, a hard disk drive (HDD), a solid state drive (SSD), or flash memory. The DB 230 may be referred to as a memory.

The processor 220 controls the overall operation of the server 200 . For example, the processor 220 may generally control the input unit (not shown), the display (not shown), the communication unit 210 , the DB 230 , and the like by executing programs stored in the DB 230 . The processor 220 may control the operation of the external server 200 by executing programs stored in the DB 230 .

The communication unit 210 may include one or more components that allow the server 200 to communicate with another device (not shown) and a server (not shown). The other device (not shown) may be a computing device such as the server 200 or a sensing device, but is not limited thereto. The communication unit 210 may receive a user input from another electronic device or data stored in the external device from an external device through a network.

The DB 230 may store a program for processing and controlling the processor 220 . For example, the DB 230 may store an instruction for providing a service. Also, the DB 230 may store data generated by the processor 220 . For example, the DB 230 may store information related to a special recording production service provided by the processor 220 . The DB 230 may store information input to or output from the server 200 .

The processor 220 is ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), controllers (controllers), microcontroller It may be implemented using at least one of (micro-controllers), microprocessors, and other electrical units for performing functions.

The DB 230 may store at least one instruction executed through the processor 220 . the at least one command provides an actor virtual reality (VR) terminal with a virtual space and a virtual character image of a counterpart moving in the virtual space; acquiring a single image of an actor wearing the actor VR terminal from a camera terminal; And it may be set to produce a special filmed image in which both the actor and the virtual character appear by using the virtual character image and the single image.

Here, the virtual space may be a virtual space implemented based on a real space in which the actor plays.

Here, the single image may include an image including the movement and dialogue of the actor in the real space.

Here, the at least one command may be further set to provide the special filmed image to the actor VR terminal.

Here, the at least one command is to obtain analysis information related to the movement of the actor from the actor VR terminal; And based on the analysis information, it may be further configured to provide information related to a part that needs to be corrected among the movement of the actor to the actor VR terminal.

3 is a diagram illustrating an embodiment of a method of operating a server for providing a special photographed material production service.

Referring to FIG. 3 , the server may provide a virtual space and a virtual character image ( S300 ). For example, the server may provide an actor virtual reality (VR) terminal with a virtual space and a virtual character image of a counterpart moving in the virtual space.

For example, the virtual space may be a virtual space implemented based on a real space in which the actor plays.

First, the environment of the filming site is actually built, and then environment information of the filming site can be obtained by a method such as 360-degree shooting. Based on the obtained environment information of the filming location, the server may perform 3D modeling. Through this, it is possible to build a virtual space of an environment such as a filming site on the actor's VR terminal.

4 is a diagram illustrating an embodiment of a screen shown in an actor VR terminal.

Referring to FIG. 4 , a virtual space may be shown on the display of the actor VR terminal 520 . A virtual character that is a role opposite to the actor may appear in the virtual space. The virtual character may include at least one character. The motion and dialogue of the virtual character may be preset. The actor may perform the performance by matching the virtual character appearing on the display of the actor VR terminal 520 while wearing the actor VR terminal 520 . email

For example, on the display of the actor VR terminal 520 , information about shooting time, information related to motion, and information related to dialogue may be further displayed.

Here, the information related to the motion may include a name of a first motion to be performed next by the actor and information about how many seconds later the first motion should be performed. The information related to the operation may include a name of a second operation to be performed next by the virtual character and information on how many seconds later the second operation is performed.

For example, information that the opponent's left fist flies after 3 seconds may be shown on the display of the actor VR terminal 520 .

Here, the information related to the dialogue may include information on the contents of the first dialogue to be performed next by the actor and information on how many seconds later the first dialogue should be performed. The information related to the dialogue may include details of the second dialogue to be performed next by the virtual character and information on how many seconds later the second dialogue will be performed.

For example, information indicating that dialogue A of the virtual character is performed after 2 seconds and dialogue B of the actor should be performed after 5 seconds may be displayed on the display of the actor VR terminal 520 . The remaining time until dialogue can be changed in real time.

For example, information related to motion and information related to lines may be reproduced as a voice that the actor can hear in the actor VR terminal 520 . That is, although not inserted into the image, information related to motion and information related to dialogue may be transmitted to the actor through a speaker included in the actor VR terminal 520 so that only the actor can hear it in the form of voice.

5 is a view showing an embodiment of the appearance of a special filming site.

Referring to FIG. 5 , in the filming site 500 , an actor 510 may perform an act by wearing the actor VR terminals 520 and 530 , and the camera terminal 550 may photograph the appearance of the actor 510 . . The actor 510 may perform acting based on a screen displayed on the actor VR terminal 520 . The actor 510 may act against a virtual character as if a virtual character existed in front in a virtual space in which the filming site 500 was 3D modeled. The actor 510 may act while wearing the actor VR terminal 530 , and the actor VR terminal 530 may provide feedback on whether the actor performs an accurate motion through vibration or the like. The actor VR terminal 530 may be omitted, or another type of actor VR terminal may be used.

Referring back to FIG. 3 , the server may acquire a single image ( S310 ). For example, the server may acquire a single image of an actor wearing the actor VR terminal from the camera terminal.

For example, the single image may include an image including movement and dialogue of the actor in the real space.

For example, the single image may be an image in which the camera terminal 550 captures the performance of the actor 510 in FIG. 5 .

The server may produce a special recorded video (S320). For example, the server may use the virtual character image and the single image to produce a special filmed image in which both the actor and the virtual character appear. Next, an embodiment of a special photographed image will be described.

6 is a diagram illustrating an embodiment of a screen in which a virtual character is synthesized in a special photographed material.

Referring to FIG. 6 , a virtual character 540 may be synthesized in a single image played by an actor 510 at a filming site 500 . By synthesizing the virtual character image into a single image based on the virtual character image provided to the actor VR terminal 520 , the virtual character seen by the actor 510 through the actor VR terminal 520 can be implemented in a special shot. Therefore, in the produced special film, both the actor 510 and the virtual character 540 acting in the background of the actual filming site 500 may appear.

Referring back to FIG. 3 , the server may provide a special recorded image (S330). For example, the server may provide the special filmed image to the actor VR terminal. Through the actor VR terminal, the actor can check the special filmed video in which the virtual character is synthesized with the acting he has performed.

The server may acquire motion analysis information (S340). For example, the server may acquire analysis information related to the movement of the actor from the actor VR terminal. For example, the actor VR terminal may collect information about the actor's motion during filming through a sensor or the like. When the server obtains the analysis information from the actor VR terminal, it can compare it with the ideal movement of the action that the actor should have originally performed. For example, the server may store information on the ideal movements of the motion to be performed by the actor in the scenario, and may perform a comparison on whether the motion is well performed based on the analysis information obtained from the actor VR terminal. have.

The server may provide feedback information (S350). For example, the server may provide the actor VR terminal with information related to a part requiring correction of the actor's movement based on the analysis information.

For example, the part that needs to be modified may include information such as that the trajectory of the left fist swung at 1:30 of the scene should be higher, and the subsequent kick should be faster.

When the actor receives feedback information from the server through the actor VR terminal, it can be reflected when the next episode is postponed. In addition, since the actor can check the special filmed video in which his acting from the previous scene and the virtual character are synthesized through the actor's VR terminal, there is an effect that he can look back on his acting clearly.

Some of the detailed steps shown in the example of FIG. 3 may not be essential steps and may be omitted. In addition to the steps shown in FIG. 3 , other steps may be added, and the order of the steps may vary. Some of the above steps may have their own technical meaning.

A method of providing a special recording production service performed by at least one server, the method comprising: providing a virtual space and a virtual character image of a counterpart moving in the virtual space to an actor virtual reality (VR) terminal; obtaining a single image of an actor wearing the actor VR terminal from a camera terminal; and producing a special filmed image in which both the actor and the virtual character appear by using the virtual character image and the single image.

Here, the virtual space may include a virtual space implemented based on a real space in which the actor plays.

Here, the single image may include an image including the movement and dialogue of the actor in the real space.

Here, the server may further include the step of providing the special filmed image to the actor VR terminal.

Here, the server, the step of obtaining the analysis information related to the movement of the actor from the VR terminal; and providing information related to a part requiring correction of the actor's movement to the actor VR terminal based on the analysis information.

Here, the virtual character image may further include shooting time information, motion-related information, and dialogue-related information.

Here, the information related to the motion may include a name of a first motion to be performed next by the actor and information about how many seconds later the first motion should be performed. The information related to the operation may include a name of a second operation to be performed next by the virtual character and information on how many seconds later the second operation is performed.

Here, the information related to the dialogue may include information on the contents of the first dialogue to be performed next by the actor and information on how many seconds later the first dialogue should be performed. The information related to the dialogue may include details of the second dialogue to be performed next by the virtual character and information on how many seconds later the second dialogue will be performed.

Here, the information related to the part requiring correction includes information related to the degree to which the start point of the actor's movement should be corrected, the extent to which the end point of the actor's movement should be corrected, and the degree to which the movement trajectory of the actor should be corrected. can do.

Here, the information related to the degree to which the motion trajectory of the actor needs to be corrected may include image information on the correct motion trajectory.

The operation according to the embodiment of the present specification may be implemented as a computer-readable program or code on a computer-readable recording medium. The computer-readable recording medium includes all types of recording devices in which data readable by a computer system is stored. In addition, the computer-readable recording medium may be distributed in a network-connected computer system to store and execute computer-readable programs or codes in a distributed manner.

When the embodiment is implemented in software, the above-described technique may be implemented as a module (process, function, etc.) that performs the above-described function. A module may be stored in a memory and executed by a processor. The memory may be internal or external to the processor, and may be coupled to the processor by various well-known means.

In addition, the computer-readable recording medium may include a hardware device specially configured to store and execute program instructions, such as ROM, RAM, and flash memory. The program instructions may include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.

Although some aspects of the invention have been described in the context of an apparatus, it may also represent a description according to a corresponding method, wherein a block or apparatus corresponds to a method step or feature of a method step. Similarly, aspects described in the context of a method may also represent a corresponding block or item or a corresponding device feature. Some or all of the method steps may be performed by (or using) a hardware device such as, for example, a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, one or more of the most important method steps may be performed by such an apparatus.

In embodiments, a programmable logic device (eg, a field programmable gate array) may be used to perform some or all of the functions of the methods described herein. In embodiments, the field programmable gate array may operate in conjunction with a microprocessor to perform one of the methods described herein. In general, the methods are preferably performed by some hardware device.

The various embodiments described above may be embodied in other specific forms without departing from the technical idea and essential characteristics thereof. Accordingly, the above detailed description should not be construed as restrictive in all respects but as exemplary. The scope of the various embodiments should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the various embodiments are included in the scope of the various embodiments. In addition, claims that are not explicitly cited in the claims can be combined to form an embodiment or included as a new claim by amendment after filing.

Claims (5)

In the method of providing a special film production service performed by at least one server,
Provide a virtual space and a virtual character image of a counterpart moving in the virtual space to an actor virtual reality (VR) terminal,
The virtual space is a virtual space implemented based on the real space in which the actor is acting, and the virtual character image includes shooting time information, motion-related information, and dialogue-related information,
The information related to the action may include a name of a first action to be performed by the actor next, information about how many seconds later the first action should be performed, a name of a second action to be performed next by the virtual character, and including information about how many seconds later the second operation is performed, and
The information related to the dialogue may include content of a first line to be performed by the actor next, information about how many seconds later the first line should be performed, content of a second line to be performed by the virtual character next, and a number of seconds After including information on whether the second metabolism is performed, a step;
acquiring a single image of an actor wearing the actor VR terminal from a camera terminal, wherein the single image includes an image including an actor's movement and dialogue in the real space;
producing a special filmed image in which both the actor and the virtual character appear by using the virtual character image and the single image;
providing the special filmed image to the actor VR terminal so that the actor can check the actor's movements and lines after the actor's movements and lines are finished;
obtaining analysis information related to the movement of the actor from the actor VR terminal; and
Based on the analysis information, information related to the part requiring correction of the actor's movement is provided to the actor VR terminal,
The information related to the part requiring the correction includes information related to the degree to which the start point of the actor's movement should be corrected, the extent to which the end point of the actor's movement should be corrected, and the degree to which the movement trajectory of the actor should be corrected,
The information related to the degree to which the movement trajectory of the actor needs to be corrected, including image information on the correct movement trajectory,
How to provide special film production service. delete delete delete delete

Images