KR102636728B1 - Performance Media Content Providing System Combining Interactive VR Technology - Google Patents

Abstract

The present invention relates to a performance media content provision system based on a 3D game engine. Specifically, it provides performance media content combining XR technology based on a 3D game engine.
The present invention provides performance media content that combines XR technology based on a 3D game engine, and has the effect of allowing performance media content to be enjoyed regardless of the environment in a new space that fuses the real world and the virtual world.

Description

Performance media content provision system combining interactive VR technology {Performance Media Content Providing System Combining Interactive VR Technology}

The present invention relates to a performance media content provision system. Specifically, it provides performance media content combining XR technology based on a 3D game engine.

AR is a concept that augments reality by displaying additional information in the real world. It is implemented by projecting reality and displaying 3D content on top of it. VR uses computer technology to create a simulation environment, completely blocking the user's view and allowing them to experience only a virtual world. MR mixed reality places real-time rendering content around real objects or people and on the floor screen, creating a rich virtual environment where people can immerse themselves in the water space. XR encompasses VR, AR, and MR and is used as a term that can also include other forms of reality that will appear in the future.

The 5G era, characterized by ultra-high speed, ultra-connectivity, and ultra-low latency, has begun in earnest, and the XR market, which encompasses VR, AR, and MR, delivers the virtual world to viewers more quickly in LIVE format and allows viewers to focus on the situation and You can immerse yourself and have a realistic experience. When linked with online and offline content, there are no limits to expression and connectivity with interactive media art is also free.

In the future, it is predicted that people will interact in a new space that fuses the real world and the virtual world in a smooth way that is not limited by the environment or technology. Accordingly, the performance media industry is also developing technology that fuses the virtual world, but this is not yet possible. The technology for this is insufficient.

Korean Patent Publication No. 10-183927 (2018.03.09)

The purpose of the present invention is to provide performance media content that combines XR technology based on a 3D game engine, so that performance media content can be enjoyed regardless of the environment in a new space that fuses the real world and the virtual world. .

In order to solve the above problems, the present invention receives shooting data captured from a camera, provides it to the 3D game engine unit 120, performs interaction between the generated 3D virtual creation and the set target, and produces a screen rendering output. An augmented reality screen output management server 100 for producing performance content to be output to the unit 700, and a screen production output unit configured to receive the rendered result from the augmented reality screen output management server 100 and constructed at a location to be transmitted. A communication network 200 for transmitting performance content to (700), a camera 300 for photographing a target set in real space including a screen production output unit 700, and a camera 300 set through a location tracking method using infrared rays. By the camera tracking unit 400 for tracking the location of the target, the lighting 600 located on one side of the screen rendering output unit 700, and the augmented reality screen output management server 100 through the communication network 200. It includes a screen rendering and output unit 700 for receiving and outputting the implemented performance content, and the augmented reality screen output management server 100 receives shooting data taken from a camera and provides space to enable space expansion. A performance media content provision system based on a 3D game engine is provided, which includes a 3D game engine unit 120 for providing interactive VR for augmentation.

The present invention provides performance media content that combines XR technology based on a 3D game engine, and has the effect of allowing performance media content to be enjoyed regardless of the environment in a new space that fuses the real world and the virtual world.

1 is a configuration diagram showing the overall configuration of the present invention.
Figure 2 is a diagram showing the configuration of the augmented reality screen output management server 100 of the present invention.
Figure 3 is a diagram schematizing the overall configuration of the present invention.
Figures 4 and 5 are diagrams showing an embodiment of the present invention.

Terms and words used in this specification and claims should not be construed as limited to their usual or dictionary meanings, and the inventor may appropriately define the concept of terms to explain his or her invention in the best way. It must be interpreted as meaning and concept consistent with the technical idea of the present invention based on principles.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings.

The present invention relates to a performance media content provision system based on a 3D game engine. Specifically, it provides performance media content combining XR technology based on a 3D game engine.

The game engine is core software that implements interactive applications with real-time graphic display functions necessary to run games such as video games on a PC.

Figure 1 is a configuration diagram showing the overall configuration of the present invention. The present invention includes an augmented reality screen output management server 100, a communication network 200, a camera 300, a camera tracking unit 400, an augmented reality equipment 500, It is comprised of lighting 600 and a screen display output unit 700.

Each component is described in detail as follows.

The augmented reality screen output management server 100 receives the captured data from the camera 300, provides it to the 3D game engine unit 120, performs interaction between the generated 3D virtual creation and the set target, and displays the data on the screen. This is to produce content to be output to the production output unit 700.

The shooting data captured by the camera 300 is data including real space recognition information about real space, set target recognition information, and set target movement information.

Figure 2 is a diagram showing the configuration of the augmented reality screen output management server 100 of the present invention.

The augmented reality screen output management server 100 includes a basic information input unit 110, a 3D game engine unit 120, a screen output setting unit 130, a rendering unit 140, and a lighting control unit 150. do.

The basic information input unit 110 is used to receive basic content information for screen output to the screen rendering output unit 700.

The basic content information includes basic event and performance information, content concept, type, synopsis, and video and audio data that become the main content of the event.

The 3D game engine unit 120 is intended to receive shooting data captured from a camera and provide interactive VR for space augmentation to enable space expansion.

The 3D game engine unit 120 is preferably an Unreal Engine, and has the latest light source effects such as perpixel lighting, perpixel specular, and HDR rendering, and soft and realistic physical characteristics to create the feeling of business cards being exchanged in real life. It provides a material system based on , continuous updates, various technical support, network support between Unreal developers, and excellent development tools, and the engine configuration is flexible so that various technologies can be combined and expanded.

Interactive VR uses Unreal Engine to easily link with the operating system and allow for real-time modifications.

In particular, the game engine is compatible with most operating systems and can be easily linked, shortening the linking time, and editing time is shortened as real-time editing is possible.

Therefore, the production time can be shortened as the linking time and modification time are shortened through the interactive VR.

The Interactive VR is interactive in stages and has very high scalability by adding visual and behavioral interactivity through an interactive device.

The screen output setting unit 130 is used to output content to be generated in a three-dimensional space to the screen of the screen production output unit 700, and to set the implemented content to be implemented in the same screen space as the actual location to be transmitted.

The content implemented in the screen output setting unit 130 can be formed in either 3D or 2D form, and can be implemented by including elements such as setting space implementation, viewing angle implementation, and viewpoint setting within the content.

The screen output setting unit 130 includes a three-dimensional space creation unit 131, a content screen implementation unit 132, and a screen implementation setting unit 133.

The three-dimensional space creation unit 131 is used to create a three-dimensional space modeled as the same space as the screen of the screen rendering and output unit 700.

That is, the 3D space creation unit 131 takes the extracted information and reorganizes the actual screen into a 3D space using a 3D graphic tool, or generates a design drawing file of the media screen corresponding to the exhibition space and uses a 3D graphic tool. It is implemented in the same space as the actual screen using . Here, the 3D space is a virtual space modeled by reducing and reconstructing the actual screen.

The three-dimensional space is a virtual object created with the same shape, space, and ratio as the actual screen, and is a virtual space created by analyzing spatial values from shooting data captured by the camera 300.

The content screen implementation unit 132 is intended to implement the content to be created in a three-dimensional space under the same conditions as when applying it to the screen rendering and output unit 700.

The screen presentation setting unit 133 is used to set the implemented content to be implemented in the same screen space (screen presentation output unit 700) as the actual location to be transmitted.

When the setting for implementing the implemented content in the same space as the actual transmission location is completed, the screen implementation setting unit 133 detects the conditions currently set on the screen and preset the same space as the actual environment where the content will be transmitted. It is possible to determine whether the setting conditions (eg, angle of view, viewpoint, space, etc.) required to create are satisfied, and to detect conditions that do not satisfy the setting conditions.

If a condition that does not satisfy the setting conditions is detected, readjustment request notification information may be provided to the user, and the unsatisfied condition may be displayed on the screen.

The screen implementation setting unit 133 includes a content priority setting function 133a and a data conversion function 133b.

The content priority setting function 133a is used to set target data to be composed and produced as VR content and to set content priorities for the set target data.

The data conversion function 133b is for converting audio data into visual data.

Specifically, the data conversion function 133b selects at least one visual data belonging to the visual data category corresponding to the audio data from among the pre-classified visual data categories based on the audio data and information about the external environment to convert the audio data into audio data. can be converted into visual data.

In addition, the pre-classified visual data category of the data conversion function 133b includes first categories classified based on the genre of the audio data, and each of the first categories includes second categories classified based on the external environment. It can be included.

For example, audio data can be classified into genres by indexing sounds by frequency domain, and visual data can be pre-classified into specific categories based on audio data classified by genre.

The screen implementation setting unit 133 further includes a setting value change function 133c, and the visitor retention value for each location of each content output through a plurality of installed screen presentation output units 700 is calculated according to the set time period [Equation 1] By calculating through , it is possible to determine whether visitors consistently visit at any given time. Through this, data about time zones in which the visitor retention value of a specific time zone has a lower standard value than the preset standard value of the visitor retention value is stored, and the saved data is analyzed to generate time zones with a visitor retention value lower than the set standard value more than a certain number of times. In this case, the content in that time zone can be changed to other content.

(Here, G is the visitor retention value, K is the total number of screen rendering output units 700, D _m is the visitor density measurement value of the mth screen rendering output unit 700, and D _a is all calculated visitor density measurement values. represents the average.)

These visitor retention figures are calculated by storing the visit density information of visitors visiting the location of each screen display output unit 700 at each set time interval, calculating the variance of the stored visit density information, and the smaller the variance, the more consistent the number of visitors. It can be determined that it is maintained, and the larger the variance, the more inconsistently the visitors are maintained.

In addition, this is preferably applied in the case where there are multiple performance spaces, each installed screen production output unit 700, and can be applied in the case where visitors can move fluidly, but does not limit the spatial elements.

For example,

1) The measurement time zone is time zone A, the total number of screen presentation output units 700 is 3, the visitor density measurement value of the first screen presentation output unit 700 is 5, and the visitor density measurement value of the second screen presentation output unit 700 is 5. If the visitor density measurement value is 4, the visitor density measurement value of the third screen display output unit 700 is 4, and the average of all calculated visitor density measurement values is 4.5,

The visitor retention figure G is 26.7.

2) The measurement time zone is time zone B, the total number of screen presentation output units 700 is 3, the visitor density measurement value of the first screen presentation output unit 700 is 5, and the visitor density measurement value of the second screen presentation output unit 700 is 5. The visitor density measurement value is 3, the visitor density measurement value of the third screen display output unit 700 is 5, and the average of all calculated visitor density measurement values is 4.2.

The visitor retention figure G is 22.05.

The smaller the variance, the larger the visitor retention value, so it can be judged as a high score, i.e., a high visitor retention value. The larger the variance, the smaller the visitor retention value, so it can be judged as a low score, i.e., a low visitor retention value.

In other words, when comparing the results of 1) and 2), it can be confirmed that visitors are maintained more consistently in time zone A because the visitor retention value of 1) is higher.

The rendering unit 140 is for rendering data set by the screen output setting unit 130.

The rendering unit 140 can simulate the generated directed image in real time and integrate it with the actual image, which has the effect of reducing errors and increasing completeness of the result.

The lighting control unit 150 is used to control the position of the lighting 600 and to match the movement of the lighting in the screen implementation setting unit 133 with the actual lighting, 600.

The communication network 200 is built at a location to be transmitted, receives the rendered result from the augmented reality screen output management server 100, and transmits the content to the screen rendering and output unit 700.

The camera 300 is used to photograph an object set in a real space including the screen rendering and output unit 700.

The camera tracking unit 400 is located on one side of the camera 300 and is used to track the location of a set target through a location tracking method using infrared rays.

The augmented reality equipment 500 is worn by the user to visualize content implemented on the screen rendering and output unit 700. The augmented reality equipment 500 typically includes VR equipment, XR equipment, etc., and is worn by a user to visualize virtual space content.

The lighting 600 is located on one side of the screen rendering and output unit 700 and is controlled by the lighting control unit 150.

The lighting 600 is formed to match the graphics and colors of the person to break down the boundary between virtuality and reality.

The screen rendering and output unit 700 is formed as a screen of a predetermined size and is used to receive and output content implemented by the screen output setting unit 130 through the communication network 200. The screen presentation output unit 700 may be formed in a predetermined number and may be any device for outputting content, such as a display.

The present invention may further include a fire detection unit 800 and a cooling system 900 (not shown).

The fire detection unit 800 uses a fire detection sensor to detect the augmented reality screen output management server 100, communication network 200, camera 300, camera tracking unit 400, augmented reality equipment 500, and lighting 600. , This is to detect whether a fire is detected by measuring the temperature of the screen display output unit 700.

The fire detection unit 800 calculates the safety value through [Equation 2], and when the calculated safety value is higher than the standard value, an operation signal for operating the cooling system 900 can be transmitted to the cooling system 900. .

(Here, W is the safety value, r is the standard value, it means.)

Here, the standard value refers to the score that is considered to be the perfect safety value. In other words, if the safety score is 50 points, the standard value is 50, and if the safety value is 10 points, the standard value is 10. This means that the perfect score of the safety value set by the person managing the system of the present invention is set as the standard value, and it is desirable that there is no limit to the size of the standard value.

In addition, the safety weight indicates a value closer to 1 when the fire detection unit 800 is operating normally, and when failure or malfunction of the fire detection unit 800 occurs frequently, it shows a value closer to 0. At this time, the safety weight may be set by the administrator, and it is desirable not to limit its value.

The operation frequency of the fire detection unit 800 refers to the number of times the fire detection sensor is activated per time unit set by the administrator, the malfunction frequency refers to the frequency with which the fire detection sensor malfunctions, and the number of measurements below the standard value is determined by the administrator. This refers to the number of times the measured temperature value was higher than the set temperature.

For example, the perfect score for the safety value is 50, the standard value is 50, the safety weight is 0.8, the number of times the temperature measured by the fire detection unit 800 is below the standard value is 15, and the frequency of malfunction of the fire detection unit 800 is 15. 3 times, when the number of operations of the fire detection unit (800) is 30 times

It can be calculated as

The safety value has a higher value as the number of measurements below the standard value is higher compared to the number of operations of the fire detection unit 800, the lower the malfunction frequency, and the higher the safety weight.

The cooling system 900 operates when a fire is detected through the fire detection unit 800 and can be installed to prevent fire.

The cooling system 900 may be a typical cooling system that operates in the event of a fire, and detailed description thereof will be omitted.

Figure 3 is a diagram schematizing the overall configuration of the present invention. Referring to this, four lights 600 are located, four screen display output units 700, which are LEDs, can be formed, and the information tracked by the camera 300 and the camera tracking unit 400 can be managed to output an augmented reality screen. It is transmitted to the server 100 and can be provided with performance media content that combines XR technology and is based on a 3D game engine.

Figure 4 is an example diagram of the LED wall and LED floor, which are the screen display output unit 700.

Figure 5 is a diagram showing an example of the location of components such as the camera 300 and the screen production output unit 700 in the system of the present invention when producing a performance stage.

The embodiments described in the present specification and the configurations shown in the drawings are only one of the most preferred embodiments of the present invention and do not represent the entire technical idea of the present invention, so various equivalents and modifications can be substituted for them. It should be understood that there may be examples.

100 Augmented reality screen output management server
110 Basic information input unit
120 3D Game Engine Department
130 Screen output setting section
131 3D space creation unit
132 Content Screen Implementation Department
133 Screen Implementation Settings Department
140 Rendering Department
150 Lighting control unit
200 communication network
300 camera
400 Camera tracking unit
500 augmented reality equipment
600 lights
700 Screen production output unit
800 fire detection unit
900 cooling system

Claims (2)

Receives shooting data captured from the camera, provides it to the 3D game engine unit 120, performs interaction between the generated 3D virtual creation and the set target, and produces performance content to be output to the screen production output unit 700. An augmented reality screen output management server (100) for
A communication network (200) built at a location to be transmitted to receive the rendered results from the augmented reality screen output management server (100) and transmit the performance content to the screen production output unit (700);
A camera 300 for photographing a target set in real space including a screen rendering and output unit 700,
A camera tracking unit 400 for tracking the location of a target set through a location tracking method using infrared rays,
Lighting 600 located on one side of the screen display output unit 700,
It includes a screen production output unit 700 for receiving and output performance content implemented by the augmented reality screen output management server 100 through a communication network 200,

The augmented reality screen output management server 100 is
A basic information input unit 110 for receiving basic content information for output to the screen display output unit 700,
A 3D game engine unit 120 for receiving shooting data captured from the camera 300 and providing interactive VR for space augmentation to enable space expansion,
In order to output content to be created in a three-dimensional space to the screen rendering and output unit 700, a screen output setting unit 130 is configured to set the implemented content to be implemented in the same space of the screen rendering and output unit 700 as the actual location to be transmitted. )and,
A rendering unit 140 for rendering data set by the screen output setting unit 130,
To control the position of the lighting 600, it includes a lighting control unit 150 to match the movement of the lighting in the screen implementation setting unit 133 and the actual lighting, 600,
The screen output setting unit 130 is
a three-dimensional space creation unit 131 for generating a three-dimensional space modeled as the same space as the screen of the screen rendering and output unit 700;
a content screen implementation unit 132 for implementing content to be created in a three-dimensional space under the same conditions as those applied to the screen production output unit 700;
It includes a screen implementation setting unit 133 for setting the implemented content to be implemented as a screen presentation output unit 700,

The screen implementation setting unit 133 is
A setting for determining whether visitors consistently visit at any given time by calculating the visitor retention value for each location of each content output through a plurality of installed screen display output units 700 using [Equation 1] for each set time zone. It further includes a value change function (133c).

[Equation 1] is

(Here, G is the visitor retention value, K is the total number of screen rendering output units 700, D _m is the visitor density measurement value of the mth screen rendering output unit 700, and D _a is all calculated visitor density measurement values. represents the average.)
ego,

The visitor retention value calculated by the above [Equation 1] stores the visitation density information of visitors visiting the location of each screen rendering output unit 700 at each set time interval, and calculates the distribution of the stored visitation density information. A performance media content provision system combined with interactive VR technology, characterized in that the smaller the variance, the more consistently visitors are determined to be maintained, and the larger the variance, the more inconsistently the visitors are determined to be maintained. According to paragraph 1,
Through the fire detection sensor, the augmented reality screen output management server (100), communication network (200), camera (300), camera tracking unit (400), augmented reality equipment (500), lighting (600), and screen rendering output unit (700) ) a fire detection unit 800 for detecting whether a fire is detected by measuring the temperature of the
A performance media content providing system combined with interactive VR technology, characterized in that it further includes a cooling system (900) installed to prevent fire, which is activated when a fire is detected through the fire detection unit (800).