KR20240003584A - Vr-based apartment experience method and system - Google Patents

Abstract

The present invention relates to a system for experiencing a cyber model house by interactively rendering a high-quality 3D model house in real time using parallel rendering technology, and provides ultra-high resolution 3D model house content using parallel GPU computing technology. It can be rendered in real time and improves immersion by providing an interactive video system through user motion recognition. The boundaries of each screen are installed continuously to form a model house space. An output module 200 consisting of a plurality of output devices; A parallel rendering unit 100 consisting of a plurality of rendering devices correspondingly connected to a plurality of output devices; It includes a motion recognition unit 300 that recognizes the user's motion, generates motion data, and transmits it to the parallel rendering unit 100, wherein the parallel rendering unit 100 is one of a number of rendering devices. The rendering device is configured as a server, and the remaining rendering devices are configured as clients. A model house image (500) and segmentation data (600) are stored in each rendering device, and the model house image (500) is used to create an indoor space. Spatial data 512, view data 511 corresponding to the view image seen from the window of the indoor spatial data 512, furniture data 513 placed within the indoor spatial data 512, and lighting data 514 by a light source ), and the rendering device includes a database unit 160 in which the model house image 500 and the partition data 600 are stored, and a model house image 500 stored in the database unit 160. ), a content loading unit 110 that loads the partition area data 600 stored in the database unit 160, a partition loading unit 120 that receives motion data and matches it with the corresponding motion data in the database unit 160. A motion processing unit 130 that loads motion command data, a rendering unit 140 that divides the model house image 500 according to the division area data 600 and renders the divided model house contents based on the motion command data 700. ), a split content transmission unit 150 that transmits the divided model house image rendered by the rendering unit 140 to an output device connected to the rendering device, and the content loading unit 110 transmits the loaded model house image A configuration further including an object extraction unit 111 that extracts 500 for each object 510 is disclosed.

Description

VR-BASED APARTMENT EXPERIENCE METHOD AND SYSTEM}

The present invention relates to a system for experiencing a cyber model house by interactively rendering a high-quality 3D model house in real time using parallel rendering technology.

Most cyber apartment model houses currently being serviced by large corporations and small and medium-sized construction companies are image-based VR (Image-Based Virtual Reality) panoramic video systems produced by filming the interior of the actual model house after construction.

This image-based VR experience system has the disadvantage of lowering the sense of presence and immersion because it is unable to provide the experiencer with views from various locations and provide high-quality video services due to limitations in production technology.

In addition, the image-based VR video experience system has many difficulties in implementing an interactive experience system that can change the internal structure and replace interior materials and furniture in real time, which is possible in the modeling-based VR (Modeling-Based Virtual Reality) experience system. As a result, it is unable to meet the diverse needs of the experiencers.

Therefore, the present invention is intended to solve the problems described above, using parallel GPU computing technology to render ultra-high resolution 3D model house content in real time and interact through user motion recognition to improve immersion. We would like to provide a video system that is capable of this.

In order to solve the above problems, the present invention

It includes a motion recognition unit 300 that recognizes the user's motion, generates motion data, and transmits it to the parallel rendering unit 100, wherein the parallel rendering unit 100 is one of a number of rendering devices. The rendering device is configured as a server, and the remaining rendering devices are configured as clients, and the model house image 500 and segmented area data 600 are stored in each parallel rendering device, and the model house image 500 constitutes an indoor space. indoor space data 512, view data 511 corresponding to the view image seen from the window of the indoor space data 512, furniture data 513 placed within the indoor space data 512, and It consists of an object 510 containing lighting data 514, and the parallel rendering device is a database unit 160 in which model house image 500 and partition data 600 are stored, and stored in database unit 160. A content loading unit 110 that loads the model house image 500, a partition loading unit 120 that loads the partition data 600 stored in the database unit 160, and a motion data received and corresponding data in the database unit 160. The motion processing unit 130 retrieves motion command data matching the motion data, divides the model house image 500 according to the segmentation area data 600, and divides the model house image divided based on the motion command data 700. It includes a rendering unit 140 that renders, a split content transmission unit 150 that transmits the segmented model house image rendered by the rendering unit 140 to an output device connected to the rendering device, and the content loading unit 110 It is characterized in that it further includes an object extraction unit 111 that extracts the loaded model house image 500 for each object 510,

The rendering unit 140 includes a screen dividing unit 141, which extracts a model house image divided into rectangular areas composed of the coordinates of the segmented area data 600, with the center of the model house image 500 as the origin, and a motion A motion command data processing unit 142 generates rendering data to render a model house image based on command data 700. If the rendering device is a client, the rendering data generated by the motion command data processing unit 142 is sent to the server. and, if the rendering device is a server, the rendering data generated in the motion command data processing unit 142 or the rendering data transmitted from another rendering device is transmitted to the remaining rendering devices by the synchronization unit 143. It is characterized by including a GPU parallel processing unit 143 that renders the model house image 500 using parallel GPU computing using the transmitted rendering data,

The partition data 600 is the three-dimensional coordinates of three of the four corners of the screen of the output device connected to the rendering device when the origin is the center point on the screen of the output device located in the center of the output module 200. We present an interactive high-quality video system based on real-time parallel rendering, which is characterized in that it consists of:

The present invention has the following effects.

In other words, ultra-high resolution 3D model house content can be rendered in real time using parallel GPU computing technology, and immersion can be improved by providing an interactive video system through user motion recognition.

Meanwhile, the effects of the present invention are not limited to the effects mentioned above, and various effects may be included within the range apparent to those skilled in the art from the contents described below.

1 is a block diagram of the overall configuration of the present invention.
Figure 2 is a block diagram of the configuration of a rendering device among the components of the present invention.
Figure 3 is a system configuration diagram for one embodiment.
Figure 4 is a system configuration diagram for another embodiment.

Hereinafter, the 'VR-based apartment experience method and system' according to the present invention will be described in detail with reference to the attached drawings. The described embodiments are provided so that those skilled in the art can easily understand the technical idea of the present invention, and the present invention is not limited thereto. In addition, the matters expressed in the attached drawings may be different from the actual implementation form in the schematic drawings to easily explain the embodiments of the present invention.

Meanwhile, each component expressed below is only an example for implementing the present invention. Accordingly, other components may be used in other implementations of the present invention without departing from the spirit and scope of the present invention.

Additionally, each component may be implemented purely through hardware or software configurations, but may also be implemented through a combination of various hardware and software configurations that perform the same function. Additionally, two or more components may be implemented together by one hardware or software.

Additionally, the expression 'including' certain components is an 'open' expression and simply refers to the existence of the corresponding components and should not be understood as excluding additional components.

The present invention relates to an interactive high-quality video system based on real-time parallel rendering, and is composed of a parallel rendering module 100, an output module 200, and a motion recognition unit 300, as shown in FIG. 1.

The output module 200 is composed of a plurality of output devices (200a, 200b, 200c, ...). Multiple output devices that output divided model house images are connected to produce one model house image (500) as a whole. It serves to output .

The boundaries of each output device screen are installed continuously to form a model house space. The display can be installed along the interior wall, or temporary facilities such as a screen are installed like a wall as shown in Figure 6 to create a model house space.

This can also be formed.

In particular, the output device is installed so that the boundaries of the screens of each output device are in contact so that the output module 200 forms one large screen. For example, the LED/LCD display is grid-shaped (see FIG. 3) or horizontal. It can be each display when connected in a row (see Figure 4) or vertically, and when multiple screens are connected in a grid, horizontally or vertically, and multiple projectors project divided images to each screen, each projector It may be a combination of and screen. However, the meaning of 'line' is that it is connected in a single line when viewed from the front, and also includes the connected display being curved when viewed from another direction (such as from above or from the side) (see Figure 4). Additionally, the screens may be connected in four or multiple angles to surround the front, forming a space where the model house image 500 is displayed.

The motion recognition unit 300 recognizes the user's motion, generates motion data, and transmits it to the parallel rendering unit 100. It uses the Kinect sensor to detect the user's motion in the space where the output module 200 is installed. It can recognize gaze, hand movements, body movements, etc.

The parallel rendering unit 100 is composed of a plurality of rendering devices (100a, 100b, 100c, ...) connected to a plurality of output devices (200a, 200b, 200c, ...). In other words, the output device and rendering device are connected 1:1.

At this time, one rendering device (100a) among multiple rendering devices (100a, 100b, 100c, ...) is designated as the server, and the remaining rendering devices (100b, 100c, ...) are designated as clients. connected by a network. This is for synchronization of rendering, which will be described later.

The rendering device 100a stores the model house image 500 and segmentation data 600 consisting of the coordinate values of the images output from each output device on the model house image 500, thereby rendering the model house image 500. It is configured to divide according to the division area data 600, render the divided model house image, and then transmit the rendered divided model house image to a connected output device.

Each rendering device (100a, 100b, 100c, ...) includes a content loading unit 110, a partition loading unit 120, a motion processing unit 130, a rendering unit 140, and a divided content transmission unit, as shown in FIG. 2. It consists of (150) and a database unit (160). The addition of an alphabet to the identification number of the above configuration means the configuration of a specific rendering device 100b (rendering unit 140b, database unit 160b, etc.), and no alphabet is added (an identification number consisting of numbers only). refers to the components of each rendering device (100a, 100b, 100c, ...) (the rendering units (140a, 40b, 140c, ...) are referred to as the rendering unit 140, etc.).

The model house image 500 and partition data 600 are stored in the database unit 160. Additionally, motion command data 700 matching specific motion data is also stored.

The split area data 600 is composed of three-dimensional coordinates for three of the four corners of the screen of the output device connected to the rendering device when the origin is the center point on the screen of the output device located in the center of the output module 200. do. Referring to FIG. 4, when the center of the second output device (200b) among the three output devices connected horizontally is set to the coordinate origin point (0,0,0), the division area of the second output device (200b) Data 600b consists of Pc2(-8,5,0) in the upper left, Pa2(-8,-5,0) in the lower left, and Pb2(8,-5,0) in the lower right.

The division data 600a of the first output device 200a is Pc1(-22,5,8) at the top left, Pa1(-22,-5,8) at the bottom left, and Pb1(-8,) at the bottom right. -5,0), and similarly, the partition data 600c of the third output device 200c is Pc3(8,5,0) at the upper left, Pa3(8,-5,0) at the lower left, It is composed of Pb3(22,-5,8) in the lower right.

The model house image 500 is indoor space data that constitutes the indoor space such as windows, pillars, and walls of the model house image 500.

(512), view data 511, indoor space data corresponding to the view image seen outside the window of indoor space data 512

It is composed of an object 510 including furniture data 513 placed within 512 and lighting data 514 by a light source. This is to change data for each object when rendering.

The motion command data 700 is command data that matches specific motion data and induces a change in the model house image 500. For example, the motion data for 'raising one hand' is matched with the motion command data 'turn on the interior light', and the motion data for 'raising both hands' is matched with the motion command data for 'turning off the interior light'. , The motion data of 'movement of pushing one hand to the left or right' is matched with the motion command data of 'moving the position of a specific piece of furniture according to the direction of movement of the hand', and the motion data of 'movement of turning one's head' is matched with the motion command data of 'moving one's head.' Motion command data such as 'change the viewpoint of the currently visible screen depending on the direction of rotation' may be matched. In particular, specializing in model houses, the indoor space data 512 can be changed to change the wallpaper or flooring, the furniture data 513 can be changed to change the type and arrangement of furniture, or the view data 511 can be changed to change the degree of indoor lighting. , the foreground can be compared, and by changing the lighting data 514, the indoor space atmosphere according to the indoor lighting brightness and color can be checked (see Figure 7).

The content loading unit 110 serves to load the model house image 500 stored in the database unit 160. In particular, it may further include an object extraction unit 111 that extracts the loaded model house image 500 for each object 510, including view data 511, indoor space data 512, furniture data 513, The lighting data 514 is extracted separately.

The partition loading unit 120 serves to load partition data 600 stored in the database unit 160. In the embodiment of FIG. 4, the first rendering device 100a has a partition area composed of Pc1 (-22, 5, 8), Pa1 (-22, - 5, 8), and Pb1 (-8, - 5, 0). The data 600a is loaded, and the second rendering device 100b and the third rendering device 100c similarly load their respective partition data 600b and 600c.

The motion processing unit 130 receives motion data and retrieves motion command data matching the motion data from the database unit 160, and is composed of a motion data receiving unit 131 and a motion command data generating unit 132. do.

The motion data receiving unit 131 receives motion data from the motion recognition unit 300, and the motion command data generating unit 132 searches the motion data received from the motion data receiving unit 131 in the database unit 160. The motion command data (700) matching this is loaded. For example, when the motion data of ‘raising one hand’ is received, the corresponding motion command data of ‘turn on the interior light’ is retrieved.

The rendering unit 140 serves to divide the model house image 500 according to the division area data 600 and render the divided model house image based on the motion command data 700, and displays the screen as shown in FIG. 2. It consists of a division unit 141, a motion command data processing unit 142, a synchronization unit 143, and a GPU parallel processing unit 144.

The screen splitter 141 extracts a model house image divided into a rectangular area composed of the coordinates of the split area data 600 when the center of the model house image 500 is the coordinate origin (0,0,0). . 4, in the first rendering device 100a, Pc1(-22,5,8), Pa1(-22,-5,8), Pb1(-8,-5, A rectangular area with 0) as its three corners is extracted, and in the second rendering device 100b, a rectangular area with the coordinates of the segment data 600b as its three corners is extracted, and in the third rendering device 100c, A rectangular area with three corners as coordinates of the partition data 600c is extracted.

The motion command data processing unit 142 serves to generate rendering data to render a model house image based on the motion command data 700. For example, rendering data to change the illumination settings in a segmented model house image is generated according to the motion command data such as 'Turn on the interior lights', or according to the motion command data such as 'Move the position of a specific piece of furniture according to the direction of hand movement'. Rendering data that extracts specific furniture and creates rendering data to move, or changes the camera viewpoint setting that renders the model house image (500) according to the motion command data of 'change the viewpoint of the currently visible screen depending on the direction of turning the head'. etc. to create .

The synchronization unit 143 transmits the rendering data generated in the motion command data processing unit 142 to the server when the rendering device is a client, and the rendering data generated in the motion command data processing unit 142 when the rendering device is a server. Alternatively, it serves to transmit rendering data transmitted from another rendering device to the remaining rendering devices.

In the embodiment of FIG. 4, the case where the first rendering device 100a is a server, and the second rendering device 100b and the third rendering device 100c are clients are described as follows.

When rendering data is generated in the motion command data processing unit 142 of the first rendering device 100a, it is transmitted to the second and third rendering devices 100b and 100c, which are clients. In contrast, when rendering data is generated in the motion command data processing unit 142 of the second rendering device 100b, it is transmitted to the first rendering device 100a, which receives it from other clients, That is, it is transmitted to the third rendering device 100c. Therefore, all rendering devices have synchronized rendering data.

The synchronization unit 143 renders the model house content 500 using parallel GPU computing using the rendering data transmitted by the synchronization unit 143. In other words, numerous objects can be processed in real time by rendering them in parallel using the GPU. Therefore, it is natural that the rendering device must have multiple GPUs built in to enable parallel GPU computing, as shown in Figure 3.

The split content transmission unit 150 serves to transmit the split model house image rendered by the rendering unit 140 to an image output device connected to the rendering device. That is, it is transmitted from the first rendering device 100a to the first output device 200a, and from the second parallel rendering device 100b to the second output device 200b. This creates an experiential model house space as shown in Figure 5.

So far, the present invention has been examined focusing on its preferred embodiments. A person skilled in the art to which the present invention pertains will understand that the present invention may be implemented in a modified form without departing from the essential characteristics of the present invention. Therefore, the disclosed embodiments should be considered from an illustrative rather than a restrictive perspective. The scope of the present invention is indicated in the claims rather than the foregoing description, and all differences within the equivalent scope should be construed as being included in the present invention.

Claims (3)

An output module 200 consisting of at least one output device that is installed continuously so that the boundaries of each screen come into contact with each other to form a model house implementation space;
A parallel rendering unit 100 composed of a plurality of rendering devices correspondingly connected to the at least one output device;
It includes a motion recognition unit 300 that recognizes the user's motion, generates motion data, and transmits it to the parallel rendering unit 100,
The parallel rendering unit 100 is
Among multiple rendering devices, one specific rendering device is configured as a server, and the remaining rendering devices are configured as clients.
The model house image 500 and the segmentation area data 600 are stored in each rendering device, and the model house image 500 is
Indoor spatial data (512) that constitutes indoor space,
View data 511 corresponding to the view image seen from the window of the indoor spatial data 512, furniture data 513 placed within the indoor spatial data 512,
It consists of an object 510 containing illumination data 514 by a light source, and the rendering device
The database unit 160, which stores the model house image 500 and the partition data 600, the content loading unit 110, which loads the model house image 500 stored in the database unit 160, and the database unit 160. A partition loading unit 120 that loads the stored partition data 600,
A motion processing unit 130 that receives motion data and loads motion command data matching the motion data from the database unit 160;
A rendering unit 140 that divides the model house image 500 according to the division area data 600 and renders the divided model house image based on the motion command data 700;
It includes a split content transmission unit 150 that transmits the split model house image rendered by the rendering unit 140 to an output device connected to the rendering device,
The content loading unit 110 further includes an object extraction unit 111 that extracts the loaded model house image 500 for each object 510.
Interactive high-quality video system based on real-time parallel rendering
The method of claim 1, wherein the rendering unit 140
A screen splitter 141 that extracts a model house image divided into rectangular regions composed of the coordinates of the division area data 600 when the center of the model house image 500 is taken as the origin;
A motion command data processing unit (142) that generates rendering data to render a model house image based on the motion command data (700),
If the rendering device is a client, the rendering data generated in the motion command data processing unit 142 is transmitted to the server. If the rendering device is a server, the rendering data generated in the motion command data processing unit 142 or another parallel rendering device is transmitted. A synchronization unit 143 that transmits the transmitted rendering data to the remaining rendering devices,
Characterized by comprising a GPU parallel processing unit 143 that renders the model house image 500 using parallel GPU computing using the rendering data transmitted by the synchronization unit 143.
An interactive, high-quality video system based on real-time parallel rendering.
The method of claim 1 or 2, wherein the partition data 600 is
When the center point on the screen of the output device located in the center of the output module 200 is taken as the origin, it is characterized in that it is composed of three-dimensional coordinates for three of the four corners of the screen of the output device connected to the rendering device.
An interactive, high-quality video system based on real-time parallel rendering.