KR20240034986A - Method and device for ray tracing to route light based on distance between objects - Google Patents

Abstract

According to an embodiment of the present disclosure, a ray tracing-based rendering method for metaverse implementation generates a primary ray projected from a viewpoint toward a virtual space and creates a first intersection point (hit) where the primary ray intersects a specific object. step of checking point); According to the attribute information of the specific object, a secondary ray occurring at the first intersection is generated, a second intersection point where the secondary ray intersects another object is identified, and the distance between the first intersection point and the second intersection point is determined. step of understanding; and determining a path of the secondary ray and a sub-ray derived from the secondary ray based on the distance and deriving a color value of a pixel corresponding to the first intersection according to the path, wherein the virtual An acceleration structure that divides the space into a hierarchical tree form is built in advance, and the object is an object included in the virtual space or a primitive that constitutes the object.

Description

Ray tracing method and device for setting the path of light according to the distance between objects {METHOD AND DEVICE FOR RAY TRACING TO ROUTE LIGHT BASED ON DISTANCE BETWEEN OBJECTS}

This disclosure relates to a ray tracing method and device for setting a light path according to the distance between objects to implement a metaverse.

With the advancement of computer technology, it has become possible to create a virtual space of hyper reality. In order to create a space of hyper reality, the process of rendering real objects and moving them to virtual space is necessary.

Rendering is a computer graphics technology used in games, etc. to express the real world or the virtual world. It refers to the process of creating an image from a model using a computer program. This model is expressed in a data structure or a strongly defined language. This is a description of a constructed three-dimensional object.

It contains geometry, perspective, texture mapping, lighting, and shading information. These images are called digital images or raster graphics images. In the graphics pipeline, rendering is the final step in providing the final look of models and animations.

Through this rendering technology, it is possible to have a realistic experience like seeing an object or place in person without having to create or visit it.

Recently, due to the prolonged non-face-to-face situation and the surge in real estate prices, it has become increasingly difficult to set up exhibitions, galleries, or model houses in real spaces. On the other hand, technological limitations have recently been overcome, making it possible to create exhibitions, galleries, or model houses in virtual spaces. Users can view exhibitions, galleries, or model houses that are no different from the real thing in a virtual space. However, because the cost and time to implement a virtual space are excessive, general users must entrust the implementation of the virtual space to a professional company. In addition, modifying various objects in the virtual space is inconvenient because you have to request one by one from a professional company. Therefore, there is a need for technology that allows ordinary users to easily create a virtual space and hold a virtual exhibition, gallery, or model house.

Meanwhile, rendering technology includes rasterization, which creates an image by projecting a 3D object onto the screen, and creating an image by tracing the path of light by projecting a ray toward each pixel on the screen from the camera or user's perspective. There are ray tracing techniques, etc.

Among these, ray tracing is a rendering method based on global illumination. It considers the effect of light reflected or refracted from other objects on the image of the current object and expresses reflection, refraction, and shadow effects, creating a realistic virtual space. can be created.

In this way, there is an advantage in securing high-quality images because the physical properties of light are naturally reflected in the rendering results. However, since the paths of numerous lights reaching from the object to the light source are traced, a relatively large amount of calculations and a wide memory bandwidth are required. It is required and there are difficulties in performing rendering at high speed.

The purpose of an embodiment of the present disclosure is to provide a ray tracing technique with a reduced computational amount by setting a different path of light reaching the light source from the primary ray in consideration of the distance between objects.

However, the technical challenges that this embodiment aims to achieve are not limited to the technical challenges described above, and other technical challenges may exist.

According to an embodiment of the present disclosure, a ray tracing-based rendering method for metaverse implementation generates a primary ray projected from a viewpoint toward a virtual space and creates a first intersection point (hit) where the primary ray intersects a specific object. step of checking point); According to the attribute information of the specific object, a secondary ray occurring at the first intersection is generated, a second intersection point where the secondary ray intersects another object is identified, and the distance between the first intersection point and the second intersection point is determined. step of understanding; and determining a path of the secondary ray and a sub-ray derived from the secondary ray based on the distance and deriving a color value of a pixel corresponding to the first intersection according to the path, wherein the virtual An acceleration structure that divides the space into a hierarchical tree form is built in advance, and the object is an object included in the virtual space or a primitive that constitutes the object.

According to one embodiment of the present disclosure, determining the distance includes forming a first sphere with a preset first radius and a second sphere with a preset second radius centered on the first intersection; Setting the inside of the first sphere as a first area, the outside of the second sphere as a second area, and the overlapping area between the outside of the first sphere and the inside of the second sphere as a third area; And if the second intersection point is located in the first area, the other object is determined to be a first object, and if the second intersection point is located in the second area, the other object is determined to be a second object, and When the second intersection is located in the third area, determining the other object as a third object, wherein the second radius is set larger than the first radius, and the first object is the specific object is an object adjacent to , the second object is an object far away from the specific object, and the third object is an object that is not involved in the color value of the pixel.

According to an embodiment of the present disclosure, the step of determining the distance includes, when a bounding box located only within the third area exists, a node corresponding to the bounding box and a child included in the node. modifying the acceleration structure by removing all nodes and objects; Searching the modified acceleration structure to identify objects included in nodes that intersect the second ray and confirming the second intersection point; and determining which of the first to third areas the second intersection point is located in.

According to an embodiment of the present disclosure, the step of determining the distance includes determining that a node to which the specific object and the other object commonly belong is located in a preset first layer in the acceleration structure and a lower layer included in the first layer. If located, the other object is determined to be the first object, and if the node to which the specific object and the other object commonly belong is located in the second layer, which is a higher layer than the first layer, the other object is determined to be the second object. If it is determined to be an object, and at least one layer exists between the first layer and the second layer, and a node to which the specific object and the other object commonly belong is located in one of the at least one layer, and determining the other object as a third object.

According to an embodiment of the present disclosure, the step of deriving the color value of the pixel includes selecting a sub-ray derived from the secondary ray according to attribute information of the other object when the other object is determined to be the first object. A subsequent intersection point that intersects another object is sequentially generated and recursively checked, and if the other object is determined to be the second object, the second intersection point is a separate light source different from the light source already included in the virtual space. , and if the other object is determined to be the third object, it is considered that the secondary ray projected to the third object has not occurred.

According to an embodiment of the present disclosure, the process of recursively checking the subsequent intersection point is such that when one of the sub-rays intersects an object determined to be the third object, the one sub-ray does not occur. If one of the sub-rays intersects an object determined to be the second object, the subsequent intersection point by the one sub-ray is set as another light source.

According to an embodiment of the present disclosure, the step of deriving the color value of the pixel includes determining the second intersection point according to the distance between the viewpoint and the second intersection point when the other object is determined to be the second object. It includes setting the light source as a separate light source different from the light source already included in the virtual space, or setting it as a light source that does not reflect or refract light in the other object.

According to an embodiment of the present disclosure, the step of deriving the color value of the pixel includes the color value of the second intersection formed by intersecting the first object by the secondary ray and the second intersection formed by intersecting the second object. When all color values of two intersection points are collected, the average value of each collected color value or the color value showing the highest level among each collected color value is included in the calculation to derive the color value of the pixel.

An embodiment of the present disclosure provides a ray tracing technique that significantly reduces the amount of calculations for rendering but ensures realism, which contributes to reducing modeling costs and implementing a stable metaverse.

Figure 1 is a block diagram of a metaverse system according to an embodiment of the present disclosure.
Figure 2 is a diagram for explaining the hardware of a metaverse system according to an embodiment of the present disclosure.
Figure 3 is a flowchart showing a method of operating a metaverse system according to an embodiment of the present disclosure.
Figure 4 is a flowchart showing a method of operating a metaverse system according to an embodiment of the present disclosure.
Figure 5 is a flowchart showing the operation of the metaverse system according to an embodiment of the present disclosure.
FIG. 6 is a flowchart illustrating a method of operating a metaverse system according to an embodiment of the present disclosure.
Figure 7 is a diagram for explaining a process for visualizing information related to the amount of sunlight in virtual space according to an embodiment of the present disclosure.
Figure 8 is a diagram for explaining a process for visualizing information related to the amount of sunlight in virtual space according to an embodiment of the present disclosure.
Figure 9 is a diagram for explaining a process for visualizing information related to the amount of sunlight in virtual space according to an embodiment of the present disclosure.
Figure 10 is a diagram for explaining the operation of the metaverse system according to an embodiment of the present disclosure.
Figure 11 is a diagram for explaining the operation of the metaverse system according to an embodiment of the present disclosure.
Figure 12 shows a screen displayed on the user terminal 130 according to an embodiment of the present disclosure.
Figure 13 is a flowchart showing a method of operating a metaverse system according to an embodiment of the present disclosure.
Figure 14 is a flowchart showing a method of operating a metaverse system according to an embodiment of the present disclosure.
FIG. 15 is a flowchart illustrating a method of operating a metaverse system according to an embodiment of the present disclosure.
Figure 16 is a flowchart showing a method of operating a metaverse system according to an embodiment of the present disclosure.
Figure 17 is a flowchart showing a method of operating a metaverse system according to an embodiment of the present disclosure.
Figure 18 is a flowchart showing a method of operating a metaverse system according to an embodiment of the present disclosure.
Figure 19 is a flowchart showing the operation of the metaverse system according to an embodiment of the present disclosure.
Figure 20 is a diagram for explaining a method for displaying information about a parking space according to an embodiment of the present disclosure.
FIG. 21 is a diagram for explaining a method for providing information about a real space according to an embodiment of the present disclosure.
Figure 22 is a conceptual diagram for explaining ray tracing.
Figure 23 is a block diagram of the structure of a ray tracing device according to an embodiment of the present disclosure.
Figure 24 is a conceptual diagram for explaining the acceleration structure.
Figure 25 is a flowchart of a ray tracing-based rendering method according to an embodiment of the present disclosure.
Figure 26 is a flowchart of a method for determining the distance between objects according to an embodiment of the present disclosure.
Figure 27 is an example diagram of virtual space projected onto a plane to explain a method of determining the distance between objects according to an embodiment of the present disclosure.
Figure 28 is an example of an acceleration structure to explain another embodiment of a method for determining the distance between objects.
Figure 29 is a flowchart of a method for deriving the color value of a pixel according to an embodiment of the present disclosure.
Figure 30 is an example diagram of virtual space projected onto a plane to explain a method of deriving the color value of a pixel according to an embodiment of the present disclosure.

Advantages and features of the disclosed embodiments and methods for achieving them will become clear by referring to the embodiments described below in conjunction with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below and may be implemented in various different forms. The present embodiments are merely provided to ensure that the present disclosure is complete and to those skilled in the art to which the present disclosure pertains. It is only provided to fully inform the user of the scope of the invention.

Terms used in this specification will be briefly described, and the disclosed embodiments will be described in detail.

The terms used in this specification are general terms that are currently widely used as much as possible while considering the function in the present disclosure, but this may vary depending on the intention or precedent of a technician working in the related field, the emergence of new technology, etc. In addition, in certain cases, there are terms arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the relevant invention. Therefore, the terms used in this disclosure should be defined based on the meaning of the term and the overall content of this disclosure, rather than simply the name of the term.

In this specification, singular expressions include plural expressions, unless the context clearly specifies the singular. Additionally, plural expressions include singular expressions, unless the context clearly specifies plural expressions.

When it is said that a part "includes" a certain element throughout the specification, this means that, unless specifically stated to the contrary, it does not exclude other elements but may further include other elements.

Additionally, the term “unit” used in the specification refers to a software or hardware component, and the “unit” performs certain roles. However, “wealth” is not limited to software or hardware. The “copy” may be configured to reside on an addressable storage medium and may be configured to run on one or more processors. Thus, as an example, “part” refers to software components, such as object-oriented software components, class components, and task components, processes, functions, properties, procedures, Includes subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functionality provided within the components and “parts” may be combined into smaller numbers of components and “parts” or may be further separated into additional components and “parts”.

According to one embodiment of the present disclosure, “unit” may be implemented with a processor and memory. The term “processor” should be interpreted broadly to include general purpose processors, central processing units (CPUs), microprocessors, digital signal processors (DSPs), controllers, microcontrollers, state machines, etc. In some contexts, “processor” may refer to an application-specific integrated circuit (ASIC), programmable logic device (PLD), field programmable gate array (FPGA), etc. The term “processor” refers to a combination of processing devices, for example, a combination of a DSP and a microprocessor, a combination of a plurality of microprocessors, a combination of one or more microprocessors in combination with a DSP core, or any other such combination of configurations. It may also refer to

The term “memory” should be interpreted broadly to include any electronic component capable of storing electronic information. The terms memory include random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable-programmable read-only memory (EPROM), electrical may refer to various types of processor-readable media, such as erasable PROM (EEPROM), flash memory, magnetic or optical data storage, registers, etc. A memory is said to be in electronic communication with a processor if the processor can read information from and/or write information to the memory. The memory integrated into the processor is in electronic communication with the processor.

Below, with reference to the attached drawings, embodiments will be described in detail so that those skilled in the art can easily implement them. In order to clearly explain the present disclosure in the drawings, parts unrelated to the description are omitted.

Figure 1 is a block diagram of a metaverse system according to an embodiment of the present disclosure.

The metaverse system 100 may include a server 110, an administrator terminal 120, and a user terminal 130. The server 110 is a device in which the metaverse is implemented. In other words, the server 110 is a device in which virtual space is rendered. The server 110 may be connected to the administrator terminal 120 and the user terminal 130 by wire or wirelessly. The administrator terminal 120 and the user terminal 130 can transmit and receive data with each other through the server 110.

The metaverse system 100 can implement a metaverse using predetermined objects. The predetermined object may be received from the user terminal 130 or the manager terminal 120 to the server 110. A predetermined object may be created in the server 110 based on data received from the user terminal 130 or the manager terminal 120. Objects can refer to various objects or finishing materials that make up a virtual space. A virtual space is a three-dimensional space, and the virtual space can be filled with objects or finishing materials such as walls (wallpaper), furniture, street trees, stairs, ceilings, floors, lights, windows, and doors. Various objects or finishing materials placed in this virtual space can be called objects.

Based on a control signal from the administrator terminal 120, the server 110 can create a virtual space based on a predetermined object and fill the virtual space using a pre-stored object.

Based on the control signal from the user terminal 130, the server 110 can place the user terminal or a virtual person corresponding to the user in the virtual space. Additionally, based on the control signal from the user terminal 130, the server 110 can control the virtual character to move around the virtual space and control the virtual character to interact with objects. Additionally, during the interaction process, the server 110 may transform objects in the virtual space.

Figure 2 is a diagram for explaining the hardware of a metaverse system according to an embodiment of the present disclosure.

The server 110, administrator terminal 120, and user terminal 130 included in the metaverse system 100 may include a processor 210 and memory 220. The processor 210 may perform operations based on instructions stored in the memory 220. However, it is not limited to this, and the server 110, administrator terminal 120, and user terminal 130 may include only the processor 210 and not memory. The processor 210 may be set to output a preset signal to an output line for a preset time based on the input signal. Each component of the server 110, the administrator terminal 120, and the user terminal 130 may perform preset operations according to signals.

Hereinafter, the operations of the server 110, the administrator terminal 120, and the user terminal 130 will be described in more detail.

Figure 3 is a flowchart showing a method of operating a metaverse system according to an embodiment of the present disclosure.

The administrator terminal 120 may acquire the properties of the virtual space based on the administrator's input. The properties of virtual space are necessary to create a virtual space, such as information on the type of virtual space, size information of the virtual space, location information of the real space corresponding to the virtual space, time information of the virtual space, shape of the virtual space, and virtual space. It may include at least one of the price of the space or the owner of the virtual space.

Information on the type of virtual space may be related to the purpose of the virtual space. Information on the type of virtual space may include at least one of an exhibition hall, an art gallery, a single-family house, a multi-family house, an officetel, a studio, an apartment, a temple, and a church.

The size information of the virtual space may represent the area of the ground surface of the virtual space. Information on the size of the virtual space can be expressed as the number of pixels (voxels). For example, the size information of the virtual space may include the number of pixels (voxels) included in the horizontal side and the number of pixels (voxels) included in the vertical side. Additionally, the size information of the virtual space may be expressed as the area of the real space, and the unit may be square meters or pyeong. The size information of the virtual space may include the height of the space. The metaverse system of the present disclosure can predetermine the ratio between the length of one side of a pixel (voxel) and the unit length of real space. For example, 1 cm of real space may correspond to the length of 1 to 100 pixels lined up in one dimension. However, it is not limited to this, and in the metaverse system of the present disclosure, the ratio of the length of one side of a pixel (voxel) and the unit length of real space can dynamically change. For example, as the user enlarges the screen using the user terminal, the number of one-dimensionally arranged pixels corresponding to 1 cm of real space may increase. Additionally, as the user reduces the screen using the user terminal, the number of one-dimensionally arranged pixels corresponding to 1 cm of real space may decrease.

Location information in real space corresponding to virtual space may include GPS coordinate information or address information in real space. The server of the metaverse system of the present disclosure can use the location information of the real space to implement the virtual space as hyper reality at a location corresponding to the real space. Location information in real space may be 3D coordinate information. Location information in real space may include latitude information, longitude information, and height (altitude) information. Height information included in the location information of the real space may include information on the number of floors of the corresponding real space within the building. For example, in an apartment building, height information may include information related to which floor the actual space corresponding to one household is located on. The location information provided may further include height information in addition to GPS information, which is two-dimensional information. The height included in the location information can be used by the server 110 or the user terminal to automatically generate terrain such as a river or mountain, or a feature such as a tree or building in the virtual space based on the location information of the real space corresponding to the virtual space. The server 110 or the user terminal 130 may previously store information about terrain or features corresponding to location information in real space. Location information in virtual space may be the same as location information in real space. In other words, the virtual world can have the same coordinate system as the real world. Therefore, when a user inputs real space location information into the user terminal, the user can move to a virtual space corresponding to the real space within the virtual world. This is because the same coordinates are used in virtual space as in real space.

Time information in virtual space may represent virtual time. The virtual time may include at least one of date, day, hour, minute, or second. Based on the virtual time, the server 110 or the user terminal 130 determines the position of the sun, the position of the moon, whether street lights are turned on, whether headlights of a car are turned on, and whether the virtual space and lights around the virtual space are turned on. You can decide. In this way, the virtual space and the light around the virtual space are automatically controlled based on the time information of the virtual space, so the user can feel the virtual space as a real space using the user terminal. Time information in virtual space may be the same as the current time. However, it is not limited to this. Time information in virtual space may be determined based on location information in real space corresponding to the virtual space. That is, the server 110 can determine the time zone corresponding to the location information of the virtual space as the time information of the virtual space. The server 110 may determine time information in the virtual space to be different from the actual time based on the input of the administrator terminal 120 or the user terminal 130.

The shape of the virtual space may be a three-dimensional polyhedron shape. The shape of the virtual space may be, for example, a rectangular parallelepiped. However, it is not limited to this, and if the type information of the virtual space is an apartment, it may have a complex polyhedral shape because it includes a plurality of rooms, a living room, and a kitchen. Additionally, in the case of a virtual space with multiple layers, polyhedra can be stacked top and bottom to form one virtual space.

The process by which the administrator terminal 120 acquires the properties of the virtual space will be described in more detail later.

The server 110 may perform step 311 of acquiring virtual space properties including time information of the virtual space and location information of the real space from the administrator terminal 120. The server 110 may perform step 311 of acquiring properties of a virtual object or virtual space from the user terminal 130 or the administrator terminal 120. Since time information in virtual space and location information in real space have been previously described above, detailed explanations will be omitted. The properties of the virtual space may include information for the server 110 to create the framework of the virtual space. The server 110 can distinguish between areas that the user can access and areas that the user cannot access based on the properties of the virtual space. For example, if the virtual space is a house, the server 110 can position the user's viewpoint inside the virtual space of the house. In other words, it may be difficult for the user to move the viewpoint outside the virtual space using the user terminal. However, it is not limited to this, and the server 110 may ensure that the user's viewpoint is located within a predetermined range from the virtual space. The user's viewpoint may refer to location information within the virtual space from which the user views the virtual space.

The server 110 may perform step 321 of creating a three-dimensional virtual space based on the properties of the virtual space. The server 110 may perform step 321 of creating a three-dimensional virtual space based on the properties of the basic virtual object and the properties of the virtual space. The server 110 may obtain the shapes of a plurality of candidate virtual spaces stored in advance from memory based on the type information of the virtual space included in the properties of the virtual space. Additionally, the server 110 may obtain a plurality of candidate virtual objects that can be placed in the virtual space based on the type information of the virtual space. The server 110 determines the shape of one virtual space among the shapes of a plurality of candidate virtual spaces based on a signal from the administrator terminal 120, and determines a virtual object to be placed in the virtual space among the plurality of candidate virtual objects. You can.

The server 110 can load a three-dimensional (two-dimensional) map corresponding to location information in real space from a pre-stored virtual world. Virtual world includes virtual space. A virtual world is a larger space than a virtual space. A virtual world may be a metaverse that embodies a city, country, or earth. The virtual world may be a 3D map. Here, the virtual world may include a map that displays terrain and features in three dimensions. According to various embodiments of the present disclosure, a virtual world may be provided by a company, a national agency, or a local government. For example, the 3D map may be the same as the 3D map provided by the Seoul Metropolitan Government. In the present disclosure, the 3D map may be replaced with a 2D map.

Virtual space can have a hierarchy. As already explained, the virtual world may include a virtual space. A virtual world can be included in a virtual Earth. Additionally, the virtual Earth can be included in the virtual universe. In this way, a virtual space can include a smaller space, and a virtual space can include a larger space. Properties of virtual space may include layer information. Hierarchy information can be displayed as numbers. The larger the hierarchical information, the wider the space can be represented. However, it is not limited to this, and as hierarchical information becomes smaller, it may represent a wider space. For example, if the hierarchical information is level 1, it may indicate that the virtual space is a unit, indoor space, or room. Here, a unit may mean a unit space such as 1 square meter or 1 pyeong. Indoor space may refer to a living room, room, or kitchen. Additionally, if the hierarchical information is level 2, it can indicate that the virtual space is a unit of a house or a virtual space transaction. Additionally, if the hierarchy information is level 3, it may indicate that the virtual space is one apartment building. Additionally, if the hierarchy information is level 4, it may indicate that the virtual space is an apartment complex. Additionally, if the hierarchical information is level 5, it can indicate that the virtual space is a city. In this way, the hierarchical information of virtual space can be infinitely expanded.

The server 110 may determine the shape of the virtual space based on the shape of the virtual space included in the properties of the virtual space. The virtual space may have a polyhedral shape. The virtual space may include floors, walls, windows, doors, and ceilings. A virtual space may be a space with a single layer, but is not limited to this and may also be a space with multiple layers.

The server 110 may determine the shape of the virtual space based on the size information of the virtual space included in the virtual space properties. For example, if the shape of the virtual space is determined to be a rectangular parallelepiped, the server 110 may automatically determine the shape of the virtual space based on the size information of the virtual space. Size information of the virtual space may include horizontal width, vertical width, and height.

As already explained, location information in real space may be three-dimensional coordinate information. That is, the location in real space may include not only location information on the ground but also height information. The server 110 may place a polyhedral-shaped virtual space in a location corresponding to the location information of the real space on the 3D map.

In addition, the server 110 determines the position of the sun, the position of the moon, whether street lights are turned on, whether headlights of cars are turned on, and the lighting in the virtual space and around the virtual space based on the time information of the virtual space included in the properties of the virtual space. You can decide whether to turn it on or not. For example, if the time information in the virtual space is daytime, objects that can emit light may not light up. Additionally, if the time information in the virtual space is night, objects that can emit light may not light up. Objects that can emit light may be the sun, the moon, indoor lighting, street lights, or cars.

The server 110 can place basic virtual objects placed around the virtual space around the virtual space without receiving an administrator's input from the administrator terminal 120. The basic virtual object may be placed outside the virtual space to create the atmosphere of the virtual space and implement hyper reality. The properties of the basic virtual object include at least the object's identification information, type, texture, color, shape, size, location, manufacturer, manufacturing year, product identification number, and seller of the real object corresponding to the virtual object. It can contain one. The server 110 may place virtual objects inside and around the virtual space based on the properties of the basic virtual object. The default virtual object may be the initial setting of the virtual object for the virtual space. For example, it may include trees, sun, moon, street lights, trees, grass, terrain, structures (buildings), or cars outside the virtual space. The server 110 may store pre-researched data for each location in real space. The server 110 may automatically place basic virtual objects around the virtual space based on the location of the real space. Pre-researched data can be generated by applying an object recognition program to street view images. Based on the administrator's input received from the administrator terminal 120, the server 110 may modify the location, type, color, shape, or number of the basic virtual object.

The properties of the virtual object may include at least one of the manufacturer of the real object corresponding to the virtual object, the manufacturing year of the real object, the product identification number of the real object, the price, and the seller. Therefore, users can easily decorate the real space based on the properties of virtual objects.

Additionally, the server 110 may adjust the number of vehicles on the road based on time information in virtual space. For example, the server 110 may receive information about traffic volume from an external server. The server 110 can receive information about traffic volume according to day of the week, road, and time. Server 110 may store traffic information for a predetermined period of time. Here, the predetermined time may be between 4 weeks and 52 weeks. Additionally, the server 110 may obtain an average traffic volume based on the traffic volume obtained during a predetermined time. The server 110 may obtain the average traffic volume on roads around a location in real space corresponding to the virtual space based on time information in the virtual space. Here, time information in virtual space may be related to day of the week or time. Additionally, the server 110 may determine the number of vehicles, which are virtual objects, corresponding to the average traffic volume on surrounding roads. The server 110 may previously store a table that corresponds the average traffic volume to the number of vehicles. Additionally, the server 110 may control the user terminal 130 to display a determined number of vehicles on the road. The user terminal 130 may generate noise based on the determined number of vehicles. In other words, the larger the number of vehicles determined, the larger the noise may be. Users can experience the level of noise in a virtual space without having to go to the actual space. Additionally, if the time is night, the server 110 may cause vehicles to turn on their headlights. Therefore, the user can experience light pollution based on the headlights of the vehicle displayed on the user terminal 130, or enjoy the atmosphere of the night in real space in advance in virtual space.

Properties of the virtual space may include the price of the virtual space. Virtual space can be traded between users. The price of the virtual space can be initially determined by an administrator using the server 110 or the administrator terminal 120. The server 110 may determine the price by multiplying the size of the virtual space by a predetermined price per unit size. Prices can be set in real-world cash. However, it is not limited to this, and the price of the virtual space can be set in the virtual currency used in the virtual space. Cryptocurrency and real-world cash can be exchanged at a predetermined exchange rate.

Properties of the virtual space may include owner information of the virtual space. Owner information may include the user ID or unique identifier of the virtual space. However, it is not limited to this, and the owner information of the virtual space may include at least one of the resident registration number, name, gender, mobile phone number, and account number of the user in the real world, or may be an encrypted code generated by a combination of these. there is. The owner of the virtual space may be granted permission to modify the virtual space by the server 110. The owner of the virtual space can use the user terminal 130 to modify the size or shape of the virtual space or modify virtual objects included in the virtual space. This will be explained later.

The server 110 may perform step 331 of obtaining a purchase signal for a three-dimensional virtual space from the user terminal 130. The purchase signal for the virtual space may include identification information related to the user terminal 130. Identification information related to the user terminal may include at least one of the user terminal's unique identification code, manufacturing code, phone number, and purchaser's identification information. The buyer's identification information may include at least one of the buyer's social security number, name, gender, mobile phone number, and account number in the real world, or may be an encrypted code generated by a combination of these. Buy signals for virtual space may include amounts of virtual currency. Here, the user terminal 130 may refer to the terminal of a user who wishes to make a purchase. The amount of virtual currency may represent the amount the user will pay to purchase virtual space. Users can also purchase a virtual space by paying a predetermined amount. However, it is not limited to this, and when the user enters the amount he or she can pay, the user terminal 130 can generate a purchase signal. That is, when at least one user submits a payable virtual currency to the server 110, such as in an auction, the server 110 can transfer ownership of the virtual space to the user who offers the highest price.

When the purchase signal for the virtual space satisfies predetermined purchase conditions, the server 110 changes owner information included in the properties of the virtual space into identification information related to the user terminal included in the purchase signal for the virtual space. (322) can be performed. There may be at least one purchase condition. For example, the server 110 may determine that the purchase condition has been satisfied if the amount of virtual currency included in the purchase signal is greater than or equal to the original owner's sale price. Additionally, the server 110 can check whether the purchase signal was sent by the original owner. For example, in order to prevent the original owner from purchasing his or her own virtual space in order to increase the price of the virtual space, the server 110 may prevent the server 110 from purchasing the virtual space it owns. Additionally, the server 110 may determine that the purchase condition has been satisfied based on the original owner of the virtual space approving the purchase signal.

In this way, when the purchase signal for the virtual space satisfies the predetermined purchase conditions, the server 110 changes the owner information included in the properties of the virtual space to identification information related to the user terminal included in the purchase signal for the virtual space. Step 322 can be performed. Identification information related to the user terminal may include at least one of the user terminal's unique identification code, manufacturing code, phone number, and purchaser's identification information.

The server 110 may perform step 332 of transmitting the properties of the virtual space to the user terminal 130. Server 110 may also transmit properties of the basic virtual object. Alternatively, the server 110 may transmit the properties of the basic virtual object changed by the existing user to the user terminal 130. The user terminal 130 may create and display a virtual space and an object based on the properties of the virtual space and the properties of the virtual object transmitted from the server 110. Users can check virtual space and virtual objects and modify them. Since the user terminal 130 locally stores the properties of the virtual space and the properties of the virtual object, the virtual space and the virtual object can be modified offline. However, it is not limited to this, and the user terminal 130 receives the properties of the virtual space and the properties of the virtual object from the server 110, and sends the properties of the modified virtual space and the properties of the modified virtual object to the server 110 in real time. ) can also be sent.

The user terminal 130 may perform step 333 of transmitting virtual object modification request information to the server 110. The user terminal 130 may further perform the step of transmitting virtual space modification request information to the server 110. As already described, the user terminal 130 can display a virtual space and virtual objects, and can modify at least one of the virtual space and virtual objects based on the user's input. The user terminal 130 may reflect the modified content in at least one of the properties of the virtual space and the properties of the virtual object. Additionally, the user terminal 130 can transmit virtual object modification request information to the server 110 so that the modification contents are also reflected in the server 110. Virtual object modification request information may include properties of the modified virtual object. Additionally, the virtual space modification request information may include properties of the modified virtual space. The virtual space modification request information may include identification information related to the user terminal.

The server 110 may perform step 323 of determining whether identification information related to the user terminal included in the virtual object modification request information matches owner information included in the properties of the virtual space. The server 110 checks whether the identification information related to the user terminal included in the virtual object modification request information matches the owner information included in the properties of the virtual space stored in the server 110, and only those with modification permission can access the virtual object. Alternatively, you can change the virtual space. When the properties of the modified virtual space and the properties of the modified virtual object are reflected in the server 110, the modified virtual space and the modified virtual object may be displayed on a third-party terminal. Therefore, if anyone modifies the virtual space and virtual objects, it is a violation of the virtual space owner's right to use the virtual space. Therefore, the server 110 can allow only the virtual space owner to modify the virtual space and virtual objects.

If the identification information related to the user terminal included in the virtual object modification request information matches the owner information included in the properties of the virtual space, the server may perform step 334 of acquiring the properties of the virtual object from the user terminal. .

As already described, the virtual object modification request information may include the properties of the modified virtual object, but the virtual object modification request information may be transmitted from the user terminal 130 to the server 110 separately from the virtual object properties. The user terminal 130 may transmit the properties of the virtual object to the server 110 whenever the virtual object is changed. That is, the server 110 can reflect the changes in real time whenever the virtual object is changed, and a third-party terminal can also check the modification process in real time. The owner of the user terminal 130 has the advantage of being able to show the interior design process in the form of a performance. The server 110 may store properties of virtual objects by time. However, it is not limited to this. After obtaining a signal indicating “completion” from the user, the user terminal 130 may transmit the properties of the virtual object to the server 110. That is, the server 110 may not retain information about the change process of virtual objects in the virtual space. However, in this case, the amount of data transmission between the server 110 and the user terminal 130 may be reduced.

Here, the virtual object may refer to a virtual object placed inside or outside the virtual space. The properties of the virtual object may include at least one of the object's identification information, type, texture, color, shape, size, and location. Types of virtual objects may include backgrounds (trees, grass, flower beds, artifacts), buildings (walls, stairs, windows, doors), furniture, cars, etc. Virtual objects may include basic virtual objects. Each virtual object may include identification information. The server 110, the administrator terminal 120, and the user terminal 130 may specify the virtual object based on the identification information of the virtual object. The server 110 may store 3D models and colors for virtual objects. Descriptions of the properties of a virtual object may be applied to the properties of a virtual item. While virtual objects can be used to decorate virtual spaces, virtual items can be used to decorate virtual people.

The server 110 may previously store texture information included in the properties of the virtual object. That is, the server 110 may previously store information on a plurality of textures that can be applied to the virtual object as a texture database. The texture information may include at least one of two-dimensional information, three-dimensional information, pattern, or color of the surface of the virtual object. For example, even if it is the same wall, the texture may change if the type of wallpaper changes. Likewise, the server 110 may set the surface feel of the virtual object differently based on texture information. Texture information can only express information about a unit area. In order to cover an area larger than the unit area of the virtual object, the server 110 may repeatedly reflect texture information on the surface of the virtual object.

Texture information may be two-dimensional information, but is not limited to this and may have a three-dimensional form. If the texture information is 3D information, hyper reality can be more easily implemented. For example, texture information about an inner wall, which is a virtual object, may be a pattern or color expressed in two-dimensional information. In the real world, most interior walls are finished with two-dimensional wallpaper, so the texture information for the interior wall may be sufficient as a pattern or color expressed on a two-dimensional surface. However, in the case of real-world exterior walls, they may include bricks and joints between bricks, and may have three-dimensional curves. In order to implement such a three-dimensional texture in virtual space, the texture information of the present disclosure may include a three-dimensional shape. Texture information may be information representing the relief form. In other words, the part that is exposed to the outside in the virtual space is implemented in three dimensions, but the part that is not exposed to the outside may be flat or not implemented. Three-dimensional texture information may include surface information that has a curve with respect to a flat reference surface, and patterns and colors expressed on the surface. The reference surface may be a surface in contact with a virtual object in a virtual space, and the surface information may be a surface opposite to the reference surface, which may be a surface shown to the user in the virtual space. Surface information may be a set of points with different distances to a reference surface.

The server 110 may include at least one piece of three-dimensional texture information to represent the outer surface of the virtual object. Additionally, the server 110 may configure the surface of a virtual object of a large area by repeatedly applying three-dimensional texture information for a unit area to the surface of the virtual object. The unit area may be a square. The distance of the edge of the surface information included in the texture information of the unit area to the reference plane may be constant. Therefore, even if the texture information of multiple unit areas is combined, a sense of continuity can be created. However, it is not limited to this.

The server 110 may set initial texture information for the virtual object. The server 110 may change the texture information reflected in the virtual object based on a signal from the administrator terminal 120 or the user terminal 130. More specifically, the server 110 may receive a texture identifier for a virtual object from the administrator terminal 120 or the user terminal 130. The server 110 may reflect texture information corresponding to the received texture identifier to the virtual object. Even for objects of the same shape, different texture information can be applied to provide the user with the feeling of a different virtual object. For example, when a virtual space is created by applying different texture information to the walls, users can receive different feelings from the virtual space. According to the present disclosure, a user can easily change intuition information applied to a virtual space by using information on a plurality of textures stored in a texture database, and can implement the virtual space closer to the real space. Additionally, users can feel the atmosphere emitted by the space in advance in the virtual space when different textured finishes are used in the real space.

Texture information may be classified by type of virtual object. The user terminal 130 or the administrator terminal 120 may first select the type of virtual object. The user terminal 130 or the administrator terminal 120 may display information on a plurality of textures corresponding to the type of the selected virtual object. The user terminal 130 or the administrator terminal 120 may select one texture information from among a plurality of texture information output based on the user's or administrator's input.

The server 110 may receive the properties of the virtual object from the administrator terminal 120 or the user terminal 130. The server 110 may place a virtual object in a virtual space based on the properties of the virtual object received from the administrator terminal 120 or the user terminal 130. That is, the server 110 may place a virtual object selected from the administrator terminal 120 or the user terminal 130 among a plurality of pre-stored candidate virtual objects based on the properties of the virtual object in the virtual space. Additionally, the properties of the virtual object may be input by an administrator or user, or may be automatically generated by the administrator terminal 120 or the user terminal 130. The server 110 may modify the color, shape, and size of the three-dimensional model of the virtual object based on the properties of the virtual object received from the administrator terminal 120 or the user terminal 130. The server 110 can generate an image of the virtual space in which the virtual object is pre-arranged by pre-arranging the virtual object in the virtual space, and transmit it to the user terminal 130. The user terminal 130 may have difficulty directly rendering virtual spaces and virtual objects due to insufficient graphics processing capabilities. The server 110 may generate an image of a virtual space where a virtual object is placed, compress the image, and transmit it to the user terminal 130. In addition, the server 110 receives the user's input from the user terminal 130, changes the viewpoint or direction of view, creates an image of the virtual space where the virtual object is placed, and then compresses the image and sends it to the user terminal 130. You can also send it. Therefore, an image of a virtual space where virtual objects are placed can be displayed even on a user terminal 130 that has insufficient processing power. Location information included in the properties of a virtual object may mean the location of the virtual object within the virtual space. Location information included in the properties of a virtual object can be expressed in a three-dimensional coordinate system. Location information included in the properties of the virtual object may include orientation information of the virtual object. Orientation information may include the placement direction and rotation degree of the virtual object in virtual space. The placement direction and degree of rotation can also be expressed in a three-dimensional coordinate system.

The server 110 may perform step 324 of placing a virtual object based on the properties of the virtual object in a three-dimensional virtual space. Here, the properties of the virtual object may be properties of the virtual object modified by the user terminal 130. If the user does not modify the properties of the existing virtual object, the properties of the existing virtual object may be maintained.

When performing the step of placing a virtual object based on the properties of the virtual object in a virtual space, the server 110 may further perform the following process. The server 110 may transmit an image of a virtual object placed in a virtual space to the user terminal 130. When transmitting, the server 110 may compress the video. Alternatively, the server 110 may transmit the properties of the virtual space or the properties of the virtual object to the user terminal 130. The user terminal 130 may directly render the virtual space and virtual object based on the properties of the virtual space or the properties of the virtual object. Additionally, the user terminal 130 may display rendered virtual space and virtual objects.

The server 110 can control the virtual object to be displayed together with the virtual space. The user terminal 130 and a third-party terminal can display virtual space and virtual objects on the screen.

According to various embodiments of the present disclosure, information about a three-dimensional virtual space may include properties of the virtual space and properties of a virtual object. Therefore, if the server 110 transmits only information about the virtual space to the user terminal 130, the user terminal 130 can display the virtual space and virtual objects implemented in the server 110 on the display.

According to various embodiments of the present disclosure, the server 110 may transmit information about a three-dimensional virtual space to the user terminal 130. The server 110 may transmit information about the three-dimensional virtual space to a third-party terminal. The server 110 may transmit some of the information about the three-dimensional virtual space to the user terminal 130 or a third-party terminal. For example, the server 110 may transmit only the information necessary for the user terminal 130 or a third-party terminal to display a virtual space and a virtual object. The user terminal 130 or a third-party terminal can receive information about the three-dimensional virtual space. The user terminal 130 or a third-party terminal can display virtual space and virtual objects on the display based on information about the three-dimensional virtual space.

According to various embodiments of the present disclosure, the user terminal 130 may perform a step of obtaining information about a virtual person. The user terminal 130 may obtain information about the virtual person from the memory of the user terminal 130 or receive it from the server 110.

Information about the virtual person may include rendering information about the virtual person's appearance. The virtual character's appearance may include the virtual character's skin color, body type model, and face model. Body type models may include child types, youth types, adult types, and elderly types. The body type model may include height information. The face model includes eye position, distance between eyes, eye size, eye shape, eye color, nose position, nose shape, nose height, mouth position, mouth shape, hair style, or This may include hair color, etc.

Information about a virtual person may include information about a possessed item (item). Possessed items may include clothes, accessories, bags, etc. worn by the virtual person. Information about the virtual person may include at least one of clothing identification information and clothing color. Identification information for clothes may vary depending on the shape of the clothes. Information about the virtual person may include at least one of the identification information of the accessory, the color of the accessory, the identification information of the bag, and the color of the bag. The user terminal 130 may display a virtual person in a virtual space based on information about the virtual person. In this disclosure, items are used to decorate virtual characters, and objects are used to decorate virtual spaces.

The user terminal 130 may perform step 335 of displaying at least one of a virtual space and a virtual object on the display. Additionally, the user terminal 130 may perform a step of displaying at least one of a virtual person, a virtual space, and a virtual object on the display based on at least one of information about a three-dimensional virtual space and information about a virtual person. . When the user terminal 130 implements a virtual space from a first-person perspective, virtual people may not be displayed on the display. The user can experience the virtual space from a first-person perspective through the user terminal 130. When the user terminal 130 implements a virtual space from a third-person perspective, a virtual person may be displayed on the display. Users can feel the atmosphere of the virtual space by experiencing their virtual character in the virtual space. Multiple users' virtual characters can operate in a virtual space. However, it is not limited to this, and only one user's virtual character can be active in the virtual space.

The user terminal 130 may perform the following process before performing step 335 of displaying at least one of a virtual space and a virtual object on the display. The user terminal 130 can display a three-dimensional (two-dimensional) map. Users can look around the 3D (2D) map and check the area for which they want to see information. The user terminal 130 may display items related to information about the real space displayed on the three-dimensional (two-dimensional) map based on the user's input. The user terminal 130 may display items related to information about the real space in a pop-up window. Alternatively, the user terminal 130 may display information about the real space on a three-dimensional (two-dimensional) map based on the user's input. The user terminal 130 may display information about the real space on at least one of the left, right, bottom, or top of the screen. Additionally, the user terminal 130 can display information about the real space at a location corresponding to the real space on a 3D (2D) map. The above description focuses on the 3D map, but it is not limited thereto. In the present disclosure, the 3D map may be replaced with a 2D map.

Additionally, the user terminal 130 may display items for viewing the virtual space displayed on the three-dimensional (two-dimensional) map based on user input. When the user selects an item to view the virtual space, the user terminal 130 may display at least one of the virtual space and the virtual object on the display. That is, the user terminal 130 may be separately equipped with a mode for displaying a virtual space and a mode for displaying information about a real space on a map.

The server 110 may perform a step of acquiring properties of the virtual space including location information of the real space. The server 110 may perform a step of obtaining the properties of the virtual object from the user terminal 130 or the administrator terminal 120. The user terminal 130 may perform a step of obtaining information about the real space that is previously stored based on the location information of the real space from the server 110. The user terminal 130 may perform the step of selecting a mode for displaying a virtual space and a mode for displaying information about a real space based on the user's input. Additionally, based on the selected mode, the user terminal 130 may display at least one of information about virtual space, virtual object, and real space.

The step of displaying at least one of information about virtual space, virtual object, and real space may include the following process.

When a mode for displaying a virtual space is selected, the server 110 may perform a step of creating a three-dimensional virtual space in a portion of the three-dimensional virtual world based on the properties of the virtual space. The server 110 may perform the step of placing a virtual object based on the properties of the virtual object in a virtual space. The server 110 may transmit an image of a virtual object placed in a virtual space to the user terminal 130. When transmitting, the server 110 may compress the video. Alternatively, the server 110 may transmit the properties of the virtual space or the properties of the virtual object to the user terminal 130. The user terminal 130 may directly render the virtual space and virtual object based on the properties of the virtual space or the properties of the virtual object. Additionally, the user terminal 130 may display at least one of a virtual space and a virtual object on the display.

When a mode for displaying information about the real space is selected, the user terminal 130 may perform a step of displaying the information about the real space along with a two-dimensional map on the display. At least one of information about a 2D map and real space may be received from the server 110.

In this way, a mode for displaying virtual space and a mode for displaying information about real space are separately provided, so that users can quickly check desired information, such as information about real space or visualized virtual space.

Figure 4 is a flowchart showing a method of operating a metaverse system according to an embodiment of the present disclosure.

In FIG. 3 , step 334 in which the server 110 acquires the properties of a virtual object may include the following process. The administrator terminal 120 or the user terminal 130 may perform step 411 of acquiring a source image containing a real object or real item using an image acquisition device. The image acquisition device may be a 2D camera or a 3D camera. When the image acquisition device is a 2D camera, the administrator terminal 120 or the user terminal 130 can acquire source images for six sides of a real object or real item. The six sides may include front view, rear view, left side view, right side view, top view, and bottom view. The administrator terminal 120 or the user terminal 130 may generate information about the three-dimensional shape of a rectangular parallelepiped using a six-sided view of a real object or real item. When the image acquisition device is a 3D camera, the administrator terminal 120 or the user terminal 130 can obtain information about the 3D shape of a real object or real item as a source image. Information about the 3D shape may be 3D modeling data. Information about a three-dimensional shape may be about a closed polyhedron.

A source image containing a real object or real item may include at least one of information about a tag, a six-sided view, and a three-dimensional shape of the real object or real item. A tag for a real object or real item may be at least one term for describing the real object or real item. The tag may include at least one of a type or brand of an actual object or an actual item. A tag may be a mark such as a QR or barcode attached to a real object or real item.

The manager terminal 120 or the user terminal 130 may perform step 412 of acquiring properties of a real object or properties of a real item from a source image. The properties of the real object or the properties of the real item may correspond to the properties of the already described virtual object or virtual item. The properties of a real object or a real item may include at least one of identification information, type, texture, color, shape, size, and location of the object or item. The description of the information included in the properties of the real object or the properties of the real item is omitted because it overlaps with the description of the information included in the properties of the virtual object or the properties of the virtual item.

The administrator terminal 120 or the user terminal 130 may receive properties of a real object or properties of a real item from the server 110. For example, the manager terminal 120 or the user terminal 130 may acquire an image of a tag of an actual object. The administrator terminal 120 or the user terminal 130 may include a photographing unit, and may acquire an image of the tag using the photographing unit. The manager terminal 120 can obtain identification information of the actual object from the tag image. The identification information of the real object may be related to at least one of the item, serial number, brand, or manufacturing date of the real object. The administrator terminal 120 or the user terminal 130 may transmit identification information of the actual object to the server 110. The server 110 may obtain the properties of the real object corresponding to the identification information of the real object from a database in which the real object is stored in advance. The server 110 may transmit the properties of the virtual object to the administrator terminal 120 or the user terminal 130. The user or administrator can check the properties of the received virtual object. If there are no properties of the real object corresponding to the identification information of the real object in the pre-stored database, the server 110 may transmit a signal requesting the properties of the real object to the administrator terminal 120 or the user terminal 130. You can.

The administrator terminal 120 or the user terminal 130 may acquire some of the properties of the actual object based on the source image. For example, the administrator terminal 120 or the user terminal 130 may obtain a picture or shape displayed on an actual object based on the source image. More specifically, if the actual object is a sign, the manager terminal 120 or the user terminal 130 may be able to automatically obtain letters, shapes, or pictures displayed on the sign based on the source image.

The administrator terminal 120 or the user terminal 130 may obtain the type of object by applying an object recognition algorithm to the source image. Object recognition algorithms can be implemented based on machine learning models. The administrator terminal 120 or the user terminal 130 may display the type of recognized object. Additionally, when receiving a user's confirmation signal, the manager terminal 120 or the user terminal 130 may determine the type of the actual object as the type of the recognized object.

The administrator terminal 120 or the user terminal 130 may receive identification information included in the properties of the actual object from the administrator or user. Alternatively, the administrator terminal 120 or the user terminal 130 may request the server 110 to generate identification information, and the server 110 may automatically generate the identification information. The administrator terminal 120 or the user terminal 130 may determine the automatically generated identification information as the identification information of the actual object.

The manager terminal 120 or the user terminal 130 may receive the type of the actual object from the manager or user. The administrator terminal 120 or the user terminal 130 can obtain the texture and color of the actual object from the source image. The administrator terminal 120 or the user terminal 130 may extract the texture or color of the actual object from the area included in the object of the source image. The administrator terminal 120 or the user terminal 130 may obtain the shape of the actual object from information about the three-dimensional shape. The administrator terminal 120 and the user terminal 130 may generate the shape of an actual object using a professional 3D modeling tool. Additionally, the manager terminal 120 or the user terminal 130 may obtain the size of the actual object from the manager or user, or from information about its three-dimensional shape. The administrator terminal 120 or the user terminal 130 may obtain the location of the actual object from the administrator or user.

The server 110 may perform step 413 of receiving the source image and the properties of the real object or the properties of the real item from the manager terminal 120 or the user terminal 130. The server 110 may perform step 421 of storing the source image together with the properties of an actual object or an actual item. The server 110 may perform step 422 of acquiring the properties of the virtual object or the properties of the virtual item based on at least one of the source image, the properties of the real object, and the properties of the real item. More specifically, the server 110 may determine the properties of a real object or a property of a real item as the properties of a virtual object or a property of a virtual item. In the above manner, the server 110 can accumulate and store the properties of the virtual object or the properties of the virtual item. Additionally, the server 110 may fill the virtual space using at least one of the properties of a plurality of virtual objects based on a control signal from the administrator terminal 120 or the user terminal 130. Additionally, the server 110 may decorate a virtual character using at least one of the properties of a plurality of virtual items based on a control signal from the user terminal 130. When a user or another user clicks (touches) a virtual item held by a virtual person using a user terminal, the three-dimensional shape of the virtual item may pop up. For example, a 3D shape could be the source image of six sides of a real item. The source image for the six faces can be displayed on the surface of the hexahedron. In this case, the reality of the item may be reduced in the virtual space, but it allows the user to show off the items he or she actually owns in the virtual space, an effect that can easily satisfy the user's desire to show off without causing major problems with the item's license. There is.

Hereinafter, the step 335 in which the user terminal 130 displays at least one of a virtual person, a virtual space, and a virtual object on the display will be described in more detail with reference to FIG. 5.

Figure 5 is a flowchart showing the operation of the metaverse system according to an embodiment of the present disclosure.

Referring to FIG. 5, the user terminal 130 may further perform the following process to perform step 335 of displaying at least one of a virtual person, a virtual space, and a virtual object on the display.

The user terminal 130 may perform step 510 of locating the virtual space in some of the 3D maps stored in advance, based on the location information of the real space included in the information about the 3D virtual space. . In other words, just as the server 110 places the virtual space in a location corresponding to the location information of the real space in the 3D map, the user terminal 130 corresponds to the location information of the real space in the pre-stored 3D map. You can place the virtual space in a location that is suitable for you. The 3D map used by the server 110 may be the same as the 3D map used by the user terminal 130. The user terminal 130 may receive and store the 3D map in advance from the server 110.

Just as the server 110 places basic virtual objects around the virtual space, the user terminal 130 can place basic virtual objects in the virtual space. As already explained, the basic virtual object may be placed outside the virtual space to form the atmosphere of the virtual space and implement hyperreality. For example, it may include trees, sun, moon, street lights, trees, grass, terrain, structures (buildings), or cars outside the virtual space. The server 110 may store pre-searched data for each location in real space, and the pre-searched data may be transmitted to the user terminal 130. The user terminal 130 can automatically place basic virtual objects around the virtual space based on the location of the real space and pre-researched data received from the server. Additionally, the user terminal 130 may modify the location, type, or number of the basic virtual object based on the user's input. Location information in real space may be 3D location information. That is, it can include values for three axes. Additionally, location information of the space used by the user terminal 130 may correspond to location information of real space. The process by which the server 110 determines the number of vehicles around the virtual space has already been described, and the user terminal 130 can also perform the same process.

The user terminal 130 may perform step 520 of determining the direction and intensity of light in the virtual space based on the time information of the virtual space included in the information about the three-dimensional virtual space. For example, the user terminal 130 may determine the position of the sun and the moon based on time information in virtual space. It can also determine the size of the moon. Here, the sun and moon in virtual space may be included in the basic virtual objects. The user terminal 130 may render the direction and intensity of light in virtual space using at least one of the position of the sun, the position of the moon, and the size of the moon. In this way, the direction of light and intensity of light are determined similarly to reality, so users can feel hyper reality in virtual space.

The user terminal 130 may perform step 530 of placing at least one virtual object in the virtual space based on the properties of the virtual object included in the information about the three-dimensional virtual space. Location information included in the properties of a virtual object may mean the location of the virtual object within the virtual space. Location information included in the properties of a virtual object can be expressed in a three-dimensional coordinate system. Location information included in the properties of the virtual object may include orientation information of the virtual object. Orientation information may include the placement direction and rotation degree of the virtual object in virtual space. The placement direction and degree of rotation can also be expressed in a three-dimensional coordinate system.

The user can use the user terminal 130 to experience a hyper-reality virtual space that embodies a real space. Since the virtual space is implemented in a virtual location that corresponds to the location of the real space, the user has the effect of being able to experience the real space in advance. For example, objects placed in virtual space may be past objects in real space. Therefore, users can experience the past appearance of real space in virtual space. Additionally, objects placed in virtual space may be expected future objects in real space. Therefore, users can experience the future appearance of real space in virtual space. For example, the server 110 may store properties of virtual objects corresponding to a plurality of time zones. Additionally, when the server 110 obtains date information or era information from the user terminal 130, the server 110 may obtain properties of a virtual object corresponding to the date information or era information. Additionally, the server 110 may place virtual objects in the virtual space based on the properties of the acquired virtual objects. Therefore, users can easily experience changes in space over time.

FIG. 6 is a flowchart illustrating a method of operating a metaverse system according to an embodiment of the present disclosure. Figure 7 is a diagram for explaining a process for visualizing information related to the amount of sunlight in virtual space according to an embodiment of the present disclosure. Figure 8 is a diagram for explaining a process for visualizing information related to the amount of sunlight in virtual space according to an embodiment of the present disclosure. Figure 9 is a diagram for explaining a process for visualizing information related to the amount of sunlight in virtual space according to an embodiment of the present disclosure.

Referring to FIGS. 6 and 7 , step 520 of the user terminal 130 determining the direction and intensity of light in the virtual space may further include the following processes. The explanation below is based on the winter solstice or summer solstice, but is not limited to this. The winter solstice or summer solstice day can be replaced with a day entered by the user among the 365 days included in a year.

The user terminal 130 may perform step 610 of setting time information in the virtual space to the predetermined measurement time of the winter solstice. The predetermined measurement time may include at least one of noon, the time when the sun is at mid-south altitude, 6 a.m., or 6 p.m. The predetermined measurement time may be set by the server 110, the administrator terminal 120, or the user terminal 130.

The user terminal 130 performs a step 620 of acquiring a two-dimensional winter solstice shape formed on the floor of the virtual space by light entering the interior of the virtual space 730 without being obscured by virtual objects placed in the virtual space. You can. Referring to FIG. 7 , the user terminal 130 may determine the position of the sun 710 at a predetermined measurement time on the winter solstice. Additionally, the user terminal 130 can simulate the path 721 of light entering the virtual space 730 according to the position of the sun 710. Additionally, the user terminal 130 can obtain a two-dimensional winter solstice figure 720 formed on the floor of the virtual space 730 through the light path 721. Here, the shape may mean a shape formed on the floor of the virtual space 730 by light that is not obscured by the virtual object. In step 620, not being obscured by a virtual object may include not only a virtual object located inside the virtual space 730 but also a virtual object located outside the virtual space 730. A virtual object located inside the virtual space 730 may be an opaque object such as furniture, a wall, or the like. Additionally, a virtual object located outside the virtual space 730 may mean a tree, another building, a street lamp, or a mountain located outside the virtual space.

The user terminal 130 obtains the maximum distance from the edge 731 in the direction of light at the bottom of the virtual space 730 to the edge 722 of the two-dimensional winter solstice shape 720 as the winter solstice solar distance 740. Step 630 may be performed. The solar solstice distance 740 may have the same meaning as the maximum solar solstice distance on the winter solstice.

Referring to FIGS. 6 and 8 , the user terminal 130 may perform step 640 of setting time information in the virtual space to a predetermined measurement time of the summer solstice. The predetermined measurement time may include at least one of noon, the time when the sun is at mid-south altitude, 6 a.m., or 6 p.m. The predetermined measurement time may be set by the server 110, the administrator terminal 120, or the user terminal 130.

The user terminal 130 performs a step 650 of acquiring a two-dimensional summer solstice shape formed on the floor of the virtual space by light entering the interior of the virtual space without being obscured by the virtual object placed in the virtual space 830. You can. Referring to FIG. 8, the user terminal 130 may determine the position 810 of the sun at a predetermined measurement time on the day of summer solstice. The position 710 of the sun at a predetermined measurement time on the winter solstice may be different from the position 810 of the sun at a predetermined measurement time on the summer solstice. Additionally, the user terminal 130 may determine the path 821 of light entering the virtual space 830 according to the position 810 of the sun. Additionally, the user terminal 130 can obtain a two-dimensional summer solstice shape 820 formed on the floor of the virtual space 830 through the light path 821. Here, the shape may mean a shape formed on the floor of the virtual space 830 by light that is not obscured by the virtual object. In step 650, not being obscured by a virtual object may include not only a virtual object located inside the virtual space 830 but also a virtual object located outside the virtual space 830. A virtual object located inside the virtual space 830 may be an opaque object such as furniture, a wall, or the like. Additionally, a virtual object located outside the virtual space 830 may mean a tree, another building, a street lamp, or a mountain located outside the virtual space.

The user terminal 130 performs a step 660 of obtaining the maximum distance from the edge 831 of the floor of the virtual space 830 to the edge 822 of the two-dimensional summer solstice shape 820 as the summer solstice day sunlight distance 840. It can be done. The summer solstice solar distance 840 may have the same meaning as the summer solstice maximum solar distance.

The user terminal 130 may perform a step of displaying at least one of the solar solstice distance 740 and the summer solstice solar distance 840. The user can check the winter solstice sunlight distance 740 and the summer solstice sunlight distance 840 of the user terminal 130 and know the amount of sunlight in the real space corresponding to the virtual space. Additionally, users can consider purchasing actual space by considering the amount of sunlight.

The explanation above focuses on the winter solstice and summer solstice days, but is not limited to these. The user terminal 130 may determine time (date) information of the virtual space based on the user's input. The user terminal 130 may obtain the position of the sun, the position of the moon, or the position of a street light corresponding to time information in virtual space. Additionally, the user terminal 130 may obtain a light path and a shape formed on the floor of the virtual space based on the position of the sun, the position of the moon, or the position of a street light. Users can consider purchasing a real space corresponding to the virtual space by considering the light path of the virtual space and the shapes formed on the floor.

In addition, step 520 of the user terminal 130 determining the direction and intensity of light in the virtual space may further include the following processes.

Referring to FIG. 9 , the user terminal 130 may perform a step of obtaining a plurality of predetermined measurement times of the summer solstice or winter solstice using time information of the virtual space 930. The user terminal 130 is a plurality of two-dimensional summer solstice images formed on the floor of the virtual space by light entering the room of the virtual space without being obscured by virtual objects placed in the virtual space at a plurality of predetermined measurement times of the summer solstice or winter solstice day. Steps for acquiring shapes 921, 922, and 923 or a plurality of two-dimensional winter solstice shapes 921, 922, and 923 may be performed. The user terminal 130 may acquire a plurality of sun positions 911, 912, and 913 respectively corresponding to a plurality of predetermined measurement times of the summer solstice or winter solstice. The user terminal 130 may simulate light paths corresponding to a plurality of sun positions 911, 912, and 913. The user terminal 130 may obtain a plurality of two-dimensional summer solstice shapes (921, 922, 923) or a plurality of two-dimensional winter solstice shapes (921, 922, 923) formed on the floor based on the simulated light path. . The plurality of shapes 921, 922, and 923 may represent the movement of light entering the virtual space 930 by the sun within a day.

The user terminal 130 may perform the step of displaying a plurality of two-dimensional summer solstice shapes 921, 922, and 923 or a plurality of two-dimensional winter solstice shapes 921, 922, and 923 in the virtual space 930. The plurality of shapes 921, 922, and 923 may have predetermined transparency. Therefore, the user can check the overlapped portion of the plurality of shapes 921, 922, and 923. In FIG. 9 , the overlapping portions of the plurality of figures 921, 922, and 923 are darkened, but the display is not limited thereto. Areas where multiple shapes 921, 922, and 923 overlap may become brighter. The plurality of figures 921, 922, and 923 may have different colors, but are not limited thereto.

The explanation above focuses on the winter solstice and summer solstice days, but is not limited to these. The user terminal 130 may determine time (date) information of the virtual space based on the user's input. The user can consider purchasing a real space corresponding to the virtual space by considering the light path of the virtual space and the plurality of shapes 921, 922, and 923 formed on the floor.

According to an embodiment of the present disclosure, the step 335 of the user terminal 130 displaying at least one of a virtual space and a virtual object on the display may further include the following process.

The user terminal 130 may perform a step of obtaining information about the real space that is previously stored based on the location information of the real space from the server 110. For example, the user terminal 130 may display a virtual world. The virtual world may be a 2D map or a 3D map. As already explained, a virtual world may include a virtual space. That is, when displaying a virtual world, the user terminal 130 may display an icon representing the virtual space at the location of the virtual space. When a user touches an icon displayed at a virtual space location in the virtual world, the user terminal 130 can obtain information about the real space from the server 110. Real space may mean a real space corresponding to the virtual space. For example, if the virtual space is about apartment A, the real space can refer to apartment A in reality. The location of Apartment A, a virtual space, in the virtual world can correspond to the location of Apartment A in the real world. Additionally, the shape of Apartment A, which is a virtual space in the virtual world, can correspond to the shape of Apartment A in the real world.

Additionally, the user terminal 130 may display information about the real space on the display along with at least one of the virtual space and the virtual object. The user terminal 130 may provide some information about the real space along with an icon displayed at the location of the virtual space in the virtual world. Additionally, if the user wants to view information about the real space in detail, the user can click the icon one more time. The user terminal 130 may display a screen that displays detailed information about the real space based on the user's input. The user terminal 130 can display all information about the real space on a screen that displays detailed information. However, if all information about the actual space is displayed, there may be too much information and it may not be easy for users to view. Accordingly, the user terminal 130 can display items on a screen that displays detailed information. The items include at least one of the following: apartment information, ranking and ranking analysis information, actual property inquiry information, regional regulation information, real estate study (investment) information, investment hint information, big data analysis information, market strength-related information, and commercial/obsolescence information. It may be about. When the user selects an item, the user terminal 130 may display detailed information about the item. The user terminal can display information included in the item in the form of graphs, text, numbers, etc.

Information on actual space includes apartment information, ranking and ranking analysis information, actual listing inquiry information, regional regulation information, real estate study (investment) information, investment hint information, big data analysis information, market strength-related information, and commercial/obsolescence status. It may contain at least one piece of information.

Apartment information includes actual transaction price, lease rate, transaction turnover rate, type, sale and lease gap, metropolitan bus information, floor space, rental business rate, residents' opinions, price, monthly rental yield, public price, ownership tax, number of households, parking space, ambience map, and occupancy. It may include at least one of the following: year, presence/absence of entrance/heating, floor area ratio, price per pyeong, change in transaction volume by outsiders, building-to-land ratio, and land area.

Apartment information may include information about parking lots. Information about the parking lot includes at least one of the total number of parking spaces, size of the parking spaces, number of parking spaces per household, information about the type of parking lot, information about how to access the parking space, and type of parking space. may include.

Information about parking may include the total number of parking spaces included in the apartment. In the present disclosure, a parking space refers to a space where one vehicle can park, and a parking lot may refer to a set of at least one parking space. Information about the parking lot may include the size of the parking space. Additionally, information about parking lots may include the number of parking spaces by size. Additionally, information about parking lots may include information about the number of parking spaces per household. Information on the number of parking spaces per household may be the number of parking spaces included in the apartment complex divided by the number of households included in the apartment complex. Information about the parking lot may include information about the type of parking lot. For example, information about the parking lot may include whether it is an outdoor parking space, whether it is an indoor (underground) parking space, or whether there is a parking tower. Information about parking lots may include at least one of the number of outdoor parking spaces, the number of indoor (underground) parking spaces, or the number of parking spaces in a parking tower. The difference between an outdoor parking space and an indoor parking space can be whether there is a roof or not. Additionally, an outdoor parking space may mean a single-story parking space. A parking tower may refer to a case where a parking lot is provided in the form of an independent building of two or more stories. Additionally, in the case of an indoor parking space or a parking tower, information about the parking lot may further include information about the number of floors of the parking lot. The number of floors may be provided as from several floors underground to several floors or from several floors above ground to several floors. Information about the parking lot may include information about how to access the parking space. For example, a method for accessing a parking space may include at least one of a road and a vehicle elevator. Information about parking may include the number of accessible parking spaces depending on the access method. That is, information about parking lots may include the number of parking spaces accessible by road and the number of parking spaces accessible by vehicle elevator. Information about the parking lot may include the type of parking space. For example, information about parking spaces may include general parking spaces, disabled parking spaces, women-only parking spaces, or mechanical parking spaces. Additionally, information about parking lots may include the number of general parking spaces, the number of disabled parking spaces, the number of women-only parking spaces, and the number of mechanical parking spaces.

Figure 20 is a diagram for explaining a method for displaying information about a parking space according to an embodiment of the present disclosure.

Information about parking spaces can be displayed in a hierarchical tree structure. Referring to (A) of FIG. 20, the user terminal 130 may display information about the parking space based on information from the server 110. The user terminal 130 may display the total number of parking spaces (N0) at the highest level. Additionally, the user terminal 130 may display the number of parking spaces for each type of parking lot in the lower layer. For example, the number of outdoor parking spaces (N1), the number of indoor parking spaces (N2), and the number of parking spaces in the parking tower (N3) can be displayed. If N1, N2, or N3 is equal to 0, it may not be displayed. N0=N1 + N2 + N3. Additionally, the user terminal 130 may include information on how to access the parking space in the lower layer of the type of parking lot. For example, the number of parking spaces accessible by road (N11, N21, N31) and the number of parking spaces accessible by elevator (N12, N22, N32) can be displayed. N11+N12 may be greater than or equal to N1 since the parking space may be accessible both by road and elevator. N21+N22 can be greater than or equal to N2. N31+N32 can be greater than or equal to N3. The user terminal 130 may display the number of parking spaces by type in the lower layer of the access method to the parking space. For example, the number of parking spaces by type may include the number of general parking spaces (N111, N121, N211, N221, N311, N321) and mechanical parking spaces (N112, N122, N212, N222, N312, N322). there is. Although not displayed in (A) of FIG. 20, the user terminal 130 can further display the number of disabled parking spaces and the number of women-only parking spaces.

The user terminal 130 may display the hierarchy differently. The user terminal 130 may display the measurement differently based on the user's input. Figure 20 (A) displays the types of parking spaces in the lower layer of the approach method to the parking lot. However, it is not limited to this. As shown in (B) of FIG. 20, the user terminal 130 may display the access method to the parking lot in the lower layer of the type of parking space. Additionally, although not shown in FIG. 20, the user terminal 130 may display the number of parking spaces for each size of the parking space in a tree format.

Ranking and ranking analysis information may include at least one of price per pyeong, large lot, highest price, move-in period, transaction volume, price increase rate, transaction turnover rate, price increase rate, sale and lease gap, unsold units, lease rate, and move-in.

The actual property inquiry information may include at least one of the actual transaction price of apartment sales/monthly rent, the actual transaction price of villas, studio apartments, officetels, and start-up offices.

Regional regulatory information may include at least one of speculation areas, overheated speculation districts, adjustment target areas, and loan subscription tax resale restrictions.

Real estate study (investment) information includes increase or decrease in listings, areas with increased foreign investment, buying sentiment, apartments purchased in large quantities, apartments with the highest price increase, apartment resident communities, highest price apartments, comparison of school districts, price comparison of competitive sales complexes, price comparison of multiple apartments, prices of two complexes. It may include at least one of comparison, cyber model house, real estate big data, apartment supply volume, gap investment increase area, and unsold apartments.

Investment hint information includes reported price, academy price, sales complex, change, commercial district, reconstruction stage, population (moving in and out), salary of office workers, auction/public auction, supply (quantity of supply), favorable development, news, slope, individual complex, transportation network, commute ( It may include at least one of the following: analysis of commute time by car), real-time popular apartments, housing price sentiment index, transaction volume, and information about the currently viewing user (level of interest).

Big data analysis information may include at least one of the following items: market price, occupancy, transaction, population, economy, and transfer. Market price items included in big data analysis information include at least one of sales, lease (area, annual leave), market price increase/decrease, increase/decrease rate, sale/lease gap, sale/lease gap increase/decrease, lease rate, lease rate increase/decrease, market intensity, and market intensity increase/decrease. It can contain one. Occupancy items included in big data analysis information may include information on apartments, apartment complexes, and unsold units. Transaction information included in big data analysis information may include at least one of sales (transaction volume, area, annual leave) and lease (transaction volume, area, annual leave). Population items included in big data analysis information may include at least one of population (number, increase/decrease, increase/decrease rate, age range) or generation (number, increase/decrease number, increase/decrease rate, age range). Economic items included in big data analysis information may include at least one of information on small businesses and corporations. The transfer/exit (age range) item included in the big data analysis information may include at least one of information on population, generation, transfer/exit, and net migration of the region.

Information related to market strength may include at least one of information on market strength, transaction volume, price increase rate, unsold units, transaction ratio, move-ins, unsold units, population/number of households, apartment increase rate, and subscription.

Commercial/obsolete information includes deterioration, area drawing and analysis (access rate, lake density, number of households, parcels within the area), press release summary, building search and AI addition, market report, new construction sales information, subscription competition rate, and scheduled sales complex. , it may include at least one of the expected complex, 3D complex tour, building history and building information, and availability of monthly rent card payment.

The user terminal 130 may provide a real transaction report alarm for the real space corresponding to the virtual space based on the user's settings. Additionally, the user terminal 130 may provide information related to daily events, simulated real estate investment, and point accumulation based on user activity based on the user's selection.

FIG. 21 is a diagram for explaining a method for providing information about a real space according to an embodiment of the present disclosure.

Information about real space can be expressed as a radial graph (chart). The user terminal 130 can display information about real space in a radial graph. The user terminal 130 can display the latest transaction price of the real space, the year of approval for use of the real space (apartment) complex, the number of households included in the real space (apartment) complex, the number of parking spaces available, and the floor area ratio of the real space in a radial graph. . In other words, the plurality of axes included in the radial graph can each represent the latest transaction price, year of approval for use, number of households, number of parking spaces, and floor area ratio. The number of parking spaces available may refer to the number of parking spaces available per household. The higher the latest transaction price, floor area ratio, number of available parking spaces, and number of households, the farther they can be displayed from the center of the radial graph. The older the year of approval for use, the further away it can be displayed from the center of the radial graph. Figure 21 shows the latest transaction price, floor area ratio, number of parking spaces, and number of households in a graph, but is not limited to this. The graph may display the amount of sunlight, average monthly transaction volume, building-to-land ratio, land area, or price.

The server 110 or the user terminal 130 may obtain the graph 2110 for each real space (apartment) complex. Additionally, the server 110 or the user terminal 130 may determine the area of the graph 2110 as an evaluation score. That is, the metaverse system determines a higher evaluation score as the area of the graph 2110 is larger, and determines a lower evaluation score as the area of the graph 2110 becomes smaller.

The server 110 or the user terminal 130 can sort real space (apartment) complexes according to evaluation scores. The user terminal 130 may sort real space (apartment) complexes in ascending or descending order according to evaluation scores based on the user's input. The user terminal 130 or the server 110 may determine the ranking of the actual space (apartment) complex based on the evaluation score. The user terminal 130 can display the ranking in real space.

Additionally, the user terminal 130 may determine the score obtained by dividing the evaluation score by the price per pyeong of the actual space (apartment) as the price-related evaluation score. The user terminal 130 may sort real space (apartment) complexes in ascending or descending order of price/evaluation score based on the user's input. The user terminal 130 may determine the ranking of the actual space (apartment) complex based on the price/evaluation score. Users can easily determine the value of the actual space (apartment) based on the evaluation score or price comparison score. The server 110 or the user terminal 130 may sort real space (apartment) complexes in descending or ascending order according to transaction volume ranking. The Metaverse system provides radial graphs and rankings of real spaces (apartments) like this, allowing users to easily judge the investment value of real spaces (apartments).

The Metaverse system can provide a user interface that allows users to compare real spaces (apartments). The user terminal 130 may select a plurality of real space (apartment) complexes based on the user's input. The user terminal 130 can display at least one of the increase or decrease in listings, use approval year, number of households, floor area ratio, building-to-land ratio, site area, parking area per household, or heating method of a plurality of selected real space (apartment) complexes side by side. For example, the user terminal 130 organizes in a table at least one of the increase or decrease in listings, year of approval for use, number of households, floor area ratio, building-to-land ratio, site area, parking area per household, or heating method of a plurality of selected real space (apartment) complexes. It can be displayed. For example, the user terminal 130 displays at least one of the increase or decrease in listings, year of approval for use, number of households, floor area ratio, building-to-land ratio, site area, parking area per household, or heating method of a plurality of selected real space (apartment) complexes in a bar graph, It can be organized and displayed in a radial graph, etc.

Figure 10 is a diagram for explaining the operation of the metaverse system according to an embodiment of the present disclosure.

The metaverse system 100 may further include an installed camera that photographs a predetermined direction at a predetermined location in real space. The predetermined location may be near a location in real space corresponding to the virtual space. Additionally, the direction the camera faces may be a direction for photographing a real space corresponding to a virtual space. In other words, the camera included in the metaverse system 100 can capture images of a real space corresponding to a virtual space. The camera is connected to the server 110 and can transmit captured images to the server 110. The server 110 can receive images from a camera.

Step 335 of the user terminal 130 displaying at least one of a virtual space and a virtual object on the display may further include the following process.

When the user's input indicates that he or she wants to view the actual image of the virtual space, the user terminal 130 may receive real-time image data 1020 from the camera from the server 110. Real-time video data may be photos or videos. The present disclosure focuses on real-time video data, but is not limited thereto. Real-time video data can be replaced with video data captured by a camera and stored in advance on the server 110.

The user terminal 130 may perform a step of moving the viewpoint from a position in the virtual space or virtual world corresponding to a predetermined location to a direction in the virtual space corresponding to a predetermined direction. The user terminal 130 may receive the location of the camera in real space from the server 110. As already explained, location information in real space can be expressed in the same way as location information in virtual space. The user terminal can move the viewpoint in virtual space or virtual world based on location information on the camera in real space. The viewpoint may refer to the position from which the virtual space is viewed. Additionally, the user terminal 130 may obtain a predetermined direction of the camera from the server 110. The user terminal 130 may display on the display a virtual space 1010 looking in a direction corresponding to the predetermined direction of the camera from the viewpoint within the virtual space. As already explained, since the virtual space identically replicates the real space, the appearance of the virtual space displayed on the user terminal 130 may be similar to the real-time image captured by the camera. However, the real-time video captured by the camera may display animals, people, vehicles, etc. passing around the real space. However, virtual space may only represent predetermined virtual objects.

The user terminal 130 may perform a step of displaying real-time image data 1020 from a camera with predetermined transparency on the virtual space 1010 displayed on the display. The predetermined transparency can be between 0 and 100%. That is, the predetermined transparency may be completely transparent or completely opaque. The user terminal 130 may display an overlap image 1030 by overlapping the real-time image data 1020 on the virtual space 1010. Additionally, the user terminal 130 may output only the virtual space 1010 or only real-time image data 1020 based on the user's selection. In this way, the user can feel the atmosphere of the real space by simultaneously viewing the real-time image data 1020 and the virtual space 1010.

Additionally, the user terminal 130 may acquire an image 1021 of only a moving object in a real-time image. Moving objects in real-time video may be people, animals, vehicles, etc. The user terminal 130 can extract an image 1021 of only a moving object through frame-by-frame comparison from real-time image data. The user terminal 130 may compare real-time image data and the virtual space 1010 to extract an image 1021 of only the moving object. The user terminal 130 may determine the portion where the pixel difference is greater than a predetermined threshold as the image 1021 of only the moving object through the difference between the real-time image data and the pixels of the virtual space 1010. The user terminal 130 may display an overlap image 1040 by overlapping the image 1021 of only the moving object on the virtual space 1010. An image 1021 of only a moving object may have a predetermined transparency. In this way, the user can feel the atmosphere of the real space by simultaneously viewing the real-time image data 1020 and the virtual space 1010. Therefore, the user can know the atmosphere of the real space without going to the real space through the overlap images 1030 and 1040 and can consider purchasing the real space.

Figure 11 is a diagram for explaining the operation of the metaverse system according to an embodiment of the present disclosure.

Referring to FIG. 11 , the user terminal 130 may perform step 1110 of controlling a virtual character to move in a virtual space and interact with a virtual object based on the user's input. Virtual characters can have predetermined interactions with virtual objects. Predetermined interactions may change the posture of a virtual character or the position, color, or shape of a virtual object. For example, if the virtual object is a chair, the virtual person can interact by sitting on the virtual object. The virtual character can make the motion of sitting on the virtual object. Additionally, if the virtual object is a light, the virtual person can take actions to turn the light on or off. Virtual objects can turn on or off in response to the virtual person's movements. If the virtual object is a seesaw, the virtual person can interact by sitting on the virtual object, and the virtual object, the seesaw, can move in response to the virtual person's sitting motion.

The user terminal 130 records the location of the virtual person in the virtual space, the time the virtual person stayed at the location in the virtual space, the virtual object displayed on the display of the user terminal at the virtual person's location, and the history of interactions with the virtual object. A step 1120 of acquiring log information may be performed.

The location of a virtual person in virtual space can be expressed as three-dimensional coordinates. The user terminal 130 may obtain the location of the virtual person at a predetermined period. The predetermined period may be 5 seconds, 10 seconds, or 1 minute, or 10 minutes, etc.

The user terminal 130 allows the virtual character to divide the virtual space into at least one area. For example, if the virtual space is a house, at least one area may be divided into an entrance, living room, kitchen, room, or veranda. The user terminal 130 may determine the time the virtual person stays in at least one area as the time the virtual person stays in the location in the virtual space. The user terminal 130 may obtain identification information of at least one area and time spent in at least one area. The time a virtual character stays in at least one area may have units of minutes or seconds.

The user terminal 130 may obtain a virtual object displayed on the display of the user terminal at the location of the virtual person. The user terminal 130 may obtain identification information of the virtual object displayed on the user terminal 130 at a predetermined period even if the virtual person does not interact with the virtual object. The predetermined period may be 5 seconds, 10 seconds, or 1 minute, or 10 minutes, etc. For example, if the virtual space is a park and a virtual character is sitting on a park bench, the virtual character is interacting with the park bench, which is a virtual object. The user terminal 130 can display not only an image of a virtual person sitting on a bench, but also trees, grass, and streetlights around the virtual person. The user terminal 130 may periodically obtain identification information of virtual objects displayed on the user terminal while being around the virtual person. This displayed virtual object identification information can be utilized as the user's predicted preferred object. In other words, the displayed virtual object identification information can be used as information indicating the user's taste.

The server 110 may previously store at least one of the properties of a virtual space or a virtual object according to time in a database. At least one of the properties of the virtual space or the properties of the virtual object stored by the server 110 may be a replica of a building that existed in real space. In reality, a building changes over time. Additionally, there are cases where one building is rebuilt into another building. In other words, real space and real objects applied to real space change in size, shape, color, etc. depending on time. The server 110 may store attributes of a plurality of virtual spaces or attributes of a plurality of virtual objects according to changes in time. The properties of one virtual space or the properties of one virtual object may be intended to replicate the real space and real object at a specific point in time.

The user terminal 130 can acquire virtual time based on user input. Users can enter their desired virtual time. Users can also select a virtual time from predefined views. Virtual time may be the same as time information in virtual space. Virtual time may be time used in virtual space. The user terminal 130 may request the server 110 for properties of a virtual space or properties of a virtual object corresponding to the virtual time. The server 110 may receive time information (virtual time) of the virtual space from the administrator terminal 120 or the user terminal 130. The server 110 may perform a step of obtaining the properties of the virtual space and the properties of the virtual object corresponding to the virtual time from the database. The server 110 may perform the step of creating a three-dimensional virtual space in a portion of the three-dimensional virtual world based on the properties of the virtual space corresponding to the virtual time. The server 110 may perform the step of placing a virtual object based on the properties of the virtual object corresponding to the virtual time in a three-dimensional virtual space. The user terminal 130 may display at least one of a virtual space and a virtual object on the display.

The user terminal 130 may obtain information about the real space corresponding to the virtual time from the server 110 and provide it to the user. The server 110 may accumulate and store information about the actual space by time. The user terminal 130 may perform the step of selecting a mode for displaying a virtual space and a mode for displaying information about a real space based on the user's input. When a mode for displaying a virtual space is selected, the user terminal 130 may perform a step of displaying at least one of the virtual space and virtual object corresponding to the virtual time.

When a mode for displaying information about the real space is selected, the user terminal 130 may perform a step of acquiring information about the real space corresponding to the virtual time based on the location information included in the properties of the virtual space. there is. The user terminal 130 may display information about the real space corresponding to the virtual time along with a two-dimensional map on the display.

The process of obtaining log information will be described in more detail with brief reference to FIG. 12.

Figure 12 shows a screen displayed on the user terminal 130 according to an embodiment of the present disclosure.

The user terminal 130 may further perform the following process to perform the step 1120 of acquiring log information. The user terminal 130 may perform a step of obtaining the resolution of the display 1210 of the user terminal 130. The resolution of the display 1210 may mean the maximum resolution of the display. Resolution may include the number of horizontal pixels and the number of vertical pixels. The user terminal 130 may perform a step of setting a predetermined area of the display as the area of interest 1220 based on the resolution of the display 1210. For example, the user terminal 130 may determine that the area of interest 1220 has an area of 1/8 to 1/4 of the resolution of the display 1210. Additionally, the user terminal 130 may determine that the midpoint of the region of interest 1220 matches the midpoint of the display 1210. The user terminal 130 may perform a step of including virtual object of interest information including identification information of a virtual object displayed within the area of interest for more than a predetermined threshold time in log information. Here, the threshold time may have a value of 10 seconds or more. Placing a virtual object in the center of the screen for at least a certain period of time may indicate that the user has expressed interest in the virtual object. Accordingly, the user terminal 130 may include identification information of the virtual object displayed in the area of interest for more than a threshold time in the log information. The user terminal 130 or the server 110 may later use the log information as big data to derive a preferred virtual object according to the user. Accordingly, the user terminal 130 or the server 110 can provide a user-customized virtual object.

Referring again to FIG. 11, the user terminal 130 may perform step 1130 of transmitting log information along with identification information of the user terminal to the server 110. The user terminal 130 may collect log information for a predetermined period of time. Additionally, the user terminal 130 may transmit log information collected during a predetermined period of time to the server 110. The user terminal 130 may transmit log information to the server 110 in response to user identification information. The user terminal 130 may delete log information transmitted to the server 110. The server 110 may accumulate and store log information. By deleting log information from the user terminal 130 in this way, the memory of the user terminal 130 can be saved. In addition, log information is only used as big data in the server 110 and is not used in the user terminal 130, so it has the effect of being stored only where it is needed.

Figure 13 is a flowchart showing a method of operating a metaverse system according to an embodiment of the present disclosure.

FIG. 13 includes a configuration overlapping with FIG. 3 . More specifically, Figures 13 and 3 all show step 311, step 321, step 332, step 333, step 323, step 334, step 324, and step 335. Contains at least one of Therefore, the description of the components in FIG. 13 that overlap with FIG. 3 will be omitted.

The server 110 may perform step 1325 of issuing a current (original) non-fungible token (NFT) for the properties of the virtual object. NFTs can be issued based on Ethereum. The properties of virtual objects may also be included in NFTs. However, it is not limited to this, and the server 110 may receive the properties of the virtual object from the user terminal 130. Additionally, the server 110 may encrypt and store the properties of the virtual object. The server 110 may generate address information for the properties of the stored virtual object. The address information may be a location for the user terminal 130 or a third-party terminal to access the properties of the virtual object. Address information may include at least one of an IP address, a port number, information about a communication protocol, and directory (folder) information. NFT may contain address information of the properties of the virtual object.

The server 110 may issue an NFT based on the user's NFT creation signal received from the user terminal 130. The server 110 may determine the owner of the NFT using identification information related to the user terminal. Identification information related to the user terminal may be for the user terminal 130 that transmitted the properties of the virtual object to the server 110 in step 334. In the metaverse system 100, ownership of an NFT may mean having the copyright of the properties of the virtual object. In other words, selling an NFT to another person may mean that the copyright of the properties of the virtual object has been transferred to another person.

The step 1325 in which the server 110 currently issues a non-fungible token (NFT) may include the following processes. The server 110 may perform a step of obtaining transaction details for at least one NFT previously issued by the user. Transaction history for at least one NFT may include transaction history for selling the properties of a virtual object corresponding to the NFT. Selling the properties of a virtual object means selling information about at least one of the object type and arrangement included in the properties of the virtual object. In the present disclosure, NFT may have the same meaning as copyright for the properties of a virtual object. Copyright holders can make money by selling the properties of virtual objects. Just because a copyright holder sells the properties of a virtual object does not mean that the copyright is transferred to the buyer. In this way, ownership of the NFT will continue to belong to the original owner or copyright holder, and the copyright holder (or original owner) can continue to earn profits by selling the properties of the virtual object.

However, it is not limited to this, and the owner can also sell the copyright. At this time, the ownership of the NFT may change. Transaction history for at least one NFT may include details of changes in the owner of the NFT.

The server 110 may automatically determine the price of the current NFT based on transaction history for at least one previously issued NFT. In other words, if a user has continuously created at least one attribute of a virtual object in virtual space and made an NFT for the attribute of the virtual object, a price for the NFT may be established. Creating properties of at least one virtual object for a virtual space may mean decorating the virtual space. In other words, it may mean performing the interior of a virtual space. The server 110 can automatically determine the price of the currently issued NFT based on the price of past NFTs created by the same user. For example, the server 110 may determine the average or median price of at least one previously issued NFT as the price of the currently issued NFT. Additionally, the price of the currently issued NFT can be determined based on the selling price of the properties of at least one virtual object corresponding to at least one existing NFT. The server 110 may store an NFT price determination table that matches the average or median sales price of the attributes of at least one virtual object with the price of the NFT. The NFT price determination table can be determined by the administrator. The server 110 can automatically determine the price of the currently issued NFT by applying the average or median value of the sales price of the attributes of at least one virtual object created by the same user to the NFT price determination table. However, it is not limited to this, and the price of the currently issued NFT may be determined by user input.

Additionally, the server 110 may perform a step of automatically determining the current price of the NFT based on the user's reputation information. The server 110 may store the user's reputation information in the user's account. A user's reputation information may be determined based on the user's activities within the user's metaverse system. The server 110 may grant basic reputation information to new users. If the user has continuously created the properties of at least one virtual object for the virtual space, he or she can receive evaluation scores from other users for the at least one created virtual object. When the server 110 obtains evaluation scores from other users exceeding a predetermined threshold, the server 110 may average the evaluation scores. The server 110 may increase or decrease reputation information based on the average evaluation score. For example, points may be added or subtracted in proportion to the average evaluation score for the first virtual space. The correspondence between the added and subtracted scores and the average evaluation score may be written in a predetermined table. The added or subtracted points can be negative or positive. If the average evaluation score is the minimum point, the addition or subtraction score will be a negative number, and if the average evaluation score is the maximum point, the addition or subtraction score may be a positive number. The server 110 may obtain the user's reputation score by adding the addition or subtraction score to the basic reputation information. The addition or subtraction score can be determined for each virtual space. When a user creates virtual object properties for multiple virtual spaces, multiple addition or subtraction points can be obtained. The server 110 may obtain a reputation score by adding all of the plurality of addition and subtraction points to the basic reputation information. The server 110 can determine the current price of the NFT corresponding to the reputation score. The correspondence between the current NFT price and the user's reputation score may be listed in a predetermined table.

Figure 14 is a flowchart showing a method of operating a metaverse system according to an embodiment of the present disclosure.

The third-party terminal may perform step 1410 of receiving preview information about the three-dimensional virtual space in which the virtual object is placed from the server 110. The third-party terminal may be a terminal owned by a user different from the user who created the virtual object attribute. The server 110 may not transmit virtual object properties to a third-party terminal. This is because when a third-party terminal receives the properties of a virtual object, the virtual object can be easily modified. The server 110 may generate an image in which a virtual object is placed in a virtual space based on the properties of the virtual space and the properties of the virtual object. Additionally, the server 110 may transmit an image of a virtual object placed in a virtual space to a third-party terminal as preview information. A third party can appreciate the virtual object placed in the virtual space by looking at the preview information displayed on the third party terminal. Additionally, a third party may decide to purchase the properties of the virtual object. This is because you want to apply the virtual object applied to the virtual space to your real space. As already described, the properties of the virtual object may include at least one of the object's identification information, type, texture, color, shape, size, and location. In other words, if it has the properties of a virtual object, it is possible to easily place a real object in real space that is identical to the virtual object placed in virtual space. The property of the virtual object is to become a kind of interior design drawing.

The server 110 may perform step 1420 of obtaining a purchase signal for the attributes of the virtual object from a third-party terminal. The purchase signal may include at least one of a signal indicating a purchase, identification information related to a third-party terminal, and the amount of virtual currency to be paid for the purchase.

If the purchase signal for the properties of the virtual object satisfies a predetermined purchase condition, the server 110 may perform step 1430 of transmitting the properties of the virtual object and the properties of the virtual space to the third-party terminal. Purchase conditions can be set automatically by the server 110 or by the owner of the virtual object's properties. There may be at least one purchase condition. For example, if the amount of virtual currency included in the purchase signal is greater than or equal to the price of the virtual object attribute, the server 110 may determine that the purchase condition has been satisfied. Additionally, the server 110 may check whether the owner of the attribute of the virtual object has transmitted the purchase signal. For example, in order to prevent the owner of the properties of the virtual object from buying his or her own virtual space to increase the price of the virtual space, the server 110 can prevent the owner from purchasing the properties of the virtual object that he or she owns. there is. Additionally, the server 110 may determine that the purchase condition has been satisfied based on the owner of the attribute of the virtual object approving the purchase signal.

If the purchase condition is satisfied, the server 110 may transmit the properties of the virtual object and the properties of the virtual space to the third-party terminal. A third party can decorate their real space based on the properties of the virtual object. The form of the third party's real space may be the same as the form of the virtual space. Additionally, third parties can modify the properties of virtual objects. Therefore, a third party can modify the properties of the received virtual object and reflect it in their real space.

FIG. 15 is a flowchart illustrating a method of operating a metaverse system according to an embodiment of the present disclosure.

The steps included in Figure 15 may be performed after the steps in Figure 14.

After transmitting the properties of the virtual object and the properties of the virtual space to the third-party terminal, the server 110 may perform a step 1510 of obtaining the properties of the modified virtual object from the third-party terminal. The properties of the modified virtual object may include at least one of the object's identification information, type, texture, color, shape, size, and location. Step 1510 of the server 110 acquiring the properties of the modified virtual object from the third-party terminal may correspond to step 334 of FIG. 3 . Therefore, redundant explanations are omitted.

The server 110 may perform step 1520 of placing a virtual object based on the properties of the modified virtual object in a three-dimensional virtual space. This step 1520 may correspond identically to step 324 in FIG. 3 . Therefore, redundant explanations are omitted.

The server 110 may perform a step 1530 of issuing a chained NFT for the properties of the modified virtual object. Step 1530 may be similar to step 1325. Step 1530 will be explained focusing on differences from step 1325.

Chain NFTs can be issued based on Ethereum. Properties of modified virtual objects may also be included in NFTs. However, it is not limited to this, and the server 110 may receive the properties of the modified virtual object from a third-party terminal. Additionally, the server 110 may encrypt and store the properties of the modified virtual object. The server 110 may generate address information for the properties of the modified virtual object. The address information may be a location for the user terminal 130 or a third-party terminal to access the properties of the modified virtual object. Address information may include at least one of an IP address, a port number, information about a communication protocol, and directory (folder) information. Chain NFT may contain address information of the properties of the modified virtual object.

The server 110 may issue a modified NFT based on the user's chain NFT creation signal received from the third terminal. The server 110 may determine the owner of the chained NFT using identification information related to the third-party terminal. The identification information related to the third-party terminal may be for the third-party terminal that transmitted the properties of the virtual object modified in step 1510 to the server 110. In the metaverse system 100, ownership of a chain NFT may mean having the copyright of the properties of the virtual object. In other words, selling a chain NFT to another person may mean that the copyright of the properties of the virtual object has been transferred to another person. A chained NFT may be an NFT created by chaining to the current (original) NFT. In other words, some of the profits from chained NFTs can go to the owner of the original NFT. For this purpose, the following process can be further performed.

The server 110 may determine whether the difference between the virtual object and the modified virtual object is greater than a certain range before issuing a chained NFT. For example, the properties of an existing virtual object may need to be changed by more than 50%. Here, change may mean the following: The server 110 may determine that a change of 100/N (%) has occurred when one virtual object is replaced with another object. Here, N may mean the number of all objects placed in the virtual space. Additionally, when an existing virtual object is deleted, the server 110 may determine that there is a change of 1/2*100/N (%). The server 110 may determine the change percentage for all virtual objects, add them all up, and then determine whether a predetermined threshold percentage has been exceeded. The predetermined threshold percentage may be 50%, for example. The server 110 may issue a chained NFT if the total change percentage is greater than a predetermined threshold percentage.

The server 110 may record the relationship between the current (original) NFT and the chained NFT in the ledger of the current (original) NFT and the chained NFT. The server 110 may perform a step 1540 of recording information about the chained NFT in the secondary work information of the current NFT ledger. Additionally, the server 110 may perform a step (1550) of recording information about the current NFT in the original information of the ledger of the chained NFT. Additionally, cryptocurrency nodes can check whether information about the chain NFT has been recorded in the secondary work information of the current NFT's ledger and whether information about the current NFT has been recorded in the original information of the chain NFT's ledger. Cryptocurrency nodes can currently share the ledger of NFTs and the ledger of chained NFTs. Through this, the relationship between the original copyright holder and the using copyright holder can be clarified. Since the original copyright holder can make profits from serial copyrights, he or she will want to actively use his or her copyright. In other words, the original copyright holder will actively disclose his/her creation. Through this, creative activities in the interior design industry become more active, and the metaverse system 100 of the present disclosure can promote the development of the interior design industry.

The metaverse system 100 of the present disclosure may not recognize third-party works. The server 110 may issue multiple chained NFTs based on the current (original) NFT. However, the server 110 may not be able to create a new chain NFT based on the chain NFT. The relationship between multiple chain NFTs can be equal.

However, it is not limited to this, and chain NFTs may be issued. At this time, it can be confirmed whether the properties of the grandchild virtual object corresponding to the grandchild chain NFT are different from the properties of the original virtual object and the modified virtual object. The server 110 may issue a grandchild chain NFT when the properties of the grandchild virtual object differ from the properties of the original virtual object by more than a critical percentage and also differ from the properties of the modified virtual object by more than a critical percent.

A third party may want to decorate their real space based on the properties of the purchased virtual object. To this end, the metaverse system 100 can provide an interface for easily applying virtual objects corresponding to virtual space to real space for the convenience of third parties.

More specifically, the third-party terminal may perform the step of extracting a list of virtual objects included in the three-dimensional virtual space. A virtual space may include at least one virtual object. A third-party terminal can extract a list of virtual objects included in the virtual space. A third-party terminal can obtain a list of virtual objects using the properties of the virtual object. A third-party terminal can only obtain a list of specific types of virtual objects based on the third-party input. For example, if a third party selects the “furniture” type, the third party terminal can obtain a list of virtual objects of the furniture type. As already described, the properties of the virtual object may include at least one of the manufacturer of the real object corresponding to the virtual object, the manufacturing year of the real object, the product identification number of the real object, the price, and the seller. The list of virtual objects may include at least one of the manufacturer of the real object corresponding to the virtual object, the manufacturing year of the real object, the product identification number of the real object, the price, and the seller. Here, the price may be the price of each seller. Danmakri, a third party, can obtain prices for each seller from the server 110 based on the product identification number. Additionally, a third-party Danmakri may obtain information about purchase availability from the server 110 based on the product identification number. The list of virtual objects may include information about availability for purchase.

The third-party terminal may perform the step of displaying a list of virtual objects. The third-party terminal may transmit a virtual object list to the sales site based on the third-party input. Additionally, third-party terminals can receive a reply from the sales site about the total cost and availability of the product.

The third-party terminal may perform the step of transmitting a list of virtual objects to the server 110. A third party may change some virtual objects after reviewing the list of virtual objects before sending them to the server 110.

The server 110 may transmit a list of virtual objects and preview information of a 3D virtual space to at least one company terminal. Company terminals may include PCs, tablets, or smartphones from interior design companies or construction supply retailers.

The server 110 may perform a step of receiving quotation information for a list of virtual objects from at least one company terminal. The server 110 may perform the step of transmitting quotation information to a third-party terminal. A third party can select an interior design company by checking the quotation information displayed on the third party terminal. In this way, a third party can easily perform the interior of a real space using the virtual object properties and virtual space properties provided by the metaverse system 100 of the present disclosure.

Figure 16 is a flowchart showing a method of operating a metaverse system according to an embodiment of the present disclosure.

As already described, the metaverse system may include a user terminal 130 and a server 110. The user terminal 130 may perform step 1611 of obtaining authentication information by scanning a tag attached to an item owned by the user. Items owned by the user may be clothing, bags, electronics, shoes, etc. that the user actually owns in the real world. Items owned by the user may have tags attached to them. The tag may include a serial number for items owned by the user. The serial number may contain unique identification and authentication information for the item. In other words, if the serial number has a specific pattern, the item may be genuine, and if the serial number does not have a specific pattern, the item may not be genuine.

According to an embodiment of the present disclosure, the user terminal 130 can photograph a tag using a camera. The user terminal 130 may obtain authentication information based on the image for the tag. For example, a tag can be a barcode or QR code. However, it is not limited to this.

Additionally, the tag may include a communication unit. The communication unit of the tag can communicate with the user terminal 130. The communication method can be NFC (Near Field Communication) or Bluetooth. The user terminal 130 can obtain authentication information by communicating with the tag.

According to another embodiment of the present disclosure, the authentication information may be One Time Password. Authentication information may have values that change over time. For example, an item owned by a user may include a battery and a power generation unit that generates electricity by light. The control unit of the item owned by the user can be supplied with power by the battery and power generation unit. Additionally, the control unit of the item owned by the user can apply a predetermined algorithm to the current time to generate an authentication number that changes with time. The server 110 can also generate an authentication number that changes with time by applying a predetermined algorithm to the current time. That is, the authentication number generated by the item owned by the user may be the same as the authentication number generated by the server 110. The user terminal 130 can automatically or manually obtain an authentication number generated by an item owned by the user.

The user terminal 130 may perform step 1612 of transmitting authentication information, user identification information, and user terminal identification information to the server 110 . The user's identification information may be unique information for each user. The user's identification information may include information about at least one of name, biometric information (iris, fingerprint), resident registration number, date of birth, or phone number. Additionally, the user's identification information may be unique information generated based on information about at least one of name, resident registration number, date of birth, or phone number. Additionally, the user's identification information may be unique information entered by the user.

The identification information of the user terminal may be unique information for each user terminal. The identification information of the user terminal may include information about at least one of a phone number, device serial number, manufacturer, production number, production date, or model name. Additionally, the identification information of the user terminal may be unique information generated based on information about at least one of a phone number, device serial number, manufacturer, production number, production date, or model name.

In step 1612, the user terminal 130 may further transmit a command requesting authentication to the server 110. The server 110 may perform authentication for the user terminal 130 based on a command requesting authentication. However, it is not limited to this. A command requesting authentication from the user terminal 130 may not be transmitted to the server 110. The server 110 may perform authentication for the user terminal when receiving authentication information, user identification information, and user terminal identification information.

In step 1612, the user terminal 130 may encrypt the authentication information, the user's identification information, and the user terminal's identification information, and the server 110 may encrypt the authentication information, the user's identification information, and the user terminal's identification information. can be decrypted.

The server 110 may perform a step 1621 of determining whether the authentication information corresponds to a pre-stored virtual item and is legitimate. The server 110 may store identification information of the virtual item in advance. Additionally, the server 110 may store authentication information to correspond to identification information of the virtual item. The server 110 may compare the stored authentication information and virtual item identification information with the authentication information and virtual item identification information received from the user terminal 130 . If the stored authentication information and virtual item identification information are the same as the authentication information received from the user terminal 130 and the virtual item identification information, the server 110 determines that the authentication information corresponds to the virtual item identification information and is legitimate. You can decide.

If the stored authentication information and identification information of the virtual item are not the same as the authentication information received from the user terminal 130 and the identification information of the virtual item, the server 110 determines that the authentication information does not correspond to the identification information of the virtual item. It may be decided that it is not or is not justified. If the pre-stored authentication information is different from the authentication information received from the user terminal 130, the server 110 may control the user terminal 130 to display an error message.

In addition, if the identification information of the virtual item stored in advance is different from the identification information of the virtual item received from the user terminal 130, the server 110 selects the item owned by the user among the virtual items stored in the server 110. The most similar virtual item can be determined as legitimate. The server 110 may store pre-stored virtual items by dividing them into a plurality of groups. The server 110 may control the user terminal 130 to display an unlicensed virtual item that belongs to the same group as the identification information of the virtual item received from the user terminal 130. Here, non-licensed may mean a virtual item that does not cause licensing problems. For example, it may mean a virtual item that can be freely used by a pre-contract with the metaverse system provider or a license-free virtual item. The server 110 may determine one virtual item as a legitimate item based on the user's selection information received from the user terminal 130.

If the authentication information corresponds to the identification information of the virtual item and is determined to be legitimate, the server 110 stores the identification information of the virtual item and the identification information of the user terminal in the authenticated user item list corresponding to the user's identification code ( 1622) can be performed. The server 110 may generate an authenticated user item list for each user. The server 110 may obtain a list of authenticated user items corresponding to the user's identification code. Additionally, the server 110 may store the identification information of the virtual item and the identification information of the user terminal in an authenticated user item list corresponding to the user's identification code. The authenticated user item list may include identification information of the authenticated virtual item. That is, identification information of virtual items that have not been authenticated by the server 110 may not be included in the list of authenticated user items. The reason for simultaneously storing the identification information of the virtual item and the identification information of the user terminal in the user item list may be to determine whether the user's virtual person can wear the virtual item. The server 110 has received a request from the user terminal 130 corresponding to the identification information of the user terminal, and may allow the virtual person to wear the virtual item corresponding to the identification information of the virtual item included in the user item list. there is. Therefore, the virtual item can only be used in a specific user terminal 130. Through this, virtual items can be prevented from being used in multiple user terminals 130, and the rights of the copyright holder or brand licensor can be protected.

If the authentication information does not correspond to the identification information of the virtual item or is determined to be invalid, the server 110 may transmit information indicating that authentication has not been completed to the user terminal 130. The user terminal 130 may help the user perform authentication normally by outputting information indicating that authentication has not been completed.

The server 110 may perform step 1623 of transmitting information about the authenticated user item list to the user terminal 130. Information about the authenticated user item list may include information about authenticated virtual items. Information on the list of authenticated user items includes the copyright holder of the authenticated virtual item or item owned by the user, the creator, the item's creation time, the item's release date, the item's production order, the brand, the item's item, the item's shape, or , may include information about at least one of the materials of the item. The user terminal 130 may perform a step of outputting information about the authenticated user item list. In this way, users can use items they own in the metaverse. Therefore, users can use their favorite items in both real and virtual spaces, which can increase satisfaction.

The step 332 of displaying virtual people, virtual spaces, and virtual objects on the display may further include the following processes. The user terminal 130 may perform step 1631 of acquiring information about a virtual person using at least one of at least one item included in the user item list based on the user's selection. As previously described, the information about the virtual person may include information about the items owned by the virtual person, and at least one virtual item included in the user item list may be determined to be an item owned by the virtual person.

Authenticated virtual items can be checked in virtual space using the user terminal.

Figure 17 is a flowchart showing a method of operating a metaverse system according to an embodiment of the present disclosure.

Figure 17 can be performed after Figure 16. The server 110 may perform step 1710 of requesting authentication information obtained by scanning a tag from the user terminal at a predetermined period. The predetermined period may be one week, one month, one year, or three years. The predetermined cycle may become increasingly longer based on the time elapsed since the release of the virtual item or item owned by the user. That is, the predetermined cycle may increase in proportion to the time elapsed from the release of the virtual item or the item owned by the user. For example, the predetermined cycle up to one month after the release of a virtual item or a user-owned item may be one week. However, after one month after the release of the virtual item or the user-owned item, the predetermined cycle may be one month, and after one year after the release of the virtual item or the user-owned item, the predetermined cycle may be one year. Additionally, there may be no predetermined cycle after a critical time has passed after the release of a virtual item or an item owned by a user. That is, after the critical time has passed, authentication for virtual items or items owned by the user may not be performed.

Additionally, the server 110 may perform additional authentication regardless of the predetermined period. The server 110 may perform re-authentication in the manner described above when a user terminal that is not stored in the user's authenticated user item list attempts to access a virtual item.

The server 110 may set a predetermined cycle for each authenticated user item list. When a predetermined period arrives, the server 110 may request authentication information from the user terminal based on the identification information of the user terminal stored in the authenticated user item list.

In response to the request from the server 110, the user terminal 130 may perform step 1721 of outputting information indicating that authentication information must be obtained by scanning the tag. Based on the output information, the user can scan the tag using the user terminal 130. The user terminal 130 may obtain authentication information.

The server 110 may perform step 1722 of re-receiving authentication information, user identification information, and user terminal identification information obtained by scanning the tag from the user terminal. In FIG. 16, the server 110 has received the authentication information, the user's identification information, and the user terminal's identification information, so in FIG. 17, the information received by the server 110 for re-authentication is divided into re-received authentication information and re-received This is referred to as user identification information and re-received user terminal identification information.

The re-received authentication information corresponds to the identification information of the re-received virtual item and is legitimate, and the identification information of the re-received virtual item and the re-received user terminal within the authenticated user item list corresponding to the re-received user identification code. If identification information is included, the server 110 may perform a step of maintaining the identification information of the virtual item and the user terminal in the authenticated user item list.

Unlike FIG. 16, the server 110 further determines whether the identification information of the re-received virtual item and the identification information of the re-received user terminal are included in the authenticated user item list corresponding to the re-received user identification code. You can judge. Additionally, if the condition is satisfied, the server 110 may maintain the identification information of the virtual item and the user terminal in the authenticated user item list.

The re-received authentication information corresponds to the identification information of the re-received virtual item and is legitimate, and the identification information of the re-received virtual item and the re-received user terminal within the authenticated user item list corresponding to the re-received user identification code. If the identification information is not included, the server 110 may store the identification information of the re-received virtual item and the identification information of the re-received user terminal in the authenticated user item list. The server 110 stores the identification information of the re-received virtual item and the identification information of the re-received user terminal in the authenticated user item list, so that the user forgets his or her virtual item and does not register it with the server 110. can be prevented. Additionally, even if the identification information of the virtual item is deleted from the user item list of the server 110 due to an error, the identification information of the virtual item can be recovered through the above process. Therefore, users may not feel any inconvenience even if an error occurs.

If the re-received authentication information does not correspond to the identification information of the re-received virtual item or is invalid, the server 110 may perform a step of deleting the identification information of the virtual item from the authenticated user item list. Therefore, even if the user later requests a virtual item using the user terminal 130, the server 110 may not provide virtual item data to the user terminal 130. Therefore, the virtual character used by the user may not be able to wear the corresponding virtual item.

As already described, the server 110 may perform step 1710 of requesting authentication information from the user terminal 130 at a predetermined period. The user terminal 130 may output information indicating that authentication information must be obtained. However, since the user does not have an item owned by the user, authentication information may not be obtained from the item owned by the user.

If the server 110 does not receive a reply from the user terminal within a predetermined period after requesting authentication information, a step of deleting at least one of the identification information of the virtual item and the identification information of the user terminal from the authenticated user item list is performed. You can. Here, the predetermined period may be a week, a month, a year, or 3 years. However, it is not limited to this. The server 110 may control the predetermined period to be increased in proportion to the time elapsed from the release of the virtual item or the item owned by the user. For example, the predetermined period of time up to one month after the virtual item is released may be a week. However, after one month after the launch of the virtual item or the item owned by the user, the predetermined period may be one month, and after one year after the launch of the virtual item or the item owned by the user, the predetermined period may be one year. Additionally, there may not be a predetermined period of time after a critical time has passed after the release of a virtual item or an item owned by a user. That is, after the critical time has passed, authentication for virtual items or items owned by the user may not be performed.

The user terminal 130 may perform a step of receiving an input from the user for requesting the use of at least one virtual item among the information on the authenticated user item list. As already described, the user terminal 130 may receive information about an authenticated user item list from the server 110. The user terminal 130 may output information about a list of authenticated user items. Output can be video or sound. Information on the authenticated user item list includes at least the copyright holder of the authenticated virtual item, the creator, the item's creation time, the item's release date, the item's production order, the brand, the item's item, the item's shape, or the item's material. It can contain information about one thing. The user may use the input unit of the user terminal 130 to input into the user terminal 130 one virtual item among the information on the authenticated user item list.

In response to receiving an input for using at least one virtual item, the user terminal 130 sends use request information including identification information for the requested virtual item, user identification information, and identification information of the user terminal to the server ( 110) can be performed.

According to an embodiment of the present disclosure, the user terminal 130 can be used by the server 110 simply by transmitting the identification information for the requested virtual item, the user identification information, and the identification information of the user terminal to the server 110 without a separate command. ) can be seen to be requesting the use of a virtual item. According to another embodiment of the present disclosure, the user terminal 130 may transmit a command requesting use to the server 110 along with identification information for the requested virtual item, user identification information, and identification information of the user terminal. . The server 110 may know that the user terminal 130 is requesting use based on the command requesting use.

The server 110 may perform a step of determining whether the identification information for the requested virtual item and the identification information for the user terminal are included in the authenticated user item list corresponding to the user's identification information included in the use request information. The server 110 may obtain the user's authenticated user item list based on the user's identification information included in the use request information. Additionally, the server 110 may determine whether identification information for the requested virtual item and identification information for the user terminal included in the playback request information are included in the acquired authenticated user item list.

If the authenticated user item list includes identification information for the requested virtual item and identification information for the user terminal, the server 110 sends data of the virtual item corresponding to the identification information for the requested virtual item to the user terminal 130. ) can be performed. Data on virtual items may be stored by the server 110. However, it is not limited to this, and the server 110 may control a separate virtual item cloud server to provide virtual item data to the user terminal 130. Here, the data of the virtual item may include modeling data of the virtual item.

The data of the virtual item may include content that is not included in the item owned by the user. For example, music or video may not be played on an item owned by the user, but music or video may be played when a virtual person wears the virtual item. Accordingly, the server 110 can induce users to authenticate to the server 110 the items they actually own.

Figure 18 is a flowchart showing a method of operating a metaverse system according to an embodiment of the present disclosure.

The purchaser terminal 1810 may perform step 1821 of acquiring authentication information. The buyer terminal 1810 may be different from the user terminal 130. Additionally, the owner of the purchaser terminal 1810 may be different from the owner of the user terminal 130. The user may be the first user to purchase an item owned by the user. Additionally, the user terminal 130 may be the user terminal that first authenticated the item owned by the user. A buyer may be a person who purchases an item owned by the user from the user. The buyer terminal 1810 may be a buyer's terminal.

According to an embodiment of the present disclosure, the process by which the purchaser terminal 1810 acquires authentication information may be the same as the process by which the user terminal 130 acquires authentication information. For example, the purchaser terminal 1810 can obtain authentication information from a tag using a camera. The tag may include a communication unit. The communication unit of the tag can communicate with the purchaser terminal 1810. The communication method can be NFC (Near Field Communication) or Bluetooth. The buyer terminal 1810 can obtain authentication information by communicating with the tag. As already explained, the authentication information may be One Time Password.

According to an embodiment of the present disclosure, the process by which the purchaser terminal 1810 acquires authentication information may be different from the process by which the user terminal 130 acquires authentication information.

The user terminal 130 may request transmission of authentication information to the purchaser terminal 1810. When the user terminal 130 requests transmission of authentication information to the purchaser terminal 1810, the user terminal 130 distributes and shares the transmission and reception history information for the authentication information to the server, user terminal, and purchaser terminal. It can be done. Servers, user terminals, and buyer terminals may be interconnected through a blockchain network.

Here, the transmission/reception history information may include detailed information on the rights holder including the name of the copyright holder or trademark holder for the virtual item related to the authentication information. Transmission/reception history information may include detailed information about virtual items related to authentication information. The transmission/reception history information may include detailed rights information, including trademark status, trademark holder, copyright holder, copyright expiration date, limit on the number of resaleable items, or resale validity period information for the virtual item related to the authentication information. The transmission/reception history information may include detailed information about the user terminal and buyer terminal that purchased or sold the virtual item related to the authentication information.

The server 110, the user terminal 130, and the purchaser terminal 1810 may each perform a step of deriving a hash value based on transmission/reception history information for authentication information.

The server 110, the user terminal 130, and the buyer terminal 1810 may generate a hash value by processing transmission/reception history information of virtual item data using a hashing algorithm.

If the hash values of the server 110, the user terminal 130, and the purchaser terminal 1810 match each other, the purchaser terminal 1810 may perform a step of transmitting payment information for authentication information to the server. In order to determine whether the hash values of the server 110, the user terminal 130, and the purchaser terminal 1810 match, the hash values generated by each device may be shared with each other. That is, the hash value generated in the server 110 can be shared with the user terminal 130 and the buyer terminal 1810, and the hash value generated in the user terminal 130 can be shared with the server 110 and the buyer terminal 1810. It can be shared, and the hash value generated in the purchaser terminal 1810 can be shared with the server 110 and the user terminal 130. The server 110, the user terminal 130, and the purchaser terminal 1810 may determine that the mutually shared hash values match each other.

Payment information for authentication information may be payment information for a purchaser to purchase data of an item or virtual item owned by the user from the user using the purchaser terminal 1810. Payment information may include information necessary for payment. For example, payment information may include payment amount, credit card information, or payment bank information. The fact that the buyer terminal 1810 transmits payment information for authentication information to the server 110 may mean that the buyer terminal 1810 has paid the payment amount to the server 110.

When the server 110 receives payment information, the server 110 may perform a step of transmitting transaction approval information to the user terminal 130. Transaction approval information may indicate that the server 110 has approved the transaction of virtual item data between the user terminal 130 and the buyer terminal 1810. The server 110 may deliver to the user the amount received from the purchaser terminal, minus the service provider's fee or the copyright fee for the copyright holder.

Based on the transaction approval information, the user terminal 130 may transmit authentication information to the purchaser terminal 1810. That is, the user terminal 130 receives transaction approval information, and when the transaction approval information is determined to be valid, it can transmit authentication information to the buyer terminal 1810. Virtual item data may not be transmitted between the user terminal 130 and the purchaser terminal 1810. The buyer terminal 1810 may receive data on the virtual item through the server 110 based on the authentication information.

So far, the process by which the purchaser terminal 1810 receives authentication information has been described. Hereinafter, a process in which the purchaser terminal 1810 updates the user item list authenticated in the server 110 based on authentication information will be described with reference to FIG. 18 .

Referring to FIG. 18, the purchaser terminal 1810 transmits authentication information, identification information of a virtual item based on a physical item owned by the user, identification information of the purchaser, and identification information of the purchaser terminal to the server 110 ( 1822) can be performed. Step 1822 may be the same as step 1612 or step 1722, except that the subject of execution is different. Therefore, redundant explanations are omitted.

The server 110 may perform step 1831 of determining whether the authentication information corresponds to the identification information of the virtual item and is legitimate. This step may be the same as step 1621. Therefore, redundant explanations are omitted.

If the authentication information corresponds to the identification information of the virtual item and is determined to be legitimate, the server 110 may perform step 1832 of deleting the identification information of the virtual item from the list of authenticated user items corresponding to the user's identification code. there is. Since the right to use the virtual item has been transferred from the user to the purchaser, the server 110 deletes the identification information of the virtual item from the list of authenticated user items corresponding to the user's identification code to prevent the user from using the virtual item any longer. can do. Before deleting the identification information of the virtual item from the authenticated user item list corresponding to the user's identification code, the server 110 sends information indicating that the identification information of the virtual item will be deleted from the authenticated user item list to the user terminal 130. ) can be transmitted to. Accordingly, the user can clearly see that he or she will not be able to use the virtual item.

The server 110 may perform step 1833 of storing the identification information of the virtual item and the identification information of the purchaser terminal 1810 in the authenticated purchaser item list corresponding to the purchaser's identification code. Since the buyer has secured a legitimate right to the virtual item, the server 110 stores the identification information of the virtual item and the identification information of the buyer terminal in the buyer item list, thereby allowing the buyer terminal 1810 to use the virtual item. You can.

The server 110 may perform step 1834 of transmitting information about the authenticated buyer item list corresponding to the buyer's identification code to the buyer terminal 1810. Step 1834 may be similar to step 1623. Therefore, redundant explanations are omitted. Additionally, the buyer terminal 1810 acquires information about the buyer's virtual person using at least one item included in the buyer item list corresponding to the buyer's identification code based on the buyer's selection (1841). can be performed. Step 1841 may be similar to step 1631. Therefore, redundant explanations are omitted.

Figure 19 is a flowchart showing the operation of the metaverse system according to an embodiment of the present disclosure.

Step 332, in which the server 110 transmits the properties of the three-dimensional virtual space to the user terminal 130, may include the following process. The administrator terminal 120 performs a step (1911) of transmitting to the server 110 the identification information of the user terminal 130, the user's identification information, and information related to the sale or rental contract of a real space corresponding to the virtual space. can do. The manager may be one of a real space lessor, a real space seller, or a real space broker. Information related to a sales or rental contract of a real space corresponding to a virtual space may include the contract date, identification information of the contract subject, price of the contract subject, information on the parties to the contract and contract contents. Here, the subject of the contract may be actual space. Additionally, the identification information of the contract subject may include the location, land area, or area of the actual space. The content of the contract may include a lease contract or a sales contract. The parties to the contract may include at least one of the buyer, seller, lessor, or lessee. The administrator terminal 120 may transmit contract-related information to the server 110 in order to conclude a contract. In other words, information related to a contract may include various information required before concluding a contract. Through this process, the manager terminal 120 can subscribe to a contract.

The server 110 may perform a step 1921 of transmitting information related to the sale or rental contract of a real space to the user terminal 130. The server 110 may store information related to a sales or rental contract of a real space corresponding to a virtual space. The metaverse system of the present disclosure requires the administrator terminal 120 to transmit contract-related information to the user terminal 130 through the server 110, thereby reducing the possibility of falsification of contract contents.

The user terminal 130 may perform a step 1931 of transmitting consent information for information related to a sales or rental contract of a real space, identification information of the user terminal, and user identification information to the server 110. That is, the user can accept the contract through the user terminal 130.

When the server 110 receives consent information, the identification information of the user terminal and the user's identification information received from the administrator terminal 120 are combined with the identification information of the user terminal and the user's identification information received from the user terminal 130. A step 1941 of determining whether they are identical may be performed. The user can also input the user's identification information and terminal's identification information into the administrator terminal 120. That is, users and administrators may need to be in the same space. If the identification information of the user terminal and the user's identification information received from the administrator terminal 120 are the same as the identification information of the user terminal and the user's identification information received from the user terminal 130, the server 110 determines that the consent information is legitimate. You can decide that it is. The server 110 may receive and store the identification information of the administrator terminal and the administrator identification information from the administrator terminal 120, and may receive and store the identification information of the user terminal and the user identification information from the user terminal 130.

However, it is not limited to this. The server 110 receives consent information, receives and stores identification information of the administrator terminal and administrator identification information from the administrator terminal 120, and receives identification information of the user terminal and user identification information from the user terminal 130. You can save it. The server 110 may determine that the acceptance information is legitimate without any confirmation.

If the identification information of the user terminal and the user's identification information received from the administrator terminal 120 are the same as the identification information of the user terminal and the user's identification information received from the user terminal 130, the server 110 ), the step 1942 of transmitting information about the three-dimensional virtual space can be performed. Therefore, the user can use the virtual space using the user terminal 130. Since the user's movement around the virtual space using the virtual character has already been described in step 410, redundant description will be omitted. In addition, regardless of whether the identification information of the user terminal and the user's identification information received from the administrator terminal 120 are the same as the identification information of the user terminal and the user's identification information received from the user terminal 130, the server When acceptance information is received from the user terminal 130, a step 1942 of transmitting information about the three-dimensional virtual space to the user terminal 130 may be performed.

When the server 110 receives consent information, at least one of information related to the sale or rental contract of a real space corresponding to the virtual space, user identification information, user terminal identification information, administrator identification information, and administrator terminal identification information May be stored in an unmodifiable storage of the server 110. The storage may be IPFS (Inter-Planetary File System). The server can issue two NFTs (Non Fungible Tokens) for information related to the sale or rental contract of real space. The issued NFT can be transmitted to the administrator terminal and user terminal, respectively. The administrator terminal 120 and the user terminal 130 can access information related to the sale or rental contract of real space stored in IPFS using the hash value of the NFT, but may not arbitrarily change the contract details. Therefore, the possibility of falsification of contract contents can be greatly reduced.

As already described, the server 110 may perform step 1942 of transmitting information about the three-dimensional virtual space to the user terminal 130. After that, the user can manipulate the virtual character in the virtual space using the user terminal 130. Additionally, the user can communicate with other users in the virtual space using the user terminal 130. Other users may also be users who received information about the 3D virtual space from the server through the above process.

More specifically, in Korea's apartment subscription system, users cannot experience life in an apartment at all after receiving it and until the apartment is built. However, the metaverse system according to the present disclosure can implement an apartment complex to be built in virtual space. Therefore, users who have purchased an apartment can enjoy life in a real apartment while moving around the apartment in virtual space. In other words, the metaverse system of the present disclosure provides a virtual space where users can experience the apartment complex in advance before moving in, and the virtual space can be used as a space to provide various information related to the situation before moving in. For example, the Metaverse system of the present disclosure can inform users of construction progress, or inform them of the intermediate payment, the method of paying the balance, the payment date, etc. Additionally, the metaverse system of the present disclosure can be used as a community of apartments in virtual space. Residents can use Metaverse to engage in various social activities in virtual space and receive online services.

The user terminal 130 may perform step 420 of acquiring log information. More specifically, the user terminal 130 may perform a step of obtaining the resolution of the display 1210 of the user terminal 130. The resolution of the display may be pre-stored in memory. The user terminal 130 may perform the step of setting a predetermined area of the display as the area of interest 1220 based on the resolution of the display. The user terminal 130 may perform a step of including virtual object of interest information including identification information of a virtual object displayed within the area of interest for more than a predetermined threshold time in log information. Here, the threshold time may have a value of 10 seconds or more. Placing a virtual object in the center of the screen for at least a certain period of time may indicate that the user has expressed interest in the virtual object. Accordingly, the user terminal 130 may include identification information of the virtual object displayed in the area of interest for more than a threshold time in the log information. The user terminal 130 or the server 110 may later use the log information as big data to derive a preferred virtual object according to the user. Accordingly, the user terminal 130 or the server 110 can provide a user-customized virtual object.

The user terminal 130 may perform a step of including virtual object information of interest, including the virtual object with which the virtual person interacted and the content of the interaction, in the log information. The interaction between virtual characters and virtual objects may be predetermined. For example, a predetermined interaction may change the posture of a virtual character or change the position, color, or shape of a virtual object. For example, if the virtual object is a chair, the virtual person can interact by sitting on the virtual object. The virtual character can make the motion of sitting on the virtual object. Additionally, if the virtual object is a light, the virtual person can take actions to turn the light on or off. Virtual objects can turn on or off in response to the virtual person's movements. If the virtual object is a seesaw, the virtual person can interact by sitting on the virtual object, and the virtual object, the seesaw, can move in response to the virtual person's sitting motion. Predetermined interactions may be assigned identification information. The user terminal 130 may include identification information of the virtual object with which the virtual person interacted and identification information of the interaction in the virtual object information of interest.

The user terminal 130 may perform a step of including the contents of the virtual character's conversation with other users' virtual characters or NPC (non-player character) in the log information. The content of the conversation may include things that need to be improved in the virtual space or content about virtual objects of interest. If the virtual space is an apartment complex before moving in, users who have received the apartment can experience life after moving into the apartment in the virtual space provided by the metaverse system of this disclosure. Additionally, in virtual space, the commercial space of an apartment complex can be used as a space where residents can receive coupons for promotional purposes. For example, a user can move to a specific space in virtual space using the user terminal 130. The specific space may be a space at a location designated by the server 110. Specific spaces may be included in commercial space. The server 110 can issue coupons to virtual people in the commercial space. Coupons may include a serial number, QR code, or barcode. Users can use coupons received from Metaverse at real stores. Additionally, a coupon may be a digital discount coupon. The user terminal 130 may transmit the coupon to the server 110. The server 110 may control the user terminal 130 to display at least one virtual item corresponding to the coupon. The user may select one of at least one virtual item using the user terminal 130. The server 110 may store the identification information of the selected virtual item and the user terminal in the user item list in response to the user's selection. Through the above process, the metaverse system can induce virtual characters to move around in virtual space.

The server 110 may accumulate and store the movements of virtual characters. If there is a route among the virtual characters' movements that is more than a predetermined distance from a pre-designed road, the server 110 may transmit the movement route to the administrator terminal 120. The administrator terminal 120 can display the corresponding route on the display. Among the virtual characters' movements, those that are more than a predetermined distance from a pre-designed road may be the optimal route for users when going from one point to another. However, if the road is not designed along the route, it may cause inconvenience to residents in reality. Managers can decide whether a road design is needed for the route in question.

The server 110 may perform a step of determining information about a crowded location that a plurality of users frequently hear in a virtual space, based on log information. As already explained, log information can periodically obtain the user's location. The server 110 can obtain the distribution of virtual characters at the same time. The server 110 may determine the location of congestion over time based on the distribution of virtual people. For example, if more than a critical number of virtual people are gathered around a location at a certain point in time, the location may be determined to be crowded at that point in time. Additionally, if there are less than a critical number of virtual people around a location at a point in time, it may be determined that the location is not crowded at a point in time.

The server 110 may perform a step of determining the rent or sale price of a commercial building to be higher the closer it is to the location of the actual space corresponding to the crowded location. In other words, the patterns of real users can be simulated based on the patterns of virtual people in virtual space, and through this, the rent or sale price around locations predicted to be highly crowded can be determined to be high. Therefore, the commercial sale price can be realized.

The server 110 may determine a virtual object frequently used by a plurality of users based on virtual object information of interest included in the log information. Since the virtual object information of interest has already been described, redundant description will be omitted. Virtual object information of interest may be obtained for a plurality of virtual characters. The server 110 may obtain virtual object information of interest for a plurality of virtual characters and determine the virtual object information of the highest interest or frequently used. Additionally, the server 110 may perform a step of transmitting a frequently used virtual object to the manager terminal 120. If the virtual space is still under construction in reality, administrators can implement frequently used virtual objects in reality. For example, if a virtual person often sits on a bench in a specific location, the number of benches in that location in real life can be increased. Additionally, if you have a high preference for one design among the various bench designs in the virtual space, you can implement a bench in the complex with the bench with the highest preference in reality.

The user terminal 130 performs a step of transmitting a purchase signal for a purchase object included in a plurality of candidate virtual objects stored in the server 110, the user's identification information, and the user terminal's identification information to the server 110. You can. A purchase signal for a purchase object may include identification information for the purchase object.

Server 110 may perform steps to determine whether the purchase signal is legitimate. For example, the server 110, the user terminal 130, and the administrator terminal 120 may each perform a step of deriving a hash value based on transmission/reception history information about a purchase signal for a purchase object. The server 110, the user terminal 130, and the administrator terminal 120 can generate a hash value by processing the transmission/reception history information of the purchase signal for the purchase object using a hashing algorithm. If the hash values of the server 110, the user terminal 130, and the administrator terminal 120 match each other, the server 110 may determine that the purchase signal is legitimate. Additionally, the user terminal 130 may transmit payment information for a purchase signal for a purchase object to the server 110.

If the purchase signal for the object is determined to be legitimate, the server 110 may perform a step of storing the identification information of the purchase object and the identification information of the user terminal 130 in the user object list corresponding to the user's identification information. . Therefore, the user can take possession of the purchase object electronically.

If the purchase signal is determined to be legitimate, the server may include activating the purchase object by the deadline of the purchase object included in the properties of the purchase object. In other words, a purchase object may have a deadline. The deadline may be infinite. However, the deadline may be finite. If the deadline has expired, the server 110 may delete the identification information of the purchase object from the user object list. Additionally, the server 110 may determine the purchase object as a virtual object of interest. The server 110 may store virtual objects of interest as log information. Additionally, the server 110 may determine the virtual object of interest as a virtual object with high preference. The server 1100 may transmit the virtual object of interest to the manager terminal 120, allowing the manager to implement a real object corresponding to the virtual object of interest in the real world.

The user terminal 130 may perform step 332 of displaying at least one of a virtual person, a virtual space, and a virtual object on the display. At this time, the user terminal 130 may perform a step of displaying the activated purchase object. In other words, the virtual object purchased by the user can be displayed in virtual space. Conversely, virtual objects that the user has not purchased may not be displayed in virtual space or may only display a translucent outline.

More specifically, the purchase object may be a virtual object in the shape of a temple tile. When a user purchases roof tiles through the above process, the roof tiles purchased by the user may be displayed as a virtual object as part of a temple building in virtual space. When a user or other users click (touch) a tile that is a virtual object using their terminal, buyer information of the virtual object may be displayed on the display. For example, buyer information may be the buyer's name and wish message. Through this, as users purchase tiles, which are virtual objects, the temple building in virtual space can be completed. Therefore, it can encourage user participation. Additionally, users can feel a sense of accomplishment by seeing the process of building a building being completed in a virtual space through their contributions, and can feel satisfied because other users can confirm their contributions.

Additionally, the purchase object may be a virtual object in the shape of a temple tile. When a user purchases roof tiles through the above process, the roof tiles purchased by the user may be displayed as a virtual object as part of a temple building in virtual space. When a user or other users click (touch) a tile that is a virtual object using their terminal, buyer information of the virtual object may be displayed on the display. For example, buyer information may be the buyer's name and wish message. Through this, as users purchase tiles, which are virtual objects, the temple building in virtual space can be completed. Therefore, it can encourage user participation.

The above explanation was based on virtual objects, but the same explanation may be possible for virtual space. More specifically, the user terminal 130 transmits a purchase signal for a purchase space included in a plurality of candidate virtual spaces stored in the server 110, the user's identification information, and the user terminal's identification information to the server 110. can be performed. The purchase signal for the purchase space may include identification information for the purchase space. When a user purchases a purchased space, the server 110 can control the user terminal 130 to display the purchased space. The server 110 can control the movement of virtual characters in virtual space. Additionally, the user terminal 130 may control placement of at least one object included in the user object list into the purchase space.

Meanwhile, according to an embodiment of the present disclosure, the server 110 or the user terminal 130 may include a ray tracing device, and rendering using the ray tracing technique may be performed accordingly. In other words, when displaying an object placed in a virtual space, a more realistic virtual space can be realized by determining the color by tracing the path of light from the light source through the object to the user's viewpoint.

Hereinafter, an embodiment of the ray tracing device and ray tracing-based rendering method of the present disclosure will be described in detail using the attached FIGS. 22 to 30.

Figure 22 is a conceptual diagram for explaining ray tracing.

Referring to FIG. 22, in virtual space modeling, a ray tracing device can determine a viewpoint 2210 to generate a 3D image. The location of the viewpoint 2210 may correspond to a camera installed to take pictures in a predetermined direction at a predetermined location included in the metaverse system 100. Additionally, the position and direction of the viewpoint 2210 may be set differently for each frame according to the user's input through the user terminal 130.

When the viewpoint 2210 is determined, the ray tracing device can determine the screen 2220 of the display according to the determined viewpoint 2210, and each ray projected for each pixel of the screen 2220 from the viewpoint 2210. can be created. For example, as shown in FIG. 22, when the resolution of the display is 9*8, rays for 72 pixels can be generated respectively. Hereinafter, a ray representing one pixel will be described, and the same principle applies to other pixels constituting the screen 2220.

First, a primary ray (2240) is generated from the viewpoint 2210. The first ray 2240 passes through the screen 2220 and intersects a specific object 2230. Here, the object may be an object included in the virtual space or a primitive, which is the most basic unit area that constitutes the object. For example, a primitive may be in the form of a polygon including triangles and squares. In other words, an object is composed of a set of primitives, and a primitive can be the most basic unit of rendering processing.

At the intersection (hit point) where the primary ray 2240 intersects the object 2230, a reflection ray (2251), a refraction ray (2252), and a shadow ray (2253) can be generated. and these are referred to as secondary rays. The secondary ray is determined according to the properties of the object, and the properties of the object correspond to the above-described object properties and may include, but are not limited to, the reflectance, refractive index, transmittance, and original color value of the object.

A shadow ray 2253 is generated from the intersection point in the direction of the light source 2260. Here, the light source 2260 may be a virtual light source that has been created in advance and included in the virtual space, and its location may be determined based on the viewpoint 2210. The ray tracing device can determine whether the intersection point is exposed to the light source 2260 through the shadow ray 2253. For example, if another object exists in the path of the shadow ray 2253 and meets it, a shadow may be formed on the pixel corresponding to the intersection point.

The reflection ray 2251 is generated in a direction corresponding to the angle of incidence of the primary ray 2240, and a weight according to the reflectance of the object 2230 may be applied. The refractive ray 2252 is generated in a direction corresponding to the incident angle of the primary ray 2240 and the refractive index of the object 2230, and a weight according to the refractive index may be applied. That is, when the reflectance and refractive index of the object 2230 are “0”, the reflection ray 2251 and the refraction ray 2252 for the object 2230 do not occur.

Additionally, the ray tracing device determines whether the reflected ray 2251 and the refracted ray 2252 reach other objects. If the reflection ray 2251 reaches another object, the ray tracing device can check the coordinates of the intersection point and the color value of the other object. Additionally, a ray tracing device can identify tertiary rays that occur at intersections with other objects. Tertiary rays can also include reflected rays, refracted rays, and shadow rays, and when the reflected rays and refracted rays meet another object, another intersection may occur. In this way, sub-rays derived from secondary rays can be created and their paths tracked until all rays generated in response to a pixel reach the light source or leave the virtual space, and subsequent intersection points by each sub-ray can be confirmed. It can be.

As a result, the ray tracing device can identify the primary ray 2240 for a pixel and the path of all rays derived from the primary ray 2240 and derive the color value of the corresponding pixel according to the path analysis result. In other words, the color value of the pixel is the color of the object 2230 at the intersection by the primary ray, the object color at each intersection by all generated reflection rays 2251 or refraction rays 2252, and all generated shadow rays 2253 are the light source. Whether or not 2260 is reached is collectively determined, and the ray tracing device can construct the screen 2220 by performing the above process for all pixels.

Meanwhile, hereinafter, the specific object 2230 where the primary ray 2240 intersects is defined as an “object,” and an object different from the specific object where the secondary ray intersects is defined as an “object.” Additionally, the intersection point between the object 2230 and the primary ray 2240 is defined as a “first intersection point,” and the intersection point between the object and the secondary ray is defined as a “second intersection point.” In addition, rays that are derived from the secondary ray and occur at the second intersection are defined as “sub rays,” and the point where the sub ray intersects with another object is defined as a “subsequent intersection.”

Figure 23 is a block diagram of the structure of a ray tracing device according to an embodiment of the present disclosure.

Referring to FIG. 23, the ray tracing device 2300 according to an embodiment of the present disclosure includes a ray generation unit 2310, a Traversal (TRV) unit 2320, an Intersection (IST) unit 2330, and a shading unit. It may include a unit 2340, a memory 2350, and an acceleration structure generation unit 2360.

According to one embodiment, the TRV unit 2320, IST unit 2330, and acceleration structure generation unit 2360 may be implemented as separate hardware included in the server 110 or the user terminal 130. Additionally, the memory 2350 may be constructed in the form of an internal or external memory or database included in the server 110 or the user terminal 130. In addition, FIG. 2 shows only components related to one embodiment of the present disclosure, and of course, other general-purpose components related to ray tracing may be further included.

The TRV unit 2320 according to one embodiment receives information on the rays generated from the ray generation unit 2310. For example, in the case of a primary ray, the TRV unit 2320 may receive information about the viewpoint and direction of the primary ray. Additionally, in the case of secondary rays and sub-rays, the TRV unit 2320 can receive information about the starting point and direction of each ray. Here, the starting point means the first intersection where the first ray was hit in the case of the secondary ray. The starting point and starting point can be set as coordinates, and the direction can be implemented as a vector.

The TRV unit 2320 reads the acceleration structure (AS) from the memory 2350, performs a search, and cross-checks each node included in the acceleration structure (AS) with the ray. In this regard, an embodiment of the acceleration structure and a method for searching the acceleration structure will be described in detail with reference to FIG. 24.

Figure 24 is a conceptual diagram for explaining the acceleration structure.

According to an embodiment of the present disclosure, the process of finding intersections between rays and objects requires a large amount of energy and computation, and accordingly, the ray tracing device 2300 performs the intersection confirmation process by hierarchically dividing the 3D virtual space. can be processed quickly.

To this end, the acceleration structure generator 2360 according to an embodiment may generate an acceleration structure (AS), and the generated acceleration structure (AS) may be stored in the memory 2350. Here, the acceleration structure (AS) refers to a space division structure in which the virtual space displayed on the display is divided into a hierarchical tree node form. The acceleration structure (AS) contains location information of objects included in a three-dimensional virtual space and can be created in various forms. For example, the acceleration structure generator 2350 uses the data structures of KD-tree (K-dimensional tree), BVH (Bounding Volume Hirearchy), SBVH, OSAH, and AOBVH to represent the relationship between objects in three-dimensional space. Acceleration structures can be created. However, the above example data structure does not limit the present invention.

According to one embodiment, the acceleration structure generator 2360 can hierarchically divide the 3D virtual space based on objects placed in the virtual space and set each divided area as a bounding volume. . The bounding volume preferably has a simple geometric shape, and a representative example is a hexahedral-shaped bounding box. The bounding box may include at least one object or primitive included in the virtual space.

For example, assuming that the 3D virtual space to be implemented on the screen is an office and the overall structure of the office is a state in which desks are arranged, the acceleration structure generator 2360 can divide the office by setting a bounding box for each desk. . In addition, the space of each desk can be divided by setting bounding boxes for various office supplies such as monitors and computers placed on each desk. In this way, the space can be divided hierarchically by gradually setting the bounding box in detail from large-scale objects to small objects, and ultimately, the bounding box can be set in primitive units.

According to one embodiment, since the virtual space is gradually divided into details by the bounding box, the structure of the three-dimensional virtual space can be implemented as an acceleration structure (AS) in the form of a hierarchical tree node as shown in FIG. 24. Here, each node included in the tree may correspond to a divided area, that is, a bounding box.

Referring to FIG. 24, the acceleration structure (AS) may include a top node, an inner node, a leaf node, and a primitive.

The top node is a node of the highest level that has no upper nodes and only lower nodes. In Figure 24, node number 1 corresponds to this. In the example of the virtual space mentioned above, an office may correspond to this, and the three-dimensional frame in which the virtual space is generally constructed can be set as the top node as the largest bounding box.

An internal node is a node that has both upper and lower nodes. Nodes 2 to 7 in FIG. 24 correspond to this. Additionally, internal nodes may be configured across multiple layers. As shown in Figure 24, node 2 has internal nodes 4 and 5 as child nodes, and node 3 has internal nodes 6 and 7 as child nodes. That is, even if they are the same internal nodes, nodes 2 and 3 are placed in a layer one level higher than nodes 4 to 7. In the example mentioned above, nodes 2 and 3 may correspond to bounding boxes containing each desk, and nodes 4 to 7 may correspond to each office supply placed on each desk.

A leaf node is the lowest node that has only upper nodes and no lower nodes. Nodes 8 to 15 in Figure 24 correspond to this. Each leaf node may include at least one object, and as shown in FIG. 24, the object at this time can be set in primitive units. In the example mentioned above, nodes 8 to 15 may correspond to bounding boxes containing each part or area that makes up each office supply.

According to one embodiment, the ray tracing device 2300 first performs an intersection check between a ray and a bounding box (node) through an acceleration structure (AS), and calculates the intersection point with an object included in the intersected bounding box. can be checked quickly. At this time, the ray tracing device 2300 compares the direction of the ray and the coordinates of the starting point with the coordinates of the object, so that complex calculations may not be required for intersection inspection.

In this regard, the TRV unit 2320 may search the acceleration structure (AS) and perform cross-examination between each node and the ray. For example, the cross-check between nodes and rays may be, but is not limited to, AABB (Axis-Aligned Bounding Box) or OBB (Oriented Bounding Box) collision testing. Additionally, the TRV unit 2320 can read the direction of the ray and the coordinates of the starting point from the memory 2350 and determine which nodes (bounding boxes) the ray has intersected and which nodes it has not intersected based on this.

According to one embodiment, the acceleration structure can be simplified through cross-examination of nodes and rays. For example, using Figure 24, when the first ray passes through node 1 and intersects node 2, the tree structure of number 3 or less is excluded from the intersection test. After this, if the first ray intersects node 4, the tree structure of node 5 or lower is excluded from the intersection test. Finally, when it intersects with node 8, node 9 is excluded from the intersection test, and node 8 can be identified as the node containing the first intersection.

In this way, the TRV unit 2320 may continue to search the acceleration structure (AS) until it detects a leaf node or a bounding box of a primitive where a ray intersects. The TRV unit 2320 searches for the acceleration structure (AS) along one path of the ray, and if no intersecting leaf nodes are found on the path, the TRV unit 2320 may search for the acceleration structure (AS) through another path.

Referring again to FIG. 23, the IST unit 2330 according to one embodiment may receive a node (bounding box) where rays intersect from the TRV unit 2320. The IST unit 2330 can read information about objects included in the corresponding node from the memory 2350 and perform a cross-check between each read object and the ray. At this time, information about the object read may include geometry data of the object. For example, geometric data may include information about the vertices and coordinates of primitives constituting an object, but is not limited thereto.

After identifying the object on which the ray was hit, the IST unit 2330 checks which point of the identified object the ray intersected and outputs the intersection point. The intersection point may be stored in coordinate form and transmitted to the shading unit 2340.

The shading unit 2340 according to one embodiment derives the color value of the corresponding pixel based on information about the intersection received from the IST unit 2330 and attribute information of the object corresponding to the intersection. That is, the shading unit 2340 may determine the color value of the pixel corresponding to the first intersection by combining information about each intersection by all rays derived from the first ray.

Meanwhile, despite quickly identifying the intersection between rays and objects using the acceleration structure (AS) as described above, the amount of calculation for ray tracing processing may still be excessively required depending on the type of virtual space and the number and configuration of objects. In particular, ray tracing determines the color of an object by combining the colors of other objects reflected or refracted through secondary rays and sub-rays, so it is necessary to calculate even the color of objects that have little effect on the quality of virtual space or the user's sense of reality. The process can proceed. In addition, due to the nature of the metaverse, there may be many cases where the color in reality and the color visually identified by the user through the virtual space are different, resulting in a sense of reality. For example, when a user looks at a specific object on the screen, the colors of objects around the object must be comprehensively considered as they are easily judged by the naked eye, while objects far away from the object require the user to determine their association with the object. Since it is difficult to perceive, it may appear more realistic that only the color of a single object is reflected on the object.

In this regard, the ray tracing device 2300 of the present disclosure can perform a rendering method that performs ray tracing by considering the distance between objects, and an embodiment of this will be described below using FIGS. 25 to 30. .

Figure 25 is a flowchart of a ray tracing-based rendering method according to an embodiment of the present disclosure.

Referring to FIG. 25, in step 2510, the ray tracing device 2300 generates a primary ray projected from the viewpoint of the current frame toward the virtual space and confirms the first intersection point where the generated primary ray intersects the object. . Here, the process of generating the first ray and confirming the first intersection point is replaced with the above-described content.

In step 2520, the ray tracing device 2300 generates a secondary ray generated at the first intersection according to attribute information of the object. That is, the ray tracing device 2300 can read the reflectance and refractive index of the object from the memory 2350 and generate a reflection ray and a refractive ray by applying weights to each reflectance and refractive index. Of course, shadow rays incident on a light source can also be generated.

The ray tracing device 2300 can check the second intersection point where each generated secondary ray intersects the object and determine the distance between the first intersection point and the second intersection point. Preferably, it is possible to determine how far the object is from the object.

According to one embodiment, since the virtual world may have the same coordinate system as the real world, the distance between the first intersection and the second intersection may be determined based on location information of the real space corresponding to the virtual space. The ray tracing device 2300 may transmit information about the first intersection and the second intersection to the server 110 or the user terminal 130. The server 110 or the user terminal 130 may extract GPS coordinate information of the first intersection and the second intersection, calculate the distance between them, and transmit it to the ray tracing device 2300. In this case, there is an advantage that the exact distance between the objects can be used for rendering.

However, the processing speed of rendering has a significant impact on the quality of the metaverse and the user's immersion, and in cases where dynamic objects exist in successive frames or the user's viewpoint is actively changing, the distance is based on the actual coordinate system. This method requires communication between devices, so there is a high risk of delay.

In this regard, the ray tracing device 2300 according to one embodiment displays the relationship between the distance between the object and the object in the virtual space or uses an acceleration structure (AS), thereby automatically connecting the object and the object without communicating with the outside. The distance can be determined. Hereinafter, an embodiment of this will be described using FIGS. 26 and 27.

FIG. 26 is a flowchart of a method for determining the distance between objects according to an embodiment of the present disclosure, and FIG. 27 is a flow chart showing a virtual space on a plane to explain a method of determining the distance between objects according to an embodiment of the present disclosure. This is a projected example.

Looking at Figure 26, in the process of determining the distance between the first intersection and the second intersection, first, the ray tracing device 2300 has a first sphere and a second radius centered on the first intersection. A second district can be formed. Here, the first radius and the second radius are preset values according to the frame of the virtual space, and the second radius is a larger value than the first radius. The first radius is set as a distance to specify the surrounding area close to the object, and the second radius is set as a distance to specify an area far away from the object.

According to a further embodiment, the first radius may be changed based on attribute information of the virtual space and the object. For example, the ray tracing device 2300 is a main scene of the virtual space (the target of the object) based on at least one of the type, size, and shape of the virtual space and the location of the real space corresponding to the virtual space. ) objects can be specified. The ray tracing device 2300 can calculate a reference length, which is the average value of the separation distance between specific objects. Afterwards, when the object is confirmed through cross-examination, the first radius can be finally set based on the attribute information and acceleration structure (AS) of the object. For example, the ray tracing device 2300 may establish an error range of the reference length based on the identification information, type, shape, size, and location of the object. In addition, by analyzing the acceleration structure (AS), the number of objects located within the reference length of the object can be identified, and depending on the number of identified objects, a specific length within the error range can be determined as the first radius. If there are many objects located within the standard length, the first radius can be set to be relatively small, and if there are few objects located within the standard length, the first radius can be set to be relatively large.

According to a further embodiment, the second radius may be changed based on attribute information of the virtual space and the object. For example, the ray tracing device 2300 determines the object based on at least one of the type, size, shape, and location of the real space corresponding to the virtual space and the identification information, type, shape, size, and location of the object. Objects that become background scenes can be specified based on . Here, objects that become the background scene may include objects that are default to the background (the primary ray does not directly intersect). Thereafter, the ray tracing device 2300 may identify the object closest to the object among the objects that become the background scene and set the distance between the identified object and the object as the second radius.

When the first sphere and the second sphere are formed, the ray tracing device 2300 uses the inside of the first sphere as the first area, the outside of the second sphere as the second area, and the outside of the first sphere and the inside of the second sphere overlap. The area can be set as the third area. Here, the first area is an area highly related to the color value of the object and can be stored in the memory 2350. The second area may be stored in the memory 2350 as an area that is related to the color value of the object but does not generate sub-rays. The third area is unrelated to the color value of the object and may be stored in the memory 2350 as an area not used for ray tracing. In other words, the third area refers to an area that does not affect or rather degrades the realism of the object. By specifying the third area and excluding tracking, the amount of rendering calculations can be significantly reduced and efficiency increased.

According to one embodiment, the first sphere and the second sphere are formed to overlap in the virtual space within the system, but are not actually visible visually. In addition, the first to third areas are also reflected in the acceleration structure (AS), so it is possible to determine which area each node is distributed over.

Once each area is set, the ray tracing device 2300 can determine in which area the second intersection point is located. The ray tracing device 2300 determines the object as a first object when the second intersection point is located in the first area, determines the object as a second object when the second intersection point is located in the second area, and determines the object as a second object when the second intersection point is located in the second area. If located in this third area, the object can be determined as a third object. Here, the first object is an object adjacent to the object, the second object is an object far away from the object, and the third object is an object that is not involved in the color value of the pixel corresponding to the first intersection.

Taking the office virtual space mentioned above as an example again, assuming that the object is an office supply located on a specific desk, the first object may be other office supplies located on the desk. The secondary object can be a building, person, terrain, or natural environment displayed outside the office door or window. The third object may be an object that is located inside the office but appears to the user to be unrelated to the object. For example, this could be another desk, office supplies placed on another desk, other furniture, appliances, tools, people, etc.

The embodiment of FIG. 26 that determines the distance using FIG. 27 will now be described. The solid arrow line represents the primary ray, and the dotted arrow line represents the secondary ray.

The ray tracing device 2300 projects a primary ray from a viewpoint (not shown) to the virtual space 2700. The first ray intersects the object 2710, and the ray tracing device 2300 confirms the first intersection point 2711. The ray tracing device 2300 forms a first sphere (S1) and a second sphere (S2) centered on the first intersection 2711 and projects them into the virtual space 2700. The ray tracing device 2300 sets a first area, a second area, and a third area according to the first sphere S1 and the second sphere S2.

Afterwards, the ray tracing device 2300 generates a secondary ray generated at the first intersection 2711. Secondary rays may include reflected rays or refracted rays as shown. The ray tracing device 2300 determines the location of the second intersection formed when the secondary ray intersects the object. When the second intersection 2721a is located in the first area, the ray tracing device 2300 determines that the object 2720a reached by the second ray is the first object. The criterion for determining the area is the intersection point, not the location of the object. In other words, an object may be completely contained in one area, but may also be contained across two or more areas. As shown, when the second intersection 2721a is in the first area, the object 2720a is determined to be the first object even if it spans the third area. When the second intersection 2721b is located in the second area, the ray tracing device 2300 determines the object 2720b to which the secondary ray has arrived as the second object. When the second intersection 2721c is located in the third area, the ray tracing device 2300 determines that the object 2720c reached by the second ray is the third object.

In relation to this embodiment, according to an embodiment of the present disclosure, processing amount can be further reduced in the process of confirming the second intersection. Since the third area is an area where ray tracing is not performed and is stored in the memory 2350, the effectiveness of searching for nodes distributed only in the third area is low.

Looking at FIG. 27, there may be a bounding box B that exists only in the third area. The ray tracing device 2300 can identify such a bounding box (B), and if identified, can perform a process of removing all the corresponding node and its included child nodes and objects (P) from the acceleration structure (AS). there is. The ray tracing device 2300 can identify the second intersection point by searching the modified acceleration structure (AS) and identifying objects included in the node that intersect the second ray. Additionally, it is possible to determine which area of the first to third areas the second intersection point is located in. In other words, the efficiency of cross-checking can be increased by removing nodes and objects that are not required for rendering before performing cross-checking between secondary rays and nodes.

According to an embodiment of the present disclosure, the ray tracing device 2300 uses an acceleration structure (AS) to determine objects close to the object (first object), objects far away (second object), and objects unrelated to rendering. Objects (third objects) can be judged. In this regard, another embodiment of a method of determining the distance between the first intersection and the second intersection will be described using FIG. 28.

Figure 28 is an example of an acceleration structure to explain another embodiment of a method for determining the distance between objects.

First, looking at the acceleration structure (AS) illustrated in FIG. 28, nodes are distributed up to a total of 4 layers, starting with node 1, which is the top node, as the top 1 layer and going to lower nodes. Internal nodes are distributed in layers 2 and 3, and leaf nodes are located in layer 4. Of course, the acceleration structure (AS) in FIG. 28 is only an example, and leaf nodes may be distributed in the same layer as internal nodes.

The ray tracing device 2300 determines that the object is the first object when the node in common between the object and the object is located in a first layer and a lower layer included in the first layer within the acceleration structure (AS). can do. In Figure 28, if the object 2801 belongs to node 8, which is a leaf node, and the first layer is set to layer 3, the object belonging to node 8 and the objects 2802 belonging to node 9 are the first object. It is judged as That is, the objects 2802 included in the bounding box of node 4 are recognized as objects adjacent to the object 2801 and are stored in the memory 2350. Of course, if the first layer is set to layer 4, only objects included in the same leaf node can be determined as the first object.

The ray tracing device 2300 may determine the object to be a second object when the node to which the object and the object belong in common is located in a second layer, which is a higher layer than the first layer. In FIG. 28, the objects 2803 belonging to node 3 belong in common only to the object 2801 and node 1, which is the top node. If the first layer is the 3rd layer and the second layer is set to the 1st layer, which is a layer higher than the 3rd layer, the objects 2803 belonging to node 3 are determined to be the 2nd object. That is, the objects 2803 included in the bounding box of node 3 are recognized as objects far away from the object 2801 and are stored in the memory 2350.

The ray tracing device 2300 operates when at least one layer exists between the first layer and the second layer and the node to which the object and the object commonly belong is located in one of the layers between the first layer and the second layer, The object can be determined to be a third object. In Figure 28, when the first layer is set to 3 layers and the second layer is set to 1 layer, there are 2 intermediate layers between them. Additionally, the objects 2804 belonging to node 5 are common with the object 2801 in node 2 of the second layer. Accordingly, the objects 2804 included in the bounding box of node 5 have little relevance to the color of the object 2801 and are recognized as objects not involved in the color value of the pixel and are stored in the memory 2350.

In this way, since the ray tracing device 2300 has an acceleration structure (AS) in which the virtual space is hierarchically divided, the distance between the objects can be adjusted to a certain extent by utilizing the tree form of the acceleration structure (AS) without complex calculations. It can be determined quickly.

Referring again to FIG. 25, in step 2530, the ray tracing device 2300 determines the path of the secondary ray and the sub-ray based on the distance determined in step 2520 and determines the path of the pixel corresponding to the first intersection according to the determined path. Color values can be derived. An embodiment of this will be described using FIG. 29.

Figure 29 is a flowchart of a method for deriving the color value of a pixel according to an embodiment of the present disclosure.

First, when the second intersection point is located in the first area and the object is determined to be the first object, the ray tracing device 2300 may sequentially generate sub-rays derived from the secondary ray according to the attribute information of the object. . In other words, subsequent intersection points that occur when each sub-ray intersects an object can be sequentially confirmed. At this time, the ray tracing device 2300 can determine the distance from the first intersection for each subsequent intersection (nth intersection, n is a natural number greater than 2) and determine the path of light according to the determined distance.

For example, if the third intersection point where the third ray intersects the object is located in the first area, the ray tracing device 2300 generates the fourth ray and confirms the fourth intersection point. If the reflectance and refractive index of the object including the third intersection are “0”, only a shadow ray may be generated and end up in a path incident to the light source without the fourth intersection. If the reflectance and refractive index of the object are not “0”, the ray tracing device 2300 determines in which area the fourth intersection point is located. That is, when all subsequent intersection points are located in the first area, the ray tracing device 2300 controls the color of each object corresponding to the second intersection to the nth intersection until all generated sub-rays enter the light source or leave the screen. The color value of the pixel corresponding to the first intersection is determined by combining the value, the original color of the object, and whether the shadow ray at each intersection is exposed to the light source.

If the third intersection point is located in the second area, the third intersection point is set as a separate light source different from the previously set light source. That is, the ray tracing device 2300 synthesizes the color value of the object corresponding to the second intersection, the original color of the object, and whether the shadow ray at each intersection is exposed to the two light sources, respectively, to determine the pixel corresponding to the first intersection. Determine the color value of .

If the third intersection point is located in the third area, the corresponding third ray is considered not to have occurred. That is, the color value of the pixel corresponding to the first intersection is determined by combining the color value of the object corresponding to the second intersection, the original color of the object, and whether the shadow ray of each intersection is exposed to the existing light source.

If the second intersection point is located in the second area and the object is determined to be a second object, the ray tracing device 2300 may set the second intersection point as a separate light source different from the light source already included in the virtual space.

According to one embodiment, when the object is determined to be a second object, the ray tracing device 2300 may determine the distance between the viewpoint and the second intersection. For example, if the distance between the viewpoint and the second intersection is greater than or equal to a preset reference value, the ray tracing device 2300 determines that the color of the object is a factor that does not affect the user's immersion or the reality of the virtual space and determines the second intersection. The intersection can be set to a separate light source from the light source already included in the virtual space. In this case, the ray tracing device 2300 may determine the color value of the pixel corresponding to the first intersection by combining the original color of the object and whether the shadow ray of the first intersection is exposed to the two light sources.

On the other hand, if the distance between the viewpoint and the second intersection is less than a preset standard value, the color of the object may be judged as a factor that should be considered for the user's immersion or the reality of the virtual space. However, in terms of quality and realism, considering light generated from objects far away from the object reduces rendering efficiency, so the ray tracing device 2300 does not cause reflection or refraction of light from the object. It can be set to . That is, if the object is determined to be a second object and the distance between the viewpoint and the second intersection is less than or equal to a preset reference value, the object may be set as an object with reflectance and refractive index of "0" rather than as a light source. In this case, the ray tracing device 2300 may determine the color value of the pixel corresponding to the first intersection by combining the color value of the object, the original color of the object, and whether the shadow ray at each intersection is exposed to the existing light source. there is.

If the second intersection point is located in the third area and the object is determined to be a third object, the ray tracing device 2300 may consider that the secondary ray projected to the third object has not occurred. In other words, among the secondary rays, reflected rays or refracted rays projected to an object are considered not to have occurred in the first place and can be excluded from the rendering process. In this case, the color value of the pixel corresponding to the first intersection may be determined by combining only the original color of the object and whether the shadow ray of the first intersection is exposed to the existing light source.

The embodiment of FIG. 29 that determines the distance using FIG. 30 will now be described.

Figure 30 is an example diagram of virtual space projected onto a plane to explain a method of deriving the color value of a pixel according to an embodiment of the present disclosure. The solid line of the arrow represents the first ray, the dotted line of the arrow represents the second ray, and the dashed line of the arrow represents the sub-lay (3rd ray). Additionally, the object 3030 including the subsequent intersection 3031 shown in FIG. 30 is assumed to be a material with reflectance and refractive index of “0” for explanation purposes.

The ray tracing device 2300 projects a primary ray from a viewpoint (not shown) to the virtual space 3000. The first ray intersects the object 3010, and the ray tracing device 2300 confirms the first intersection point 3011. The ray tracing device 2300 forms a first sphere (S1) and a second sphere (S2) centered on the first intersection 3011 and projects them into the virtual space 3000. The ray tracing device 2300 sets a first area, a second area, and a third area according to the first sphere S1 and the second sphere S2.

Afterwards, the ray tracing device 2300 generates a secondary ray generated at the first intersection 3000. Secondary rays may include reflected rays or refracted rays as shown. The ray tracing device 2300 determines the location of the second intersection formed when the secondary ray intersects the object.

When the second intersection 3021 is located in the first area, the ray tracing device 2300 determines that the object 3020 that the second ray reaches is the first object. The ray tracing device 2300 generates a sub-ray according to attribute information of the object 3020 at the second intersection 3021 and confirms a subsequent intersection 3031 where the sub-ray intersects another object 3030. Since the reflectance and refractive index of the object 3030 are "0", the ray generated at the subsequent intersection 3031 is directed to the light source L1, and the ray tracing device 2300 traces the path of all rays generated at each intersection starting from the first ray. It is synthesized and stored in memory 2350.

When the second intersection point (3041, 3051) is located in the second area, the ray tracing device 2300 determines the object (3040, 3050) to which the secondary ray has arrived as the second object. If it is determined that the distance between the viewpoint and the second intersection 3041 is less than or equal to a preset reference value, the ray tracing device 2300 sets the reflectance and refractive index of the corresponding object 3040 to “0”. Accordingly, the ray generated at the second intersection 3041 is directed to the light source L1, and the ray tracing device 2300 compiles the paths of all rays generated at each intersection starting from the first ray and stores them in the memory 2350. .

Conversely, if it is determined that the distance between the viewpoint and the second intersection 3051 is greater than or equal to a preset reference value, the ray tracing device 2300 sets the object 3050 as a separate light source L2. Accordingly, the shadow ray generated at the first intersection 3011 is directed in two directions: the existing light source L1 and the set light source L2.

When the second intersection point 3061 is located in the third area, the ray tracing device 2300 determines that the object 3060 reached by the secondary ray is a third object. Accordingly, the secondary ray directed to the object 3060 determined to be a third object is considered not to have occurred from the beginning. Accordingly, the ray tracing device 2300 compiles only the paths of the first ray and the second ray and stores them in the memory 2350.

According to one embodiment, in step 2530, when both the color value of the second intersection formed by intersecting the first object and the color value of the second intersection formed by intersecting the second object are collected by the secondary ray, the ray tracing device (2300) may include the average value of each collected color value or a color value showing the highest level among each collected color value in the calculation for deriving the color value of the pixel corresponding to the first intersection. Both a reflected ray and a refractive ray may be generated at the first intersection, and at least two or more second intersections may be generated accordingly. The ray tracing device 2300 preprocesses the color value of each object through the above process to create the first intersection. The time to derive the color value of the intersection can be reduced.

Referring to FIG. 30, for example, a reflection ray may be generated at the first intersection 3011 and intersect the first object 3021, and at the same time, a refractive ray may be generated and intersect the second object 3040. . In this case, the ray tracing device 2300 displays the color of the object 3030 that intersects the sub-ray, the original color of the object 3021 that the reflected ray of the secondary ray intersects, and the second intersection 3021 and subsequent intersection points ( The color value of the second intersection 3021 is determined by comprehensively determining whether the shadow ray of 3031) is exposed to the light source L1. In addition, the ray tracing device 2300 synthesizes the original color of the object 3041 that intersects the refracted ray among the secondary rays and whether the shadow ray at the second intersection 3041 is exposed to the light source L1 to create a second Determine the color value of the intersection point (3041). Afterwards, the average value of the color values of each second intersection can be obtained and set as the final color value of the second intersection, or the larger color value among the color values of each second intersection can be set as the final color value of the second intersection. there is.

As an additional embodiment of the present disclosure, before step 2510, a process of reducing the number of primary rays projected to the screen by a preset ratio may be further performed, and after step 2530, a preset deep learning-based scaler or scaler may be used. Image quality can be improved by performing up-sampling. For example, the amount of computation can be greatly reduced by reducing the number of rays by 1/4. At this time, the ray tracing device 2300 may select one pixel to project the first ray among four adjacent pixels. The ray tracing device 2300 can search attribute information of objects, extract original color values of objects corresponding to four pixels, and select feature values representing them. For example, the average of four color values can be calculated, and the color value closest to the average among the four color values can be selected as the feature value. Alternatively, among the four color values, the color value closest to the basic color included in the object information corresponding to the four pixels may be selected as the feature value. Alternatively, among the four color values, the color value that shows the greatest difference from the color of the object closest to the corresponding object can be selected as the feature value. The ray tracing device 2300 may project a primary ray to a pixel corresponding to a feature value. Thereafter, when the color value of each pixel is determined through ray tracing through steps 2510 to 2530, the ray tracing device 2300 converts the determined color value of each pixel into a tensor and inputs it to the previously learned artificial neural network model. can do. Through this, pixels compressed to 1/4 can be interpolated and an image of the original size can be output. Here, the artificial neural network model is preferably a GAN (Generative Adversarial Network) algorithm, including CGAN (Deep Convolution GAN) and SRGAN (Super Resolution GAN), or a deep learning super sampling (Dlss) technique using GAN. It is not limited to this. In this way, through this additional embodiment, it is possible to achieve the effect of reducing the amount of computation and bandwidth while maintaining resolution.

As an additional embodiment of the present disclosure, the ray tracing device 2300 may further perform a process of comparing the load of the TRV unit 2320 and the IST unit 2330 and the load on the acceleration structure generation unit 2360 in real time. You can. The ray tracing device 2300 may determine the state of the load by comparing the speed at which the TRV unit 2320 and the IST unit 2330 process frames with the speed at which the acceleration structure generation unit 2360 processes frames. If the processing speed of the TRV unit 2320 and the IST unit 2330 is lower, it may be determined that more load is applied to the TRV unit 2320 and the IST unit 2330, and at this time, the ray tracing device 2300 ) can be controlled to lower the quality of rendering. For example, in step 2520, the ray tracing device 2300 sets the first radius centered on the object smaller to reduce the number of objects determined to be the first object, makes the first radius smaller, or makes the second radius smaller. By making the radius larger, the number of third objects that are not used in rendering can be increased. Conversely, if the processing speed of the acceleration structure generator 2360 is lower, it may be determined that more load is placed on the acceleration structure generator 2360, and in this case, the ray tracing device 2300 may use the acceleration structure (AS). ) can be controlled to reduce the complexity. For example, the ray tracing device 2300 may rebuild the acceleration structure (AS) by increasing the maximum number of primitives or decreasing the depth of the tree. In this way, through this additional embodiment, delays that may occur when implementing a virtual space can be minimized in real time for each frame.

So far, we have looked at various embodiments. A person skilled in the art to which the present invention pertains will understand that the present invention may be implemented in a modified form without departing from the essential characteristics of the present invention. Therefore, the disclosed embodiments should be considered from an illustrative rather than a restrictive perspective. The scope of the present invention is indicated in the claims rather than the foregoing description, and all differences within the equivalent scope should be construed as being included in the present invention.

Meanwhile, the above-described embodiments of the present invention can be written as a program that can be executed on a computer, and can be implemented in a general-purpose digital computer that operates the program using a computer-readable recording medium. The computer-readable recording media includes storage media such as magnetic storage media (eg, ROM, floppy disk, hard disk, etc.) and optical read media (eg, CD-ROM, DVD, etc.).

10: Enterprise terminal
20: Job seeker terminal
100: server
130: processor
2210: Viewpoint
2220: screen
2230: object
2240: 1st ray
2251: Reflected Ray
2252: Refracted Ray
2253: Shadow Ray
2260: light source
2300: Ray tracing device
2700: Virtual space
3000: Virtual space

Claims (8)

In a ray tracing-based rendering method for implementing the metaverse,
Generating a primary ray projected from a viewpoint toward virtual space and confirming a first intersection (hit point) where the primary ray intersects a specific object;
According to the attribute information of the specific object, a secondary ray generated at the first intersection is generated, a second intersection point where the secondary ray intersects another object is identified, and the distance between the first intersection point and the second intersection point is determined. step of understanding; and
Determining a path of the secondary ray and a sub-ray derived from the secondary ray based on the distance and deriving a color value of a pixel corresponding to the first intersection according to the path,
An acceleration structure that divides the virtual space into hierarchical tree nodes is built in advance,
A ray tracing-based rendering method, wherein the object is an object included in the virtual space or a primitive constituting an object. According to clause 1,
The step of determining the distance is,
forming a first sphere having a preset first radius and a second sphere having a preset second radius around the first intersection;
Setting the inside of the first sphere as a first area, the outside of the second sphere as a second area, and the overlapping area between the outside of the first sphere and the inside of the second sphere as a third area; and
If the second intersection point is located in the first area, the other object is determined to be a first object, and if the second intersection point is located in the second area, the other object is determined to be a second object, and the other object is determined to be a second object. 2. When the intersection point is located in the third area, determining the other object as a third object;
The second radius is set larger than the first radius,
The first object is an object adjacent to the specific object, the second object is an object far away from the specific object, and the third object is an object that is not involved in the color value of the pixel, ray tracing-based rendering. method. According to clause 2,
The step of determining the distance is,
If a bounding box located only within the third area exists, modifying the acceleration structure by removing all nodes corresponding to the bounding box and lower nodes and objects included in the node;
Searching the modified acceleration structure to identify objects included in nodes that intersect the second ray and confirming the second intersection point; and
A ray tracing-based rendering method comprising determining which of the first to third areas the second intersection point is located in. According to clause 1,
The step of determining the distance is,
If a node to which the specific object and the other object commonly belong is located in a first layer and a lower layer included in the first layer within the acceleration structure, determining the other object to be a first object,
When a node that the specific object and the other object commonly belong to is located in a second layer, which is a higher layer than the first layer, the other object is determined to be a second object,
If at least one layer exists between the first layer and the second layer, and a node to which the specific object and the other object commonly belong is located in one of the at least one layer, the other object is 3Includes the step of determining that it is an object,
The first object is an object adjacent to the specific object, the second object is an object far away from the specific object, and the third object is an object that is not involved in the color value of the pixel, ray tracing-based rendering. method. According to claim 2 or 4,
The step of deriving the color value of the pixel is,
When the other object is determined to be the first object, a sub-ray derived from the secondary ray is sequentially generated according to the attribute information of the other object, and the subsequent intersection point where the sub-ray intersects another object is recursively created. Check,
When the other object is determined to be the second object, the second intersection is set as a separate light source different from the light source already included in the virtual space,
A ray tracing-based rendering method comprising the step of considering that the secondary ray projected to the third object did not occur when the other object is determined to be the third object. According to clause 5,
The process of recursively checking the subsequent intersection is,
If one of the sub-rays intersects an object determined to be the third object, the one sub-ray is considered not to have occurred,
A ray tracing-based rendering method where, when one of the sub-rays intersects an object determined to be the second object, a subsequent intersection by the one sub-ray is set to another light source. According to claim 2 or 4,
The step of deriving the color value of the pixel is,
When the other object is determined to be the second object, the second intersection is set as a separate light source different from the light source already included in the virtual space depending on the distance between the viewpoint and the second intersection, or the other object is set. A ray tracing-based rendering method, which includes setting that light reflection and refraction phenomena do not occur. According to claim 2 or 4,
The step of deriving the color value of the pixel is,
When both the color value of the second intersection formed by intersecting the first object and the color value of the second intersection formed by intersecting the second object are collected by the secondary ray, the average value or collection of each color value collected A ray tracing-based rendering method that includes the color value showing the highest level among each color value in the calculation for deriving the color value of the pixel.