WO2022186417A1

WO2022186417A1 - Method and system for providing immersive content by using ultra-high-definition image

Info

Publication number: WO2022186417A1
Application number: PCT/KR2021/003542
Authority: WO
Inventors: 박길홍; 김선욱; 이서문
Original assignee: (주)디피어소시에이츠
Priority date: 2021-03-03
Filing date: 2021-03-23
Publication date: 2022-09-09

Abstract

A method for realizing an ultra-high-definition masterpiece image, according to the present invention, may comprise the steps of: receiving a plurality of different fragment images corresponding to a target image; combining the plurality of fragment images on the basis of the similarity of each of the plurality of fragment images; applying a canvas map to an original image generated as a result of the combination, so that the original image has a pre-specified texture; and applying a normal map to the original image to which the canvas map has been applied, so that reflection characteristics of virtual light incident on a virtual space are reflected.

Description

Method and system for providing immersive content using ultra-high-resolution images

The present invention relates to a method and system for realizing an ultra-high quality image, and a method and system for providing an immersive virtual exhibition space.

As technology develops, users consume content using various electronic devices and methods.

In particular, technologies such as virtual reality (VR, Virtual Reality), augmented reality (augmented reality, AR), extended reality (XR, eXtended Reality), or mixed reality (MR, Mixed Reality) provide users with more lively content. has been provided

Virtual reality is a compound word of Virtual, meaning virtual, and Reality, meaning reality. In the field of virtual reality, various efforts are being made to enable spatial and temporal experiences close to reality by directly acting on the user's five senses beyond simply realizing a virtual space.

Furthermore, augmented reality is a technology that overlays a 3D virtual image on a real image or background and displays it as a single image. Augmented reality is a technology that places a virtual object in the real space or recognizes a real object and configures a virtual space around it. effort is being made

In addition, extended reality is a technology that implements various more advanced services such as spatial mapping and object tracking beyond simple content display. Users can experience a 360-degree virtual view without using a headset, and a natural-like experience can be experienced by providing spatial sound. Similarly, in the case of extended reality, efforts are being made to provide realistic and lively content.

It is very important to provide lively content to users for various content providing technologies including virtual reality, augmented reality and extended reality, and through this, users can obtain a sense of presence and vitality as if they actually visited, even if they do not actually go to a specific space. can

For example, consider an art gallery or museum where you can appreciate works of art. Masterpieces such as Leonardo da Vinci's Mona Lisa and Vincent Willem van Gogh's Starry Night are on display in the Art Department or Museum Department. To see these masterpieces, users around the world have to plan a long trip. However, the various technologies discussed above allow a user to appreciate an art work such as a painting without directly going to an art gallery or a museum.

However, an important factor in providing content through content providing technology such as virtual reality, augmented reality, or extended reality is a sense of reality and vitality as if the user is actually in the space. For example, when a masterpiece is provided through a low-resolution image, there is a difference between the image and the masterpiece provided, so the user does not feel the satisfaction of going and appreciating the masterpiece in person, and still thirsts to see the real masterpiece. feel

Accordingly, in accordance with the trend of various technological developments, there is a need for a method capable of providing a feeling of a user visiting a corresponding site and consuming content, that is, seeing and experiencing it with their own eyes.

In addition, in recent years, as smart devices become popular, the cases of using the built-in cameras, processors, and applications of smart devices are increasing in exhibition spaces such as museums and art galleries visited by users.

For example, Patent Publication No. 2000-0054344 relates to a three-dimensional virtual reality museum/shopping mall system and an operating method thereof through the Internet, and is an invention for providing a virtual art museum.

However, such a published patent only provides an art museum in virtual reality, and the user does not feel a sense of realism and vitality as if he were actually in the space.

Therefore, in line with the trend of various technological developments, there is a need for a method that can provide a feeling of a user visiting an actual exhibition space and consuming content such as famous paintings, that is, seeing and experiencing it with their own eyes. do.

SUMMARY OF THE INVENTION One object of the present invention is to provide a method and system for realizing an ultra-high-resolution image that can feel a sense of presence and realism.

In particular, the present invention is to provide a method and system for realizing an ultra-high-definition image capable of generating an ultra-high-quality image that realistically expresses famous paintings and various works of art.

Another object of the present invention is to provide a method and system for realizing an ultra-high-definition image that can provide an experience for a user to see an actual work or an exhibition in an art gallery or museum.

SUMMARY OF THE INVENTION One object of the present invention is to provide a method and system for providing an immersive virtual exhibition space that can feel a sense of presence and realism.

In particular, it is an object of the present invention to provide a method and system for providing an immersive virtual exhibition space in which a user can provide an experience such as actually viewing works or exhibits in an art gallery or museum.

Another object of the present invention is to provide a method and system for providing an immersive virtual exhibition space in which a user can directly design a virtual exhibition space.

In order to solve the above problems, the ultra-high-definition masterpiece image implementation method according to the present invention includes the steps of receiving a plurality of different sculptural images corresponding to a target image, and based on the similarity of each of the plurality of sculptural images, the plurality of Combining the pieces of images, applying a canvas map to the original image so that the original image generated as a result of the combination has a predetermined texture, and applying the canvas map to the original image to which the canvas map is applied , applying a normal map to reflect reflection characteristics of virtual light incident on the virtual space.

Furthermore, the plurality of fragment images may be images satisfying a preset resolution criterion, and any one of the plurality of fragment images may include image information that at least partially overlaps with the other one.

Furthermore, in the generating of the original image, edge blending may be performed on the combined fragment images.

Furthermore, the edge blending may be such that the mutually overlapping image information of each of the first fragment image and the second fragment image including mutually overlapping image information among the plurality of fragment images overlap.

Furthermore, in the step of applying the canvas map, the original image to which the canvas map is applied may be generated by synthesizing a layer corresponding to the predetermined texture with the original image.

Furthermore, the step of applying the normal map may be performed based on light source information defining at least one of a location, a shape, an intensity, and a direction of light for a virtual light source.

Furthermore, in the method for realizing an ultra-high-definition masterpiece image according to the present invention, in the step of applying the normal map, the reflection angle for each pixel constituting the original image is based on the light source information and a normal vector included in the normal map. The method may further include calculating information and calculating a new pixel value for each pixel constituting the original image based on the reflection angle information.

The method for realizing an ultra-high-definition masterpiece image according to the present invention includes changing the light source information, and based on the changed light source information and a normal vector included in the normal map, reflection angle information for each pixel constituting the original image. The method may further include re-calculating a new pixel value for each pixel constituting the original image based on the re-calculation step and the re-calculated reflection angle information.

The method for realizing an ultra-high-definition masterpiece image according to the present invention further comprises the step of providing the original image to which the normal map is applied, to an electronic device that reproduces the virtual space, in the providing step, based on the reflection angle information An original image including the calculated new pixel values may be provided.

On the other hand, the ultra-high-definition image implementation system according to the present invention, a communication unit for receiving a plurality of different pieces of images corresponding to a target image, and a control unit for combining the plurality of pieces of images based on the similarity of each of the plurality of pieces of images Including, wherein the control unit applies a Canvas Map to the original image so that the original image generated as a result of the combination has a predetermined texture, and to the original image to which the Canvas Map is applied, A normal map may be applied to reflect the reflection characteristics of virtual light incident on the virtual space.

On the other hand, the program stored in the computer-readable recording medium according to the present invention comprises the steps of receiving a plurality of different pieces of image corresponding to a target image, and based on the similarity of each of the plurality of pieces of images, Combining fragment images, applying a Canvas Map to the original image so that the original image generated as a result of the combination has a predetermined texture, and to the original image to which the Canvas Map is applied, The method may include applying a normal map to reflect the reflection characteristics of virtual light incident on the virtual space.

In addition, in order to solve the above problems, the method for providing an immersive virtual exhibition space according to the present invention includes the steps of specifying main content to be displayed in a virtual exhibition space, and using the visual characteristics of the main content, the virtual exhibition space generating the background content of , generating the immersive content so that the main content is superimposed on at least a part of the background content, and transferring the immersive content to the electronic device so that the immersive content is provided in the virtual exhibition space It may include the step of transmitting.

Furthermore, the generating of the background content may include extracting a visual characteristic of the main content, and setting a pattern, texture, or color of the background content by using the extracted visual characteristic.

Furthermore, the composition information of the background content is matched to the visual characteristic, and the background content may be generated using the composition information matched to the extracted visual characteristic.

Furthermore, an image corresponding to the illuminated light may be formed on the background content, and the light direction of the image corresponding to the light may be changed over time.

Furthermore, a reflection characteristic of the light incident on the virtual exhibition space may be set to match the light shining on the background content, and a pixel value of the main content may be changed to reflect the reflection characteristic.

Furthermore, the background content may be generated by reflecting weather information.

Furthermore, the weather information may be information corresponding to a region in which the electronic device is located.

Furthermore, the background content may include glass disposed on at least one of a wall and a ceiling of the virtual exhibition space.

Furthermore, the generating of the immersive content may include arranging the main content in one area of the background content to generate the immersive content.

Meanwhile, the system for providing an immersive virtual exhibition space according to the present invention specifies main content to be displayed in a virtual exhibition space, generates background content of the virtual exhibition space by using the visual characteristics of the main content, and the background and a controller for generating the immersive content so that the main content is superimposed on at least a portion of the content, and a communication unit for transmitting the immersive content to an electronic device so that the immersive content is provided in the virtual exhibition space.

Meanwhile, the program stored in the computer-readable recording medium according to the present invention includes the steps of specifying main content to be displayed in a virtual exhibition space, and using the visual characteristics of the main content to select the background content of the virtual exhibition space. generating the immersive content so that the main content is superimposed on at least a portion of the background content, and transmitting the immersive content to an electronic device so that the immersive content is provided in the virtual exhibition space. may include

As described above, the method and system for realizing an ultra-high-resolution image according to the present invention can provide content with a sense of reality and vitality by generating an ultra-high-resolution image. Through this, the user can easily access works displayed in art galleries or museums at any time. Through this, users can enjoy rich cultural and artistic contents. Furthermore, art galleries or museums may provide more users with content corresponding to works, thereby enhancing user interest.

Furthermore, the method and system for realizing an ultra-high-definition image according to the present invention can generate an ultra-high-definition image exceeding the maximum resolution by using a plurality of different fragment images including at least a part of the image. The ultra-high-resolution image is made of a high-resolution image in which the image quality is not impaired even when enlarged, so that details of an actual object (eg, a masterpiece) can be expressed intact.

When such a super-high-resolution image is provided to the user by a method according to various technologies such as virtual reality, augmented reality, mixed reality, or extended reality, the user selects an object corresponding to the image (eg, a masterpiece) on-site ( It can convey a vivid feeling as if you were actually seeing it in a museum or art gallery, for example.

Also, as described above, the method and system for providing an immersive virtual exhibition space according to the present invention can generate immersive content by generating background content of the virtual exhibition space using the visual characteristics of the main content. Accordingly, the user can feel the texture or color of the main content more realistically, and can be provided with the feeling of viewing an exhibition in an actual exhibition space.

Furthermore, the method and system for providing an immersive virtual exhibition space according to the present invention can provide a different environment of the virtual exhibition space according to time and weather. Therefore, if the user wants to view an art work such as a masterpiece on a rainy day, he sets the atmosphere of the virtual exhibition space to the same as on a rainy day, and if he wants to view an art work at night, sets the atmosphere of the virtual exhibition space to night. You can have fun with it and experience it instead of an experience you can't actually experience.

When such immersive content is provided to a user in a method according to various technologies such as virtual reality, augmented reality, mixed reality, or extended reality, the user can easily access works displayed in art galleries or museums at any time. Through this, users can enjoy rich cultural and artistic contents. Furthermore, art galleries or museums may provide more users with content corresponding to works, thereby enhancing user interest.

1A is a conceptual diagram for explaining an art museum in a virtual space provided by an ultra-high-definition image realization system according to the present invention.

1B is a block diagram of an ultra-high-definition image realization system according to the present invention.

2 is a flowchart of an ultra-high-definition image implementation method according to the present invention.

3 is a conceptual diagram for explaining an ultra-high-resolution image according to the present invention.

4A, 4B, and 4C are conceptual diagrams for explaining an original image according to the present invention.

5 is a conceptual diagram for explaining an original image to which a canvas map according to the present invention is applied.

6 and 7 are conceptual diagrams for explaining an original image to which a normal map according to the present invention is applied.

8 and 9 are conceptual diagrams for explaining an original image to which an AO map according to the present invention is applied.

10 is a conceptual diagram for explaining a final image according to the present invention.

11 is a conceptual diagram illustrating virtual reality, augmented reality, mixed reality, or extended reality provided by the method and system for providing an immersive virtual exhibition space according to the present invention.

12 is a block diagram of a system for providing an immersive virtual exhibition space according to the present invention.

13 is a flowchart illustrating a method for providing an immersive virtual exhibition space according to the present invention.

14, 15, 16, and 17 are conceptual diagrams for explaining immersive content according to the present invention.

18, 19, 20, 21, and 22 are conceptual diagrams for explaining a user interface.

Hereinafter, the embodiments disclosed in the present specification will be described in detail with reference to the accompanying drawings, but the same or similar components will be given the same reference numerals regardless of reference numerals, and redundant descriptions thereof will be omitted. The suffixes "module" and "part" for components used in the following description are given or mixed in consideration of only the ease of writing the specification, and do not have distinct meanings or roles by themselves. In addition, in describing the embodiments disclosed in the present specification, if it is determined that detailed descriptions of related known technologies may obscure the gist of the embodiments disclosed in this specification, the detailed description thereof will be omitted. In addition, the accompanying drawings are only for easy understanding of the embodiments disclosed in this specification, and the technical idea disclosed herein is not limited by the accompanying drawings, and all changes included in the spirit and scope of the present invention , should be understood to include equivalents or substitutes.

Terms including ordinal numbers such as first, second, etc. may be used to describe various elements, but the elements are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

When a component is referred to as being “connected” or “connected” to another component, it may be directly connected or connected to the other component, but it is understood that other components may exist in between. it should be On the other hand, when it is said that a certain element is "directly connected" or "directly connected" to another element, it should be understood that the other element does not exist in the middle.

The singular expression includes the plural expression unless the context clearly dictates otherwise.

In the present application, terms such as “comprises” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It is to be understood that this does not preclude the possibility of the presence or addition of numbers, steps, operations, components, parts, or combinations thereof.

The present invention relates to a method and system for realizing an ultra-high-definition image in order to provide a feeling that a user visits a site and consumes content, that is, a feeling of seeing and experiencing with his/her own eyes. A method of generating an ultra-high-resolution image will be described with the accompanying drawings.

Meanwhile, the ultra-high-resolution image described in the present invention may mean an image that satisfies a preset resolution or a preset pixel criterion. Here, the preset resolution or the preset pixel reference may mean that the resolution of the image is equal to or higher than (or exceeds) the resolution of a specific value, or that the pixels of the image are pixels (pixels per inch) of the specific value or more.

First, FIG. 1A is a conceptual diagram for explaining an art museum in a virtual space provided by the ultra-high-definition image realization system according to the present invention. 1b is a block diagram of an ultra-high-definition image implementation system according to the present invention, FIG. 2 is a flowchart of an ultra-high-definition image implementation method according to the present invention, and FIG. 3 is a conceptual diagram for explaining an ultra-high-definition image according to the present invention, 4A, 4B, and 4C are conceptual diagrams for explaining an original image according to the present invention, FIG. 5 is a conceptual diagram for explaining an original image to which a canvas map according to the present invention is applied, and FIGS. 6 and 7 are this view It is a conceptual diagram for explaining an original image to which a normal map according to the present invention is applied, FIGS. 8 and 9 are conceptual diagrams for explaining an original image to which an AO map according to the present invention is applied, and FIG. 10 is a final image according to the present invention. It is a conceptual diagram for

The ultra-high-definition image realization system 100 according to the present invention may operate in conjunction with an electronic device capable of outputting (or reproducing) an image. As shown in FIG. 1A , the ultra-high-definition image realization system 100 according to the present invention may operate in conjunction with the head mounted display 10a, the XR glasses 10b, and the electronic device 10c. The ultra-high-definition image realization system 100 according to the present invention outputs an ultra-high-resolution image to the electronic device 10c on the head mounted display 10a or XR glass 10b, so that the user can make art in an art gallery or museum 10d. It is possible to provide an experience similar to actually appreciating the work 10e.

Here, the image means a digital image, and may also be referred to as image data. For example, information obtained by capturing an art work such as a painting with a scanner or camera and turning it into data can be called an image.

On the other hand, a head mounted display (10a, hereinafter referred to as 'HMD') (10a) is a generic term for image display devices that can be worn like glasses and enjoy images, and recently also referred to as a face mounted display (FMD). call The HMD serves to show the virtual world to the user's eyes after blocking the user's sight and hearing from the outside. As an example, the HMD is equipped with a microphone, a stereo speaker, and various sensors in the form of a face worn with a display in front of the eyes.

In particular, such an HMD may display content such as VR (Virtual Reality). Furthermore, the HMD may display either augmented reality (AR) content or mixed reality (MR) content.

Here, the content refers to data, data, or information of signs, characters, figures, colors, voices, sounds, images, and images (or a complex of at least two of them). Hereinafter, an image will be described as an example.

On the other hand, XR Glass (eXtended Reality Glasses) (10b) allows you to view any content uploaded to the app in the electronic device (10c) through the lens, floating in a virtual space. Baseball broadcasts, YouTube, music videos, news, and art works can be viewed on the train or bus, on the sofa or on the bed by adjusting the size of the screen.

Furthermore, the electronic device 10c includes at least one of a smart phone, a mobile phone, a tablet PC, a kiosk, a computer, a notebook computer, a digital broadcasting terminal, a personal digital assistant (PDA), and a portable multimedia player (PMP). can contain one

The user can enjoy the image provided by the ultra-high-definition image realization system 100 according to the present invention by wearing it on the head mounted display 10a or the XR glasses 10b. Through this, the user may be provided with the feeling of consuming content by visiting the site of an art gallery or museum, that is, seeing and experiencing the content with their own eyes.

Hereinafter, in order to provide a more realistic image through virtual reality, augmented reality, mixed reality and extended reality (hereinafter referred to as extended reality, etc.) to the user, a method of implementing a super high-resolution image for a work of art Let me explain in detail.

As shown in FIG. 1B , the ultra-high-definition image realization system 100 according to the present invention may include at least one of a communication unit 110 , a storage unit 120 , and a control unit 130 . At this time, the ultra-high-definition image realization system 100 according to the present invention is not limited to the above-described components, and may further include components that perform the same or similar roles as the functions according to the description of the present specification.

The communication unit 110 may transmit/receive data to and from the head mounted display 10a, the XR glasses 10b, the electronic device 10c, and at least one external server described above. For example, the communication unit 110 may receive an image obtained by photographing (or scanning) a work of art from an external server. Also, the communication unit 110 may receive the fragment image 200a from an external server.

Here, the fragment image 200a may be understood as an image including at least a portion of the target image. For example, the target image is divided into left and right areas based on the center, the left area is taken or scanned with the camera as a first piece image, and the right area is captured or scanned with the camera as a second piece image. can be understood as

Each of the plurality of fragment images 200a is generated by scanning a portion of the target image with a preset resolution. The N pieces of image 200a are generated by scanning the target image N times with a preset resolution. Accordingly, each of the plurality of fragment images 200a may have a preset resolution.

In this case, each of the plurality of fragment images 200a may be an ultra-high-resolution image. That is, each of the plurality of fragment images 200a may mean an image satisfying a preset resolution or a preset pixel criterion.

Furthermore, any one of the plurality of fragment images 200a may include image information that at least partially overlaps with the other one. Specifically, when the target image is scanned multiple times, a portion of the target image is scanned multiple times. For example, each of the upper left scan image of the target image and the upper center scan image of the target image includes an image corresponding to a specific region of the target image.

Here, the overlapping image information may mean an image corresponding to a specific region of the target image, which is included in each of one and the other of the plurality of fragment images. For example, referring to FIG. 4C , the fifth fragment image 415 and the sixth fragment image 416 equally include one

region

415a and 416a of the target image. As such, a region of the target image that is identically included in the plurality of fragment images may be understood as an overlapping image path.

The target image described in the present invention may be understood as a work of art such as a painting. The target image may mean the picture itself, or it may mean information obtained by converting the picture into data.

Furthermore, the storage unit 120 may also be referred to as a database (DB), and may be configured to store images used in the present invention. For example, the storage unit 120 may store images according to the present invention using a cloud server.

Furthermore, the controller 130 may be configured to generally control the components described above, the head mounted display 10a, the XR glasses 10b, and the electronic device 10c shown in FIG. 1A .

Meanwhile, the controller 130 may generate an original image by combining the fragment image 200a received through the communication unit 110 .

Here, the original image may be understood as an image generated by combining a plurality of fragment images 200a based on the target image. That is, the original image may correspond to the target image.

In this case, the original image may be an ultra-high-resolution image. That is, the original image may mean an image that satisfies a preset resolution or preset pixel criteria.

Further, the control unit 130 to the original image 210, a canvas map (Canvas Map) 220, a normal map (Normal Map) 230 , a mask map (Mask Map) 240 and a coating map (Coat Map) By sequentially applying 250 , a final image 200b may be generated (see FIG. 2 ).

Here, the image is made of a plurality of pixels, and includes coordinate information of each of the plurality of pixels and color information matching the coordinate information. In an embodiment, the color information corresponding to each pixel may include at least three channels (eg, color value channels for each of R, G, and B).

Here, the “map” includes data corresponding to each pixel constituting the original image 210 and is used for the purpose of imparting a specific effect to the image. Like the original image 210 , the map consists of a plurality of pixels, and each pixel corresponds to each pixel constituting the image. A pixel of the map corresponding to a specific pixel constituting an image includes coordinate information corresponding to the specific pixel, and effect information corresponding to an effect to be applied to the specific pixel is matched to the pixel of the map.

In an embodiment, each of the effect information corresponding to each pixel constituting the map may include at least three channels, and each channel has a value corresponding to an effect to be applied to the original image pixel corresponding to the corresponding pixel. This can be saved.

The map may be understood as an image having the same size as the original image 210 , but it is difficult to be used as a meaningful image by itself. In this specification, an image corresponding to the map is expressed as a layer superimposed on the original image.

In this specification, “applying a map to an image” may be understood as synthesizing a layer corresponding to the map on the image. More specifically, synthesizing the map layer with the image may be changing the image information based on the map information. For example, applying a map to an image means that i) a pixel value corresponding to a specific pixel of the image and ii) effect information corresponding to the pixel included in the map are substituted into a preset function to obtain a new pixel value. This may mean acquiring new pixel values for all pixels included in the image by repeating the process.

Furthermore, the canvas map 220 described in the present invention may be understood as information representing the material of the object on which the target image is drawn. That is, the canvas map 200 may be understood as a layer corresponding to a predetermined texture.

More specifically, if i) the object on which the target image is drawn is a canvas (fabric used when drawing a picture such as an oil painting), the canvas map may be a layer corresponding to the canvas material, and ii) the object on which the target image is drawn is wood ( If it is a wooden board used when drawing woodblock prints, etc.), the canvas map may be a layer corresponding to the material of the wood, and iii) if the object on which the target image is drawn is a copper plate (a plank-shaped board made of copper), the canvas map Silver may be a layer corresponding to the copper material.

The normal map 230 described in the present invention defines the reflection characteristics of each pixel constituting the original image. Specifically, the normal map includes a pixel corresponding to each pixel constituting the original image. Each pixel included in the normal map defines a normal vector of a pixel of the corresponding original image.

Here, the normal vector is a vector defining a direction perpendicular to the tangent line passing a certain point (pixel) on the curve of the plane. In other words, the normal vector is a vector defining the direction of a normal line, which is a reference in defining the angle of incidence and the angle of reflection when light is incident on a specific point (pixel) of an object.

More specifically, each pixel included in the normal map 230 stores normal vector information (X, Y, and Z angles of 180 degrees on both sides) in a storage space (R, G, B color channels of 0-255 values) in the form of a texture image. ) may contain information stored in That is, the normal map includes information that enables it to be calculated at which angle the light will be reflected when light is incident on a specific pixel of the original image at a specific angle.

The normal vector does not definitively define the direction of light reflected from a specific pixel of the original image, but allows it to be calculated in which direction the light will be reflected when light is incident on a specific pixel at a specific angle.

In an embodiment, the normal vector may be configured to define a specific direction in 3D.

Furthermore, the mask map 240 described in the present invention may include a metallic map 241 , an ambient occlusion map 242 and a smoothness map 243 . can More specifically, the mask map 240 includes effect information corresponding to each of a metallic map 241 , an ambient occlusion map 242 , and a smoothness map 243 . can The mask map includes a plurality of pixels, and each pixel corresponds to each of the pixels constituting the original image. Each pixel constituting the mask map includes coordinate information, and the coordinate information corresponds to coordinate information of a pixel of an original image to which the corresponding pixel corresponds. Effect information including three channels may be matched to each pixel constituting the mask map. Effect information corresponding to each of a metallic map 241 , an ambient occlusion map 242 , and a smoothness map 243 may be stored in each of the three channels.

For example, by applying effect information corresponding to the metallic map 241 to the Red channel, effect information corresponding to the OA map 242 to the Green channel, and effect information corresponding to the smooth map 243 to the Alpha channel, one You can create a map.

Through this, one mask map includes effect information corresponding to at least three types of maps through one map. Since effects corresponding to three types of maps can be applied to data allocated to generate one map, the mask map 240 has an advantage in that unnecessary data can be reduced.

Here, the metallic map 241 includes information capable of expressing a unique color (eg, gloss, mirror-like reflection) of a metal material included in a target image using specular reflection of light.

An AO map (Ambient Occlusion Map) 242 described in the present invention may include information about an occlusion value between an object (or pixel) and another object (or pixel) in a target image. That is, the AO map 242 may include information on whether the region of the model should receive strong or weak indirect lighting.

Furthermore, the smoothness map 243 described in the present invention may include information capable of expressing a feeling of spreading light.

The modified image described herein is, in the original image, i) Canvas Map 220, ii) Normal Map 230, ii) Metallic Map 241, iv ) AO map (Ambient Occlusion Map) 242 and vi) may mean a result to which both the smoothness map (Smoothness Map) 243 are applied.

Furthermore, the coating map (Coat Map) 250 described in the present invention is the original image 210 generated by merging the fragment images, that is, the original image 210 itself to which no map is applied, or the original image. It may mean information about (210).

Furthermore, the final image 200b described in the present invention is i) a canvas map 220, ii) a normal map 230, iii) a metallic map 241, iv ) AO Map (Ambient Occlusion Map) (242), v) Smoothness Map (243) and vi) Coating Map (250) are applied to an image that expresses the texture of the target image and at the same time expresses the original You can keep the data values of the image.

Hereinafter, a method for realizing an ultra-high-definition image according to the present invention along with a flowchart will be described in more detail.

In the present invention, a process of receiving a plurality of different pieces of images corresponding to the target image may be performed (S310). That is, the communication unit 110 may receive a plurality of different pieces of images corresponding to the target image.

The target image may be understood as a work of art such as a painting. The target image may mean the picture itself, or it may mean information obtained by converting the picture into data.

The fragment image 410 may be understood as an image including at least a portion of the target image. For example, the target image is divided into left and right areas based on the center, the left area is taken or scanned with the camera as a first piece image, and the right area is captured or scanned with the camera as a second piece image. can be understood as

Furthermore, the plurality of fragment images 410 may be ultra-high-resolution images. That is, each of the plurality of fragment images 410 may be images satisfying a preset resolution criterion.

Furthermore, any one of the plurality of fragment images 410 may include image information that at least partially overlaps with the other one.

region

Meanwhile, in the present invention, based on the similarity of each of the plurality of fragment images, a process of combining the plurality of fragment images may be performed ( S320 ). The controller 130 may combine a plurality of fragment images to generate an original image corresponding to the target image.

More specifically, the controller 130 may merge the fragment images by connecting the fragment images based on the target image. The controller 130 may match pieces of images adjacent to each other based on the target image, and connect the matched pieces of images.

In this case, the controller 130 may combine the fragment images based on image information overlapping each other among the plurality of fragment images. The controller 130 may combine the fragment images including the overlapping images by arranging them adjacent to each other. For example, the second fragment image 412 including image information overlapping the upper portion of the fifth fragment image 415 illustrated in FIG. 4B may be disposed above the fifth fragment image 415 . . Also, the sixth fragment image 412 including image information overlapping the right portion of the fifth fragment image 415 may be disposed on the right side of the fifth fragment image 415 .

Furthermore, the controller 130 may naturally connect the boundaries of the fragment images by performing edge blending on the fragment images combined with the combined fragment images to generate an original image.

Here, edge blending refers to performing interpolation on the boundary of the fragment images, overlapping parts of two images with each other so that the two images can be viewed as one smooth image. it means. Edge blending is performed by moving the position of any one of the two images at a preset interval until overlapping image information included in each of the two images overlaps each other.

In an embodiment, the controller 130 may correct the transparency of the adjacent engraving images differently. The controller 130 may overlap one piece image having a different transparency on one piece image. The controller 130 may change the position of the fragment image having different transparency until the region corresponding to the overlapped image information included in each of the two overlapping fragment images overlaps by more than a preset reference. When the same area included in the two overlapping pieces of images overlaps by more than a preset reference, the controller 130 may adjust the transparency of the two overlapping pieces of images to 0. Through this, the two sculptural images can be connected to each other. The controller 130 may generate an original image by performing edge blending on all of the plurality of different fragment images.

Edge blending will be described in more detail together with FIGS. 4B and 4C. The controller 130 may connect the fifth engraved image 415 and the sixth engraved image 416 adjacent to the fifth engraved image 415 by matching each other as shown in FIG. 4C (a) . In this case, the fifth fragment image 415 and the sixth fragment image 416 may include the

same portions

415a and 416a of the target image. The controller 130 may correct the transparency of the sixth piece image 416 to 90% in order to accurately overlap the

same parts

415a and 416a. The control unit 130 overlaps the sixth engraving image 416 with different transparency on the fifth engraving image 415, and until the

same parts

415a and 416a match more than a preset standard, the sixth engraving image ( 416) can be changed. When the

same parts

415a and 416a match more than a preset criterion, the controller 130 sets the transparency of the fifth engraving image 415 and the sixth engraving image 416 to the original of the sixth engraving image 416 . By adjusting the transparency value to be the same, the fifth fragment image 415 and the sixth fragment image 416 may be connected. Furthermore, the controller 130 may generate an original image corresponding to the target image by performing edge blending on all of the nine fragment images 410 .

The ultra-high-definition image realization system 100 according to the present invention generates an original image by performing such edge blending, thereby minimizing waste data of a misaligned portion of a connected fragment image or a boundary portion of the fragment images.

Furthermore, compared to the case of generating a single image by photographing a work of art once, a sculpture image is generated by shooting each area of the work of art, and edge blending is performed on the sculpture image to generate an original image, resulting in a higher quality You can create an image.

Through this, the ultra-high-definition realization system 100 according to the present invention can provide a more realistic artwork to the user through extended reality or the like.

Meanwhile, in the present invention, a canvas map may be applied to the original image so that the original image generated as a result of the combination has a predetermined texture ( S330 ).

Here, the canvas map may be understood as information representing the material of an object on which a target image such as a picture is drawn. More specifically, if i) the object on which the target image is drawn is a canvas (fabric used when drawing a picture such as an oil painting), the canvas map may be a layer corresponding to the canvas material, and ii) the object on which the target image is drawn is wood ( If it is a wooden board used when drawing woodblock prints, etc.), the canvas map may be a layer corresponding to the material of the wood, and iii) if the object on which the target image is drawn is a copper plate (a plank-shaped board made of copper), the canvas map Silver may be a layer corresponding to the copper material.

The controller 130 may generate a canvas map based on the object on which the target image is drawn.

More specifically, the controller 130 may use an image of the object itself on which the target image is drawn as a canvas map. For example, if the target image is an oil painting, the controller 130 may use an image obtained by photographing a canvas as a canvas map.

Furthermore, the controller 130 may obtain a material of the object on which the target image is drawn by using an image obtained by photographing the object on which the target image is drawn, and may generate a canvas map by converting the obtained material into data. For example, if the target image is an oil painting, the control unit 130 obtains the material of the canvas (eg, a material woven by replacing the fabric) from the photographed image of the canvas, and generates information representing the material of the canvas by generating information representing the material of the canvas. You can create maps.

Furthermore, the controller 130 may predict the material of the object on which the target image is drawn from the original image. In this case, the controller 130 may use the artificial intelligence model generated by machine learning. The controller 130 may configure the information of the canvas map so that the predicted material is expressed.

The controller 130 may generate an original image to which the canvas map is applied by synthesizing a layer corresponding to a predetermined texture in the original image.

That is, the controller 130 may generate a canvas map image by applying the generated canvas map to the original image. That is, the controller 130 may process the mapping (or blending, overlay) between the original image and the canvas map.

More specifically, the control unit 130 substitutes i) a pixel value corresponding to a specific pixel of the original image and ii) effect information corresponding to the specific pixel among information included in the canvas map to the first blending function, You can change the pixel value for a specific pixel in . The controller 130 applies the above-described method to all pixel values constituting the original image.

If the pixel value of the original image before the change is a pixel value that does not express the texture of the object on which the target image is drawn, the pixel value of the changed original image may be a pixel value that expresses the texture of the object on which the target image is drawn. For example, suppose the value of a particular pixel contained in the original image before the change was (3, 99, 5). If the object on which the target image is drawn is a canvas, the canvas map includes effect information corresponding to the specific pixel value, and the pixel value of the changed original image to which it is applied may be (34, 104, 29). Also, if the object on which the target image is drawn is copper, the canvas map has information on the copper material, and the pixel value of the changed original image to which this is applied may be (8, 52, 124).

Furthermore, by adjusting the parameter of the first blending function, the controller 130 may adjust the ratio in which the original image to which the canvas map is applied expresses the material of the object on which the target image is drawn. For example, suppose that the pixel value of a specific pixel included in the original image before the change is (3, 99, 5), and the object on which the target image is drawn is a canvas. When the material of the object is expressed by 3 in the target image, the controller 130 adjusts the parameter value of the first blending function to set the pixel value of a specific pixel included in the original image to which the canvas map is applied (34, 104, 29). ) can be changed to In addition, when the material of the object is expressed by 97 in the target image, the controller 130 adjusts the parameter value of the first blending function to set the pixel value of a specific pixel included in the original image to which the canvas map is applied (101, 164). , 29) can be changed.

However, this is only an example for convenience of description, and information included in the original image to which the canvas map is applied is not limited thereto.

FIG. 5 is a diagram illustrating the original image 530 to which the canvas map is applied by applying the original image 510 and the canvas map 520 . More specifically, FIG. 5( a ) is a conceptual representation of the original image 510 which fails to express the material of the object on which the target image is drawn or does not reflect the material of the object on which the target image is drawn. 5( b ) is a conceptual representation of a canvas map including material information of an object on which a target image is drawn. 5( c ) is a conceptual representation of the original image 530 to which the canvas map is applied, which expresses the material of the object on which the target image is drawn by applying the canvas map 510 to the original image 510 . That is, the user does not recognize the material of the object on which the target image is drawn through the original image 510 of FIG. 5(a), but the target image is drawn through the original image 530 to which the canvas map of FIG. The material of the object can be recognized.

Therefore, when the original image 530 to which the canvas map is applied is displayed, compared to when the original image 510 is displayed in extended reality, etc., the user feels the texture of the art work, thereby experiencing the same experience as viewing the art work in an actual art museum. can be provided.

Hereinafter, a method for realizing a more sophisticated, ultra-high-definition image similar to a real work of art will be described.

On the other hand, in the present invention, the process of applying a normal map (Nnormal Map) to the original image to which the Canvas Map is applied so that the reflection characteristics of the virtual light incident on the virtual space are reflected (S340). That is, the controller 130 may apply the normal map to the original image to which the canvas map is applied.

The normal map includes information that enables it to be calculated at which angle the light will be reflected when light is incident on a specific pixel of the original image at a specific angle.

Each pixel constituting the normal map corresponds to a pixel constituting the original image. When the normal map is applied to the original image, vector information matched to a pixel corresponding to a specific pixel of the original image among pixels constituting the normal map is matched to the original image.

However, when there is no light source information, even if the normal map is applied to the original image, it has no effect on the original image.

Here, the light source information is information defining a virtual light source, and is used to calculate the incident angle of the virtual light incident on each of the pixels constituting the original image. Specifically, the light source information defines at least one of a position, a shape, an intensity, and a light traveling direction with respect to the virtual light source.

For example, when the virtual light source is a point light source, the light source information may include location information of the point light source and information defining that light is emitted in all directions from the point light source.

By using the light source information, the incident angle of the virtual light incident on each of the pixels constituting the original image may be calculated. When the incident angle is applied to an original image to which the normal map is applied (or a normal vector matched to each pixel constituting the original image), information on the reflection angle of light from a specific pixel may be calculated. By applying the calculated reflection angle to a second blending function, a new pixel value of the specific pixel may be calculated. If the above-described process is applied to all pixels included in the original image, the original image corresponding to the virtual light source may be generated.

Meanwhile, the light source information may be changed according to a user's request. Accordingly, the reflection angle information for each pixel constituting the original image may be recalculated based on the changed light source information, and the pixel values of the pixels constituting the original image may be recalculated based on the recalculated reflection angle information. .

As described above, even in the original image to which the same normal map is applied, when the light source information is different, different images may be generated. Through this, according to the present invention, various light source effects can be applied to the original image.

Meanwhile, the normal map may be generated based on the original image. The normal map is generated by forming a normal vector for each pixel constituting the original image. A normal vector for a specific pixel constituting the original image may be generated based on a brightness value of the specific pixel and a brightness value of a pixel adjacent to the specific pixel.

In an embodiment, the specific pixel is plotted in a 3D space by using the X and Y-axis coordinates of the specific pixel and the brightness value of the specific pixel as the Z-axis coordinates. Two pixels adjacent to the specific pixel are also plotted in a 3D space. In this case, three points may be plotted on a three-dimensional space, and a surface including all three points may be specified. The specified plane may be specified as a normal plane of the specific pixel. A vector in a direction perpendicular to the normal plane is specified as a normal vector of the specific pixel.

When the above-described process is applied to all pixels included in the original image, a normal map is generated. The normal map may provide target images in various environments.

Specifically, the target image, such as a work of art displayed in a museum, etc., depends on i) the amount of light according to the displayed time, ii) from which direction the light shines in which direction, iii) the place where the target image is displayed, etc. , the amount and angle of the reflected light are different. When the normal map and the light source information are used together, it becomes possible to provide target images in various environments.

The normal map allows the three-dimensional effect of the target image to be implemented differently according to the position of the light source and the intensity of light emitted from the light source. That is, the user may feel a different three-dimensional effect with respect to the same target image.

6 is a conceptual diagram showing the extreme amount of light reflected on

target images

610 and 620 according to the direction of light.

6 (a) is a view of light entering from the left (a) direction of the target image 610 . In FIG. 6A , the first area 610a that is the left area of the target image 610 reflects light, and the second area 610b that is the right area of the target image 610 does not reflect light. . Accordingly, the user can feel the three-dimensional effect of the target image in the first region 610a, but cannot feel the three-dimensional effect of the target image in the second region 610b.

6 (b) is a view of light entering from the right (b) direction of the target image 620 . In FIG. 6B , the first area 620a that is the left area of the target image 620 does not reflect light, and the second area 620b that is the right area of the target image 620 reflects light. . Accordingly, the user cannot feel the three-dimensional effect of the target image in the first region 620a, but can feel the three-dimensional effect of the target image in the second region 620b.

As such, the user may feel the three-dimensional effect of the target image differently according to the amount of light illuminating the target image and the angle of light reflected from the target image.

For example, it is assumed that light enters only a specific region 710a of the target image 710 together with FIG. 7A . The first area 710a of the target image 710 reflects light, and the second area 710b does not reflect light. The effect of the normal map is applied to the first region 710a, and the effect for the normal map is not applied to the second region 710b. Through this, the first region 730a of the original image 730 to which the normal map is applied may express a three-dimensional effect, whereas the second region 730b may not express the three-dimensional effect.

It is assumed that light enters a specific area 740b of the target image 740 together with FIG. 7B . The first area 740a of the target image 740 does not reflect light, and the second area 740b reflects light. The effect of the normal map is applied to the second region 740b, and the effect for the normal map is not applied to the first region 740a. Through this, the first region 760a of the original image 740 to which the normal map is applied does not express a three-dimensional effect, whereas the second region 760b may express a three-dimensional effect.

However, for convenience of explanation, it has been described that one area of the target image reflects light and the other area does not reflect light. can

Light source information according to these various extended reality situations (weather, time, place, etc.) may be generated. The light source information may be selectively applied to the original image according to an exhibition environment to be implemented.

Hereinafter, a method for realizing an ultra-high-definition image that is more sophisticated and more similar to a real work of art will be described.

Meanwhile, in the present invention, by applying a mask map to a normal map image, a process of generating an ultra-high-resolution image more similar to an image of a work of art may be performed.

Here, the mask map may include a metallic map, an ambient occlusion map (AO), and a smoothness map. More specifically, the mask map refers to a map in which only core data can be used by applying the Metalic Map to the Red channel, OA to the Green channel, and the Smoothness Map to the Alpha channel. The mask map has an advantage in that unnecessary data can be reduced.

Meanwhile, in the present invention, a process of applying the metallic map, the AO map, and the smooth map to the original image to which the normal map is applied may be performed. The metallic map, the AO map, and the smooth map may be applied in any order. However, in the present invention, for convenience of explanation, a process of applying a metallic map, an AO map, and a smooth map to an original image to which a normal map is applied will be described.

In the present invention, by applying the metallic map to the original image to which the normal map is applied, it is possible to express the texture of the metallic material included in the target image.

Here, the metallic map includes information that can express a unique color (eg, gloss, mirror-like reflection) of a metal material included in the target image by using the specular reflection of light.

That is, in the original image to which the metallic map is applied, the metal-included portion of the target image may be expressed as a unique color of the corresponding metal.

More specifically, the controller 130 may apply the metallic map to the original image to which the normal map is applied, and may express a unique color of the metal by changing the pixel values of the original image to which the normal map is applied. The controller 130 may change the pixel values of the original image to which the normal map is applied by substituting i) the pixel values of the original image to which the normal map is applied and ii) the metallic map to the third blending function. If the pixel value of the original image with the normal map applied before the change is a pixel value that does not reflect the unique color of the metal material in the target image, the pixel value of the original image with the changed metallic map applied is the original color of the metal material in the target image. This may be a reflected pixel value.

For example, suppose that the target image is an oil painting, and pixel values (245, 167, 59) of the original image to which the normal map is applied. The controller 130 may substitute the pixel values 245 , 167 , and 59 of the original image and the metallic map to the third blending function. Through this, the original image to which the metallic map is applied may have pixel values of (249, 167, 24). The pixel value of the original image to which the normal map was applied before the change does not reflect the specular reflection of the target image, so if the unique color of the metal material is not expressed, the pixel value of the original image to which the changed metallic map is applied is the specular reflection of the target image. may be a pixel value in which a unique color of a metal material is expressed by reflecting .

On the other hand, the specular reflection of the target image also differs according to the above-described i) the amount of light according to the time the target image is displayed, ii) from which direction the light shines in which direction, iii) the place where the target image is displayed, etc. material can be expressed. For example, if the amount of light is large, the metal material may appear whiter (shiny).

The controller 130 may adjust the amount of light shining on the target image and the specular reflection in the target image expressed by the original image to which the metallic map is applied by adjusting the parameter of the third blending function.

These various extended reality situations (weather, time, place, etc.) and the corresponding metallic map (or parameter of the third preset blending function) may be matched with each other and stored in the storage unit 120 as matching information.

The controller 130 may control a parameter of the metallic map or a third preset blending function through matching information based on the situation of the extended reality to be implemented.

Meanwhile, the controller 130 may generate matching information using an artificial intelligence model generated by machine learning, and may generate an original image to which a metallic map is applied based on the matching information.

As such, in the present invention, by applying the metallic map to the original image (ie, the original image to which the normal map is applied), a unique color of the metal material of the target image can be expressed. The user can recognize the unique color of the metallic material included in the target image through the original image to which the metallic map is applied.

Therefore, when the original image to which the metallic map is applied is displayed, compared to when the original image to which the metallic map is not applied is displayed in extended reality, etc., the user can feel , you can get the same experience as seeing art works in real art galleries.

On the other hand, in the present invention, by applying an AO map (Ambient Occlusion Map) to the original image to which the metallic map is applied, it is possible to create a shadow in a corner of the target image, thereby expressing a more natural and realistic feeling.

However, as described above, the order of the Metallic Map, the Ambient Occlusion Map, and the Smoothness Map does not matter. Therefore, before applying the metallic map, the AO map may be applied. have. In this case, the AO map may be applied to the original image to which the normal map is applied.

Here, the AO map (Ambient Occlusion Map) may include information on an occlusion value between each point on the surface of an object (or pixel) and another object (or pixel) in the target image.

That is, the original image to which the AO map is applied may be understood as an image in which light and dark portions are displayed as a shaded image.

More specifically, the controller 130 may apply the AO map to the original image to which the metallic map is applied, and may express the shading by changing the pixel values of the original image to which the metallic map is applied. The controller 130 may change the pixel values of the original image to which the metallic map is applied by substituting i) the pixel values of the original image to which the metallic map is applied and ii) the AO map to the fourth preset blending function. If the pixel value of the original image to which the metallic map was applied before the change is a pixel value that does not reflect the occlusion value (that is, it is not shaded), the pixel value of the original image to which the changed AO map is applied is the pixel value that reflects the occlusion value (that is, shaded) pixel values.

For example, the original image to which the metallic map is applied may have pixel values of ((255, 255, 255), (255, 30, 20), (255, 30, 20)) as shown in (a) of FIG. have. In this case, i) a pixel value of 30 (811) located in row 2, column 2, ii) a pixel value of 20 (812) located in row 2, column 3, iii) a pixel value of 30 (813) located in row 3, column 3, and iv) 3 row 3 For the pixel value 20 (814) located in the column, a shadow (shading) may be generated due to environmental shielding by the pixel value 255 other than the pixel value 255 . In this case, iv) the pixel value 20 (814) located in the third row and third column generates the largest amount of shading by the surrounding pixel values, and thus the pixel value 20 (814) may be changed to the pixel value 3 (824). Further, ii) the pixel value 20 (812) located in row 2, column 3, and iii) the pixel value 30 (813) located in row 3, column 3 produces relatively less shading, and therefore a pixel value of 5 (822) and a pixel value of 7, respectively (823) may be changed. Further, i) a pixel value of 30 (811) located in row 2 and column 2 generates relatively the least shading, and thus may be changed to a pixel value of 10 (821). That is, in the original image to which the AO map of FIG. 8(b) is applied, the dark part may be expressed to be darker.

9(a) is a view showing the original image 910 to which the metallic map is applied. By applying the AO map 920 of FIG. 9B to the original image 910 to which the metallic map is applied, the original image 930 to which the AO map is applied as shown in FIG. 9C may be generated. In the original image 930 to which the AO map is applied, a shadow 911 may be generated by blocking pixels.

As such, in the present invention, by applying the AO map to the original image (that is, the original image to which the metallic map is applied), the shadow effect is applied and the ambient light is shielded, thereby expressing the texture of the detailed brush touch of the picture.

Therefore, when the original image to which the AO map is applied is displayed, compared to when the original image to which the AO map is not applied is displayed in extended reality, etc., the user can feel the texture of the detailed brush touch included in the art work, so as to feel the texture of the detailed brush touch included in the art work, etc. You can be provided with the same experience as looking at a work of art.

Meanwhile, in the present invention, by applying a smoothness map to the original image to which the AO map is applied, the smooth surface of the target image can be made softer and the rough surface can be expressed more rough.

Here, the smoothness map includes information capable of expressing a feeling of spreading light.

More specifically, the controller 130 applies the smooth map to the original image to which the AO map is applied, and by changing the pixel values of the original image to which the AO map is applied, a smooth surface can be expressed more softly and a rough surface can be expressed more rough. . The controller 130 may change the pixel values of the original image to which the AO map is applied by substituting i) the pixel values of the original image to which the AO map is applied and ii) the smooth map to the fifth preset blending function. Compared to the pixel values of the original image to which AO map is applied, the pixel values of the original image to which the AO map is applied have a smoother surface, and the rough surface of the target image has a rougher value. can have

On the other hand, the roughness or softness of the surface of the target image also depends on i) the amount of light according to the time the target image is displayed, ii) whether the light is shining from which direction, iii) the place where the target image is displayed, etc. Accordingly, different metal materials can be expressed. For example, if the amount of light is high, a smooth surface may appear smoother.

The controller 130 may adjust the roughness of the surface according to the amount and angle of light illuminating the target image, expressed by the original image to which the smooth map is applied, by adjusting the parameters of the fifth preset blending function.

These various extended reality situations (weather, time, place, etc.) and the corresponding smooth map (or parameters of the fifth preset blending function) may be matched with each other and stored in the storage unit 120 as matching information.

The controller 130 may control the parameters of the smooth map or the fifth preset blending function through matching information based on the situation of the extended reality to be implemented.

Meanwhile, the controller 130 may generate matching information using an artificial intelligence model generated by machine learning, and may generate an original image to which a smooth map is applied based on the matching information.

As such, in the present invention, by applying the smooth map to the original image (ie, the original image to which the AO map is applied), the smooth surface of the target image can be expressed more softly and the rough surface of the target image can be expressed more rough. The user can recognize the texture of the surface of the target image through the original image to which the smooth map is applied.

Therefore, when the original image to which the smooth map is applied is displayed, compared to when the original image to which the smooth map is not applied is displayed in extended reality, etc., the user can feel the soft and rough texture of the surface of the art work to see the art work in an actual art museum. You can get the same experience.

On the other hand, in the present invention, in the original image to which all of Canvas Map, Normal Map, Metallic Map, AO Map (Ambient Occlusion Map) and Smoothness Map are applied, the coating map ( Coat Map) can be applied to create the final image.

Here, the original image to which the Canvas Map, Normal Map, Metallic Map, Ambient Occlusion Map, and Smoothness Map are all applied is called “changed image”. do.

Also, the coating map may refer to an original image generated by merging fragment images, that is, an original image to which no map is applied.

In addition, the final image is a Canvas Map, Normal Map, Metallic Map, AO Map (Ambient Occlusion Map), Smoothness Map and Coating Map applied. As an image, the data value of the original image can be maintained while expressing the texture of the target image.

More specifically, the controller 130 may blend the changed image and the original image. The controller 130 may substitute i) the pixel value of the changed image and ii) the pixel value of the original image into the sixth function. The final image generated through this process can maintain the data value of the original image while expressing the texture of the target image.

10( a ) is a diagram illustrating an original image to which a smooth map is applied or a modified image 1010 . By applying the coating map 930 of FIG. 10(b) to the modified image 1010, a final image 1030 as shown in FIG. 10(c) may be generated.

As such, in the present invention, by generating the final image, it is possible to express the texture of the target image and maintain the data value of the original image.

Accordingly, when the final image is displayed compared to when the original image is displayed in extended reality or the like, the user may be provided with an experience such as viewing a work of art in an actual art museum.

On the other hand, immersive content is generated using the ultra-high-definition image generated by the ultra-high-definition image realization system 100 according to the present invention, and the immersive content is converted into virtual reality (VR, Virtual Reality), augmented reality (AR, Augmented). Reality), extended reality (XR, eXtended Reality), or mixed reality (MR, Mixed Reality) can be output to electronic devices.

More specifically, the present invention, by outputting immersive content to an electronic device, virtual reality (VR, Virtual Reality), augmented reality (AR, Augmented Reality), extended reality (XR, eXtended Reality) or mixed reality (MR, Mixed Reality) (hereinafter referred to as virtual reality, etc.), to provide a feeling that the user visits to the site of an exhibition space such as an art gallery or museum and consumes the exhibits, that is, the feeling of seeing and experiencing with their own eyes. It relates to a method and system for providing an immersive virtual exhibition space for Hereinafter, a method of generating immersive content will be described with accompanying drawings.

Here, virtual reality (VR, Virtual Reality) is a compound word of Virtual, meaning virtual, and Reality, meaning reality, and refers to an artificial environment similar to the real one but not the real one. In the field of virtual reality, various efforts are being made to enable spatial and temporal experiences close to reality by directly acting on the user's five senses beyond simply realizing a virtual space.

Furthermore, augmented reality (AR, Augmented Reality) is a technology that overlays a 3D virtual image on a real image or background and displays it as a single image. Augmented reality is a technology that places a virtual object in the real space or recognizes a real object and configures a virtual space around it. effort is being made

In addition, extended reality (XR, eXtended Reality) is a technology that implements various advanced services such as spatial mapping and object tracking beyond simple content display. Users can experience a 360-degree virtual view without using a headset, and a natural-like experience can be experienced by providing spatial sound. Similarly, in the case of extended reality, efforts are being made to provide realistic and lively content.

The method for providing an immersive virtual exhibition space according to the present invention is for generating immersive content that can provide a sense of presence and vitality as if the user actually visited the exhibition space by using virtual reality, augmented reality and extended reality. As such, the following will be described in detail together with the accompanying drawings.

11 is a conceptual diagram illustrating virtual reality, augmented reality, mixed reality, or extended reality provided by the method and system for providing an immersive virtual exhibition space according to the present invention. 12 is a configuration diagram of a system for providing an immersive virtual exhibition space according to the present invention, and FIG. 13 is a flowchart illustrating a method for providing an immersive virtual exhibition space according to the present invention, FIGS. 14, 15, 16, and 17 is a conceptual diagram for explaining immersive content according to the present invention, and FIGS. 18, 19, 20, 21, and 22 are conceptual diagrams for explaining a user interface.

The immersive content generation 100 according to the present invention may operate in conjunction with an electronic device capable of outputting (or playing) immersive content. 11 , the system 100 for providing an immersive virtual exhibition space according to the present invention may operate in conjunction with the head mounted display 10a, the XR glasses 10b, and the electronic device 10c. The system 100 according to the present invention outputs immersive content to the head mounted display 10a, XR glasses 10b, or electronic device 10c, so that the user can display objects ( 10e) can provide the same experience as actually appreciating it.

Here, “realistic content” refers to something that can be provided to a user through an electronic device capable of outputting immersive content, and includes signs, characters, figures, colors, voices, sounds, images and images (or at least two of them). data, data or information of the above complex).

On the other hand, a head mounted display (10a, Head mounted display, hereinafter referred to as 'HMD') is a generic term for image display devices that can be worn like glasses to enjoy an image, and is recently also called a face mounted display (FMD). The HMD serves to show the virtual world to the user's eyes after blocking the user's sight and hearing from the outside. As an example, the HMD is equipped with a microphone, a stereo speaker, and various sensors in the form of a face worn with a display in front of the eyes.

XR Glasses (eXtended Reality Glasses) 10b allow users to view any content uploaded to an app in the electronic device 10c through a lens, floating in a virtual space. Baseball broadcasts, YouTube, music videos, news, and art works can be viewed on the train or bus, on the sofa or on the bed by adjusting the size of the screen.

However, the electronic device capable of outputting immersive content described in the present invention is not limited to the head mounted display 10a and the XR glasses 10b, and an electronic device capable of realizing virtual reality, etc. If (10c), the type is not distinguished. Hereinafter, all electronic devices capable of outputting immersive content will be collectively referred to as “electronic device outputting immersive content” or “electronic device”.

Hereinafter, together with the accompanying drawings, in order to provide a more realistic content to a user through virtual reality, augmented reality, mixed reality, and extended reality, a method of generating immersive content will be described in detail.

12 , the system 100 for providing an immersive virtual exhibition space according to the present invention includes a communication unit 110 , an input unit 120 , an output unit 130 , a storage unit 140 , and a control unit 150 . It may include at least one. In this case, the system 100 for providing an immersive virtual exhibition space according to the present invention is not limited to the above-described components, and may further include components that perform the same or similar functions as those described herein.

The communication unit 110 may transmit/receive data to and from the head mounted display 10a, the XR glasses 10b, the electronic device 10c, and at least one external server described above. For example, the communication unit 110 may transmit the immersive content according to the present invention to an electronic device to which the immersive content is output.

The input unit 120 includes a camera or an image input unit for inputting an image signal, a microphone or an audio input unit for inputting an audio signal, and a user input unit (eg, a touch key) for receiving information from a user. , a push key (mechanical key), a keyboard (Keyboard), a mouse (mouse), a joypad (joypad), etc.) may include.

Furthermore, the output unit 130 may output the immersive content to the view area (eg, the main view area) of the electronic device linked to the system 100 for providing an immersive virtual exhibition space according to the present invention.

Furthermore, the storage unit 140 may also be referred to as a database (DB), and may be configured to store contents used and generated in the present invention. For example, the storage unit 140 may store the immersive content according to the present invention by using a cloud server.

Furthermore, the storage unit 140 may store “background information” including configuration information of the background content that matches the visual information of the main content.

Here, the “main content” may be understood as an exhibit provided to the user through immersive content. For example, if the exhibit is a picture, the main content may be an image for the picture.

The “background content” described in the present invention may be understood as content forming a periphery of the main content. More specifically, the background content is content constituting a virtual exhibition space in which the main content is displayed.

The “virtual exhibition space” described in the present invention is an exhibition space implemented in virtual reality, and can be understood as a space in which main contents can be displayed, such as a virtual art gallery, a virtual museum, and a virtual exhibition hall. .

On the other hand, the “visual characteristics of the main content” described in the present invention relates to various information that can be seen and recognized by the eye, for example, i) shape, ii) color, iii) contrast, iv) There may be saturation, v) texture, vi) three-dimensionality, vii) resolution, and viii) volume.

Meanwhile, the controller 150 may be configured to generally control the components described above, the head mounted display 10a, the XR glasses 10b, and the electronic device 10c shown in FIG. 11 .

Furthermore, the controller 150 may generate immersive content by overlapping the main content with one region of the background content.

More specifically, the controller 150 may generate background content by further including light source information in which time information and weather information are reflected in the background content. That is, the controller 150 may generate background content in which the brightness and position of the light source change according to time information and weather information, and use the generated background content to generate immersive content.

Here, “light source information” may be understood as information defining a virtual light source existing in the virtual exhibition space. Specifically, the light source information may define at least one of a location, a shape, an intensity, and a direction of light for a virtual light source existing in the virtual exhibition space.

Furthermore, the controller 150 may change the light characteristic of the main content to reflect the light source information included in the background content. Therefore, when receiving immersive content generated using main content that does not reflect light source information and when receiving immersive content generated using main content reflecting light source information, the user receives the same main content You can feel three-dimensionality and texture differently. Accordingly, the user can experience the same experience as seeing the exhibits in the actual exhibition space.

Hereinafter, a method for providing an immersive virtual exhibition space according to the present invention together with a flowchart will be described in more detail.

In the present invention, a process of specifying main content displayed in the virtual exhibition space may be performed (S310). The controller 150 may specify main content displayed in the virtual exhibition space.

Here, the “virtual exhibition space” is an exhibition space realized in virtual reality, and may be understood as a space in which main contents can be displayed, such as a virtual art gallery, a virtual museum, and a virtual exhibition hall.

As shown in FIG. 14 , the virtual exhibition space may include a plurality of virtual exhibition spaces. In this case, as shown in FIG. 15 , any one virtual exhibition space may be output to the display of the electronic device on which the immersive content is output.

The controller 150 may specify the main content through the process of receiving the main content through the communication unit 110 . In this case, the communication unit 110 may receive main content from an external server or an external electronic device.

Furthermore, the control unit 150 may specify the main content based on the user's selection received through the input unit 120 . To this end, the controller 150 may output the main content sample to an electronic device such as a terminal possessed by the user or an electronic device through which immersive content is output through the output unit 140 .

The user selects the main content that he wants to view, that is, he wants to display in the virtual exhibition space among the main content samples output on the display of the electronic device, and transmits it to the communication unit 110 of the system 100 according to the present invention, or (120) can be entered.

The control unit 150 may specify the main content based on the user's selection received through the communication unit 110 or the input unit 120 .

Meanwhile, the controller 150 may specify the main content based on the user's history stored in the storage 140 . In this case, the user may mean a user logged into the system 100 according to the present invention by a user account.

The “user history” stored in the storage 140 may include a history of main content specified by the user in the past (a specified main content name, a specific number of times, a specific period, etc.).

The controller 150 may specify the main content based on the history of the main content specified by the user in the past. For example, the control unit 150 specifies as the main content content whose number of times specified by the user is equal to or greater than a preset number of times, or the main content specified by the user is not yet output to the display unit of the user's electronic device. If not, the corresponding main content can be specified. When specifying the main content based on the user's history, the controller 150 may specify the main content using an artificial intelligence model generated by machine learning.

Furthermore, in the present invention, the process of generating the background content of the virtual exhibition space may be performed using the visual characteristics of the main content (S320). That is, the controller 150 may generate the background content by using the visual characteristics of the main content.

Here, “visual characteristics of the main content” refers to various information that can be seen and recognized by the eye, for example, i) shape, ii) color, iii) contrast, iv) saturation, v) texture. , vi) three-dimensionality, vii) resolution, and viii) volume.

The controller 150 may extract a visual characteristic from the specified main content and determine the composition of the background content to maximize the extracted visual characteristic. In the step of generating the background content ( S320 ), a visual characteristic of the main content may be extracted, and a pattern, texture, or color of the background content may be set by using the extracted visual characteristic.

For example, the controller 150 may form a predetermined pattern with the shape (or pattern) of the main content, and generate the background content using the pattern. In this case, if the main content is a picture made of wood, the background content may generate the background content in a regular pattern (eg, a pattern that repeats at regular intervals) as the background content. As another example, the controller 150 may configure the color of the background content by using a complementary color of the color of the main content. When the color of the main content is yellow, the controller 150 may generate the background content using purple, which is a complementary color of yellow, and if the color of the main content is green, the controller 150 uses red, which is a complementary color of green, You can create background content. However, this is only an example for convenience of description.

Furthermore, the control unit 150 may generate background content based on the background information stored in the storage unit 140 .

The “background information” stored in the storage 140 may include configuration information of background content that matches the visual information. In this case, configuration information of the background content is matched to the visual characteristic, and the background content may be generated using configuration information matched to the extracted visual characteristic.

In this way, the controller 150 may generate the background content based on the composition information of the background content matched with the visual information extracted from the main content. For example, it is assumed that the composition information of the background content matched to the oil painting texture is the canvas texture and color (texture and white in which the fabric is interwoven) in the background information. If the visual information extracted from the main content is an oil painting texture, the controller 150 may generate the background content with a canvas texture and color based on the matching information.

Furthermore, in the present invention, the process of generating the immersive content may be performed so that the main content is superimposed on at least a part of the background content (S330). The controller 150 may generate immersive content by overlapping the main content with one region of the background content.

As described above, “realistic content” can be provided to a user through an electronic device capable of outputting immersive content, and includes signs, characters, figures, colors, voices, sounds, images and images (or these data, data, or information of at least two or more of the complexes). For example, immersive content includes VR content and XR content.

The controller 150 may generate immersive content by arranging the main content in one area of the background content. In this case, the controller 150 may arrange the main content in one area of the background content corresponding to the matching coordinates matching the main content. For example, if the coordinates matching the main content correspond to the left central region of the background content, the controller 150 arranges the main content in the left central region of the background content, and the coordinates matching the main content are on the right side of the background content If it corresponds to the central region, the controller 150 may generate the immersive content by arranging the main content in the right central region of the background content.

Furthermore, the controller 150 may generate the immersive content by arranging the main content in a part of the background content based on the user's selection received through the input unit 120 . To this end, the controller 150 outputs the main content and the background content on an electronic device such as a terminal possessed by the user or an electronic device on which the immersive content is output, such as “Place the main content on the background content”. A guide message can be printed.

The user can move the main content to one area of the background content by viewing the guide message. In this case, the user uses an electronic device or a touch key, a mechanical key, a keyboard, a mouse, a joypad, etc. of the system 100 according to the present invention, The main content may be positioned as one area of the background content. After arranging the main content in one area of the background content, the user may output a “Done” button that is output on the display of the electronic device or utter “Done” or “Generate realistic content” by voice. The voice uttered by the user may be input through an electronic device or a microphone or an audio input unit of the system 100 according to the present invention.

The control unit 150 may generate the immersive content when a preset time elapses from the time of receiving a command for generating immersive content from the user or outputting a guide message on the display. The immersive content generated in this way may be a state in which the main content is displayed by overlapping a part of the background content.

Furthermore, the controller 150 may transmit the immersive content to the electronic device so that the immersive content is provided in the virtual exhibition space. The controller 150 may control the communication unit 110 to transmit the immersive content to the electronic device, or may control the output unit 140 to output the immersive content to the display of the electronic device.

A user may view main content displayed in the virtual exhibition space through an electronic device that outputs immersive content. In the immersive content according to the present invention, since the background content is generated based on the visual characteristics of the main content, the user can more realistically feel the texture or color of the main content, and furthermore, viewing the exhibits in the actual exhibition space. You may be provided with the same feeling.

Hereinafter, a method of generating immersive content that can provide a more vivid and realistic feeling as if a user is viewing an exhibition in an actual exhibition space will be described in detail.

Meanwhile, in the present invention, the main content may be set to reflect light source information included in the background content. The controller 150 may change the light characteristic of the main content to reflect the light source information included in the background content.

First, in the present invention, a process of receiving a plurality of different pieces of images corresponding to main content may be performed. That is, the communication unit 110 may receive a plurality of different pieces of images corresponding to the main content.

The fragment image 410 may be understood as an image including at least a portion of the main content. For example, the main content is divided into left and right areas based on the center, the left area is captured or scanned with the camera as a first piece image, and the right area is captured or scanned with the camera as a second piece image. can be understood as

Furthermore, the plurality of fragment images may be ultra-high-resolution images. That is, each of the plurality of fragment images may be images that satisfy a preset resolution criterion.

Furthermore, any one of the plurality of fragment images may include image information that at least partially overlaps with the other one.

Here, the overlapping image information may mean an image corresponding to a specific region of the main content, which is included in each of the one and the other of the plurality of fragment images.

Meanwhile, in the present invention, a process of combining a plurality of sculptural images may be performed based on the similarity of each of the plurality of sculptural images. The controller 150 may generate main content by combining a plurality of pieces of images.

More specifically, the controller 150 may merge the fragment images by linking the fragment images based on overlapping image information. The controller 150 may match pieces of images adjacent to each other and connect the matched pieces of images.

In this case, the controller 150 may combine the fragment images based on image information overlapping each other among the plurality of fragment images. The controller 150 may combine the fragment images including the overlapping images by arranging them adjacent to each other.

Furthermore, the controller 150 may generate main content by performing edge blending on the combined fragment images to the combined fragment images, thereby naturally connecting the boundaries of the fragment images.

In an embodiment, the controller 150 may correct the transparency of the adjacent engraving images differently. The controller 150 may overlap one piece image having a different transparency on one piece image. The controller 150 may change the position of the fragment image having different transparency until the region corresponding to the overlapped image information included in each of the two overlapping fragment images overlaps by more than a preset reference. When the same area included in the two overlapping pieces of images overlaps by more than a preset reference, the controller 150 may adjust the transparency of the two overlapping pieces of images to 0. Through this, the two sculptural images can be connected to each other. The controller 150 may generate main content by performing edge blending on all of the plurality of different pieces of images.

The system 100 according to the present invention generates main content by performing such edge blending, thereby minimizing waste data of a misaligned portion of a connected fragment image or a boundary portion of the fragment images.

Furthermore, compared to the case of generating the main content by photographing a work of art once, a sculpture image is generated by photographing each area of the work of art, and edge blending is performed on the sculpture image to generate the main content, resulting in a higher-quality image can create

Through this, the system 100 according to the present invention can provide a more realistic exhibit to the user through virtual reality or the like.

Meanwhile, the controller 150 may apply a canvas map to the main content so that the specified main content has a predetermined texture.

Here, the canvas map may be understood as information representing the material of an object on which the main content is written or a material constituting the main content. More specifically, i) If the main content is an oil painting created on a canvas (a fabric used for drawing pictures such as oil paintings), the canvas map may be a layer corresponding to the canvas material, ii) the main content is wood (woodblock prints, etc.) If it is a woodblock print created on a wooden board used for drawing), the canvas map may be a layer corresponding to the material of the wood, iii) If the main content is a copper print created on a copper plate (a plank-shaped board made using copper), the canvas map Silver may be a layer corresponding to the copper material.

The controller 150 may generate a canvas map based on the object on which the main content is written. More specifically, the controller 150 may use an image of the object itself on which the main content is drawn as a canvas map. For example, if the main content is an oil painting drawn on a canvas, the controller 150 may use an image obtained by photographing the canvas as a canvas map.

Furthermore, the controller 150 may obtain a material of the object on which the main content is drawn by using an image obtained by photographing the object on which the main content is drawn, and may generate a canvas map by converting the obtained material into data. For example, if the main content is an oil painting, the control unit 150 obtains the material of the canvas (eg, a material woven by replacing the fabric) from the photographed image of the canvas, and generates information representing the material of the canvas by generating information representing the material of the canvas. You can create maps.

Furthermore, the controller 150 may predict the material of the object on which the main content is drawn from the main content. In this case, the controller 150 may use the artificial intelligence model generated by machine learning. The controller 150 may configure the information of the canvas map so that the predicted material is expressed.

The controller 150 may generate the main content to which the canvas map is applied by synthesizing a layer corresponding to a pre-specified texture in the main content.

That is, the controller 150 may generate a canvas map image by applying the generated canvas map to the main content. That is, the controller 150 may process the mapping (or blending, overlay) between the main content and the canvas map.

More specifically, the control unit 150 substitutes i) a pixel value corresponding to a specific pixel of the main content and ii) effect information corresponding to the specific pixel among information included in the canvas map to a first preset function, You can change the pixel value for a specific pixel in the content. The controller 150 applies the above-described method to all pixel values constituting the main content.

If the pixel value of the main content before the change is a pixel value that does not express the texture of the object on which the main content is drawn, the changed pixel value of the main content may be a pixel value that expresses the texture of the object on which the main content is drawn. For example, suppose that a specific pixel value included in the main content before the change is (3, 99, 5). If the object on which the main content is drawn is a canvas, the canvas map includes effect information corresponding to the specific pixel value, and the pixel value of the changed main content to which this is applied may be (34, 104, 29). Also, if the object on which the main content is drawn is copper, the canvas map has information about the copper material, and the pixel value of the changed main content to which this is applied may be (8, 52, 124). However, this is merely an example for convenience of description, and information included in the main content to which the canvas map is applied is not limited thereto.

Therefore, when the immersive content generated using the main content to which the canvas map is applied is displayed, compared to when the immersive content generated using the main content to which the canvas map is not applied is displayed, the user feels the texture of the exhibit. As a result, you can receive the same experience as viewing the exhibits in an actual exhibition space.

Meanwhile, in the present invention, a process of applying a normal map to the main content to which the Canvas Map is applied may be performed so that the reflection characteristics of virtual light incident on the virtual exhibition space are reflected. That is, the controller 150 may apply the normal map to the main content to which the canvas map is applied.

The normal map includes information enabling calculation of an angle at which light is reflected when light is incident on a specific pixel of the main content at a specific angle.

Each pixel constituting the normal map corresponds to a pixel constituting the main content. When the normal map is applied to the main content, vector information matched to a pixel corresponding to a specific pixel of the main content among pixels constituting the normal map is matched to the main content.

However, when there is no light source information, even if the normal map is applied to the main content, the main content is not affected.

Here, the light source information may be understood as defining a virtual light source existing in the virtual exhibition space. Specifically, the light source information may define at least one of a location, a shape, an intensity, and a direction of light for a virtual light source existing in the virtual exhibition space.

For example, when a virtual light source existing in the virtual exhibition space is a point light source, the light source information may include location information of the point light source and information defining that light is emitted in all directions from the point light source.

In the present invention, light source information existing in a virtual exhibition space may be implemented by background content. The controller 150 may form a virtual light source existing in a virtual exhibition space by arranging a light source (eg, the sun, the moon, lighting, etc.) in the background content. A detailed method of arranging the light source in the background content will be described later.

Meanwhile, by utilizing the light source information, it is possible to calculate the incident angle of the virtual light incident on each of the pixels constituting the main content. When the incident angle is applied to the main content to which the normal map is applied (or a normal vector matched to each pixel constituting the main content), information on the reflection angle of light from a specific pixel may be calculated. A new pixel value of the specific pixel may be calculated by applying the calculated reflection angle to a second preset function. If the above-described process is applied to all pixels included in the main content, the main content corresponding to the virtual light source can be generated.

Furthermore, the controller 150 may apply a mask map to the main content to which the normal map is applied, so that the main content expresses a texture more similar to the actual exhibit.

Accordingly, in the main content according to the present invention, the pixel value of the main content may be changed according to how the background content is configured. As an extreme example, if the background content does not include light source information, the main content may have the same pixel value as the main content to which the normal map or the mask map is not applied even if the normal map or the mask map is applied. On the other hand, if the background content includes light source information such as the sun, the pixel value of the main content may be changed according to the light source information of the sun.

When a user is provided with immersive content generated using the main content that does not reflect light source information and when receiving immersive content generated using main content that reflects light source information, the user provides three-dimensional effect, even with the same main content, You can feel different textures, etc. Accordingly, the user can experience the same experience as seeing the exhibits in the actual exhibition space.

Hereinafter, a process of realizing light source information of a virtual exhibition space based on background content will be described.

As shown in FIG. 14 , the virtual exhibition space may include a plurality of virtual exhibition spaces. In this case, the controller 150 may generate different background content for each virtual exhibition space.

The immersive content may include a plurality of virtual exhibition spaces having different background content. In this case, as shown in FIG. 15 , any one virtual exhibition space may be output to the display of the electronic device on which the immersive content is output.

The controller 150 may create a plurality of virtual exhibition spaces based on a user's selection. To this end, the controller 150 may output a user interface for designing a virtual exhibition space to an electronic device such as a terminal possessed by the user or an electronic device through which immersive content is output. For example, the controller 150 may output a message such as “Please select the number of virtual exhibition spaces” through the display or audio of the electronic device, so that the user can design a plurality of virtual exhibition spaces.

The user may select the desired number of virtual exhibition spaces according to the guidance message of the electronic device. In this case, the user can use an electronic device or a touch key, a mechanical key, a keyboard, a mouse, a joypad, and the like of the system 100 according to the present invention. The number of virtual exhibition spaces (ex: 6) can be selected. Also, the user may utter “six” by voice. The voice uttered by the user may be input through an electronic device or a microphone or an audio input unit of the system 100 according to the present invention.

The controller 150 may form six virtual exhibition spaces 400 as shown in FIG. 14 based on the user's selection. Hereinafter, for convenience of explanation, a plurality of virtual exhibition spaces are expressed as a first virtual exhibition space 410 and a second virtual exhibition space 420 . Furthermore, the main content displayed in the first virtual exhibition space 410 is expressed as the first main content, and the background content of the first virtual exhibition space is expressed as the first background content.

Meanwhile, as described above, each of the plurality of virtual exhibition spaces may have different background contents by using the visual characteristics of the main content displayed in each virtual exhibition space.

For example, as shown in FIG. 15A , the first virtual exhibition space 410 is a first virtual exhibition space created using first main contents 511 displayed in the first virtual exhibition space 410 . It may have one background content 510 . The first background content 510 may include eight walls 512 having dot-patterned wallpaper, a white floor 513 , and a ceiling 514 made of glass.

As another example, the second virtual space 420 may have the second background content 520 generated using the second main content 521 and that shown in FIG. 15B . The second background content 520 may have background content including eight walls 522 having comb-shaped wallpaper, a white floor 523 , and a ceiling 524 made of glass. That is, the first background content 510 and the second background content 520 may be set differently based on the respective visual characteristics of the first main content 511 and the second main content 521 .

Meanwhile, the background information stored in the storage 140 may include a plurality of pieces of background content configuration information matching the visual characteristics of the main content. For convenience of explanation, hereinafter, configuration information of a plurality of background contents matching a specific visual characteristic of the main content is expressed together with the first configuration information and the second configuration information.

For example, it is assumed that the background information may include the first composition information and the second composition information by matching the oil painting texture. The controller 150 may control the communication unit 110 to transmit the first configuration information and the second configuration information to a display of an electronic device such as a terminal possessed by a user or an electronic device on which immersive content is output. Alternatively, the controller 150 may control the first configuration information and the second configuration information to be output to the output unit 130 of the system 100 according to the present invention.

The user may select any one of the first configuration information and the second configuration information output to the output unit 130 of the electronic device or the system 100 according to the present invention. The user's selection may be transmitted from the electronic device to the communication unit 110 of the present invention or may be input through the input unit 120 of the present invention.

The controller 150 may generate background content based on the user's selection.

Furthermore, the controller 150 may generate the background content by reflecting time information in the background content. The controller 150 may generate immersive content using background content whose brightness is changed over time.

The storage unit 140 may store brightness information defining the brightness of the background content according to time. For example, suppose that the relatively darkest brightness is 0, and the relatively brightest brightness is 10. The brightness information stored in the storage unit 140 includes i) 6:00 to 8 o'clock to match brightness 2, ii) 8 o'clock to 11 o'clock to match brightness 3, iii) 11 o'clock to 13 o'clock to match brightness 5, iv ) 13 o'clock to 18 o'clock to match brightness 3, v) 18 o'clock to 22 o'clock to match brightness 1, vi) 22 o'clock to 24 o'clock to match brightness 0, vii) 24 o'clock to 5 o'clock to match brightness 0 , viii) may include a brightness of 1 by matching 5 to 6 o'clock. However, this is an example for convenience of description, and brightness information may be more subdivided.

When the immersive content is output through the electronic device, the controller 150 may generate the background content so that the brightness of the background content is changed based on the output time. For example, when the time when immersive content is output is 12:00 noon, the user can receive the background content set to the brightness of 5, thereby giving the user the same feeling as viewing the exhibits in the exhibition space during the actual daytime. As another example, when the time at which immersive content is output is 12:00 pm (24 o'clock), the user is provided with the background content set to 0 brightness, so that the user can experience the same feeling as watching an exhibition in an exhibition space at night. can receive

Furthermore, the controller 150 may change the pixel value of the main content according to the brightness of the background content that is changed based on the brightness information of the storage unit 140 . That is, the controller 150 may change the pixel value of the main content to reflect the reflection characteristics of the virtual light incident on the virtual exhibition space according to the brightness information of the background content.

For example, if the output time of immersive content is 12:00 noon, and the brightness of the background content is 5, the main content reflects the light reflection characteristics of the virtual exhibition space corresponding to the brightness 5 of the background content. Pixel values may be changed.

As another example, if the time at which the immersive content is output is 12:00 pm (24 o'clock) and the brightness of the background content is 0, the main content is the light of the virtual exhibition space corresponding to the brightness 0 of the background content. A pixel value may be changed to reflect the reflection characteristic.

Therefore, the user is provided with a different three-dimensional effect and texture when viewing the main content through the immersive content during the day and when viewing the main content through the immersive content at night. You will be provided with a lively experience such as

Meanwhile, the controller 150 may generate background content by further including light source information in which time is reflected in the background content. For example, an image corresponding to the illuminated light may be formed on the background content, and the direction in which the image corresponding to the light is illuminated may change over time.

The controller 150 may generate background content in which the brightness and position of the light source change according to the passage of time, and use the generated background content to generate immersive content. In this case, the light source may be at least one of the sun, the moon, and lighting. That is, the controller 150 generates the background content including the image corresponding to the light by reflecting information on light sources such as the sun, moon, and lighting whose position (altitude) or intensity changes over time. can do it

In this case, the controller 150 may set at least one of a wall and a ceiling of the virtual exhibition space as glass through the background content. For example, the background content may include glass disposed on at least one of a wall and a ceiling of the virtual exhibition space. Alternatively, the controller 150 may set the virtual exhibition space to be outdoors.

When the immersive content is output through the electronic device, the controller 150 may generate the background content so that the position and intensity of light sources such as the sun, moon, and lighting are changed based on the output time.

For example, when the time at which immersive content is output is 1 o'clock (13:00) in the afternoon, as shown in FIG. 16(a), the controller 150 sets the sun ( 611), the background content 610 may be generated so that light incident on the virtual exhibition space from the sun disposed at a predetermined position (altitude) is reflected. In this case, the background content 610 may be configured to be relatively bright by light incident on the virtual exhibition space from the sun 611 . Furthermore, when the time at which the immersive content is output is 3 o'clock in the afternoon (15:00), the control unit 150 controls the sun 611 of FIG. You can create background content.

As another example, when the time at which immersive content is output is 8:00 p.m. (20:00), as shown in FIG. 16(b) , the controller 150 reaches a predetermined position (altitude) with respect to the user. A light source such as 621 may be disposed, and the background content 620 may be generated such that light incident on the virtual exhibition space from the moon 621 disposed at a predetermined position (altitude) is reflected. In this case, the background content 620 may be configured to be relatively dark by light incident on the virtual exhibition space from the moon 621 . Furthermore, the controller 150 may generate the background content 620 by further including a light source such as the lighting 622 . In this case, the background content 620 may generate the background content 620 by making the area 623 illuminated by the light 622 relatively bright compared to the other surrounding areas 624 .

Furthermore, the controller 150 may change the pixel value of the main content based on the light source information included in the background content. That is, the controller 150 may change the pixel value of the main content to reflect the reflection characteristics of light incident on the virtual exhibition space due to the background content. More specifically, a reflection characteristic of light incident on the virtual exhibition space may be set to match the light shining on the background content, and a pixel value of the main content may be changed to reflect the reflection characteristic.

For example, when the time at which immersive content is output is 1:00 pm (13:00), and light source information such as the sun is included in one area of the background content, the main content is based on the light source information included in the background content. The pixel value may be changed to reflect the reflection characteristics of the light.

As another example, if the time at which immersive content is output is 8:00 p.m. (20:00), and accordingly, light source information such as the moon and lighting is included in one area of the background content, the main content is displayed in the background content. The pixel value may be changed to reflect the light reflection characteristic by the included light source information.

Meanwhile, the light source information according to the passage of time may be stored in the storage unit 140 by matching with the configuration information.

That is, in the present invention, the background content may be generated based on the specified visual information of the main content, and the pixel value of the main content may be changed based on the light source information of the generated background content.

Accordingly, the user is provided with main content reflecting the virtual light incident on the virtual exhibition space in the virtual exhibition space including the light source information according to the time of viewing the immersive content, so that the user can view the exhibition more in the actual exhibition space. You can get the same experience as you do.

Furthermore, the controller 150 may generate the background content by further including light source information in which the weather information is reflected in the background content. The controller 150 may generate background content in which the brightness and position of the light source change according to the weather, and use the generated background content to generate immersive content. In this case, the weather information may be input from the user through the input unit 120 or from an external server through the communication unit 110, weather information of a region in which the user is currently located may be received. In this way, the background content is generated by reflecting the weather information, and through this, the brightness of an image corresponding to light can be set according to the weather information.

Also, the weather information may be information corresponding to a region in which the electronic device is located. In this case, even if the time at which the immersive content is output is the same at 1 pm (13:00), the light source included in the background content when the user uses the immersive content in Seoul and when the immersive content is used in Jeju Island The information may be different. More specifically, when the weather in Seoul is clear, a user who uses immersive content in Seoul may be provided with immersive content including light source information reflecting the clear weather. On the other hand, when it rains on Jeju Island, a user who uses immersive content on Jeju Island may be provided with immersive content including light source information reflecting rainy weather.

Meanwhile, in the present invention, a process of generating immersive content may be performed by further reflecting docent content in the main content and background content. The controller 150 may generate immersive content by including the Doston content.

Here, the dost content may be understood as a function of describing the main content displayed in the virtual exhibition space.

As shown in (a) of FIG. 17 , the controller 150 may output the guide 713 to one area of the main view 710 of the electronic device on which the immersive content is output. Furthermore, when a specific main content 711 is output to a specific region of the user's main view 710 , the controller 150 may control a description of the corresponding main content 711 to be output through audio. Furthermore, the controller 150 may control the description 712 of the corresponding main content 711 to be output on one area of the main view 710 .

Here, the “main view” may mean a view area corresponding to the user's gaze among a plurality of view areas (or display areas) provided by the electronic device.

Furthermore, as shown in (b) of FIG. 17 , the controller 150 may output the button 722 together around the main content 721 displayed in the virtual exhibition space.

The user may transmit a request for explanation of the corresponding main content 721 to the system 100 according to the present invention by pressing a button 722 located around the main content 721 requiring explanation. In this case, the user's request for explanation is an electronic device that outputs immersive content or a user input unit according to the present invention (eg, a touch key, a push key, a mechanical key, a keyboard, a mouse ( mouse), joypad, etc.). Furthermore, a request for explanation of specific main content may be detected by a sensing unit (not shown) of the system 100 according to the present invention.

Here, the sensing unit (not shown) according to the present invention includes a proximity sensor, an illumination sensor, a touch sensor, an acceleration sensor, a magnetic sensor, and a gravity sensor. (G-sensor), gyroscope sensor, motion sensor, RGB sensor, infrared sensor (IR sensor: infrared sensor), fingerprint sensor (finger scan sensor), ultrasonic sensor, Optical sensor (eg, camera), microphone (microphone), battery gauge, environmental sensor (eg, barometer, hygrometer, thermometer, radiation detection sensor, thermal sensor, gas detection sensor, etc.) ), and a chemical sensor (eg, an electronic nose, a healthcare sensor, a biometric sensor, etc.).

Meanwhile, the immersive content according to the present invention may be generated based on a user's input. The immersive content generated based on the user's input has the advantage that it can be variously configured according to the user's taste and purpose (education, event, lecture, etc.). Hereinafter, a user interface provided to a user so that the user can compose immersive content will be described. For convenience of explanation, a case in which the virtual exhibition space is an art museum and the main content displayed in the virtual exhibition space is a work of art (or a picture) will be described as an example.

The control unit 150 is installed on a display unit (hereinafter, referred to as “display unit”) of an electronic device such as a terminal possessed by the user or an electronic device on which immersive content is output, as shown in FIG. 18(a), in a virtual exhibition space. You can control to display a screen asking you to set the viewing environment for (art museum). The control unit 150 is recommended in the system 100 according to the present invention and the option 811 for directly selecting the viewing environment of the virtual exhibition space together with a guide message such as “Do you want to set the museum viewing environment?” on the display unit. An option 812 for selecting a recommended environment may be output.

The user may select one of the options by looking at the options output to the display unit. In this case, the user selects an option using an electronic device or a touch key, a mechanical key, a keyboard, a mouse, a joypad, etc. of the system 100 according to the present invention. can choose Furthermore, the user may utter “direct selection 811” or “recommended environment selection 812” by voice. The voice uttered by the user may be input through an electronic device or a microphone or an audio input unit of the system 100 according to the present invention.

When the user selects the option of the direct selection 811 , the controller 150 may output various interior information constituting the background content on the display unit. For example, various interior information constituting the background content includes i) the shape of the virtual exhibition space (ex: indoor or outdoor, a hexahedral indoor space, an 8-pillar-shaped indoor space, etc.), ii) the material of the wall of the virtual exhibition space (ex: concrete material, wood material, etc.) or wallpaper (color of wallpaper, pattern of wallpaper, etc.), iii) floor material (concrete, marble, etc.) or color, iv) material of ceiling (glass, wood, concrete, etc.) or color, v) furniture (chairs, lighting fixtures, flowerpots, etc.).

The controller 150 may output various interior information to the display unit to provide an interface through which the user can directly compose the background content of the virtual exhibition space. For example, the controller 150 may provide an interface environment capable of generating one background content through a user drag-and-drop method of various interior information.

Furthermore, when the user selects the recommended environment selection 812 option, the control unit 150 may output a sample of the background content stored in the storage unit 140 to the display unit. In this case, the controller 150 may output the composition information of the background content matching the specified visual information of the main content as the

recommendation information

821 and 822 to the display unit.

The user may select any one of the

recommendation information

821 and 822 output to the display unit to generate background content.

Meanwhile, the controller 150 may receive weather information reflected in the background content from the user. In this case, as shown in (a) of FIG. 19 , the controller 150 includes a message such as “select the weather in which you want to view an art work” on the display unit, and weather emoticons (emoticon, 911, 912, 913, 914) can be output.

Based on the user selecting any one of the weather emoticons output on the display, the controller 150 may generate background content including light source information reflecting the corresponding weather information. For example, when the user selects the sun emoticon 911, the controller 150 may generate background content by including light source information reflecting sunny weather information.

Furthermore, the controller 150 may collect weather information from an external server based on the region where the user is located. In this case, as shown in (b) of FIG. 19 , the controller 150 may output a location access permission option together with a guide message such as “Do you want to allow access to the current location?” on the display unit.

When the user selects the allow option 921 output on the display unit or receives input of uttering “allow” by voice, the controller 150 detects the user's location through GPS, etc., and corresponds to the detected location. Weather information can be received from an external server. As illustrated in FIG. 19C , the controller 150 may output weather information of a location where the user is located on the display.

Furthermore, the controller 150 may receive time information reflected in the background content from the user. In this case, as shown in FIG. 20 , the controller 150 may output an interface for inputting a time together with a message such as “Please select a time to view an art work” on the display unit.

Based on the time input by the user, the controller 150 may generate background content by including light source information reflecting the corresponding time information. For example, as shown in FIG. 20 , when the user inputs 12:53 noon, the controller 150 places the sun at a position (altitude) corresponding to 12:53 noon, and arranges it at the corresponding position (altitude) Background content may be generated so that virtual light incident on the virtual exhibition space from the sun is reflected.

Furthermore, when the background content is generated including light source information to which weather information and time information received from the user are reflected, the controller 150 may change the pixel value of the main content according to the light source information included in the background content. Since the description of this process has been described above, it will be omitted.

Furthermore, the control unit 150 may receive a specific main content input from the user. To this end, the control unit 150, as shown in FIG. 21, along with a message such as “Please select an artist you want to view” on the display unit, along with the name and image of the artist (Klimt 1111, Van Gogh 1112, Renoir (Klimt 1111), Van Gogh (1112), Renoir ( 1113), etc.) can be output.

The user may select any one artist (eg, Van Gogh 1112 ) to view among the names and images of the artist output on the display unit. Alternatively, the author's name can be uttered like "Van Gogh".

When any one artist is selected by the user, the controller 150 may output the titles 1122 of the selected artist 1121 to the display unit. The user wants to view by using an electronic device or a touch key, a mechanical key, a keyboard, a mouse, a joypad, etc. of the system 100 according to the present invention. You can select at least one work title, or utter a voice, such as “Starry Night”. The voice uttered by the user may be input through an electronic device or a microphone or an audio input unit of the system 100 according to the present invention.

Meanwhile, the user may transmit an enlargement request for any one of the main contents displayed in the virtual exhibition space to the system 100 according to the present invention.

When receiving an enlargement request for a specific main content through the communication unit 110 or the input unit 120, the control unit 150 may output a guide message such as “Do you want to enlarge the selected picture?” as shown in (a) of FIG. 22 . When receiving approval of enlargement of the selected picture from the user, the control unit 150 may output the corresponding main content to the main view alone, as shown in FIG.

The user may receive a specific gesture 1210 to enlarge or reduce the specific area 1220 of the main content. For example, the controller 150 may reduce the specific area 1220 of the main content when the user closes two fingers, and enlarge the specific area 1220 of the main content when the user opens the two fingers. Such a user's gesture may be detected by a sensing unit (not shown) according to the present invention.

On the other hand, as described above, the main content in the present invention may be an ultra-high-definition image that is generated by combining a plurality of different sculptural images obtained by photographing a region of an exhibition object (eg, a work of art such as a painting). have.

Accordingly, the system 100 according to the present invention can provide a user with a high-quality, clear image without breaking the image quality of the main content even when the user enlarges the main content.

As described above, the method and system for providing an immersive virtual exhibition space according to the present invention can generate immersive content by generating background content of the virtual exhibition space using the visual characteristics of the main content. Accordingly, the user can feel the texture or color of the main content more realistically, and furthermore, the user can be provided with the feeling of viewing an exhibition in an actual exhibition space.

When the immersive content is provided to the user by a method according to various technologies such as virtual reality, augmented reality, mixed reality, or extended reality, the user can easily access works displayed in art galleries or museums at any time. Through this, users can enjoy rich cultural and artistic contents. Furthermore, art galleries or museums may provide more users with content corresponding to works, thereby enhancing user interest.

Meanwhile, the present invention described above may be implemented as a program storable in a computer-readable medium (or recording medium), which is executed by one or more processes in a computer.

Furthermore, the present invention as seen above can be implemented as computer-readable codes or instructions on a medium in which a program is recorded. That is, the present invention may be provided in the form of a program.

Meanwhile, the computer-readable medium includes all types of recording devices in which data readable by a computer system is stored. Examples of computer-readable media include Hard Disk Drive (HDD), Solid State Disk (SSD), Silicon Disk Drive (SDD), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. There is this.

Furthermore, the computer-readable medium may be a server or a cloud storage that includes a storage and that an electronic device can access through communication. In this case, the computer may download the program according to the present invention from a server or cloud storage through wired or wireless communication.

Furthermore, in the present invention, the computer described above is an electronic device equipped with a processor, that is, a CPU (Central Processing Unit, Central Processing Unit), and there is no particular limitation on the type thereof.

On the other hand, the above detailed description should not be construed as limiting in all respects, but should be considered as exemplary. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the present invention are included in the scope of the present invention.

Claims

Receiving a plurality of different pieces of images corresponding to the target image;

combining the plurality of fragment images based on the similarity of each of the plurality of fragment images;

applying a canvas map to the original image so that the original image generated as a result of the combination has a predetermined texture; and

and applying a normal map to the original image to which the Canvas Map is applied so that reflection characteristics of virtual light incident on a virtual space are reflected.
According to claim 1,

The plurality of fragment images are images that satisfy a preset resolution standard,

Any one of the plurality of fragment images comprises image information that at least partially overlaps with the other one.
3. The method of claim 2,

The step of generating the original image comprises:

An ultra-high-definition image implementation method, characterized in that edge blending is performed on the combined fragment images.
4. The method of claim 3,

The edge blending is

The super-high-definition image implementation method, characterized in that the mutually overlapping image information of each of the first fragment image and the second fragment image including mutually overlapping image information among the plurality of fragment images is overlapped.
According to claim 1,

In the step of applying the canvas map,

A method for realizing an ultra-high-definition image, characterized in that by synthesizing the original image with a layer corresponding to the specified texture, the original image to which the canvas map is applied is generated.
6. The method of claim 5,

The step of applying the normal map is,

A method for realizing an ultra-high-definition image, characterized in that it is performed based on light source information defining at least one of a location, a shape, an intensity, and a direction of light with respect to a virtual light source.
7. The method of claim 6,

In the step of applying the normal map,

calculating reflection angle information for each pixel constituting the original image based on the light source information and a normal vector included in the normal map; and

The method of claim 1, further comprising calculating a new pixel value for each pixel constituting the original image based on the reflection angle information.
7. The method of claim 6,

changing the light source information;

recalculating reflection angle information for each pixel constituting the original image based on the changed light source information and a normal vector included in the normal map; and

Based on the recalculated reflection angle information, the method further comprising the step of recalculating a new pixel value for each pixel constituting the original image.
9. The method of claim 8,

The method further comprising the step of providing the original image to which the normal map is applied, to an electronic device that reproduces the virtual space,

In the step of providing,

A method for realizing an ultra-high-definition image, comprising providing an original image including a new pixel value calculated based on the reflection angle information.
a communication unit configured to receive a plurality of different pieces of image corresponding to the target image; and

a control unit for combining the plurality of fragment images based on the similarity of each of the plurality of fragment images;

The control unit is

applying a canvas map to the original image so that the original image generated as a result of the combination has a predetermined texture,

Super high-definition image realization system, characterized in that applying a normal map (Normal Map) to the original image to which the Canvas Map is applied so that the reflection characteristics of the virtual light incident on the virtual space are reflected.