KR102623594B1 - Method for providing augmented reality service with non-fungible token content - Google Patents

Abstract

According to some embodiments of the present disclosure, a method for providing an augmented reality service using non-fungible token (NFT) content performed by a computing device including at least one processor is disclosed. The method for providing the augmented reality service includes: photographing a real space according to an input from a user to use the augmented reality service; displaying the captured real space in a first area; Constructing a library containing at least one content related to at least one NFT confirmed to be owned by the user; displaying the configured library in a second area; upon receiving a first user input in the second area, generating a virtual space based on first content included in the library; and displaying the created virtual space by overlaying it on the real space displayed in the first area.

Description

Method for providing augmented reality service using non-fungible token content {METHOD FOR PROVIDING AUGMENTED REALITY SERVICE WITH NON-FUNGIBLE TOKEN CONTENT}

This disclosure relates to a method for providing an augmented reality service, and specifically relates to a method for providing an augmented reality service using non-fungible token content.

Non-Fungible Token (NFT) is a type of token issued on a blockchain network and has a limited issuance amount, so unlike general cryptocurrency tokens, it is non-fungible. As described above, NFTs are non-substitutable and rare. Against this background, NFT is being used as a means of guaranteeing irreplaceable ownership of digital art, in-game goods, music files, etc.

Marketplaces for trading NFTs are emerging. However, the absence of a service that can utilize NFT content is a factor that reduces the motivation to own NFT.

In addition, because the ownership of NFTs is stored in the on-chain area of the blockchain, and the content files are stored in storage outside the blockchain, it is a cumbersome task to individually verify the actual ownership of the content for the user or the user's acquaintances.

Therefore, in order for NFT transactions to become more active, service provision technology that can easily utilize NFT content while meeting the needs of using NFT content is required.

Republic of Korea Open Patent 10-2022-0011385

This disclosure was developed in response to the above-described background technology, and seeks to provide a method for providing an augmented reality service that can utilize NFT content.

The technical problems of the present disclosure are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below.

According to an embodiment of the present disclosure to solve the problems described above, an augmented reality service using non-fungible token (NFT) content performed by a computing device including at least one processor is provided. A method for doing so is disclosed. The method for providing the augmented reality service includes: photographing a real space according to an input from a user to use the augmented reality service; displaying the captured real space in a first area; Constructing a library containing at least one content related to at least one NFT confirmed to be owned by the user; displaying the configured library in a second area; upon receiving a first user input in the second area, generating a virtual space based on first content included in the library; and displaying the created virtual space by overlaying it on the real space displayed in the first area. may include.

In addition, the step of configuring a library containing at least one content related to at least one NFT confirmed to be owned by the user includes at least one blockchain mainnet or transaction history of the blockchain mainnet. Accessing at least one of at least one marketplace providing information and determining the at least one NFT; Accessing the metadata storage location of the at least one NFT to obtain at least one content related to the at least one NFT - the metadata includes the name of at least one content related to the at least one NFT and at least one Contains at least one of the following storage locations for content data -; and configuring the library including the at least one content. may include.

In addition, the step of accessing the metadata storage location of the at least one NFT to obtain at least one content related to the at least one NFT includes metadata of the at least one NFT or metadata of the at least one NFT. Obtaining at least one content corresponding to the at least one NFT based on at least one of the stored URLs; may include.

In addition, the real space is captured through a depth sensing module, and the step of overlaying and displaying the virtual space on the real space displayed in the first area is based on depth information acquired through the depth sensing module. Overlaying the virtual space on the real space; It can be included.

In addition, the step of overlaying and displaying the virtual space on the real space displayed in the first area may include determining an area of at least one background object constituting the space in the captured real space; and displaying the virtual space based on the determined area of the at least one background object. may include.

In addition, the step of overlaying and displaying the virtual space on the real space displayed in the first area includes determining at least one first object excluding at least one background object constituting the space in the captured real space. steps; removing the at least one first object from the real space displayed in the first area; and displaying the virtual space by overlaying it on the real space from which the at least one first object has been removed. may include.

In addition, the step of overlaying and displaying the virtual space on the real space displayed in the first area includes determining at least one first object excluding at least one background object constituting the space in the captured real space. steps; determining whether at least one second object corresponding to the at least one first object exists in the virtual space; and when at least one second object exists, displaying the at least one second object by overlaying it on a third area where the at least one first object is located in the real space. may include.

Additionally, upon receiving a second user input in the second area, generating a third object based on second content included in the library; and displaying the third object by overlaying it on the first area. It may further include.

Additionally, the second user input may include an input for dragging and dropping the second content from the second area to the first area.

Additionally, when the location where the real space is displayed in the first area changes, the virtual space may be displayed by overlaying the real space by tracking the changed location.

Additionally, a computing device for providing an augmented reality service, comprising: a photographing unit that photographs a real space according to input from a user for using the augmented reality service; a display unit that displays the captured real space in a first area; And configuring a library including at least one content related to at least one NFT that is confirmed to be owned by the user, displaying the library configured through the display unit in a second area, and 1 A control unit that receives a user input and creates a virtual space based on first content included in the library; It includes, and the display unit can display the virtual space by overlaying it on the real space displayed in the first area.

The technical solutions obtainable from this disclosure are not limited to the solutions mentioned above, and other solutions not mentioned above will be clearly apparent to those skilled in the art from the description below. It will be understandable.

According to some embodiments of the present disclosure, it is possible to provide a method for providing an augmented reality service in which content that can confirm ownership of a user is displayed.

The effects that can be obtained from the present disclosure are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description below. .

Various aspects will now be described with reference to the drawings, where like reference numerals are used to collectively refer to like elements. In the following examples, for purposes of explanation, numerous specific details are set forth to provide a comprehensive understanding of one or more aspects. However, it will be clear that such aspect(s) may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form to facilitate describing one or more aspects.
1 illustrates an example system for performing a method for providing augmented reality services according to some embodiments of the present disclosure.
FIG. 2 is a flowchart illustrating an example of a method in which a computing device displays content by overlaying it on an object according to some embodiments of the present disclosure.
FIG. 3 is a flowchart illustrating an example of a method by which a computing device configures a library according to some embodiments of the present disclosure.
FIG. 4 is a flowchart illustrating another example of a method in which a computing device displays a virtual space by overlaying it on a real space displayed in a first area according to some embodiments of the present disclosure.
FIG. 5 is a flowchart illustrating an example of a method in which a computing device displays at least one second object by overlaying at least one first object according to some embodiments of the present disclosure.
FIG. 6 is a diagram illustrating an example of a user interface displayed on a display unit according to some embodiments of the present disclosure.
FIG. 7 is a diagram for explaining another example of a user interface displayed on a display unit according to some embodiments of the present disclosure.

Since the present invention can make various changes and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all changes, equivalents, and substitutes included in the spirit and technical scope of the present invention. While describing each drawing, similar reference numerals are used for similar components.

Terms such as first, second, A, B, etc. may be used to describe various components, but the components should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, a first component may be named a second component, and similarly, the second component may also be named a first component without departing from the scope of the present invention. The term and/or includes any of a plurality of related stated items or a combination of a plurality of related stated items.

When a component is said to be "connected" or "connected" to another component, it is understood that it may be directly connected to or connected to the other component, but that other components may exist in between. It should be. On the other hand, when it is mentioned that a component is “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between.

The terms used in this application are only used to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the technical field to which the present invention pertains. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and unless explicitly defined in the present application, should not be interpreted in an ideal or excessively formal sense. No.

In the present disclosure, a computing device can capture real space according to input from a user. The real space may be, for example, a room where the user is located, an office, or an outdoor space. The computing device may display the captured real space in the first area of the display unit. Here, the first area may be the entire area of the display unit of the computing device. Depending on the embodiment, the first area may be at least a partial area of the display unit. The computing device may display content related to a non-fungible token (NFT) that is confirmed to be owned by the user by overlaying it on the real space displayed in the first area. Here, NFT may be a virtual token issued on a blockchain network. NFT can be used to indicate uniqueness, originality, or ownership by containing an address pointing to a digital file in a token. Content related to NFT may be a digital file. Content may be a digital file such as an image, video, item, or 3D modeling file. The computing device can display content related to the NFT, which is confirmed to be owned by the user, by overlaying it in real space. Here, the meaning of overlaying can be understood as displaying content by overlapping it on the screen where the real space is displayed. Hereinafter, a method for a computing device to provide an augmented reality service will be described through FIGS. 1 to 7.

1 illustrates an example system for performing a method for providing augmented reality services according to some embodiments of the present disclosure.

Referring to FIG. 1 , the computing device 100 may include a control unit 110, a storage unit 120, a communication unit 130, a display unit 140, and a photographing unit 150. However, the above-described components are not essential for implementing the computing device 100, so the computing device 100 may have more or less components than the components listed above.

Computing device 100 may include typical computer hardware (e.g., a computer processor, memory, storage, input and output devices, and other components of a traditional computing device; electronic communication devices such as routers, switches, etc.; network A combination of electronic information storage systems (such as network-attached storage (NAS) and storage area network (SAN)) and computer software (i.e., instructions that cause a computing device to function in a certain way) The desired system performance can be achieved by using . For example, the computing device 100 is a personal computer (PC), notebook (note book), mobile terminal, smart phone, tablet PC, etc. owned by the user. It can include all types of terminals that can connect to wired/wireless networks. Alternatively, computing device 100 may include any type of computer system or computer device, such as a microprocessor, digital processor, device controller, and user terminal.

The control unit 110 can typically process the overall operation of the computing device 100. The control unit 110 processes signals, data, information, etc. input or output through the components of the computing device 100 or runs an application program stored in the storage unit 120 to provide appropriate information or functions to the user. It can be handled.

The control unit 110 may be composed of one or more cores, such as a central processing unit (CPU), general purpose graphics processing unit (GPGPU), and tensor processing unit (TPU). It may include a processor for data analysis.

In the present disclosure, the control unit 110 can photograph a real space through the photographing unit 150 based on input from a user for using an augmented reality service. When a real space is photographed through the photographing unit 150, the display unit 140 may display the photographed real space in the first area. The control unit 110 may create a virtual space based on content related to the NFT that is confirmed to be owned by the user, according to input from the user. Content related to NFT may be a digital file. Content may be a digital file such as an image, video, item, or 3D modeling file. In this disclosure, content may be content related to space. For example, content may include digital files associated with a space, such as a virtual room, virtual office, virtual store, or virtual museum. As another example, content may include digital files related to spaces such as a real room, a real office, a real store, or a real museum.

The control unit 110 may create a virtual space based on content including digital files related to the virtual space. When a virtual space is created, the display unit 140 may display the virtual space by overlaying it on the real space. Overlaying can be understood to mean that a virtual space is displayed overlapping the screen where the real space is displayed. The control unit 110 can provide an augmented reality service by overlaying and displaying the virtual space on the real space displayed on the display unit 140. Hereinafter, the method by which the control unit 110 displays virtual space by overlaying it on real space will be further described with reference to FIG. 2 .

In the present disclosure, the control unit 110 may display a library containing content related to NFT in the second area. In order to display the library, the control unit 110 may configure a library that includes at least one content related to at least one NFT that is confirmed to be owned by the user. Here, the library may provide appropriate functions/information to the user by modularizing information such as content files, content preview files, content-related descriptions, or content names of content related to NFT.

By having the control unit 110 build a library of content whose ownership has been confirmed by NFT in advance, the process for service users to apply NFT content to the augmented reality service can be made simple and easy. In addition, when the user uses NFT content for an augmented reality service by performing the library building procedure in advance by the control unit 110, other people can verify that the ownership of the user's augmented reality content has been verified by NFT without a separate verification process. Trustworthy.

Storage unit 120 may include memory and/or persistent storage media. Memory is a flash memory type, hard disk type, multimedia card micro type, card type memory (e.g. SD or XD memory, etc.), and RAM (Random Access). Memory, RAM), SRAM (Static Random Access Memory), ROM (Read-Only Memory, ROM), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, magnetic disk, optical disk It may include at least one type of storage medium.

The communication unit 130 is located between the computing device 100 and the communication system, between the computing device 100 and the server 200, between the computing device 100 and the blockchain mainnet 300, and between the computing device 100 and the marketplace. It may include one or more modules that enable communication between 400 or between computing device 100 and network 500.

The display unit 140 may display (output) information (or at least one content) processed by the computing device 100. For example, the display unit 140 may display execution screen information of an application program running on the computing device 100, or UI (User Interface) and GUI (Graphic User Interface) information according to this actual screen information. . In the present disclosure, the display unit 140 may display an augmented reality image generated through the control unit 110.

The photographing unit 150 may include one or more cameras. Image frames processed by the photographing unit 150 may be stored in the storage unit 120 or displayed on the display unit 140.

In the present disclosure, the photographing unit 150 may include a depth sensing module capable of sensing the depth of real space. For example, the photographing unit 150 may include a depth sensing module such as a Time of Flight (ToF) sensor.

Specifically, the depth sensing module is a sensor that can calculate the depth value of each pixel from a 2D image captured through a camera, and can measure the depth of real space from the captured 2D image of real space. For example, the photographing unit 150 may photograph the right wall, the left wall, and the front wall existing between the right and left walls. The right wall and the left wall may extend from the location where the user photographs the real space through the computing device 100 to the front wall. The depth sensing module can sense the depth from the location where the computing device 100 captured the real space to the front wall. The depth sensing module can generate depth information based on the sensed depth to the front wall. The depth sensing module can generate depth information by sensing the depth of the right wall extending from the location where the computing device 100 captured the real space. In this case, the control unit 110 may display the virtual space by overlaying it on the real space based on depth information.

According to some embodiments of the present disclosure, when a virtual space is displayed based on depth information, the size of at least one object displayed in the virtual space may be determined based on the depth information.

Specifically, the control unit 110 may display at least one object in a virtual space according to additional input from the user. When displaying at least one object in a virtual space, the control unit 110 may determine the size of the at least one object based on the position at which the at least one object is displayed. For example, when displaying at least one object at a first position according to an input from a user, the control unit 110 may determine the size of the at least one object as the first size. When displaying at least one object at a second location located further away from the first location according to an input from the user, the control unit 110 may determine the size of the at least one object to be a second size smaller than the first size. there is. For example, the first location may be close to the location where the computing device 100 captured the real space, and the second location may be close to the front wall. Accordingly, users can feel perspective in virtual space, and interest can improve when using augmented reality services. Hereinafter, an example in which the control unit 110 determines the size of at least one object will be described later with reference to FIG. 7.

Server 200 may include any type of computer system or computer device, such as, for example, a microprocessor, mainframe computer, digital processor, portable device, or device controller.

Server 200 may include typical computer hardware (e.g., a device that may include a computer processor, memory, storage, input and output devices, and other components of a traditional computing device; electronic communications devices such as routers, switches, etc.; network attachments) A combination of electronic information storage systems (such as network-attached storage (NAS) and storage area network (SAN)) and computer software (i.e., instructions that cause a computing device to function in a particular way). This can be used to achieve desired system performance.

In the present disclosure, the server 200 may configure a library containing content related to NFT. The server 200 may transmit the configured library based on a signal transmitted from the computing device 100.

The blockchain mainnet 300 may be a network that launches and operates a blockchain project. The blockchain mainnet 300 is composed of an unspecified number of nodes and can record transactions and verify them by the blockchain protocol. The blockchain mainnet 300 can issue tokens according to a predetermined protocol and record and verify transactions regarding token ownership. In Figure 1, a single blockchain mainnet 300 is shown for convenience, but the embodiment is not limited thereto. The computing device 100 can communicate with a plurality of different types of blockchain mainnets 300. Each blockchain mainnet 300 can be operated by a blockchain protocol that supports NFT. Blockchain protocols that support NFT include Ethereum, WAX, FLOW, and Binance Smart Chain, but embodiments are not limited thereto.

NFTs can be issued according to protocols such as Ethereum, WAX, and FLOW Binance Smart Chain mentioned above. The amount of NFT issued may be limited. For example, the issuance quantity of NFT may be 1. However, the embodiment is not limited thereto. The issuance quantity of NFT may be a number greater than 1. The issuance volume of NFTs may be significantly smaller than that of fungible tokens.

The marketplace 400 may be a platform that relays NFT transactions. The marketplace 400 provides an NFT transaction relay function and can store NFT transaction transactions that have been verified by blockchain. Computing device 100 may communicate with multiple marketplaces. Examples of the marketplace 400 include OpenSea, Rarible, Nifty Gateway, and SuperRare, but the embodiment is not limited thereto.

The network 500 may be a closed network such as a Local Area Network (LAN) or a Wide Area Network (WAN), or an open network such as the Internet. The Internet uses the TCP/IP protocol and several services existing at its upper layer, namely HTTP (HyperText Transfer Protocol), Telnet, FTP (File Transfer Protocol), DNS (Domain Name System), SMTP (Simple Mail Transfer Protocol), and SNMP ( It refers to a worldwide open computer network structure that provides Simple Network Management Protocol (NFS), Network File Service (NFS), and Network Information Service (NIS).

FIG. 2 is a flowchart illustrating an example of a method in which a computing device displays content by overlaying it on an object according to some embodiments of the present disclosure.

Referring to FIG. 2, the photographing unit 150 of the computing device 100 may photograph a real space according to an input from a user for using an augmented reality service (S110). Input from the user for using the augmented reality service may include input for logging into the service account of the augmented reality service or input for executing an application that provides the augmented reality service.

The display unit 140 may display the captured real space in the first area (S120). Here, the first area may be the entire area of the display unit 140. Alternatively, the first area may be at least a portion of the entire area of the display unit 140.

The control unit 110 may configure a library that includes at least one content related to at least one NFT that is confirmed to be owned by the user (S130). Here, confirmation that the user owns it can be understood to mean that the final ownership of at least one NFT exists in a virtual wallet owned by the user.

Specifically, the control unit 110 operates at least one of at least one blockchain mainnet 300 or at least one marketplace 400 that provides transaction history information of the blockchain mainnet 300 through the communication unit 130. You can access . The control unit 110 may perform a search for transactions that have been verified in the blockchain mainnet 300. As another example, the control unit 110 may access the block explorer and perform a search for transactions that have been verified on the blockchain mainnet 300. As another example, the control unit 110 may connect to the marketplace 400, which relays NFT transactions and records transaction history, and performs a search for transactions that have been verified on the blockchain mainnet 300. The control unit 110 may determine the NFT in which the virtual wallet address associated with the user is recorded as the final owner. When at least one NFT is determined, the control unit 110 may obtain information related to the at least one NFT. Information related to at least one NFT may include an NFT token ID or metadata storage address. Here, metadata may be data related to at least one NFT or data providing information related to content. For example, the metadata may include the name of at least one content related to at least one NFT or the storage location of at least one content data. The control unit 110 may acquire at least one content by accessing the metadata storage location of at least one NFT. Hereinafter, an example of a method by which the control unit 110 acquires at least one content related to NFT will be described with reference to FIG. 3.

As at least one content is acquired, the control unit 110 may configure a library including the at least one content. Here, the library may provide appropriate functions/information to the user by modularizing information such as content files, content preview files, content-related descriptions, or content names of content related to NFT. For example, referring to (a) of FIG. 6, the control unit 110 may configure a first library item 1441 including first content 143. The control unit 110 may configure a second library item 1442 that includes at least one content different from the first content 143. The control unit 110 may configure the library 144 including a first library item 1441 and a second library item 1442. Hereinafter, an embodiment of the library 144 will be described later with reference to FIG. 6.

Referring again to FIG. 2, the display unit 140 may display the configured library in the second area (S140). Here, the second area may be smaller than the first area. The second area may be at least a portion of the area included in the first area.

When the library is displayed in the second area, the user can select a specific item from among the items included in the library through user interaction. For example, the user can change the displayed library items through a sliding interaction on the second area. At least some of the library items may be displayed in the second area. Exemplarily, preview content or main content of at least some of the library items may be displayed in the second area. Here, the preview content may mean an image that allows the first content to be confirmed even if the first content or the entire file related to the first content is not loaded. In other words, the preview content may be an image with a smaller capacity than the first content through which the first content can be viewed. Preview content may be included as a component in a library including first content. Main content may be understood to have the same or similar meaning as first content. In other words, the main content displayed in the second area may be the first content.

As the control unit 110 receives the first user input in the second area, it can create a virtual space based on the first content included in the library (S150). The user input in the second area may be an input that changes a library item displayed through a sliding interaction. The user input in the second area may be an input for selecting the first content included in the library item displayed in the second area. As the control unit 110 receives the first user input in the second area, it may create a virtual space based on the first content.

For example, the first content may be content related to a virtual museum. Content related to a virtual museum can be understood by the user or other users as an image or 3D modeling file that embodies a museum that does not actually exist. In this case, the control unit 110 may create a virtual space including a virtual museum based on the first content.

As another example, the first content may be content related to a real-life museum. Content related to a real-life museum can be understood as an image or 3D modeling file created by a user or another user by photographing an actually existing museum. In this case, the control unit 110 may create a virtual space that virtually implements a real-life museum based on the first content.

The display unit 140 may display the generated virtual space by overlaying it on the real space displayed in the first area (S160).

Specifically, the photographed real space may include at least one background object constituting the space. At least one background object may include, for example, a background object related to a wall. The control unit 110 may determine a first background object related to the wall surface. The control unit 110 may determine the 1-1 background object corresponding to the first background object in the created virtual space. The control unit 110 may display the 1-1 background object by overlaying it on the first background object. The at least one background object may include, for example, a background object related to the floor. The controller 110 may determine a second background object related to the floor. The control unit 110 may determine the 2-1 background object corresponding to the second background object in the generated virtual space. The control unit 110 may display the 2-1 background object by overlaying it on the second background object.

According to some embodiments of the present disclosure, the control unit 110 may determine the area of at least one background object constituting the space in the captured real space. The display unit 140 may display a virtual space based on the determined area of at least one background object.

For example, the control unit 110 may determine a first background object related to a wall among at least one background object constituting a space in real space. The control unit 110 may determine the area of the determined first background object. The control unit 110 may determine the 1-1 background object corresponding to the first background object in the generated virtual space. The control unit 110 may display the 1-1 background object by overlaying it on the first object based on the area of the first background object. For example, the size of the first content or the virtual space to be created based on the first content may be preset in the metadata of the NFT related to the first content. However, if the virtual space created based on the first content is displayed overlaid on the real space with a preset size, the user may perceive that the virtual space has not been created properly. In other words, users may feel uncomfortable using augmented reality services. Accordingly, the control unit 110 may display the virtual space by overlaying it on the real space, based on the area of at least one background object constituting the space in the real space.

According to some embodiments of the present disclosure, the photographing unit 150 may photograph real space through a depth sensing module. The photographing unit 150 can photograph real space through a depth sensing module such as a Time of Flight (ToF) sensor. In this case, the control unit 110 may overlay the virtual space on the real space based on depth information obtained through the depth sensing module.

For example, the photographing unit 150 may photograph the right wall, the left wall, and the front wall existing between the right and left walls. The right wall and the left wall may extend from the location where the user photographs the real space through the computing device 100 to the front wall. The depth sensing module can sense the depth from the location where the computing device 100 captured the real space to the front wall. The depth sensing module may generate depth information of a first front wall object related to the front wall. The controller 110 may determine the 1-1 front wall object corresponding to the first front wall object in the generated virtual space. The controller 110 may display the 1-1 front wall object by overlaying it on the first front wall object, based on depth information of the first front wall object. Accordingly, users using augmented reality services can feel a sense of liveliness.

According to some embodiments of the present disclosure, when the location where the real space is displayed in the first area changes, the virtual space may be displayed by overlaying the real space by tracking the changed location.

Specifically, the location of the real space displayed in the first area may change as the user operates the computing device 100. For example, a user may operate the computing device 100 to photograph the front wall in real space. The user may operate the computing device 100 so that the right wall is captured from the front wall. Accordingly, the real space displayed in the first area may change from the front wall to the right wall. As the location where the real space is displayed in the first area changes, the controller 110 may track the changed location. The control unit 110 may overlay and display the virtual space at the changed location.

According to the above-described configuration, the computing device 100 can provide an augmented reality service by overlaying and displaying at least one NFT-related content on the real space displayed in the first area. Accordingly, the user's friends or acquaintances can trust that the augmented reality image displayed on the computing device 100 was implemented using NFT content without a separate verification process. In other words, the user's friends or acquaintances can trust that the user owns the augmented reality image displayed on the computing device 100. In order to further maximize the above-described effect, the computing device 100 may share augmented reality images or NFT content implemented using NFT content to other computing devices of other users. Here, sharing the augmented reality image with another computing device of another user may be understood as causing the augmented reality image displayed on the display unit 140 to be displayed on the display unit of the other computing device. For example, the computing device 100 may transmit an augmented reality image to another computing device through the communication unit 130, thereby causing the augmented reality image generated by the computing device 100 to be displayed on the display unit.

According to some embodiments of the present disclosure, the computing device 100 may access the metadata storage location of at least one NFT to configure a library including at least one content. When the storage location is accessed, the computing device 100 may acquire at least one content related to at least one NFT. Hereinafter, an example of a method by which the computing device 100 configures a library will be described with reference to FIG. 3.

FIG. 3 is a flowchart illustrating an example of a method by which a computing device configures a library according to some embodiments of the present disclosure.

Referring to FIG. 3, the control unit 110 of the computing device 100 operates at least one blockchain mainnet 300 or at least one market that provides transaction history information of the blockchain mainnet 300. You can access at least one of the places 400 and determine at least one NFT (S131).

The control unit 110 may perform a search for transactions that have been verified on the blockchain mainnet 300. As another example, the control unit 110 may access the block explorer and perform a search for transactions that have been verified on the blockchain mainnet 300. As another example, the control unit 110 may connect to the marketplace 400, which relays NFT transactions and records transaction history, and performs a search for transactions that have been verified on the blockchain mainnet 300. Based on the search results of the transaction, the control unit 110 may determine at least one NFT in which the virtual wallet address of the service account related to the user is recorded as the final owner. The control unit 110 may obtain information related to at least one NFT in which the virtual wallet address associated with the service account is recorded as the final owner. Information related to at least one NFT may include an NFT token ID or metadata storage address.

The control unit 110 may access the metadata storage location of at least one NFT to obtain at least one content related to at least one NFT (S132).

Specifically, the control unit 110 may acquire at least one content corresponding to at least one NFT based on at least one of the metadata of the at least one NFT or the URL stored in the metadata of the at least one NFT.

Here, metadata may be data related to at least one NFT or data providing information related to content. For example, metadata may include the name of at least one content related to at least one NFT, a content description indicating the type of at least one content, or a storage location of at least one content data. The content description may indicate at least one type of content. For example, the content description may indicate whether the content is related to a cultural space, a tourist destination, or an office. The storage location of content data may be information indicating the location where the content is stored. Depending on the embodiment, metadata may directly include a content file, a preview content file, etc. In this disclosure, metadata and content files, etc. may be stored in an off-chain area. For example, metadata and content files may be stored in IPFS (InterPlanetary File System), cloud storage, etc., but the embodiment is not limited thereto.

The control unit 110 may configure a library including at least one content (S133). Depending on the embodiment, the control unit 110 may configure a library including a plurality of library items. Here, a library item can be understood as a sub-concept of a library. Alternatively, a library item may be a unit with lower granularity than the library. For example, a first library item included in the library may include at least one first content. A second library item included in the library may include at least one second content that is different from the at least one first content.

According to some embodiments of the present disclosure, the control unit 110 may configure a library according to each type of at least one content based on metadata of at least one NFT.

For example, the control unit 110 may configure a first library including a plurality of first library items related to a cultural space. The control unit 110 may configure a second library that includes a plurality of second library items related to tourist attractions.

According to some embodiments of the present disclosure, the control unit 110 may configure library items according to each type of at least one content based on metadata of at least one NFT.

For example, the control unit 110 may configure a first library item including a plurality of contents related to a cultural space. The control unit 110 may configure a second library item including a plurality of contents related to a tourist destination. The control unit 110 may configure a library including first library items and second library items.

According to the above-described configuration, the computing device 100 can acquire at least one content by accessing at least one of at least one blockchain mainnet 300 or at least one marketplace 400. Accordingly, at least one content displayed on the computing device 100 may be content that is confirmed to be owned by the user, and uniqueness of the content may be guaranteed.

According to some embodiments of the present disclosure, the computing device 100 may determine at least one object existing in the real space when overlaying the virtual space on the real space displayed in the first area. Here, at least one object may be an object excluding a background object. For example, the at least one object may be an object such as a chair or a desk. After removing at least one object, the computing device 100 may display the virtual space by overlaying it on the real space displayed in the first area. Hereinafter, another example in which the computing device 100 overlays and displays a virtual space on the real space displayed in the first area will be described with reference to FIG. 4.

FIG. 4 is a flowchart illustrating another example of a method in which a computing device displays a virtual space by overlaying it on a real space displayed in a first area according to some embodiments of the present disclosure.

Referring to FIG. 4, the control unit 110 of the computing device 100 may determine at least one first object excluding at least one background object constituting the space in real space (S161). At least one background object may be, for example, an object related to a wall, ceiling, or floor. At least one first object excluding the background object may be, for example, a chair, a desk, a bed, or an object related to a person.

The control unit 110 may remove at least one first object from the real space displayed in the first area (S162).

The display unit 140 may display the virtual space by overlaying it on the real space from which at least one first object has been removed (S163).

If at least one first object is not removed and is displayed together with the virtual space, the user may feel a sense of discomfort in the augmented reality image. Accordingly, the control unit 110 may remove at least one first object excluding the background object from the real space displayed in the first area and then display the virtual space by overlaying it.

According to some embodiments of the present disclosure, when at least one first object is determined, the computing device 100 determines whether a second object corresponding to the first object exists in the virtual space created based on the first content. You can decide. If a second object exists, the computing device 100 may display the second object by overlaying it on the first object. Hereinafter, an example of a method in which the computing device 100 overlays and displays at least one second object on at least one first object will be described with reference to FIG. 5 .

FIG. 5 is a flowchart illustrating an example of a method in which a computing device displays at least one second object by overlaying at least one first object according to some embodiments of the present disclosure.

Referring to FIG. 5 , the control unit 110 of the computing device 100 may determine at least one first object excluding at least one background object constituting the space in the captured real space (S210). At least one background object may be, for example, an object related to a wall, ceiling, or floor. At least one first object excluding the background object may be, for example, a chair, a desk, a bed, or an object related to a person.

The control unit 110 may determine whether at least one second object corresponding to at least one object 1 exists in the virtual space (S220).

Specifically, the virtual space created based on the first content may include at least one object in addition to the background object constituting the space. For example, the virtual space may include objects such as chairs, beds, or desks. The control unit 110 may determine whether at least one second object corresponding to at least one first object exists in the created virtual space.

For example, the at least one first object may be an object related to a chair. The controller 110 may determine whether at least one second object related to the chair exists in the virtual space.

When at least one second object exists, the display unit 140 may display the at least one second object by overlaying it on a third area where the at least one first object is located in real space (S230).

Specifically, when at least one second object exists, the controller 110 may determine a third area where the at least one first object is located in real space displayed in the first area. The display unit 140 may display at least one second object by overlaying it on the third area.

At least one first object may be an object related to an object actually owned by the user of the computing device 100. Accordingly, when at least one second object existing in a virtual space is displayed by overlaying at least one first object, the interest of the user using the augmented reality service may be increased.

According to some embodiments of the present disclosure, a virtual space and a library created based on the first content may be displayed on the display unit 140. Hereinafter, an example of a user interface displayed on the display unit 140 will be described with reference to FIG. 6.

FIG. 6 is a diagram illustrating an example of a user interface displayed on a display unit according to some embodiments of the present disclosure.

Referring to (a) of FIG. 6, the display unit 140 may include a first area 141 and a second area 142.

The first area 141 may be the entire area of the display unit 140. Depending on the embodiment, the first area 141 may be at least a partial area of the display unit 140. The first area 141 may be an area where a real space photographed through the photographing unit 150 or a virtual space overlaid on the real space is displayed.

The first area 141 may include a second area 142, at least one background object 145, and at least one first object 146. However, it is not limited to this.

The second area 142 may be smaller than the first area 141 . The second area 142 may be at least a portion of the area included in the first area 141 . The second area 142 may be an area where the library 144 is displayed.

The library 144 may include a first library item 1441 and a second library item 1442. The user may select a specific item from among the items included in the library 144 through user interaction with the second area 142. The user can change the displayed library items through sliding interaction on the second area 142. For example, at least a portion of the first library item 1441 and the second library item 1442 may be displayed in the second area 142. The user can cause the second library item 1442 to be displayed through the interaction of sliding the first library item 1441 from a left area to a right area within the second area 142.

The first library item 1441 may include first content 143. The first content 143 displayed in the first library item 1441 may be preview content. Here, the preview content may mean an image that allows the first content 143 to be confirmed even if the entire file related to the first content 143 is not loaded.

At least one background object 145 may be an object related to the background constituting the real space. At least one background object 145 may be an object constituting the floor surface of real space. However, the background object is not limited to this and may include objects constituting the wall or objects constituting the ceiling.

At least one first object 146 may be an object excluding at least one background object 145. At least one first object 146 may be a multi-function printer. However, it is not limited to this and the first object may be a desk, chest of drawers, or chair.

Referring to (b) of FIG. 6, a virtual space may be displayed in the first area 141. The virtual space may include background objects corresponding to at least one background object 145 in the real space. The virtual space may include at least one virtual object 147. Depending on the embodiment, the control unit 110 may determine a specific object excluding at least one background object 145 in the captured real space. For convenience of explanation, it is assumed that the specific object is a bed. The control unit 110 may determine whether at least one second object 147 corresponding to a specific object exists in the virtual space. For example, at least one second object 147 may be a bed. If at least one second object 147 exists, the control unit 110 may overlay and display the at least one second object 147 in the third area where the specific object is located in real space.

According to some embodiments of the present disclosure, when the location where the real space is displayed in the first area 141 changes, the virtual space may be displayed by overlaying the real space by tracking the changed location.

Specifically, the location of the real space displayed in the first area 141 may change as the user operates the computing device 100. For example, a user may operate the computing device 100 to photograph the front wall in real space. The user may operate the computing device 100 so that the right wall is captured from the front wall. Accordingly, the real space displayed in the first area 141 may change from the front wall to the right wall. As the location where the real space is displayed in the first area 141 changes, the controller 110 may track the changed location. The control unit 110 can overlay and display the virtual space at the changed location.

FIG. 7 is a diagram for explaining another example of a user interface displayed on a display unit according to some embodiments of the present disclosure.

Referring to FIG. 7 , a third object created based on the second content 148 may be displayed in the first area 141 of the display unit 140.

Specifically, as the control unit 110 receives the second user input in the second area 142, the third object 1481 is generated based on the second content 148 included in the first library item 1441. can be created. Here, the second user input may be a sliding interaction to change a library item. The second user input may be an input for selecting the second content 148. The second user input may include an input for dragging and dropping the second content from the second area 142 to the first area 141. A drag and drop input may be an input that moves an object to another location by clicking on it in a user interface environment. A user may move an object to an area included in the first area 141 through a drag and drop input. The control unit 110 may display the third object 1481 by overlaying it on the first area 141 according to the second user input.

According to some embodiments of the present disclosure, the control unit 110 may overlay virtual space on real space based on depth information obtained through a depth sensing module. In this case, the size of the third object 1481 displayed in the virtual space may be determined based on depth information.

Specifically, the control unit 110 may display the third object 1481 in virtual space as it receives the second user input. When displaying the third object 1481 in a virtual space, the control unit 110 may determine the size of the third object 1481 based on the location where the third object 1481 is displayed. For example, when displaying the third object 1481 at the first position according to an input from the user, the control unit 110 may determine the size of the third object 1481 as the first size. When displaying the third object 1481 at a second location located further away from the first location according to an input from the user, the control unit 110 sets the size of the third object 1481 to a second location smaller than the first size. It can be decided by size. For example, the first location may be close to the location where the computing device 100 captured the real space, and the second location may be close to the front wall. Accordingly, users can feel perspective in virtual space, and interest can improve when using augmented reality services.

According to some embodiments of the present disclosure, the control unit 110 may share the augmented reality image displayed on the display unit 140 to another computing device of another user through the communication unit 130. Here, sharing the augmented reality image with another computing device of another user may be understood as causing the augmented reality image displayed on the display unit 140 to be displayed on the display unit of the other computing device. The augmented reality image displayed on the display unit 140 may be generated based on content related to an NFT that is confirmed to have ownership by the user of the computing device 100. Accordingly, when another user of another computing device receives an augmented reality image shared from the computing device 100, the other user can trust that the user has ownership of the NFT of contents included in the augmented reality image.

As described above with reference to FIGS. 1 to 7 , the computing device 100 may create a virtual space based on content related to at least one NFT that is confirmed to be owned by the user. The computing device 100 may provide an augmented reality service by overlaying and displaying the created virtual space on the real space displayed on the display unit 140. Accordingly, the user's friends or acquaintances can trust that the augmented reality image displayed on the computing device 100 was implemented using NFT content without a separate verification process. Depending on the embodiment, the computing device 100 may share the augmented reality image or NFT content to another computing device of another user in order to further maximize the above-described effect.

The description of the presented embodiments is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments without departing from the scope of the disclosure. Thus, the present disclosure is not limited to the embodiments presented herein but is to be interpreted in the broadest scope consistent with the principles and novel features presented herein.

Claims (11)

A method for providing an augmented reality service using non-fungible token (NFT) content performed by a computing device including at least one processor,
Photographing a real space according to input from a user for using the augmented reality service;
displaying the captured real space in a first area;
Constructing a library containing at least one content related to at least one NFT confirmed to be owned by the user;
displaying the configured library in a second area;
upon receiving a first user input in the second area, generating a virtual space based on first content included in the library; and
Overlaying and displaying the created virtual space on the real space displayed in the first area;
Including,
The step of overlaying and displaying the virtual space on the real space displayed in the first area includes:
determining at least one first object excluding at least one background object constituting the space in the photographed real space;
determining whether at least one second object corresponding to the at least one first object exists in the virtual space; and
If the at least one second object exists, displaying the at least one second object by overlaying it on a third area where the at least one first object is located in the real space;
Including,
method.
According to claim 1,
The step of configuring a library containing at least one content related to at least one NFT confirmed to be owned by the user,
Accessing at least one of at least one blockchain mainnet or at least one marketplace that provides transaction history information of the blockchain mainnet to determine the at least one NFT;
Accessing the metadata storage location of the at least one NFT to obtain at least one content related to the at least one NFT - the metadata includes the name of at least one content related to the at least one NFT and at least one Contains at least one of the following storage locations for content data -; and
configuring the library including the at least one content;
Including,
method.
According to claim 2,
The step of accessing the metadata storage location of the at least one NFT to obtain at least one content related to the at least one NFT,
Obtaining at least one content corresponding to the at least one NFT based on at least one of the metadata of the at least one NFT or the URL stored in the metadata of the at least one NFT;
Including,
method.
According to claim 1,
The real space is photographed through a depth sensing module,
The step of overlaying and displaying the virtual space on the real space displayed in the first area includes:
Overlaying the virtual space on the real space based on depth information obtained through the depth sensing module;
Including,
method.
According to claim 1,
The step of overlaying and displaying the virtual space on the real space displayed in the first area includes:
determining an area of at least one background object constituting a space in the photographed real space; and
displaying the virtual space based on the determined area of the at least one background object;
Including,
method.
According to claim 1,
The step of overlaying and displaying the virtual space on the real space displayed in the first area includes:
determining at least one first object excluding at least one background object constituting the space in the photographed real space;
removing the at least one first object from the real space displayed in the first area; and
Overlaying and displaying the virtual space on the real space from which the at least one first object has been removed;
Including,
method.
delete According to claim 1,
upon receiving a second user input in the second area, generating a third object based on second content included in the library; and
Overlaying and displaying the third object on the first area;
Containing more,
method.
According to claim 8,
The second user input includes an input for dragging and dropping the second content from the second area to the first area,
method.
According to claim 1,
The virtual space is,
When the location at which the real space is displayed in the first area changes, it is displayed overlaid on the real space in a manner that tracks the changed location.
method.
A computing device for providing augmented reality services,
a photographing unit that photographs a real space according to input from a user for using the augmented reality service;
a display unit that displays the captured real space in a first area; and
Constructing a library including at least one content related to at least one NFT that is confirmed to be owned by the user, displaying the library configured through the display unit in a second area, and first in the second area a control unit that receives a user input and creates a virtual space based on first content included in the library;
Including,
The display unit,
Displaying the virtual space by overlaying it on the real space displayed in the first area,
The control unit,
Determining at least one first object excluding at least one background object constituting the space in the photographed real space,
Determine whether at least one second object corresponding to the at least one first object exists in the virtual space,
When the at least one second object exists, controlling the display unit to overlay and display the at least one second object in a third area where the at least one first object is located in the real space.
Computing device.