KR20240074059A - Method and device for providing spatial information using augmented reality or virtual reality - Google Patents

Abstract

According to one aspect according to the technical spirit of the present disclosure, a method executed by a user device includes receiving a user identifier indicating a user; Outputting spatial information generated by integrating at least one 3D image data, wherein the 3D image data includes a unique spatial identifier; Receiving a location specified in the output spatial information; Obtaining a spatial identifier of 3D image data including a location and coordinates in the 3D image data corresponding to the location; Receiving review data about a spatial identifier of 3D image data including a location and coordinates in the 3D image data corresponding to the location; and transmitting a user identifier, review data, a spatial identifier of the 3D image data including the location, and coordinates in the 3D image data corresponding to the location to the server.

Description

Method and device for providing spatial information using virtual reality {METHOD AND DEVICE FOR PROVIDING SPATIAL INFORMATION USING AUGMENTED REALITY OR VIRTUAL REALITY}

The technical idea of this disclosure relates to a method and device for providing spatial information using virtual reality, and more specifically, review data for spatial information using AR (Augmented Reality)/VR (Virtual Reality). It relates to a method and device for providing.

With the development of the Internet and SNS, we are in the golden age of review services. Six out of 10 people enjoy cultural activities such as visiting popular cafes, restaurants, and pop-up stores at least once every six months, and the criteria for deciding whether to consume culture is reviews from others. In particular, the generation that is accustomed to using SNS has a clear tendency to decide whether to visit or experience something based on ratings and reviews of restaurants, cafes, exhibition spaces, etc.

In various spaces such as conventional restaurants, cafes, exhibition spaces, stores, and lodging establishments, information on users' opinions and evaluations of the space is accumulated through review systems provided by various platforms. However, search engines only sort search results in order of accuracy and time for the search term, and reviews of spaces provided on the platform only provide rough information about the space or user experience, and do not provide objective information about detailed spaces within the space. It does not provide details. In particular, there are not many cases where there is an evaluation of a specific spatial point, so it is often difficult to recognize the experience of the space in advance.

The technical problem that the technical idea of the present disclosure seeks to achieve is that, within content that allows users to view virtual reality for a specific space, users using the virtual reality can obtain spatial information within the space, such as tables, products, windows, and toilets within the space. The aim is to provide a method and device for receiving review data for a location that can be specified, etc.

The technical problems to be achieved by the method and device for providing spatial information using virtual reality according to the technical idea of the present disclosure are not limited to the problems mentioned above, and other problems not mentioned are clear to those skilled in the art from the description below. It will be understandable.

In order to achieve the above object, according to an aspect according to the technical idea of the present disclosure, a method executed by a user device includes the steps of receiving a user identifier indicating a user; Outputting spatial information generated by integrating at least one 3D image data, wherein the 3D image data includes a unique spatial identifier; Receiving a location specified in the output spatial information; Obtaining a spatial identifier of 3D image data including a location and coordinates in the 3D image data corresponding to the location; Receiving review data about a spatial identifier of 3D image data including a location and coordinates in the 3D image data corresponding to the location; A method is disclosed that includes transmitting a user identifier, review data, a spatial identifier of 3D image data including the location, and coordinates in the 3D image data corresponding to the location to a server.

According to an exemplary embodiment, obtaining a spatial identifier of 3D image data corresponding to a location and coordinates in the 3D image data includes extracting an object and an object identifier of the object using depth information of the 3D image data; Obtaining coordinates in the 3D image data corresponding to the location, and determining an object on the 3D image data to which the coordinates in the 3D image data belong; and obtaining a spatial identifier of the 3D image data including the location and an object identifier of the object on the 3D image data to which the coordinates in the 3D image data belong, and the spatial identifier and location of the 3D image data including the location. The step of receiving review data for coordinates in the corresponding 3D image data includes review data for a spatial identifier of the 3D image data including the location and an object identifier of the object on the 3D image data to which the coordinates in the 3D image data belong. Further comprising the step of receiving, and transmitting to the server the user identifier, the spatial identifier of the 3D image data including the review data and the location, and the coordinates in the 3D image data corresponding to the location, the user identifier, the review data, and the 3D location. It may further include transmitting the spatial identifier of the image data and the object identifier of the object on the 3D image data to the server.

According to an exemplary embodiment, the method includes extracting keywords from review data for a spatial identifier of 3D image data containing a location and a coordinate in the 3D image data corresponding to the location; Determining spatial feature keywords of coordinates in the 3D image data corresponding to the location based on at least one of a keyword in the review data, a number of recommendations in the review data, and a rank of a user identifier indicating the user device that transmitted the review data. ; and transmitting spatial feature keywords of coordinates in 3D image data corresponding to the location to the server.

According to an exemplary embodiment, the method further includes receiving location information where the user device is located, review data about a spatial identifier of the 3D image data containing the location and a coordinate in the 3D image data corresponding to the location. In the step of receiving, if the location information where the user device is located matches the location information of the spatial information, receiving review data about the spatial identifier of the 3D image data containing the location and the coordinates in the 3D image data corresponding to the location. Additional steps may be included.

According to an exemplary embodiment, the method includes determining user state information indicating a connection state of a user device; and transmitting user status information to the server, where the user status information may include a connected state of the user device and a non-connected state of the user device.

According to an exemplary embodiment, the method includes, if the user status information determines that the user device is in a connected state, setting it to a standby mode capable of receiving an inquiry message for spatial information from a server or a second user device; When receiving an inquiry message about spatial information from a server or a second user device, outputting an inquiry message about spatial information to the user device; And when a response message to the inquiry message for spatial information is received, it may further include transmitting a response message to the server or the second user device.

According to another aspect according to the technical spirit of the present disclosure, there is a method executed by a server, wherein a user identifier, review data, and coordinates in the 3D image data corresponding to the location and a spatial identifier of the 3D image data including the location are retrieved from the user device. receiving; and transmitting the user identifier, the review data, the spatial identifier of the 3D image data including the location, and the coordinates in the 3D image data corresponding to the location to the review database for storage in the review database.

According to another aspect according to the technical spirit of the present disclosure, there is provided a method executed by a user device, the step of outputting spatial information generated by integrating at least one 3D image data, wherein the 3D image data includes a unique spatial identifier. - ; Receiving a location specified in the output spatial information; Obtaining a spatial identifier of 3D image data including a location and coordinates in the 3D image data corresponding to the location; and outputting review data about the spatial identifier of the 3D image data including the location and the coordinates in the 3D image data corresponding to the location.

According to an exemplary embodiment, obtaining a spatial identifier of 3D image data corresponding to a location and coordinates in the 3D image data includes extracting an object and an object identifier of the object using depth information of the 3D image data; Obtaining coordinates in the 3D image data corresponding to the location, and determining an object on the 3D image data to which the coordinates in the 3D image data belong; and obtaining a spatial identifier of the 3D image data including the location and an object identifier of the object on the 3D image data to which the coordinates in the 3D image data belong, and the spatial identifier and location of the 3D image data including the location. The step of outputting review data for the coordinates in the corresponding 3D image data includes review data for the spatial identifier of the 3D image data including the location and the object identifier of the object on the 3D image data to which the coordinates in the 3D image data belong. An output step may be further included.

According to an exemplary embodiment, the step of outputting review data for the spatial identifier of the 3D image data including the location and the coordinates in the 3D image data corresponding to the location include keywords of the review data, the number of recommendations of the review data, and The method may further include outputting spatial feature keywords of coordinates in the 3D image data corresponding to the location determined based on at least one of the ranks of the user identifier indicating the user device that transmitted the review data.

According to another aspect according to the technical idea of the present disclosure, it includes at least one processor, a memory, an input unit, an output unit, and a communication unit, wherein the processor receives a user identifier indicating a user and sends at least one 3D image data. Controls the output unit to output spatial information generated by integrating - the 3D image data includes a unique spatial identifier -, receives a location specified in the spatial information output from the input unit, and stores the 3D image data including the location. Obtain coordinates in the 3D image data corresponding to the spatial identifier and location, receive review data about the coordinates in the 3D image data corresponding to the spatial identifier and location of the 3D image data including the location from the input unit, and receive a user identifier. Disclosed is a user device that controls a communication unit to transmit review data, a spatial identifier of 3D image data containing a location, and coordinates in the 3D image data corresponding to the location to a server.

According to an exemplary embodiment, the processor extracts an object and an object identifier of the object using depth information of the 3D image data, obtains coordinates in the 3D image data corresponding to the location, and determines that the coordinates in the 3D image data are Determine the object on the 3D image data to which it belongs, obtain the spatial identifier of the 3D image data including the location and the object identifier of the object on the 3D image data to which the coordinates in the 3D image data belong, and retrieve the 3D image data including the location from the input unit. Receive review data about the spatial identifier and the object identifier of the object on the 3D image data to which the coordinates in the 3D image data belong, and receive the user identifier, review data, the spatial identifier of the 3D image data, and the object identifier of the object on the 3D image data. The communication unit can be controlled to transmit to the server.

According to an exemplary embodiment, the processor extracts keywords from the review data for spatial identifiers of the 3D image data containing the location and coordinates in the 3D image data corresponding to the location, and recommends the keywords in the review data and the review data. Determine a spatial feature keyword of the coordinate in the 3D image data corresponding to the location based on at least one of the number, and the rank of the user identifier indicating the user device that sent the review data, and determine a spatial feature keyword in the 3D image data corresponding to the location The communication unit can be controlled to transmit spatial feature keywords of coordinates to the server.

According to an exemplary embodiment, the processor acquires location information where the user device is located, and when the location information where the user device is located matches the location information of the spatial information, a spatial identifier of the 3D image data including the location from the input unit and Review data about coordinates in 3D image data corresponding to the location can be received.

According to an exemplary embodiment, the processor determines user state information indicating the connection state of the user device, and controls the communication unit to transmit the user state information to the server, where the user state information includes the connection state of the user device and the connection state of the user device. It may include a non-connected state.

According to an exemplary embodiment, when the processor determines that the user state information is a connected state of the user device, it sets the processor to a standby mode capable of receiving an inquiry message for spatial information from the server or the second user device, and sets the processor to a standby mode capable of receiving an inquiry message for spatial information from the server or the second user device. When receiving an inquiry message about spatial information from a second user device, the output unit is controlled to output an inquiry message about spatial information to the user device, and when a response message of the inquiry message about spatial information is received, the server or the second user device The communication unit may be controlled to transmit a response message to the user device.

According to another aspect according to the technical idea of the present disclosure, a computer program stored in a computer-readable recording medium coupled with hardware to execute the above-described method is disclosed.

According to another aspect according to the technical idea of the present disclosure, a computer-readable recording medium storing a computer program in combination with hardware and executing the above-described method is disclosed.

A method and device for providing spatial information using virtual reality according to the technical idea of the present disclosure allows users who experience a specific space offline to share their experience with specific locations or objects in that space through review data. Thus, the user experience can be improved.

A method and device for providing spatial information using virtual reality according to the technical idea of the present disclosure can provide spatial feature keywords for detailed locations within a specific space to enable efficient search of the space.

A method and device for providing spatial information using virtual reality according to the technical idea of the present disclosure are provided to users who use virtual reality and offline users of the space through virtual reality content that can utilize review data about locations within a specific space. By enabling communication between users, real-time information about the space can be provided.

The effects that can be obtained by the method and device for providing spatial information using virtual reality according to the technical idea of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned can be obtained from the description below. It will be clearly understandable to those with ordinary knowledge in this technical field.

In order to more fully understand the drawings cited in this disclosure, a brief description of each drawing is provided.
1 is a flowchart of a method for providing review data about spatial information using AR/VR according to an embodiment of the present disclosure.
Figure 2 is a diagram illustrating a spherical coordinate system according to an embodiment of the present disclosure.
Figure 3 is a VR image formed based on a spherical coordinate system according to an embodiment of the present disclosure.
Figure 4 shows spatial information generated based on 3D image data according to an embodiment of the present disclosure.
Figure 5 shows a location specified in spatial information generated based on 3D image data according to an embodiment of the present disclosure.
Figure 6 shows coordinates indicating a location in spatial information according to an embodiment of the present disclosure.
Figure 7 shows review data at a location specified in spatial information according to an embodiment of the present disclosure.
Figure 8 shows a format for storing review data according to an embodiment of the present disclosure.
Figure 9 illustrates a format for storing space information where review data is stored according to an embodiment of the present disclosure.
Figure 10 is a block diagram of a user device that provides review data about spatial information using AR/VR according to an embodiment of the present disclosure.
Figure 11 is a block diagram of a system that provides review data on spatial information using AR/VR according to an embodiment of the present disclosure.

Illustrative embodiments according to the technical idea of the present disclosure are provided to more completely explain the technical idea of the present disclosure to those skilled in the art, and the examples below can be modified into various other forms. may be possible, and the scope of the technical idea of the present disclosure is not limited to the examples below. Rather, these embodiments are provided to make the present disclosure more faithful and complete and to fully convey the technical idea of the present invention to those skilled in the art.

Although the terms first, second, etc. are used in this disclosure to describe various members, regions, layers, portions, and/or components, these members, parts, regions, layers, portions, and/or components are referred to by these terms. It is obvious that it should not be limited by . These terms do not imply any particular order, superiority, inferiority, or superiority or inferiority, and are used only to distinguish one member, region, region, or component from another member, region, region, or component. Accordingly, the first member, region, portion, or component described in detail below may refer to the second member, region, portion, or component without departing from the teachings of the technical idea of the present disclosure. For example, a first component may be referred to as a second component without departing from the scope of the present disclosure, and similarly, the second component may also be referred to as a first component.

Unless otherwise defined, all terms used herein, including technical terms and scientific terms, have the same meaning as commonly understood by those skilled in the art in the technical field to which the concept of the present disclosure pertains. Additionally, commonly used terms, as defined in dictionaries, should be interpreted to have meanings consistent with what they mean in the context of the relevant technology, and should not be used in an overly formal sense unless explicitly defined herein. It should not be interpreted.

In cases where an embodiment can be implemented differently, a specific process sequence may be performed differently from the described sequence. For example, two processes described in succession may be performed substantially at the same time, or may be performed in an order opposite to the order in which they are described.

In the accompanying drawings, variations of the depicted shape may be expected, depending, for example, on manufacturing techniques and/or tolerances. Therefore, embodiments based on the technical spirit of the present disclosure should not be construed as being limited to the specific shape of the area shown in the present disclosure, but should include, for example, changes in shape that occur during the manufacturing process. The same reference numerals are used for the same components in the drawings, and duplicate descriptions thereof are omitted.

As used herein, the term 'and/or' includes each and every combination of one or more of the mentioned elements.

Hereinafter, embodiments based on the technical idea of the present disclosure will be described in detail with reference to the attached drawings.

1 is a flowchart of a method for providing review data about spatial information using AR/VR according to an embodiment of the present disclosure.

Referring to FIG. 1, the method executed by the user device of the present disclosure includes receiving a user identifier indicating a user (S110), and outputting spatial information generated by integrating at least one 3D image data (S110). S120), where the 3D image data includes a unique spatial identifier, and receiving a location specified in the output spatial information (S130), 3D image data corresponding to the spatial identifier and location of the 3D image data including the location Obtaining the coordinates (S140), receiving review data about the spatial identifier of the 3D image data including the location and the coordinates in the 3D image data corresponding to the location (S150), user identifier, review data, It includes the step of transmitting the spatial identifier of the 3D image data including the location and the coordinates in the 3D image data corresponding to the location (S160).

Specifically, the user device of the present disclosure may receive a user identifier that indicates the user. For example, the user identifier corresponding to the user device may be a unique device identifier of the user device. As another example, the user identifier may be a user ID assigned to the user.

The user device of the present disclosure can output spatial information generated by integrating at least one 3D image data. As an example, the user device may capture at least one 3D image data and generate spatial information by integrating the at least one captured 3D image data. Alternatively, the user device may receive at least one 3D image data from a server and generate spatial information by integrating the at least one received 3D image data. As another example, the user device may receive spatial information generated by integrating at least one 3D image data from a server and output it to the user device. Here, 3D image data may include a unique spatial identifier.

And the user device can receive a location specified in the output spatial information. Specifically, the user device may receive a location specified in spatial information output through the input unit. The input unit of the user device is not limited thereto, but may include, for example, a touch screen, a keypad, a proximity sensor, and an illumination sensor. For example, the user device can convert changes in pressure applied to a specific part of the display or capacitance occurring in a specific part of the display into an electrical input signal and receive it.

When the user device receives a location specified in the output spatial information, the user device can obtain the spatial identifier of the 3D image data including the location and the coordinates in the 3D image data corresponding to the location. As an example, the user device may obtain coordinates in 3D image data corresponding to the location using a spherical coordinate system.

FIG. 2 is a diagram showing a spherical coordinate system according to an embodiment of the present disclosure, and FIG. 3 is a VR image formed based on a spherical coordinate system according to an embodiment of the present disclosure.

Referring to Figures 2 and 3, in the spherical coordinate system, for the origin O and an arbitrary point P (210) on the coordinate space, the point P is orthogonally projected onto the xy plane is called Q, and the origin O and When the distance between points P is OP = ρ, the size of the angle that line segment OQ makes with the positive direction of the x-axis is θ, and the size of the angle that line segment OP makes with the positive direction of the z-axis is Φ, point P (210 ) refers to a coordinate system that represents the coordinates of P(ρ, θ, Φ).

The user device of the present disclosure can obtain coordinates corresponding to a specified location using altitude and azimuth from a 3D image. Furthermore, the user device can obtain a spatial identifier of 3D image data including the location.

And the user device of the present disclosure can receive review data about the spatial identifier of the 3D image data including the location and the coordinates in the 3D image data corresponding to the location. In one embodiment, when the user device receives a location specified in the output spatial information, the user device outputs a status window through which review data can be entered at the location, and receives review data input into the status window. there is.

The user device of the present disclosure may transmit a user identifier, review data, a spatial identifier of 3D image data including the location, and coordinates in the 3D image data corresponding to the location to the server. Furthermore, the user device may store the user identifier, review data, spatial identifier of the 3D image data including the location, and coordinates in the 3D image data corresponding to the location in the memory of the user device.

A method of providing spatial information using virtual reality according to an embodiment of the present disclosure allows the user who experiences a specific space offline to share the experience of the specific location or object of the space through review data, It can improve user experience.

According to an exemplary embodiment, the user device of the present disclosure can extract an object and its object identifier using depth information of 3D image data. Here, the object identifier is information that distinguishes an object extracted using depth information of 3D image data, and may include characteristics of the object. Additionally, the user device can obtain coordinates in the 3D image data corresponding to the location and determine the object on the 3D image data to which the coordinates in the 3D image data belong. Specifically, the user device may store a set of coordinates of an object on 3D image data, and determine the object on the 3D image data to which the coordinates in the 3D image data belong based on the set of coordinates of the stored object. In addition, the user device can obtain the spatial identifier of the 3D image data including the location and the object identifier of the object on the 3D image data to which the coordinates in the 3D image data belong.

Furthermore, the user device of the present disclosure can receive review data about the spatial identifier of the 3D image data including the location and the object identifier of the object on the 3D image data to which the coordinates in the 3D image data belong. And the user device may transmit the user identifier, review data, spatial identifier of the 3D image data, and object identifier of the object on the 3D image data to the server.

According to an exemplary embodiment, the method of the present disclosure includes extracting keywords from review data for spatial identifiers of 3D image data containing a location and coordinates in the 3D image data corresponding to the location, keywords in the review data, determining spatial feature keywords of coordinates in the 3D image data corresponding to the location based on at least one of the number of recommendations in the review data and the rank of the user identifier indicating the user device that sent the review data, and corresponding to the location. The method may further include transmitting spatial feature keywords of coordinates in the 3D image data to the server.

Specifically, the user device of the present disclosure can extract keywords from review data about the spatial identifier of 3D image data including the location and the coordinates in the 3D image data corresponding to the location. As an example, the user device may extract a morpheme that appears more than a predetermined frequency as a component in the review data, calculate an evaluation value of the component according to a predetermined evaluation standard, and extract a keyword of the review data.

And the user device of the present disclosure is configured to determine coordinates in the 3D image data corresponding to the location based on at least one of the keywords of the review data, the number of recommendations of the review data, and the rank of the user identifier that indicates the user device that transmitted the review data. Spatial feature keywords can be determined. In one example, the user device acquires one or more review data for coordinates corresponding to the location, extracts the title of the review data, generates word vectors of the review data, measures similarity between word vectors, and clusters according to similarity. You can determine spatial feature keywords of coordinates by generating . And the user device of the present disclosure can transmit spatial feature keywords of coordinates in 3D image data corresponding to the location to the server.

A method of providing spatial information using virtual reality according to an embodiment of the present disclosure can enable efficient search of space by providing spatial feature keywords for detailed locations within a specific space.

According to an exemplary embodiment, the method of the present disclosure may further include receiving location information where the user device is located. In this case, if the location information where the user device is located matches the location information of the spatial information, the user device receives review data about the spatial identifier of the 3D image data containing the location and the coordinates in the 3D image data corresponding to the location. can do.

According to an exemplary embodiment, the user device of the present disclosure may determine user state information indicating the connection state of the user device and transmit the user state information to the server. Here, the user status information may include a connected state of the user device and a non-connected state of the user device. Accordingly, when the user status information is in the connected state, information about the space can be provided in real time to the user device connected to the space where the user device is located through the virtual environment by communicating with the user device occupying the actual space.

According to an exemplary embodiment, the method of the present disclosure is to set the user device to a standby mode to receive an inquiry message for spatial information from a server or a second user device when the user status information determines that the user device is connected. step; When receiving an inquiry message about spatial information from a server or a second user device, outputting an inquiry message about spatial information to the user device; And when a response message to the inquiry message for spatial information is received, it may further include transmitting a response message to the server or the second user device.

Accordingly, the method of providing spatial information using virtual reality according to an embodiment of the present disclosure is to share the space with a user using virtual reality offline through virtual reality content that can utilize review data about a location in a specific space. By enabling communication between users, real-time information about the space can be provided.

FIG. 4 illustrates spatial information generated based on 3D image data according to an embodiment of the present disclosure, and FIG. 5 illustrates a location specified in spatial information generated based on 3D image data according to an embodiment of the present disclosure. shows.

Referring to FIG. 4, the user device can output spatial information generated based on 3D image data. Specifically, the user device may output spatial information generated by integrating at least one 3D image data.

Here, the spatial information may include review data stored for coordinates in 3D image data corresponding to a predetermined location. If there is review data of coordinates in the 3D image data corresponding to a predetermined position in the spatial information, the user device may output the review data to the corresponding coordinates in the 3D image data. At this time, the user device may output the user identifier of the user device that created the review data together with the review data. In Figure 4, review data 410, 420, and 430 on predetermined coordinates output to spatial information are displayed.

Referring to FIG. 5 , the user device may receive a location 510 specified on the spatial information output in FIG. 4 . Specifically, the user device may receive a location specified in spatial information output through the input unit. For example, the user device can convert changes in pressure applied to a specific part of the display or capacitance occurring in a specific part of the display into an electrical input signal and receive it.

When the user device receives a location specified in the output spatial information, the user device can obtain the spatial identifier of the 3D image data including the location and the coordinates in the 3D image data corresponding to the location. As an example, the user device may obtain coordinates in 3D image data corresponding to the location using a spherical coordinate system.

Figure 6 shows coordinates indicating a location in spatial information according to an embodiment of the present disclosure. Referring to FIG. 6, when the user device receives review data for a predetermined location in spatial information, the user device may record the coordinates at that location in the spatial information. Coordinates in spatial information received by the user device may be accumulated and stored.

Figure 7 shows review data at a location specified in spatial information according to an embodiment of the present disclosure. Referring to FIG. 7, the user device may display spatial information including review data of coordinates corresponding to a specific location. At this time, the user device may output the review data along with the user identifier of the user device that created the review data.

As an example, when review data for a specific location on spatial information is displayed, a user different from the user who wrote the review data may additionally store a comment on the review data. Specifically, the user device may receive comments on the review data at the bottom of the displayed review data.

Figure 8 shows a format for storing review data according to an embodiment of the present disclosure. Referring to FIG. 8, text is review data, type is the type of location where the review data was input, type id is a spatial identifier of the 3D image data at the location where the review data was input, and pano_long/lat is the location corresponding to the location. These are coordinates in 3D image data, and userid is a user identifier corresponding to the user who wrote the review data. And id is a random identifier assigned by the user device, createdAt indicates the time the review data was created, and updatedAt indicates the time the review data was modified. At this time, id, createdAt, and updatedAt are values automatically assigned by the user device.

When the user device receives review data for a specific location within the spatial information, the user identifier corresponding to the user who created the review data, the review data, the type of location where the review data was entered, and a 3D image of the location where the review data was entered. Coordinates in 3D image data corresponding to the spatial identifier and location of the data can be stored.

As another example, when the user device receives review data for a specific location within spatial information, the user device may include a user identifier corresponding to the user who created the review data, the review data, the type of location where the review data was entered, and the location where the review data was entered. The spatial identifier of the 3D image data of the location and the coordinates in the 3D image data corresponding to the location can be transmitted to the server.

Here, the type of location where review data is input may include an object located within spatial information or a background within spatial information.

Figure 9 illustrates a format for storing space information where review data is stored according to an embodiment of the present disclosure. Referring to FIG. 9, link is a link where 3D image data is stored, space_id is a space identifier indicated by spatial information, and x, y, and floor represent the location of the spatial information. And id is a random identifier assigned by the user device, createdAt indicates the time the review data was created, and updatedAt indicates the time the review data was modified. At this time, id, createdAt, and updatedAt are values automatically assigned by the user device.

FIG. 10 is a block diagram of a user device 100 that provides review data about spatial information using AR/VR according to an embodiment of the present disclosure.

The user device 100 of the present disclosure includes at least one processor 110, a memory 120, an input unit 130, an output unit 130, and a communication unit 150, and the processor 110 allows the user to Receives an indicating user identifier, and controls the output unit to output spatial information generated by integrating at least one 3D image data, where the 3D image data includes a unique spatial identifier, and the space output from the input unit 130 Receive a location specified in the information, obtain a spatial identifier of the 3D image data including the location and coordinates in the 3D image data corresponding to the location, and obtain a spatial identifier of the 3D image data including the location from the input unit 130 and To receive review data about the coordinates in the 3D image data corresponding to the location, and to transmit to the server a user identifier, review data, a spatial identifier of the 3D image data containing the location, and coordinates in the 3D image data corresponding to the location. The communication unit 115 can be controlled.

The user device 100 of the present disclosure includes a communication capable desktop computer, laptop computer, laptop, smart phone, tablet PC, and mobile phone. , smart watch, smart glass, e-book reader, PMP (portable multimedia player), portable game console, navigation device, digital camera, DMB (digital multimedia broadcasting) player. , a digital audio recorder, a digital audio player, a digital video recorder, a digital video player, and a PDA (Personal Digital Assistant).

Specifically, the processor 110 of the user device 100 of the present disclosure may receive a user identifier indicating the user. For example, the user identifier corresponding to the user device may be a unique device identifier of the user device. As another example, the user identifier may be a user ID assigned to the user.

The processor 110 of the present disclosure may control the output unit 140 to output spatial information generated by integrating at least one 3D image data. As an example, the user device 100 may generate spatial information by capturing at least one 3D image data and integrating the at least one captured 3D image data. Alternatively, the user device 100 may receive at least one 3D image data from a server and generate spatial information by integrating the at least one received 3D image data. As another example, the user device may receive spatial information generated by integrating at least one 3D image data from a server and output it to the user device. Here, 3D image data may include a unique spatial identifier.

And the user device 100 can receive a location specified in the output spatial information. Specifically, the processor 110 may receive a location specified in spatial information output through the input unit 130. The input unit 130 is not limited to this, but may include, for example, a touch screen, a keypad, a proximity sensor, and an illumination sensor. The user device 100 may convert, for example, a change in pressure applied to a specific part of the display or a change in capacitance occurring in a specific part of the display into an electrical input signal and receive it.

When the processor 110 receives a location specified in the output spatial information, the processor 110 may obtain the spatial identifier of the 3D image data including the location and the coordinates in the 3D image data corresponding to the location. As an example, the processor 110 may obtain coordinates in 3D image data corresponding to a location using a spherical coordinate system.

The processor 110 of the present disclosure can obtain coordinates corresponding to a specified location using altitude and azimuth from a 3D image. Furthermore, the processor 110 may obtain a spatial identifier of 3D image data including the corresponding location.

In addition, the processor 110 of the present disclosure may receive review data about the spatial identifier of the 3D image data including the location and the coordinates in the 3D image data corresponding to the location from the input unit 130. In one embodiment, when the user device 100 receives a location specified in the output spatial information, the user device 100 outputs a status window through which review data can be entered at the location, and Review data can be received.

The processor 110 of the present disclosure may control the communication unit 150 to transmit coordinates in the 3D image data corresponding to the spatial identifier and location of the 3D image data including the user identifier, review data, and location to the server. Furthermore, the processor 110 may store the spatial identifier of the 3D image data including the user identifier, review data, and location, and the coordinates in the 3D image data corresponding to the location in the memory 120 of the user device 100.

The user device 100, which provides spatial information using virtual reality according to an embodiment of the present disclosure, allows the user who experiences a specific space offline to share the experience of the specific location or object of the space through review data. By doing so, you can improve the user experience.

According to an exemplary embodiment, the processor 110 may extract an object and its object identifier using depth information of 3D image data. Here, the object identifier is information that distinguishes an object extracted using depth information of 3D image data, and may include characteristics of the object. Additionally, the processor 110 may obtain coordinates in the 3D image data corresponding to the location and determine the object on the 3D image data to which the coordinates in the 3D image data belong. Specifically, the user device may store a set of coordinates of an object on 3D image data, and determine the object on the 3D image data to which the coordinates in the 3D image data belong based on the set of coordinates of the stored object. The processor 110 may obtain the spatial identifier of the 3D image data including the location and the object identifier of the object on the 3D image data to which the coordinates in the 3D image data belong.

The processor 110 receives review data about the spatial identifier of the 3D image data including the location and the object identifier of the object on the 3D image data to which the coordinates in the 3D image data belong from the input unit 130, a user identifier, and review data. , the communication unit can be controlled to transmit the spatial identifier of the 3D image data and the object identifier of the object on the 3D image data to the server.

According to an exemplary embodiment, the processor 110 extracts keywords from review data for spatial identifiers of 3D image data containing a location and coordinates in the 3D image data corresponding to the location, keywords in the review data, and reviews. Determine spatial feature keywords of coordinates in the 3D image data corresponding to the location based on at least one of the number of recommendations of the data, and the rank of the user identifier indicating the user device that sent the review data, and the 3D image corresponding to the location The communication unit may be controlled to transmit spatial characteristic keywords of coordinates in the data to the server.

Specifically, the processor 110 of the present disclosure may extract keywords from review data about the spatial identifier of 3D image data including the location and the coordinates in the 3D image data corresponding to the location. As an example, the processor 110 extracts morphemes that appear more than a predetermined frequency from the review data as components, calculates the evaluation value of the component according to a predetermined evaluation standard, and extracts keywords of the review data. .

And the processor 110 of the present disclosure is configured to extract information from the 3D image data corresponding to the location based on at least one of the keywords of the review data, the number of recommendations of the review data, and the rank of the user identifier indicating the user device that transmitted the review data. Spatial feature keywords of coordinates can be determined. As an example, the processor 110 acquires one or more review data for coordinates corresponding to the location, extracts the title of the review data, generates word vectors of the review data, measures the similarity between the word vectors, and calculates the similarity. By creating a cluster according to the coordinates, spatial feature keywords can be determined. And the processor 110 of the present disclosure may control the communication unit to transmit a spatial feature keyword of coordinates in 3D image data corresponding to the location to the server.

The user device 100, which provides spatial information using virtual reality according to an embodiment of the present disclosure, can provide spatial feature keywords for detailed locations within a specific space to enable efficient search of the space.

According to an exemplary embodiment, the processor 110 obtains location information where the user device is located, and when the location information where the user device is located matches the location information of the spatial information, the processor 110 obtains 3D image data including the location from the input unit. Review data about coordinates in 3D image data corresponding to a spatial identifier and location may be received.

According to an exemplary embodiment, the processor 110 determines user state information indicating the connection state of the user device and controls the communication unit to transmit the user state information to the server, and the user state information includes the connection state and the connection state of the user device. It may include a non-connected state of the user device. Accordingly, when the user status information is in the connected state, information about the space can be provided in real time to the user device connected to the space where the user device is located through the virtual environment by communicating with the user device occupying the actual space.

According to an exemplary embodiment, when the processor 110 determines that the user status information is a connected state of the user device, it sets the processor 110 to a standby mode capable of receiving an inquiry message for spatial information from a server or a second user device; , When receiving an inquiry message about spatial information from a server or a second user device, control the output unit to output an inquiry message about spatial information to the user device, and when a response message of the inquiry message about spatial information is received, the server Alternatively, the communication unit may be controlled to transmit a response message to the second user device.

Accordingly, the user device 100, which provides spatial information using virtual reality according to an embodiment of the present disclosure, provides offline communication with users using virtual reality through virtual reality content that can utilize review data about locations within a specific space. By enabling communication between users using the space, real-time information about the space can be provided.

FIG. 11 is a block diagram of a system 1100 that provides review data for spatial information using AR/VR according to an embodiment of the present disclosure. A system 1100 that provides review data about spatial information using AR/VR includes one or more first user devices 1110, one or more second user devices 1120, and one or more servers 1130, One or more first user devices 1110, one or more second user devices 1120, and one or more servers 1130 are connected via a network 1140.

For more specific components and operating processes of one or more first user devices 1110, refer to the above-described method of FIGS. 1 to 9 and the description of the user device 100 of FIG. 10.

One or more first user devices 1110 and one or more second user devices 1120 may include all types of devices including an output unit and a communication unit capable of displaying or outputting information received from the server 1130. there is. For example, one or more first user devices 1110 and one or more second user devices 1120 may include a smartphone, tablet PC, PC, smart TV, mobile phone, personal digital assistant (PDA), laptop, media player, micro It may include servers, global positioning system (GPS) devices, digital broadcasting terminals, navigation, and other mobile or non-mobile computing devices. Additionally, the one or more first user devices 1110 and one or more second user devices 1120 may include wearable devices such as watches, glasses, hair bands, and rings with output functions and communication functions.

Meanwhile, for convenience of explanation, only one first user device 1110 and one second user device 1120 among the one or more first user devices 1110 and one or more second user devices 1120 are shown in FIG. 11 . Although shown, the number of terminals may vary depending on the embodiment. As long as the processing capacity of the server 1130 allows, the number of terminals is not particularly limited.

As shown in FIG. 11 , one or more first user devices 1110, one or more second user devices 1120, and one or more servers 1130 may be connected through a network 1140. According to one embodiment, the network 1140 refers to a connection structure that allows information exchange between nodes such as a plurality of terminals and servers. An example of such a network 1140 includes a LAN (Local Area Network), MAN (Metropolitan Area Network), GSM (Global System for Mobile Network), EDGE (Enhanced Data GSM Environment), HSDPA (High Speed Downlink Packet Access), W-CDMA (Wideband Code Division Multiple Access), CDMA (Code Division Multiple Access) ), TDMA (Time Division Multiple Access), Bluetooth, Zigbee, Wi-Fi, VoIP (Voice over Internet Protocol), LTE Advanced, IEEE802.16m, WirelessMAN-Advanced, HSPA+, 3GPP Long Term Evolution (LTE), Mobile WiMAX (IEEE 802.16e), UMB (formerly EV-DO Rev. C), Flash-OFDM, iBurst and MBWA (IEEE 802.20) systems, HIPERMAN, Beam-Division Multiple Access (BDMA) , Wi-MAX (World Interoperability for Microwave Access), and 5G, various types of wired or wireless networks may be used, but are not limited to these.

The server 1130 shown in FIG. 11 receives coordinates in the 3D image data corresponding to the spatial identifier and location of the 3D image data including the user identifier, review data, and location from the user device, and receives the user identifier and review data. A method may be performed that includes transmitting data, a spatial identifier of the 3D image data including the location, and coordinates in the 3D image data corresponding to the location to a review database for storage in the review database.

The second user device 1120 shown in FIG. 11 outputs spatial information generated by integrating at least one 3D image data, where the 3D image data includes a unique spatial identifier and a specific spatial information in the output spatial information. receiving a location, obtaining a spatial identifier of the 3D image data containing the location and coordinates in the 3D image data corresponding to the location, and obtaining a 3D coordinate in the 3D image data corresponding to the location and a spatial identifier of the 3D image data containing the location. A method comprising outputting review data for coordinates in image data is executable.

According to an exemplary embodiment, the second user device 1120 includes extracting an object and an object identifier of the object using depth information of 3D image data; Obtain the coordinates in the 3D image data corresponding to the location, determine the object on the 3D image data to which the coordinates in the 3D image data belong, and determine the spatial identifier of the 3D image data containing the location and the object to which the coordinates in the 3D image data belong. The object identifier of an object in 3D image data can be obtained. And the second user device 1120 may output review data about the spatial identifier of the 3D image data including the location and the object identifier of the object on the 3D image data to which the coordinates in the 3D image data belong.

According to an example embodiment, the second user device 1120 is located at a location determined based on at least one of a keyword in the review data, a number of recommendations in the review data, and a rating of a user identifier that indicates the user device that sent the review data. Spatial feature keywords of coordinates in the corresponding 3D image data can be output.

The description of the above-described embodiments is merely an example with reference to the drawings for a more thorough understanding of the present disclosure, and should not be construed as limiting the technical idea of the present disclosure.

In addition, it will be clear to those skilled in the art to which this disclosure pertains that various changes and modifications can be made without departing from the basic principles of the present disclosure.

Claims (18)

A method executed by a user device, comprising:
Receiving a user identifier indicating a user;
outputting spatial information generated by integrating at least one 3D image data, wherein the 3D image data includes a unique spatial identifier;
Receiving a location specified in the output spatial information;
Obtaining a spatial identifier of 3D image data including the location and coordinates in the 3D image data corresponding to the location;
Receiving review data about a spatial identifier of 3D image data including the location and coordinates in the 3D image data corresponding to the location;
Transmitting the user identifier, the review data, a spatial identifier of 3D image data including the location, and coordinates in the 3D image data corresponding to the location to a server.
How to include .
According to claim 1,
Obtaining the spatial identifier of the 3D image data corresponding to the location and the coordinates in the 3D image data,
extracting an object and an object identifier of the object using depth information of the 3D image data;
Obtaining coordinates in 3D image data corresponding to the location, and determining an object on the 3D image data to which the coordinates in the 3D image data belong; and
Obtaining a spatial identifier of the 3D image data including the location and an object identifier of an object on the 3D image data to which the coordinates in the 3D image data belong.
It further includes,
Receiving review data about the spatial identifier of the 3D image data including the location and the coordinates in the 3D image data corresponding to the location,
Receiving review data about the spatial identifier of the 3D image data including the location and the object identifier of the object on the 3D image data to which the coordinates in the 3D image data belong.
It further includes,
The step of transmitting the user identifier, the spatial identifier of the 3D image data including the review data and the location, and the coordinates in the 3D image data corresponding to the location to the server,
Transmitting the user identifier, the review data, the spatial identifier of the 3D image data, and the object identifier of the object on the 3D image data to a server.
Containing more,
method.
According to claim 1,
The above method is,
extracting keywords from review data for a spatial identifier of 3D image data including the location and coordinates in the 3D image data corresponding to the location;
Spatial feature keywords of coordinates in 3D image data corresponding to the location based on at least one of a keyword of the review data, a number of recommendations of the review data, and a rank of a user identifier indicating a user device that transmitted the review data determining; and
Transmitting spatial feature keywords of coordinates in 3D image data corresponding to the location to the server.
How to include more.
According to claim 1,
The method is:
Receiving location information where the user device is located
It further includes,
Receiving review data about the spatial identifier of the 3D image data including the location and the coordinates in the 3D image data corresponding to the location,
If the location information where the user device is located matches the location information of the spatial information, receiving review data about the spatial identifier of the 3D image data including the location and the coordinates in the 3D image data corresponding to the location.
Containing more,
method.
According to claim 1,
The above method is,
determining user state information indicating a connection state of the user device; and
Transmitting the user status information to the server
It further includes,
The user status information includes a connected state of the user device and a disconnected state of the user device,
method.
According to clause 5,
The above method is,
If the user status information determines that the user device is in a connected state, setting it to a standby mode capable of receiving an inquiry message about the space information from the server or a second user device;
When receiving an inquiry message about the spatial information from the server or the second user device, outputting an inquiry message about the spatial information to the user device; and
When a response message to the inquiry message about the spatial information is received, transmitting the response message to the server or the second user device.
How to further include .
A method executed by a server, comprising:
Receiving from a user device the user identifier, the review data, a spatial identifier of 3D image data including the location, and coordinates in the 3D image data corresponding to the location; and
Transmitting the user identifier, the review data, the spatial identifier of the 3D image data including the location, and the coordinates in the 3D image data corresponding to the location to the review database for storage in the review database.
How to include .
A method executed by a user device, comprising:
outputting spatial information generated by integrating at least one 3D image data, wherein the 3D image data includes a unique spatial identifier;
Receiving a location specified in the output spatial information;
Obtaining a spatial identifier of 3D image data including the location and coordinates in the 3D image data corresponding to the location; and
Outputting review data about the spatial identifier of the 3D image data including the location and the coordinates in the 3D image data corresponding to the location.
How to include .
According to clause 8,
Obtaining the spatial identifier of the 3D image data corresponding to the location and the coordinates in the 3D image data,
extracting an object and an object identifier of the object using depth information of the 3D image data;
Obtaining coordinates in 3D image data corresponding to the location, and determining an object on the 3D image data to which the coordinates in the 3D image data belong; and
Obtaining a spatial identifier of the 3D image data including the location and an object identifier of an object on the 3D image data to which the coordinates in the 3D image data belong.
It further includes,
The step of outputting review data about the spatial identifier of the 3D image data including the location and the coordinates in the 3D image data corresponding to the location,
Outputting review data for the spatial identifier of the 3D image data including the location and the object identifier of the object on the 3D image data to which the coordinates in the 3D image data belong.
Containing more,
method.
According to clause 8,
The step of outputting review data about the spatial identifier of the 3D image data including the location and the coordinates in the 3D image data corresponding to the location,
Spatial features of coordinates in 3D image data corresponding to the location determined based on at least one of a keyword of the review data, a number of recommendations of the review data, and a rank of a user identifier indicating a user device that transmitted the review data Steps to output keywords
Containing more,
method.
at least one processor;
Memory;
input unit;
output unit; and
Ministry of Communications
Including,
The processor,
Receive a user identifier indicating the user,
Controlling the output unit to output spatial information generated by integrating at least one 3D image data, wherein the 3D image data includes a unique spatial identifier.
Receiving a location specified in the output spatial information from the input unit,
Obtaining a spatial identifier of 3D image data including the location and coordinates in the 3D image data corresponding to the location,
Receive review data about a spatial identifier of 3D image data including the location and coordinates in the 3D image data corresponding to the location from the input unit,
Controlling the communication unit to transmit the user identifier, the review data, a spatial identifier of 3D image data including the location, and coordinates in the 3D image data corresponding to the location to a server,
User device.
According to claim 11,
The processor,
Extracting an object and an object identifier of the object using depth information of the 3D image data,
Obtaining coordinates in the 3D image data corresponding to the location, determining an object on the 3D image data to which the coordinates in the 3D image data belong,
Obtaining a spatial identifier of the 3D image data including the location and an object identifier of an object on the 3D image data to which the coordinates in the 3D image data belong,
Receiving review data about a spatial identifier of the 3D image data including the location and an object identifier of an object on the 3D image data to which the coordinates in the 3D image data belong from the input unit,
Controlling the communication unit to transmit the user identifier, the review data, the spatial identifier of the 3D image data, and the object identifier of the object on the 3D image data to a server,
User device.
According to claim 11,
The processor,
Extract keywords from review data for spatial identifiers of 3D image data containing the location and coordinates in the 3D image data corresponding to the location,
Spatial feature keywords of coordinates in 3D image data corresponding to the location based on at least one of a keyword of the review data, a number of recommendations of the review data, and a rank of a user identifier indicating a user device that transmitted the review data Decide,
Controlling the communication unit to transmit a spatial feature keyword of coordinates in 3D image data corresponding to the location to the server,
User device.
According to claim 11,
The processor,
Obtain location information where the user device is located,
If the location information where the user device is located matches the location information of the spatial information, review data about the spatial identifier of the 3D image data including the location and the coordinates in the 3D image data corresponding to the location are received from the input unit. receiving,
User device.
According to claim 11,
The processor,
Determine user state information indicating a connection state of the user device,
Controlling the communication unit to transmit the user status information to the server,
The user status information includes a connected state of the user device and a disconnected state of the user device,
User device.
According to claim 15,
The processor,
If the user status information determines that the user device is in a connected state, set to a standby mode capable of receiving an inquiry message about the spatial information from the server or a second user device,
When receiving an inquiry message about the spatial information from the server or the second user device, control the output unit to output an inquiry message about the spatial information to the user device,
When a response message to the inquiry message for the spatial information is received, controlling the communication unit to transmit the response message to the server or the second user device,
User device.
A computer program combined with hardware and stored in a computer-readable recording medium to execute the method of claim 1.
A computer-readable recording medium storing a computer program in combination with hardware to execute the method of claim 1.