KR20190036868A - System for providing of augmented reality in indoor space using visible light communication and the method thereof - Google Patents

Abstract

The present invention relates to an indoor space augmented reality system using visible light communication for providing key frame information and spatial information in a specific lighting zone by using visible light communication (Li-Fi) and providing an augmented reality experience in the lighting zone through a mobile device of a user and a method thereof. A position of the user and a posture of the mobile device can be accurately estimated as well as an indoor space can accurately be recovered.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a system and method for providing an augmented reality in an indoor space using visible light communication,

The present invention relates to a system and method for providing an augmented reality in an indoor space using visible light communication, and more particularly, to a system and method for providing an augmented reality in an indoor space using visible light communication, and more particularly, To a technique for providing an augmented reality experience in a lighting zone via a mobile device.

In order to apply the Augmented Reality (AR) technique, the relationship (position or attitude) between the user's position and the position of the object to be augmented must be acquired in real time. To solve this problem, early augmented reality techniques use markers.

Marker-based augmented reality technology quickly and simply acquires the user's position and posture based on the marker. However, markless augmented reality technology has become popular due to the development of feature point technologies such as Scale Invariant Feature Transform (SIFT), FAST, ORB (Oriented FAST, Rotated BRIET)

Simultaneous Localization and Mapping (SLAM) technology has been introduced to digitally reconstruct the indoor space with the development of this technology. With the emergence of Microsoft's Kinect, 3D reconstruction such as Kinect Fusion (3D Reconstruction) technology has been developed.

In order to apply the augmented reality in the indoor space, the interior space needs to be reproduced as an augmented space. For this purpose, SLAM technology using RGB images and 3D reconstruction techniques such as Kinect Fusion have been actively studied.

In order to effectively utilize the techniques for understanding the indoor space in the augmented reality, it is necessary to maintain the spatial information on the indoor space and to store the location of the device (smart phone, head mounted display (HMD, etc.) The posture must be accurately estimated in real time through spatial information.

However, as in the existing augmented reality technology, if SLAM is used, there is a disadvantage that a large number of key frames must be held in proportion to the size of the indoor space. For example, in order to acquire the position of the user, the obtained image is individually compared with all the key frames acquired in the SLAM to acquire the closest key frame, and then the relationship of the device is estimated inversely from the corresponding key frame. In this case, since the number of key frames increases as the indoor space becomes wider, the number of searches increases. In order to overcome this limitation, the existing augmented reality technology minimizes the number of times by using multi-dimensional search (KD tree). However, when key frames of similar images are generated, they are placed in the same search space, There is a limit to the increase in

In addition, a 3D reconstruction method such as Kinect Fusion uses a 3D model generated from a 3D reconstruction algorithm and a 3D point obtained in a current frame using an ICP (Iterative Closest Points) algorithm on a depth image obtained from an RGBD camera , And the posture is estimated by minimizing the sum of errors between points corresponding to the model. However, existing 3D reconstruction methods have a limitation that the accuracy of estimation is low as the model size increases.

In addition, existing 3D reconstruction techniques can not distinguish whether the acquired 3D points are a structure of an indoor space, a fixed object or a dynamic object (non-fixed object), and express data using a voxel, There is a disadvantage that it is inefficient because it requires a large amount of data.

Korean Patent Laid-Open Publication No. 1020160001699 (published on June 1, 2016), "Augmented Reality Interaction Service Providing Apparatus and Method"

It is an object of the present invention to provide a technique for accurately estimating a user's position and a posture of a mobile device as well as restoring an accurate indoor space by providing key frame information and spatial information according to an illumination area using visible light communication (LiFi) .

It is also an object of the present invention to provide an augmented reality experience corresponding to the movement of dynamic objects rather than a fixed virtual space.

The system for providing an augmented reality space of an indoor space using visible light communication according to an embodiment of the present invention acquires key frame information for an indoor space, restores the three-dimensional space information, and stores the key frame information and the space A spatial information generating unit for packetizing information and a data providing unit for converting the packetized information into light and providing the converted information to a mobile device of a user located in the illumination zone through a plurality of lights in the illumination zone.

Wherein the user's mobile device receives the packetized information according to the light provided in the arbitrary illumination in the illumination zone and receives the key frame information for attitude tracking and the spatial information from the packetized information can do.

The user's mobile device may receive at least one or more of the floor, wall, ceiling information, and fixed object in the allocated illumination zone from the packetized information, and experience the augmented reality using the indoor space have.

The user's mobile device can convert the packetized information according to the light using 6DoF (6 Degrees of Freedom) information of the camera and perform the augmented reality in the indoor space.

The spatial information generating unit extracts a key frame image obtained from a SLAM (Simultaneous Localization and Mapping) algorithm and minutia information of the key frame image with respect to the indoor space, and generates the key frame image and the key frame image including the minutia information Information can be obtained.

The spatial information generating unit may reconstruct the three-dimensional spatial information using a three-dimensional reconstruction algorithm (or a Kinect Fusion algorithm) for the interior space.

The space information generation unit calculates the three-dimensional point group of the space corresponding to the allocated illumination area and the geometry information of the plane according to the world coordinate based on the key frame information and the space information, It can be changed.

The data provider may provide the packetized information including the frame information and the spatial information, which are different according to the allocated illumination area, through the LED (Light Emitting Diode) illumination in the illumination area.

The data providing unit converts the packetized information including at least one of information about a floor, a wall, ceiling information and a fixed object corresponding to the allocated lighting area into data and transmits the packetized information through the LED lighting in the allocated lighting area .

According to another embodiment of the present invention, an augmented reality providing system for an indoor space using visible light communication may be configured to acquire key frame information for an indoor space, to reconstruct three-dimensional spatial information, A data provider for converting the packetized information into light and providing the converted information to a mobile device of a user located in the lighting zone through a plurality of lights in the lighting zone, And a location tracking unit for tracking the location of the user from the mobile device of the user receiving the light provided by any of the lights.

The location tracking unit may track the location and posture of the user by comparing the key frame information according to the light provided to the user's mobile device.

A method of operating a system for providing an augmented reality of an indoor space using visible light communication (LiFi) according to an embodiment of the present invention includes: acquiring key frame information on an indoor space; restoring the three- Packetizing the key frame information and the spatial information according to an illumination zone and converting the packetized information into light to provide to a mobile device of a user located in the illumination zone through a plurality of lights in the illumination zone .

Wherein the packetizing step comprises: extracting a key frame image obtained from a SLAM (Simultaneous Localization and Mapping) algorithm and minutia information of the key frame image with respect to the indoor space, and extracting the key frame image and the minutia information Frame information can be obtained.

The packetizing may restore the three-dimensional spatial information using a 3D reconstruction algorithm (or a Kinect Fusion algorithm) for the interior space.

The packetizing step may calculate the 3D point cloud and the geometry information of the plane corresponding to the allocated illumination area based on the key frame information and the spatial information according to a world coordinate, Can be packetized.

The providing of the user to the mobile device may include converting the packetized information into data by including LEDs (Light Emitting Diode) lights in the illumination area, including different frame information and the spatial information according to the allocated illumination area .

A method of operating a system for providing an augmented reality of an indoor space using visible light communication (LiFi) according to another embodiment of the present invention, includes: acquiring key frame information about an indoor space; restoring the three- Packetizing the key frame information and the spatial information according to the illuminated zone, converting the packetized information into light and providing it to a user's mobile device located in the lighting zone through a plurality of lights in the lighting zone And tracking the position of the user from the mobile device of the user receiving the light provided by any illumination in the illumination zone.

The step of tracking the position of the user may track the position and posture of the user by comparing the key frame information according to the light provided to the user's mobile device.

According to the embodiment of the present invention, by providing the key frame information and the spatial information according to the illumination area using the visible light communication (LiFi), it is possible to accurately estimate the user's position and the posture of the mobile device have.

In addition, according to the embodiment of the present invention, it is possible to provide an augmented reality experience corresponding to the motion of dynamic objects rather than a fixed virtual space.

In addition, according to the embodiment of the present invention, the user's mobile device located in the illumination area receives only the information related to the specific space and calculates data, thereby enabling quick and efficient indoor space augmented reality experience.

1 is a block diagram illustrating a configuration of an augmented reality providing system for an indoor space using visible light communication according to an embodiment of the present invention.
2 is a block diagram illustrating a configuration of an augmented reality providing system for an indoor space using visible light communication according to another embodiment of the present invention.
3 is a flowchart illustrating a method of providing an augmented reality space in an indoor space using visible light communication according to an embodiment of the present invention.
FIG. 4 is a flowchart illustrating a method of providing an augmented reality in an indoor space using visible light communication according to another embodiment of the present invention.
FIG. 5 illustrates a process of providing an augmented reality according to an embodiment of the present invention.
6 is a view for explaining an example of visible light communication in an indoor space according to an embodiment of the present invention.

Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings. However, the present invention is not limited to or limited by the embodiments. In addition, the same reference numerals shown in the drawings denote the same members.

Also, terminologies used herein are terms used to properly represent preferred embodiments of the present invention, which may vary depending on the viewer, the intention of the operator, or the custom in the field to which the present invention belongs. Therefore, the definitions of these terms should be based on the contents throughout this specification.

The present invention proposes a method for providing augmented reality experience to a user by applying a SLAM (Simultaneous Localization and Mapping) algorithm and a 3D reconstruction algorithm efficiently to an augmented reality.

Embodiments of the present invention relate to a technology for estimating a posture of a mobile device necessary for experiencing an augmented reality in an indoor space using visible light communication (LiFi) and using spatial information about an indoor space acquired from an RGBD camera, It is intended to provide a technology for accurately estimating the user's position and the attitude of the mobile device as well as restoring the accurate indoor space by transmitting large amount of data quickly, accurately and safely through LED (Light Emitting Diode) illumination in the lighting area.

In the present specification, 'indoor space' may represent a space such as an exhibition, a museum, a public relations center, a house, a game room, an experience room, and a game room, and may mean a specific space to which the augmented reality is applied.

In the present embodiments, the 'user' is located in an indoor space for experiencing the augmented reality, and the mobile device is a smart phone, a tablet, a wearable device and a head mounted display display may be a device for providing an augmented reality.

FIG. 1 is a block diagram of a system for providing an augmented reality room in an indoor space using visible light communication according to an embodiment of the present invention. FIG. 3 is a block diagram of an augmented reality providing system for an indoor space using visible light communication according to an embodiment of the present invention. ≪ / RTI >

The system for providing an augmented reality 100 in an indoor space using visible light communication according to an embodiment of the present invention provides key frame information and spatial information in a specific illumination zone using visible light communication LiFi, Thereby providing an augmented reality experience in the illumination area.

Accordingly, the system 100 for providing an augmented reality space in an indoor space using visible light communication according to an embodiment of the present invention includes a spatial information generating unit 110 and a data providing unit 120. However, the augmented reality providing system 100 according to the embodiment may include components such as more memory, bus, network interface, database, and processor than the components of Fig.

The memory may be a computer-readable recording medium and may include a permanent mass storage device such as a random access memory (RAM), a read only memory (ROM), and a disk drive, . The bus may also enable communication and data transmission among the components of the augmented reality providing system 100 and may include a highspeed serial bus, a parallel bus, a storage area network (SAN), and / Or other suitable communication technology.

The network interface may be a computer hardware component for connecting the augmented reality providing system 100 to a computer network. In addition, the database may store and maintain key frame information and spatial information about the indoor space, and may store and maintain information related to the user and the user (e.g., user's personal information and device information of the mobile device). In addition, the processor may be configured to process instructions of a computer program by performing input / output operations of the basic arithmetic, logic, and augmented reality providing system 100.

The spatial information generating unit 110 and the data providing unit 120 of the augmented reality providing system 100 for an indoor space using visible light communication according to the embodiment of the present invention shown in FIG. 320). ≪ / RTI >

In step 310, the spatial information generating unit 110 acquires key frame information on the indoor space, restores the three-dimensional spatial information, and packetizes key frame information and spatial information according to the allocated illumination area.

The spatial information generating unit 110 classifies the output obtained by performing SLAM (Simultaneous Localization and Mapping) algorithm and 3D reconstruction algorithm (or Kinect Fusion algorithm) according to the allocated illumination area Can be encoded.

For example, the spatial information generation unit 110 performs a SLAM (Simultaneous Localization and Mapping) algorithm to acquire a key frame image for the indoor space, extracts minutia information of the key frame image, It is possible to obtain the key frame information including the key frame information. In addition, the spatial information generating unit 110 may restore the three-dimensional spatial information using a three-dimensional reconstruction algorithm (or a Kinect Fusion algorithm) for the interior space.

At this time, the minutia information can be extracted corresponding to the illumination change of the RGBD camera (e.g., mobile device). The three-dimensional spatial information is obtained through the camera. The three-dimensional spatial information is obtained from the color image and the depth image, and is used to construct a structure of an indoor space such as a wall, a floor and a ceiling, a fixed object such as a desk, furniture, a frame, Can be distinguished for non-stationary objects capable of movement.

More specifically, the RGB color space is transformed from the RGB image into the HSV color space according to illumination effects, and the object is subjected to a double threshold for saturation and numerical value elements Lt; / RTI > It is also possible to set the distance from the mobile device to the threshold from the depth image to divide and distinguish the object zone from the intersection of the depth segmentation and RGB segmentation results.

Then, the spatial information generating unit 110 calculates the three-dimensional point group of the space corresponding to the illuminated area allocated based on the key frame information and the spatial information and the geometrical information of the plane according to the world coordinate, And packetized.

For example, the spatial information generating unit 110 may previously calculate the three-dimensional point group of the space excluding the object capable of movement in the allocated illumination area and the geometrical information of the plane (for example, the normal vector and the distance) And then encodes and packetizes key frame information and spatial information.

In this case, the packet includes a start header, a body part, and an end header. The start header and the end header can identify the start and end of data, and the body part can include key frame information and spatial information .

In step 320, the data providing unit 120 converts the packetized information into light and provides it to the user's mobile device located in the illumination zone via a plurality of lights in the illumination zone.

First, the indoor space is divided into a plurality of illumination zones based on sections, zones, and layers, and the lights in the divided illumination zones are arranged at a predetermined distance to provide packetized information as light data. The light may be LED (Light Emitting Diode) light, and it may be installed on the wall or the ceiling with a degree of light that does not disturb the user's view.

At this time, the system for providing an augmented reality 100 in an indoor space according to an embodiment of the present invention uses visible light communication (LiFi). The LiFi is a short-range wireless communication technology that combines light and WiFi, and transmits data including visible light. The advantage of this visible light communication is that large amounts of data can be transmitted quickly, accurately and safely through LED illumination. Furthermore, compared with the markers such as the QR Code, it is possible to operate in a frequency range which is not visible even if it is placed in an indoor space without giving a sense of heterogeneity.

In addition, in step 320, the data providing unit 120 converts packetized information including different frame information and spatial information according to the allocated illumination zone, and transmits the information through the LED (Light Emitting Diode) Can be provided. At this time, the data providing unit 120 converts the packetized information including at least one of the floor, the wall surface, the ceiling information, and the fixed object corresponding to the allocated illumination area into data, As shown in FIG.

For example, illuminations within an illuminated area, distributed by zone or by floor, and by ceiling, floor, and wall, provide packetized information that includes different key frame information and spatial information along different indoor backgrounds. Accordingly, the illumination in the divided illumination area is not limited to transforming all the key frame information and spatial information for the indoor space, but only the information of the zone in charge is partially expressed, so that the mobile device of the user located in the illumination zone, Lt; / RTI >

If the user is located in any of the lights in the allocated illumination zone, the user's mobile device receives the bit-shaped packetized information of 0 and 1 according to the light provided in any illumination, It is possible to receive key frame information and spatial information for attitude tracking. At this time, the user's mobile device may experience at least one of the floor, wall, ceiling, and object information in the illuminated area allocated from the packetized information to experience the augmented reality utilizing the indoor space. The object information may be information on a fixed object such as a desk, a furniture, a frame, or the like, or an unfixed object capable of moving a book, a cup, a pollen, or the like.

Further, the user's mobile device converts the packetized information received from the illumination into 6DoF (6 Degrees Of Freedom) information of the camera, without needing to estimate the spatial information from each frame-based depth image for the indoor space So that an augmented reality experience capable of interacting with an object can be performed.

Accordingly, the system for providing the augmented reality 100 of the indoor space using the visible light communication according to the embodiment of the present invention can track the position of the user by comparing the minimum key frame information, There is no need to consider relocalization, one of the problems. In addition, the augmented reality providing system 100 provides a structure of floor, wall and ceiling information of the space in the illumination zone allocated through the illumination, and object information about the fixed object and the non-fixed object, Even mobile devices can experience augmented reality using indoor space. Further, the spatial information can be converted into image coordinates through the position of the user estimated through the key frame information and the attitude of the mobile device.

FIG. 2 is a block diagram of a system for providing an augmented reality room in an indoor space using visible light communication according to another embodiment of the present invention. FIG. 4 is a view And shows a flow chart of a method of providing reality.

The system for providing an augmented reality room 200 using visible light communication according to another embodiment of the present invention provides key frame information and spatial information in a specific illumination zone using visible light communication LiFi, Track the location from the user's mobile device.

Accordingly, the indoor space augmented reality providing system 200 using the visible light communication according to another embodiment of the present invention includes a spatial information generating unit 210, a data providing unit 220, and a location tracking unit 230 . However, the augmented reality providing system 200 according to the embodiment may include components such as a memory, a bus, a network interface, a database and a processor more than the components of Fig.

The memory may be a computer-readable recording medium and may include a permanent mass storage device such as a random access memory (RAM), a read only memory (ROM), and a disk drive, . The bus may also enable communication and data transmission among the components of the augmented reality providing system 100 and may include a highspeed serial bus, a parallel bus, a storage area network (SAN), and / Or other suitable communication technology.

The network interface may be a computer hardware component for connecting the augmented reality providing system 100 to a computer network. In addition, the database may store and maintain key frame information and spatial information about the indoor space, and may store and maintain information related to the user and the user (e.g., user's personal information and device information of the mobile device). In addition, the processor may be configured to process instructions of a computer program by performing input / output operations of the basic arithmetic, logic, and augmented reality providing system 100.

The spatial information generating unit 210, the data providing unit 220, and the position tracking unit 230 of the augmented reality providing system 200 of the indoor space using visible light communication according to another embodiment of the present invention shown in Fig. May be configured to perform steps 410, 420, 430 of FIG.

In step 410, the spatial information generating unit 210 obtains key frame information on the indoor space, restores the three-dimensional spatial information, and packetizes key frame information and spatial information according to the allocated illumination area.

In step 420, the data providing unit 220 converts the packetized information into light and provides it to the user's mobile device located in the illumination zone through a plurality of lights in the illumination zone.

However, the configuration of the spatial information generating unit 210 and the data providing unit 220 in steps 410 and 420 is different from that of the spatial information generating unit 110 and the data The detailed description of the spatial information generating unit 210 and the data providing unit 220 will be omitted.

At step 430, the location tracking unit 230 tracks the location of the user from the user's mobile device receiving the light provided by any of the lights in the illumination zone.

For example, the location tracking unit 230 compares the key frame information and the spatial information of the indoor space with the key frame information according to the light provided to the user's mobile device located in the allocated illumination area, You can track the position of the mobile device.

FIG. 5 illustrates a process of providing an augmented reality according to an embodiment of the present invention.

5, in the augmented reality providing system according to the embodiment of the present invention, a SLAM (Simultaneous Localization and Mapping) algorithm and a 3D reconstruction algorithm (or a Kinect Fusion algorithm) are performed Acquires key frame information on the indoor space, and restores the three-dimensional space information. In addition, the augmented reality providing system according to the embodiment of the present invention distributes the result according to the allocated illumination area, and controls the result to be expressed in bit form of 0 and 1 in illumination.

Then, in the process (b), the illumination in the illumination area allocated in the indoor space provides the lighted packetized information received from the augmented reality providing system according to the embodiment of the present invention.

(c), if the user is located in the allocated illumination area, the mobile device of the smartphone or the head mounted display (HMD) including the illuminance sensor displays the key frame information for attitude tracking And receives spatial information. In this case, since the spatial information can be corrected through the posture of the mobile device and the position of the user, the user can experience the augmented reality without the depth camera.

6 is a view for explaining an example of visible light communication in an indoor space according to an embodiment of the present invention.

Referring to FIG. 6, the indoor space 600 may include a plurality of lights 610 provided on at least one of a floor, a wall surface, and a ceiling. At this time, the illumination 610 can perform visible light communication (LiFi), which provides data by including visible light (visible light) as an LED (Light Emitting Diode) illumination.

In addition, to distinguish between zones or floors and lights in a lighting zone that are distributed along the ceiling, floor, and wall, the lighting 610 may have a different unique ID, Frame information and spatial information can be provided as data.

According to the embodiment, when a situation occurs in which the illumination 610 is turned off, the provided light is weak or the light is blocked by an obstacle, the augmented reality providing system according to the embodiment of the present invention May discontinue or discard the packetized information provided through the illumination 610 and may retransmit the packetized information if an operation of the normal illumination 610 is subsequently detected. According to the embodiment, for retransmission of packetized information, the augmented reality providing system according to the embodiment of the present invention can store and maintain key frame information and spatial information according to the illuminated area allocated in the indoor space in real time , Key frame information, and spatial information may be transmitted to an external server for database conversion.

6, if the user is located below the ceiling light 610 in the interior space 600, the mobile device 620 may extract the information from the packetized information, which is converted into light, Key frame information and spatial information for the augmented reality experience can be received.

6 uses a smart phone as the mobile device 620. However, at least one of a tablet, a wearable device, and a head mounted display capable of experiencing an augmented reality So that it is not limited.

The apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the apparatus and components described in the embodiments may be implemented within a computer system, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA) A programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For ease of understanding, the processing apparatus may be described as being used singly, but those skilled in the art will recognize that the processing apparatus may have a plurality of processing elements and / As shown in FIG. For example, the processing unit may comprise a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of the foregoing, and may be configured to configure the processing device to operate as desired or to process it collectively or collectively Device can be commanded. The software and / or data may be in the form of any type of machine, component, physical device, virtual equipment, computer storage media, or device , Or may be permanently or temporarily embodied in a transmitted signal wave. The software may be distributed over a networked computer system and stored or executed in a distributed manner. The software and data may be stored on one or more computer readable recording media.

The method according to an embodiment may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions to be recorded on the medium may be those specially designed and configured for the embodiments or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape, optical media such as CDROMs and DVDs, magnetic optical media such as floppy disks, magnetooptical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. For example, it is to be understood that the techniques described may be performed in a different order than the described methods, and / or that components of the described systems, structures, devices, circuits, Lt; / RTI > or equivalents, even if it is replaced or replaced.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

100, 200: Augmented reality providing system
600: Indoor space
610: Lighting
620: Mobile device

Claims (19)

A spatial information generating unit for obtaining key frame information for the indoor space, restoring the three-dimensional spatial information, and packetizing the key frame information and the spatial information according to the allocated illumination area; And
Converting the packetized information into light and providing the converted data to a mobile device of a user located in the lighting zone through a plurality of lights in the lighting zone;
And a display unit for displaying an image of the augmented reality. The method according to claim 1,
The user's mobile device
Receiving the packetized information in accordance with the light provided in the arbitrary illumination in the illumination zone, extracting key frame information for attitude tracking from the packetized information and indoor Space Augmented Reality System. 3. The method of claim 2,
The user's mobile device
Visual information is received from the packetized information and information on at least one of floor, wall surface, ceiling information and fixed object in the allocated illumination zone is received, and the augmented reality experience using the indoor space is made possible. Augmented reality providing system for indoor space. The method of claim 3,
The user's mobile device
An augmented reality providing system for an indoor space using visible light communication that converts the packetized information according to the light using 6DoF (6 Degrees of Freedom, 6 degrees of freedom) information of the camera and performs the augmented reality in the indoor space . The method according to claim 1,
The spatial information generating unit
Extracting a key frame image obtained from a SLAM (Simultaneous Localization and Mapping) algorithm and minutia information of the key frame image with respect to the indoor space, and extracting the key frame image and the key frame information including the minutia information A System for Providing Increased Reality of Indoor Space Using Communication. 6. The method of claim 5,
The spatial information generating unit
And a visual space communication system for reconstructing the three-dimensional spatial information using a three-dimensional reconstruction algorithm (or a Kinect Fusion algorithm) for the indoor space. The method according to claim 6,
The spatial information generating unit
Dimensional point group of a space corresponding to the allocated illumination zone and geometric information of a plane according to a world coordinate on the basis of the key frame information and the spatial information, A system for providing an augmented reality room in an indoor space using visible light communication. The method according to claim 1,
The data provider
And wherein the packetized information including the frame information and the spatial information different from each other according to the allocated illumination area is converted into data and provided through LED (Light Emitting Diode) illumination in the illumination area. Space Augmented Reality System. 9. The method of claim 8,
The data provider
Providing information through at least one of a floor, a wall surface, ceiling information, and a fixed object corresponding to the allocated illumination area, and providing the packetized information through the LED illumination in the allocated illumination area Augmented Reality System for Indoor Space using. A spatial information generating unit for obtaining key frame information for the indoor space, restoring the three-dimensional spatial information, and packetizing the key frame information and the spatial information according to the allocated illumination area;
Converting the packetized information into light and providing it to a mobile device of a user located in the lighting zone through a plurality of lights in the lighting zone; And
A location tracking unit for tracking the location of the user from the mobile device of the user receiving the light provided by any illumination in the illumination zone;
And a display unit for displaying an image of the augmented reality. 11. The method of claim 10,
The position-
And the position and attitude of the user are tracked by comparing the key frame information according to the light provided to the user's mobile device. An operation method of a system for providing an augmented reality of an indoor space using visible light communication (LiFi)
Acquiring key frame information for the indoor space, restoring the three-dimensional space information, and packetizing the key frame information and the spatial information according to the allocated illumination area; And
Converting the packetized information into light and providing it to a mobile device of a user located in the lighting zone through a plurality of lights in the lighting zone
The method comprising the steps of: 13. The method of claim 12,
The step of packetizing
Extracting a key frame image obtained from a SLAM (Simultaneous Localization and Mapping) algorithm and minutia information of the key frame image with respect to the indoor space, and extracting the key frame image and the key frame information including the minutia information Method for Providing Augmented Reality in Indoor Space Using Communication. 14. The method of claim 13,
The step of packetizing
And reconstructing the three-dimensional spatial information using a three-dimensional reconstruction algorithm (or a Kinect Fusion algorithm) for the interior space. 15. The method of claim 14,
The step of packetizing
Dimensional point group of a space corresponding to the allocated illumination zone and geometric information of a plane according to a world coordinate on the basis of the key frame information and the spatial information, A method for providing an augmented reality room in an indoor space using visible light communication. 13. The method of claim 12,
The step of providing the user with a mobile device
And the packetized information including the different frame information and the spatial information according to the allocated illumination area is converted into data and provided through LED (Light Emitting Diode) illumination in the illumination area. A method for providing an augmented reality in an indoor space. An operation method of a system for providing an augmented reality of an indoor space using visible light communication (LiFi)
Acquiring key frame information for the indoor space, restoring the three-dimensional space information, and packetizing the key frame information and the spatial information according to the allocated illumination area;
Converting the packetized information into light and providing it to a mobile device of a user located in the lighting zone through a plurality of lights in the lighting zone; And
Tracking the position of the user from the user ' s mobile device receiving the light provided by any illumination in the illumination zone
The method comprising the steps of: 18. The method of claim 17,
The step of tracking the location of the user
And comparing the key frame information according to the light provided to the mobile device of the user to track the position and attitude of the user. 19. A computer program stored in a computer-readable medium for performing the method of any one of claims 12 to 18.

Images