KR102314787B1 - Method and apparatus for providing virtual room - Google Patents

Abstract

The virtual space providing method performed by the disclosed virtual space providing apparatus includes the steps of selecting any one virtual space model corresponding to an image of a real space from among a plurality of predefined virtual space models; Creating a virtual space by assigning properties to the selected virtual space model based on the texture extracted from a partial region, and generating one or more objects in the virtual space using a predefined object database according to interaction through the user interface including placing.

Description

Method and device for providing virtual space {METHOD AND APPARATUS FOR PROVIDING VIRTUAL ROOM}

The present invention relates to a method and apparatus for providing a virtual space, and more particularly, to a method and apparatus for providing a virtual space for providing a virtual space corresponding to an image of a real space.

As is well known, virtual reality creates a specific environment or situation through computing operations, creating the illusion that a person using it is interacting with a real surrounding situation or environment. Such virtual reality may display a virtual space using various images or computer graphics, and change the virtual space according to a user's motion so that the user feels as if he or she is in the virtual space.

On the other hand, in the field of virtual reality technology, efforts are being made to improve the sense of reality on the virtual reality screen. The sense of reality on the virtual reality screen changes depending on how similar it is to this real space.

According to the prior art, an image of a real space is analyzed to detect all objects existing in the image, a 3D virtual model corresponding to the detected objects is generated, and a virtual space including the generated 3D virtual model is generated. created to provide a virtual reality screen.

However, according to the prior art, when the objects existing in the image of the real space are not accurately detected, the similarity of the virtual space to the real space is lowered, so that the sense of reality on the virtual reality screen is significantly lowered. In addition, although a virtual space with a low sense of reality is provided, many calculations are required to detect objects existing in the real space. There was a problem that the space could not be provided.

Republic of Korea Patent Publication No. 10-2016-0125439, publication date October 31, 2016.

According to an embodiment of the present invention, by providing a virtual space by selecting a virtual space model corresponding to an image of a real space from among a plurality of predefined virtual space models, a small amount of computing power is required, so the computing power Even when this is limited, real-time processing is enabled, and the sense of reality for the virtual reality screen including the virtual space is always maintained above the basic level.

The problems to be solved of the present invention are not limited to those mentioned above, and other problems to be solved that are not mentioned will be clearly understood by those of ordinary skill in the art to which the present invention belongs from the following description.

According to an aspect of the present invention, a method for providing a virtual space performed by an apparatus for providing a virtual space includes selecting any one virtual space model corresponding to an image of a real space from among a plurality of predefined virtual space models; creating a virtual space by assigning attributes to the selected virtual space model based on a texture extracted from a partial region of the image for the real space; and placing one or more objects in the virtual space using a defined object database.

An apparatus for providing a virtual space according to another aspect of the present invention includes an image acquisition unit for acquiring an image of a real space, an input unit for providing a user interface for interaction, and a storage unit for storing a plurality of virtual space models and object databases and an output unit for outputting a processing space corresponding to the image of the real space, and a controller, wherein the controller includes any one virtual space corresponding to the image of the real space from among the plurality of virtual space models. A space model is selected, a virtual space is created by assigning properties to the selected virtual space model based on a texture extracted from a partial region of an image for the real space, and according to an interaction through the user interface, the object One or more objects are placed in the virtual space using a database.

According to an embodiment of the present invention, by providing a virtual space by selecting a virtual space model corresponding to an image of a real space from among a plurality of predefined virtual space models, a small amount of computing power is required, so the computing power Even when this is limited, real-time processing is enabled, and the sense of reality for the virtual reality screen including the virtual space is always maintained above the basic level.

In addition, since the structural features are obtained based on the result of extracting the linear component from the image of the real space for the virtual space, it is okay even if the state of the real space is not neatly arranged when the image of the real space is acquired. This has the effect of creating a virtual space by projecting not only the private space managed by the individual but also various spaces that the individual cannot participate in.

1 is a block diagram of an apparatus for providing a virtual space according to an embodiment of the present invention.
2 is a flowchart illustrating a method for providing a virtual space performed by an apparatus for providing a virtual space according to an embodiment of the present invention.
3 is a flowchart illustrating a process of selecting a virtual space model in a method for providing a virtual space performed by an apparatus for providing a virtual space according to an embodiment of the present invention.
4 is a flowchart illustrating a process of extracting structural features of a real space in a method for providing a virtual space performed by an apparatus for providing a virtual space according to an embodiment of the present invention.
5 is an exemplary diagram for explaining a process of extracting a straight line component from a real space by an apparatus for providing a virtual space according to an embodiment of the present invention.
6 is an exemplary diagram for explaining a process in which a virtual space providing apparatus creates a virtual space by assigning attributes to a virtual space model according to an embodiment of the present invention.
7 is an exemplary diagram for explaining a process of arranging an object in a virtual space by an apparatus for providing a virtual space according to an embodiment of the present invention.

Advantages and features of the present invention and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only these embodiments allow the disclosure of the present invention to be complete, and common knowledge in the art to which the present invention pertains It is provided to fully inform those who have the scope of the invention, and the present invention is only defined by the scope of the claims.

In describing the embodiments of the present invention, if it is determined that a detailed description of a well-known function or configuration may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. In addition, the terms to be described later are terms defined in consideration of functions in an embodiment of the present invention, which may vary according to intentions or customs of users and operators. Therefore, the definition should be made based on the content throughout this specification.

1 is a block diagram of an apparatus for providing a virtual space according to an embodiment of the present invention.

As shown in FIG. 1 , the apparatus 100 for providing a virtual space according to an embodiment includes an image acquisition unit 110 , an input unit 120 , a storage unit 130 , an output unit 140 , and a control unit 150 . .

The image acquisition unit 110 acquires an image of a real space. For example, the image acquisition unit 110 may acquire 2D image information and depth information about the real space.

The image acquisition unit 110 may include a camera that captures the real world, and may further include a depth sensor that senses depth information of a frame photographed by the camera. In this case, the image acquisition unit 110 may directly generate 2D image information and depth information for a real space using a camera and a depth sensor.

Alternatively, the image acquisition unit 110 may include a communication interface capable of receiving information from a camera and a depth sensor provided separately from the virtual space providing apparatus 100 . In this case, two-dimensional image information is generated by a separate camera, and the image acquisition unit 110 may receive and acquire two-dimensional image information through a communication interface. In addition, depth information of a frame corresponding to 2D image information is generated by a separate depth sensor, and the image acquisition unit 110 may receive and acquire depth information through a communication interface.

The input unit 120 is an interface means through which a user can input a user command for interaction with the virtual space providing apparatus 100 . For example, the input unit 120 may include a keypad provided on one side of the virtual space providing apparatus 100 . Also, as the input unit 120 , a graphic user interface may be provided through the output unit 140 .

The storage unit 130 stores a plurality of predefined virtual space models, and information on structural features of the plurality of virtual space models may be stored together. Also, the storage unit 130 may store a plurality of objects that can be arranged in the virtual space and an object database including textures corresponding to each object. In addition, the storage unit 130 may store computer-executable instructions that, when executed by the controller 150 , perform a process capable of controlling the virtual space provision process as a whole. For example, the storage unit 130 may include at least one of a flash memory type, a hard disk type, a multimedia card micro type, a card type memory, a RAM, and a ROM. It may include a computer-readable recording medium of the type.

The output unit 140 may output a screen including a virtual space according to a virtual space model corresponding to the real space according to a control signal according to a result of the process performed by the controller 150 . As such, an object corresponding to a texture that can be extracted from an image of a real space is disposed in the virtual space output through the output unit 140 , and an attribute corresponding to the texture may be assigned to the virtual space model. For example, the output unit 140 may include a flat panel display device such as a liquid crystal display (LCD).

The control unit 150 selects any one virtual space model corresponding to the image for the real space obtained by the image acquisition unit 110 from among the plurality of virtual space models stored in the storage unit 130, and Based on the characteristics extracted from the partial region of the image for the image, a virtual space is created by assigning a property to the selected virtual space model, and according to the interaction with the user through the input unit 120 , the object database stored in the storage unit 130 . can be used to place one or more objects in a virtual space.

The control unit 150 analyzes the image of the real space obtained by the image acquisition unit 110 to extract structural features of the real space, and the structural features extracted from the plurality of virtual space models stored in the storage unit 130 . A virtual space model having the highest degree of similarity to the virtual space model may be selected as the virtual space model to be provided through the output unit 140 .

In addition, when extracting structural features of real space, the controller 150 extracts a plurality of linear components from an image for real space, acquires depth information of an image for real space, and extracts a plurality of linear components extends to detect a vanishing point, and defines a wall surface of the virtual space from among the surfaces that can be formed by a plurality of linear components based on the position of the detected vanishing point, and a wall surface defined from among the surfaces that can be formed by a plurality of linear components It is possible to define a window in the interior, and obtain information about the defined wall and information about the defined window as structural features.

In addition, the controller 150 may execute a computer-executable command that may be stored in the storage unit 130 to perform a process capable of controlling the virtual space provision process as a whole. For example, the controller 150 may include a processor such as a central processing unit (CPU) or a graphics processing unit (GPU).

2 is a flowchart illustrating a method for providing a virtual space performed by an apparatus for providing a virtual space according to an embodiment of the present invention.

As shown in FIG. 2 , in the method of providing a virtual space according to an embodiment, after acquiring an image of a real space, any one virtual space corresponding to the image of the real space from among a plurality of predefined virtual space models and selecting a model (S200, S300).

Next, the method for providing a virtual space according to an embodiment includes the steps of creating a virtual space by assigning attributes to a selected virtual space model based on a texture extracted from a partial region of an image of the real space (S400, S500). include more

In addition, the method of providing a virtual space according to an embodiment further includes arranging one or more objects in the virtual space using a predefined object database according to an interaction through a user interface (S600).

3 is a flowchart illustrating a process of selecting a virtual space model in a method for providing a virtual space performed by an apparatus for providing a virtual space according to an embodiment of the present invention.

As shown in FIG. 3 , the step of selecting a virtual space model in the method for providing a virtual space according to an embodiment ( S300 ) includes analyzing an image of the real space to extract structural features of the real space ( S310 ) and a plurality of and selecting, as any one virtual space model, a virtual space model most similar to a structural feature from among the virtual space models ( S320 ).

4 is a flowchart illustrating a process of extracting structural features of a real space in a method for providing a virtual space performed by an apparatus for providing a virtual space according to an embodiment of the present invention.

As shown in FIG. 4 , in the method for providing a virtual space according to an embodiment, the step of extracting the structural features of the real space ( S310 ), the step of extracting a plurality of linear components from the image of the real space ( S311 ), the reality A step of obtaining depth information of an image for space (S312), a step of detecting a vanishing point by extending a plurality of linear components (S313), a surface where a plurality of linear components can be formed based on the position of the detected vanishing point Among these, the step of defining a wall surface of the virtual space (S14), the step of defining a window within the wall defined among the surfaces that can be formed by a plurality of linear components (S315), information on the defined wall surface and information on the defined window and acquiring as a structural feature of real space (S316).

Hereinafter, a virtual space providing method performed by the virtual space providing apparatus according to an embodiment of the present invention will be described in more detail with reference to FIGS. 1 to 7 .

First, the image acquisition unit 110 acquires an image of the real space (S200). For example, the image acquisition unit 110 may acquire 2D image information and depth information for a real space by using a camera and a depth sensor.

Then, the controller 150 selects any one virtual space model corresponding to the image for the real space acquired by the image acquisition unit 110 from among a plurality of predefined virtual space models stored in the storage unit 130 . (S300).

Here, in order to select a virtual space model corresponding to the image for the real space, the controller 150 analyzes the image for the real space obtained in step S200 and extracts structural features of the real space ( S310 ).

As such, when the control unit 150 extracts the structural features of the real space, a plurality of linear components are first extracted from the image for the real space obtained in step S200. For example, the control unit 150 analyzes a high frequency component included in an image for real space using an edge detection method or a feature point method to generate linear components leading to a straight line. can be defined (S311).

Then, the controller 150 acquires the depth information of the image for the real space. For example, when acquiring an image for real space, the controller 150 may calculate a depth map by using a feature point method or a matching method as disparity as the amount of movement of regions in the image. . Alternatively, depth information may be acquired by receiving a sensing value from a depth sensor constituting the image acquisition unit 110 (S312).

Next, the control unit 150 detects a vanishing point by extending the plurality of linear components extracted from the image for real space (S313), and among the surfaces that can be formed by the plurality of linear components based on the position of the detected vanishing point. Define the wall of the virtual space. For example, the control unit 150 may define a space that can be most widely occupied among surfaces that can be formed by a plurality of straight line components at a position where the vanishing point is detected as the first wall surface, and intersect the first wall surface at a predetermined angle. A space that can occupy the widest area among the surfaces formed by the method may be defined as the second wall surface, and the widest surface among the spaces opposite to the second wall may be defined as the third wall surface (S314). For example, the wall of the virtual space may be defined as two walls or three walls according to an angle at which the image of the real space is acquired when the image of the real space is acquired in step S200. 5 shows an example in which two wall surfaces 401 and 402 are defined as shown in (B) for an image of a real space as shown in (A).

Then, the control unit 150 defines a window in the wall surface defined in step S314 among the surfaces that can be formed by the plurality of linear components. For example, the controller 150 may define a surface within the wall surface defined in step S314 as a window among surfaces that can be formed by a plurality of linear components ( S315 ). For example, as shown in FIG. 5 , a surface in the wall 401 may be defined as the window 403 , and although an example of defining one window 403 is shown in FIG. 5 , the process of defining the window 403 in this way can be repeated several times to define multiple windows.

Next, the controller 150 acquires the information on the wall defined in step S314 and the information on the window defined in step S315 as structural features of the real space ( S316 ).

In addition, the controller 150 selects a virtual space model most similar to the structural characteristics obtained in step S316 from among the plurality of virtual space models stored in the storage unit 130 as a virtual space model for the virtual space to be provided to the user. Here, the controller 150 searches for a virtual space model having the wall defined in step S314 and the window defined in step S315 from among the plurality of virtual space models stored in the storage unit 130 and selects the most similar virtual space model. For example, the control unit 150 calculates the similarity between the structural features of the real space extracted in step S310 and the plurality of virtual space models stored in the storage unit 130 , respectively, and provides the virtual space model with the highest similarity to the user. You can select it as a virtual space model for the virtual space you want to do.

Next, the controller 150 generates a virtual space by assigning properties to the virtual space model selected in step S300 based on the texture extracted from a partial region of the image for the real space, and the output unit 140 generates the virtual space. 150) and output the virtual space created by the screen. Here, the control unit 150 extracts a texture from a region corresponding to a region predefined in the virtual space in real space, and generates an object corresponding to the extracted texture in the virtual space using the object database of the storage unit 130 . , an attribute corresponding to the extracted texture may be assigned to the corresponding virtual space (S500). For example, as shown in FIG. 6 , the bed 501 may be created as an object corresponding to the texture extracted from the floor surface of the real space by searching the object database. In addition, the frame 502 and the closet 503 may be generated as objects corresponding to the texture extracted from the wall of the real space.

Meanwhile, the user may cause an interaction with the virtual space providing apparatus 100 by using a user interface such as the input unit 120 , and the controller 150 uses the object database of the storage unit 130 according to the interaction to create a virtual One or more objects are arranged in space (S600). For example, as shown in FIG. 7 , the objects 601 , 602 , and 603 designated by the user may be created at the location designated by the user.

As described so far, according to an embodiment of the present invention, a small amount of computation is required by selecting a virtual space model corresponding to an image of a real space from among a plurality of predefined virtual space models and providing a virtual space. Hereinafter, real-time processing is possible even when computing power is limited, and the sense of reality for a virtual reality screen including a virtual space is always maintained at a basic level or higher.

Combinations of each block in the block diagram attached to the present invention and each step in the flowchart may be performed by computer program instructions. These computer program instructions may be embodied in a processor of a general-purpose computer, special-purpose computer, or other programmable data processing equipment, such that the instructions executed by the processor of the computer or other programmable data processing equipment may be configured in the respective blocks in the block diagram or in the flowchart. Each step creates a means for performing the described functions. These computer program instructions may also be stored in a computer-usable or computer-readable memory that may direct a computer or other programmable data processing equipment to implement a function in a particular manner, and thus the computer-usable or computer-readable memory. The instructions stored in the block diagram may also produce an item of manufacture containing instruction means for performing a function described in each block of the block diagram or each step of the flowchart. The computer program instructions may also be mounted on a computer or other programmable data processing equipment, such that a series of operational steps are performed on the computer or other programmable data processing equipment to create a computer-executed process to create a computer or other programmable data processing equipment. It is also possible that instructions for performing the processing equipment provide steps for carrying out the functions described in each block of the block diagram and each step of the flowchart.

Further, each block or each step may represent a module, segment, or portion of code that includes one or more executable instructions for executing the specified logical function(s). It should also be noted that in some alternative embodiments it is also possible for the functions recited in blocks or steps to occur out of order. For example, it is possible that two blocks or steps shown one after another may in fact be performed substantially simultaneously, or that the blocks or steps may sometimes be performed in the reverse order according to the corresponding function.

The above description is merely illustrative of the technical spirit of the present invention, and various modifications and variations will be possible without departing from the essential characteristics of the present invention by those skilled in the art to which the present invention pertains. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical spirit of the present invention, but to explain, and the scope of the technical spirit of the present invention is not limited by these embodiments. The protection scope of the present invention should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

According to an embodiment of the present invention, a virtual space is provided by selecting a virtual space model corresponding to an image of a real space from among a plurality of predefined virtual space models.

The present invention can be used not only in the technical field of providing a virtual space through a mobile device such as a smart phone, but also in various technical fields of providing a virtual reality screen including a virtual space.

100: virtual space providing device
110: image acquisition unit 120: input unit
130: storage unit 140: output unit
150: control unit

Claims (9)

A virtual space providing method performed by a virtual space providing device, comprising:
extracting structural features of the real space by analyzing the image of the real space;
selecting any one virtual space model most similar to the structural feature from among a plurality of predefined virtual space models;
creating a virtual space by assigning attributes to the selected virtual space model based on a texture extracted from a partial region of the image for the real space;
according to an interaction through a user interface, arranging one or more objects in the virtual space using a predefined object database;
The step of extracting the structural features,
extracting a plurality of linear components from the image for the real space;
obtaining depth information of an image for the real space;
detecting a vanishing point by extending the plurality of linear components;
defining a wall surface of the virtual space based on an area among surfaces that can be formed by the plurality of linear components based on the position of the detected vanishing point;
defining a window in the defined wall surface among the surfaces that the plurality of linear components can form;
Comprising the step of acquiring information about the defined wall and information about the defined window as the structural feature
How to provide virtual space. delete delete The method of claim 1,
The generating of the virtual space may include generating an object having a property corresponding to the texture from among object models stored in the object database in the virtual space.
How to provide virtual space. In order for the processor to perform the virtual space providing method according to claim 1 or 4
A computer program stored in a computer-readable recording medium. An image acquisition unit for acquiring an image of a real space,
an input unit providing a user interface for interaction;
A storage unit in which a plurality of virtual space models and object databases are stored;
an output unit for outputting a processing space corresponding to the image of the real space;
includes a control unit,
The control unit extracts structural features of the real space by analyzing the image of the real space, selects any one virtual space model most similar to the structural features from among the plurality of virtual space models, A virtual space is created by assigning properties to the selected virtual space model based on a texture extracted from a partial region of an image for disposing one or more objects within, when extracting the structural features, extracting a plurality of linear components from the image for the real space, acquiring depth information of the image for the real space, and obtaining the plurality of linear components extends to detect a vanishing point, and defines a wall surface of the virtual space based on an area among surfaces that can be formed by the plurality of linear components based on the position of the detected vanishing point, and the plurality of linear components are formed Defining a window in the defined wall among possible surfaces, and acquiring information about the defined wall and information about the defined window as the structural feature
Virtual space provider. delete delete 7. The method of claim 6,
The controller is configured to generate an object having a property corresponding to the texture from among the object models stored in the object database in the virtual space.
Virtual space provider.

Images