KR20190065933A - Method for interior platform based on spatial awareness - Google Patents

Abstract

The present invention is to solve problems of the conventional technique. A user who wants an interior directly takes a picture, makes an object available for deletion and restoration within an image by separating an object from a photographed image and obtaining the dimension of the object, and arranges various interior products. In addition, with respect to the interior to be installed or built, by using an image of real space, the user can directly check the appearance after completion by virtually performing the internal construction and installation and presenting the virtual internal construction and installation to the user.

Description

METHOD FOR INTERIOR PLATFORM BASED ON SPATIAL AWARENESS [0002]

The present invention relates to a method of providing an interior platform based on spatial recognition, and more particularly, to a method of recognizing an object in an image photographed through a camera of a terminal and simulating changes, installation, This is about a possible platform.

Generally, when an individual desires to decorate his / her home, relocates the address of the company, or changes his / her office, a consumer who does not have expertise related to the interior requests the contractor, -line), or by asking for a job online and asking for work.

Typically, an individual tries to change the interior of a residence, an office space, or the like in accordance with his / her taste or fashion, and a company that wants to relocate the address, who wants to renovate, Procedures and renovations are canceled or completed. However, individuals or companies without interior expertise have difficulty in guessing time and space constraints due to interior construction and the costs incurred by construction work.

In addition, there is a disadvantage in that, when proceeding with the interior decoration, it is necessary to rely on a method of arranging a similar house or drawing, placing it in a virtual space, or simply checking with a picture.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems of the related art, and it is an object of the present invention to provide an image processing apparatus, So that it can be deleted and restored, and various interior products can be arranged.

Also, with regard to the installation and the interior of the construction site, it is possible to use the image of the actual space to manually change the finish material, perform product installation and change the space, and visually check the result after completion To assist in making the best purchase, installation, and construction request decisions.

According to an aspect of the present invention, there is provided a method for providing an interior platform based on spatial recognition performed by a server, comprising the steps of: (a) receiving an image photographed through a camera of a user terminal; ; (b) recognizing dimensions of objects and objects in the image and providing the dimensions to the user terminals; (c) providing the user terminal with at least one list of interior product list, installation and construction list; And (d) receiving a selection input for one or more of the items in the list and providing a quotation, wherein the dimensions include one or more of horizontal, vertical, height and three-dimensional coordinates of the object Wherein the step (b) refers to a depth value obtained through a camera of the user terminal, and recognizes the dimension based on a depth value of the feature points in the image.

According to another aspect of the present invention, there is provided a server for performing an interior platform providing method based on spatial recognition, comprising: a memory for storing an interior platform providing program based on spatial recognition; And a processor for executing the program, wherein the processor, in response to execution of the program, receives an image photographed through a camera of a user terminal, recognizes dimensions of objects and objects in the image, Providing a list of one or more of an interior product list, installation and construction list to the user terminal, receiving a selection input for one or more items in the list, and providing a quotation, The depth, the height, and the three-dimensional coordinates of the user terminal, and refers to the depth value acquired through the camera of the user terminal, and recognizes the dimension based on the depth value of the feature points in the image can do.

The present invention relates to a method for providing an interior platform based on spatial recognition, in which a user directly uploads an image of a desired place within a platform, performs virtual placement, installation, and construction of an interior product in the image , And a platform that allows you to directly see how the actual place changes.

In addition, virtual placement, installation, and construction are performed You can choose your favorite interior, request a quote for the interior, and get a quote.

1 is a configuration diagram of a system for performing an interior platform providing method based on spatial recognition according to an embodiment of the present invention.
2 is a block diagram of a user terminal for performing a method for providing an interior platform based on spatial recognition according to an embodiment of the present invention.
3 is an illustration of object recognition of a method for providing an interior platform based on spatial recognition, in accordance with an embodiment of the present invention.
FIGS. 4 to 5 illustrate an object range setting and dimension recognition of a method for providing an interior platform based on spatial recognition, according to an embodiment of the present invention.
Figure 6 is an illustration of a change through selection within a list of methods of providing an interior platform based on spatial awareness, in accordance with an embodiment of the present invention.
Figures 7 to 9 are illustrations of a platform of a method for providing an interior platform based on spatial awareness, in accordance with an embodiment of the present invention.
10 is a flowchart illustrating a method of providing an interior platform based on spatial recognition according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, which will be readily apparent to those skilled in the art. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

Throughout the specification, when a part is referred to as being "connected" to another part, it includes not only "directly connected" but also "electrically connected" with another part in between . Also, when an element is referred to as "comprising ", it means that it can include other elements as well, without departing from the other elements unless specifically stated otherwise.

In this specification, the term " part " includes a unit realized by hardware, a unit realized by software, and a unit realized by using both. Further, one unit may be implemented using two or more hardware, or two or more units may be implemented by one hardware. On the other hand, 'to' is not limited to software or hardware, 'to' may be configured to be an addressable storage medium, and may be configured to play one or more processors. Thus, by way of example, 'parts' may refer to components such as software components, object-oriented software components, class components and task components, and processes, functions, , Subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables. The functions provided in the components and components may be further combined with a smaller number of components and components or further components and components. In addition, the components and components may be implemented to play back one or more CPUs in a device or a secure multimedia card.

The "terminal" referred to below can be implemented as a computer or a portable terminal that can access a server or other terminal through a network. Here, the computer may be, for example, a laptop, a desktop, a laptop, a VR HMD (e.g., HTC VIVE, Oculus Rift, GearVR, DayDream, PSVR etc.) equipped with a web browser . ≪ / RTI > Here, the VR HMD can be used for PC (eg HTC VIVE, Oculus Rift, FOVE, Deepon, etc.), mobile (eg GearVR, DayDream, And stand alone models (e.g., Deepon, PICO, etc.) that are independently implemented. [0002] A portable terminal is a wireless communication device that is guaranteed to be portable and portable, for example, as well as a smart phone, a tablet PC, a wearable device, a Bluetooth (BLE), an NFC, an RFID, , An infrared ray, a WiFi, a LiFi, and the like. The term "network" refers to a connection structure capable of exchanging information between nodes such as terminals and servers. The network includes a local area network (LAN), a wide area network (WAN) (WWW: World Wide Web), wired / wireless data communication network, telephone network, wired / wireless television communication network and the like. Examples of wireless data communication networks include 3G, 4G, 5G, 3GPP, LTE, WIMAX, Wi-Fi, Bluetooth, infrared, Communications, Visible Light Communication (VLC), LiFi, and the like.

Hereinafter, a system for performing a method of providing an interior platform based on spatial recognition according to an embodiment of the present invention will be described in detail with reference to FIG.

A system for performing a method of providing an interior platform based on spatial recognition according to an embodiment of the present invention may include a server 100 and a user terminal 200 and may include a server 100 and a user terminal 200, Lt; RTI ID = 0.0 > network. ≪ / RTI >

According to an embodiment of the present invention, the server 100 includes a processor for executing a program and a memory in which an interior platform providing program based on spatial recognition is stored, And provides the user terminal 200 with a list of one or more of an interior product list, an installation, and a construction list. Receiving a selection input for one or more items in the list and providing a quotation, wherein the dimensions are detailed specifications including one or more of the horizontal, vertical, height and three-dimensional coordinates of the object, the user terminal 200 ), The dimension can be recognized based on the depth value of the feature points in the image.

The user terminal 200 may be a computing device located at a specific location, and may be a device owned by a user. Also, the user terminal 200 may have a built-in interior platform providing program based on the spatial recognition.

In addition, the user terminal 200 may have a built-in imaging device, and the imaging device may be a device capable of extracting a depth value, such as a stereo camera, for example.

That is, the user terminal 200 may refer to a device that recognizes the state of the space by a photographed image capturing technique, and a program in which a program capable of partial and total correction is performed.

Hereinafter, the structure of the user terminal 200 according to an embodiment of the present invention will be described in detail with reference to FIG.

The user terminal 200 according to an exemplary embodiment of the present invention may be provided with a structure for incorporating the following seven runtime components into an app in order to perform a method of providing an interior platform based on a space recognition through an application have.

As shown in FIG. 2, a platform abstraction layer unit, an application state transition management unit, a session management unit, a transmission / reception management unit, an object and spatial dimension analysis unit, a simulation data analysis unit, and a display control unit may be provided.

The platform abstraction layer can perform processing of detailed routines dependent on the terminal platform to support applications (apps) to be executed independently on terminal platforms (HW, Embedded O / S, etc.) of various models.

The app state transition manager can perform overall management of the state transition of dynamically changed terminal apps.

The session management unit may play a role of managing a session interworking state between engine parts distributed to a terminal and a server while a terminal user executes a virtual placement simulation through an application. That is, it can perform functions to create and terminate a necessary communication session between the engines, and to monitor the connection state between the engines, thereby restoring the communication session at the end of the unintended communication session.

The transmission / reception management unit can perform the function of transmitting the information generated from the user terminal 200 to the server 100 and can perform the function of receiving information generated from the server 100 by the user terminal 200 have.

The object and spatial dimension analyzing unit, and the actual dimension information of the object and the space recognized by the photographing apparatus (camera-based scanner) of the user terminal 200. [

The simulation data analyzing unit is a runtime module that is separated from the simulation engine of the server 100 during execution and is downloaded through the transmission / reception management unit. The simulation data analyzing unit is a module for analyzing the placement and the placement of objects corresponding to events such as touching and dragging, It can play a role of interpreting the method.

The display arrangement unit may control the terminal display output of the image based on the arrangement of the space and the object, and the result of the posture control simulation execution analysis.

In addition, the structure of data elements for carrying out the method of providing an interior platform based on spatial recognition will be described in detail.

Contains common GUI property elements consisting of images, buttons, text input windows, check boxes, etc., and the detailed methods of how objects and space elements are displayed on the screen. It is composed of Shape, Size, ), Orientation, Display Ordering, Correlation between Data, and so on.

In addition, GUI attribute elements and object and space elements of a space including a GUI attribute element of an object composed of a manufacturer, a product and a model, and a stock quantity, an installer capable of constructing a space, a quotation item, These elements are dynamic GUI property elements consisting of Movement, Rotation, Collision, etc., and additional functions related to objects and space elements. , Painting (Painting), Callback (Callback), Save (Save), and so on.

Referring now to Figures 3-6, a method for identifying objects and recognizing dimensions of objects according to an embodiment of the present invention will be described in detail.

The server 100 performing the method of providing an interior platform based on spatial recognition according to an embodiment of the present invention stores shape information including image information and standard information of a plurality of objects in advance in a server, The shape information corresponding to the object can be mechanically learned and stored.

As shown in FIG. 3, in identifying an object, an image received through the photographing device of the user terminal 200 is analyzed to identify an object, and the feature points in the image are analyzed to obtain one or more objects .

At this time, the object can be all the movable furniture, lighting, carpet, etc. of all interior structures and interior products such as walls, floors, ceilings, staircases, and built-in furniture, and can identify objects included in the image as objects.

That is, the server 100 can identify the classified object based on the previously stored shape information through machine learning in advance, and determine the object.

Figs. 4 to 5 are diagrams showing examples of a screen in which a wall and a window are identified as objects, a range to be designed or changed is set, and dimensions are recognized. Fig.

Dimension recognition of a method for providing an interior platform based on spatial recognition according to an embodiment of the present invention includes extracting a depth value for a feature point in an image with respect to an image received from the user terminal 200, Dimension recognition can be performed.

At this time, when a part of the object in the image is not identified, the server 100 can estimate the dimension of the object based on the data acquired through the machine learning.

As shown in FIG. 4, dimensional recognition for a wall is based on a depth value for a feature point of an object in the image corresponding to the user input when receiving a predetermined user input for the image from the user terminal 200 , And provide a dimension for the object corresponding to the user input.

Dimension recognition for a window as shown in Fig. 5 may be provided in a manner that recognizes dimensions for objects in an image corresponding to user input, such as dimension recognition for the wall described above.

Hereinafter, with reference to FIG. 6, description will be made in detail of providing a list of methods for providing an interior platform based on spatial recognition and changing a selection of items in a list according to an embodiment of the present invention.

The server 100 may provide the user terminal 200 with one or more lists of the interior product list, installation and construction list, and may receive selection inputs for one or more items in the list.

When the selection input for one or more of the items in the list is received through the user terminal 200, the selected item can be overlapped within the image, thereby virtually performing placement, installation, or construction of the interior product It can be possible.

In addition, when receiving a preset user input for an object in the image from the user terminal 200, the object is temporarily removed from the image, and after receiving the user input for selecting the replacement object, You can display an image for an object.

That is, upon receiving a selection input and removal input for a particular object (e.g., table, chair, TV, cabinet, etc.) contained within the image, the object is removed from the image, , The background may be inserted at the position where the object is removed, and the background may be provided through the user terminal 200.

When receiving a user input for a curtain in the list provided through the user terminal 200, the user recognizes the dimensions for the window area in which the curtain will be installed in the image, and determines the user input for a particular curtain in the curtain list corresponding to the user input And may overlay curtains corresponding to the user's drag input received from the user terminal 200 on the image.

At this time, the user can repeatedly perform the selection for the curtain, and the image of the curtain overlapping by the user's selection can be changed.

Thereafter, by a user request, an estimate of the overlapped curtain can be requested.

In addition, when receiving the user input for the paint in the installation and construction list, it recognizes the dimension and area information for the wall area to be painted in the image, and responds to the drag input of the user received from the user terminal 200 Color and texture to the image.

At this time, the color and texture may be changed by user input, and overlapping new colors and textures when receiving selection inputs for overlapped colors and textures and other colors and textures.

Thereafter, an estimate of the overlapped color and texture may be requested, and the estimate may be provided divided into material cost estimates and fill project estimates.

In the case of receiving the user input for the construction in the installation and construction list, the 3D modeling is performed on the recognized objects through the image, the conversion is made so that it can be independently selected and removed, the detailed list of the construction items is provided, Receive selection inputs for one or more items in the detail list, and overlap selected input items within the image.

If user input for the finish and color is received for the overlapped item, the finish and color can be mapped, the finish and color can be changed by user input, and if changed, the changed finish and color are mapped . It is also possible to change part or all of the recognized interior structure objects, and to select new shapes and materials of the structure.

Hereinafter, a method for providing an interior platform based on spatial recognition according to an embodiment of the present invention will be described in detail with reference to FIG.

The server 100 may store shape information including image information and standard information for a plurality of objects in advance in the server, and may store the shape information corresponding to the plurality of objects by machine learning.

In order to perform the method of providing the interior platform based on the user terminal 200, an image photographed through the camera of the user terminal 200 may be received (S310).

Thereafter, the server 100 recognizes the dimensions of objects and objects in the image and provides the dimensions to the user terminal 200 (S320).

At this time, it is possible to recognize the dimension based on the depth value of the feature points in the image by referring to the depth value acquired through the camera (e.g., a stereo camera) of the user terminal 200, So that one or more objects can be distinguished within the image.

In addition, when recognizing the divided objects based on the shape information previously stored in the server 100 and recognizing the dimensions of the objects, if a part of the objects in the image is not identified, So that the dimensions of the object can be estimated.

The server 100 may provide the user terminal 200 with one or more lists of the list of interior products, installation and construction list (S330).

When receiving a predetermined user input for an image from the user terminal 200, with the list provided, it is possible to determine, based on the depth value for the feature point of the object in the image corresponding to the user input, Dimensions can be provided.

Thereafter, the user terminal 200 may receive a selection input for one or more items in the list and provide a quotation (S340).

At this time, the object corresponding to the selected input item can be overlapped in the image. When receiving the preset user input for the object in the image from the user terminal 200, the object is temporarily removed from the image, After receiving the user input to select, the image of the substitute object can be displayed in the area where the object is removed.

It is possible to provide the interior platform service based on the spatial recognition to the user through steps S310 to S340, and steps S330 to S340 may be repeatedly performed by user input.

In addition, as a further embodiment of the present invention, the user terminal 200 may be a terminal capable of providing an HMD or VR and an AR, and may recognize and change real- It can be provided as a virtual reality.

For example, it is possible to wear a terminal such as an HMD or a pair of eyeglasses to realistically recognize an object, while realizing a virtual display of interior accessories, installation and construction according to user input.

If a stage design is needed, visitors can enjoy various stage design through a specific device on the stage without installing the actual stage as before, and it is possible to see through a simple device such as glasses instead of HMD.

In another embodiment of the present invention, when a particular building or structure is to be installed not only in the interior of the internal structure but also in the square, for example, it is possible to design and model the structure through the interior platform based on the space recognition, It can be projected onto the square.

The molding or structure projected in the actual space may be observed through the terminal 200 or projected onto the actual space through a separate projector for projecting the modeled structure into the space.

One embodiment of the present invention may also be embodied in the form of a recording medium including instructions executable by a computer, such as program modules, being executed by a computer. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, the computer readable medium may include both computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

While the methods and systems of the present invention have been described in connection with specific embodiments, some or all of those elements or operations may be implemented using a computer system having a general purpose hardware architecture.

It will be understood by those skilled in the art that the foregoing description of the present invention is for illustrative purposes only and that those of ordinary skill in the art can readily understand that various changes and modifications may be made without departing from the spirit or essential characteristics of the present invention. will be. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive. For example, each component described as a single entity may be distributed and implemented, and components described as being distributed may also be implemented in a combined form.

The scope of the present invention is defined by the appended claims rather than the detailed description and all changes or modifications derived from the meaning and scope of the claims and their equivalents are to be construed as being included within the scope of the present invention do.

10: System
100: server 200: user terminal

Claims (13)

A method for providing an interior platform based on spatial perception performed by a server,
(a) receiving an image photographed through a camera of a user terminal;
(b) recognizing dimensions of objects and objects in the image and providing the dimensions to the user terminals;
(c) providing the user terminal with at least one list of interior product list, installation and construction list; And
(d) receiving a selection input for one or more of the items in the list and providing a quotation;
, ≪ / RTI &
The dimensions,
A detail specification including at least one of the horizontal, vertical, height and three-dimensional coordinates of the object,
The step (b)
And the dimension is recognized based on the depth value for the feature points in the image, with reference to the depth value obtained through the camera of the user terminal.
A method of providing an interior platform based on spatial awareness.
The method according to claim 1,
Before the step (a)
The server comprises:
Storing configuration information including image information and standard information for a plurality of objects in a server and mechanically learning the configuration information corresponding to the plurality of objects,
A method of providing an interior platform based on spatial awareness.
3. The method of claim 2,
The step (b)
Further comprising analyzing feature points in the image to distinguish one or more objects within the image.
A method of providing an interior platform based on spatial awareness.
The method of claim 3,
The step (b)
Further comprising identifying the segmented object based on the shape information previously stored in the server.
A method of providing an interior platform based on spatial awareness.
5. The method of claim 4,
The step (b)
In recognizing the dimensions of the object,
If a portion of the object in the image is not identified,
Further comprising estimating a dimension of the object based on the data acquired through the machine learning,
A method of providing an interior platform based on spatial awareness.
The method according to claim 1,
The step (c)
When receiving a predetermined user input for the image from the user terminal,
And providing a dimension for an object corresponding to the user input based on a depth value for a feature point of the object in the image corresponding to the user input.
A method of providing an interior platform based on spatial awareness.
The method according to claim 1,
The step (d)
Further comprising overlapping an object corresponding to the selected entry in the image.
A method of providing an interior platform based on spatial awareness.
The method of claim 4, wherein
The step (d)
When receiving a predetermined user input for the object in the image from the user terminal,
Further comprising the step of temporarily removing the object from the image and displaying an image for the substitute object in an area from which the object has been removed after receiving a user input for selecting the substitute object,
A method of providing an interior platform based on spatial awareness.
The method according to claim 1,
When receiving user input for the curtain in the installation and construction list,
The step (d)
(d-1) recognizing a dimension for a window region in which the curtain in the image is to be installed;
(d-2) receiving a user input for a particular curtain in a curtain list corresponding to the user input;
(d-3) overlapping the curtain corresponding to the drag input of the user received from the user terminal with the image; And
(d-4) receiving an estimate request for the overlapping curtain;
Further comprising:
Wherein the steps (d-2) to (d-3) are repeatedly performed.
A method of providing an interior platform based on spatial awareness.
The method according to claim 1,
When receiving user input for the paint in the installation and construction list,
The step (d)
Recognizing dimension and area information for a wall area on which the in-image painting is to be performed;
Overlapping a color and a texture corresponding to a drag input of a user received from the user terminal with the image;
Receiving an estimate request for the overlapped color and texture; And
Further comprising:
The quotation above,
Which is divided into material cost estimates and polish construction estimates,
A method of providing an interior platform based on spatial awareness.
The method according to claim 1,
Wherein when receiving a user input for the construction in the installation and construction list, the step (d)
Performing 3D modeling on an object recognized through the image to convert the selected object to be independently selected and removed; providing a detailed list of the construction item; receiving selection input for one or more items in the detail list; ;
Overlapping the selected input item in the image; And
Mapping the finishing material and the color when a user input for the finishing material and color is received for the overlapped item; And
Receiving an estimate request for the overlapped item;
Further comprising:
The quotation above,
Which is divided into material cost estimates and construction estimates,
A method of providing an interior platform based on spatial awareness.
A server for performing an interior platform providing method based on spatial recognition,
A memory in which an interior platform providing program based on spatial recognition is stored; And
And a processor for executing the program,
The processor, upon execution of the program,
Receiving an image photographed through a camera of a user terminal, recognizing dimensions of objects and objects in the image and providing the user terminal with one or more lists of interior products list, installation and construction list Receiving a selection input for one or more of the items in the list, and providing a quotation,
The dimensions,
A detail specification including at least one of the horizontal, vertical, height and three-dimensional coordinates of the object,
And the dimension is recognized based on the depth value for the feature points in the image, with reference to the depth value obtained through the camera of the user terminal.
A server that performs a method of providing an interior platform based on spatial awareness.
A computer program product for performing a method of providing an interior platform based on spatial recognition according to claim 1.

Images