KR102312413B1 - artificial neural network-based space interior curating system and method therefor - Google Patents

Abstract

The present invention relates to an artificial neural network-based space interior curating system and a method thereof. The space interior curating system includes a management server provided to access the space interior curating system, analyze the indoor space, and provide recommended interior information, The management server includes a 3D spatial information generation unit; a user information acquisition unit; an interior concept recommendation information generation unit; and an interior prediction image providing unit.

Description

Artificial neural network-based space interior curating system and method therefor

The present invention relates to an artificial neural network-based spatial interior curating system and method, which analyzes and processes information received from a user using a deep learning model based on a plurality of human neural networks, thereby providing 3D spatial information for an indoor space, interior items By providing the user with the recommended interior recommendation information and the corresponding interior prediction image, the detailed recommendation information for each interior item is automatically provided based on the area information for each space in the room and the desired interior cost, and an interior company optimized for this is automatically provided. It is an object of the present invention to provide a system and method that can provide reasonable and transparent interior items and estimates.

As people's perception of a living space such as a house changes from a conventional residential space where they simply sleep to a place where they can enjoy hobbies and leisure life, interest in the interior of the house is also increasing.

In particular, as the demand for decorating living spaces according to individual design tendencies and personalities increases, various interior services and information are overflowing online. When wanting to change the interior of a new home, the resident has been used to request a quote for interior construction or service from an interior company.

However, in the case of the existing interior services provided, most of the cases are that the interior company directly visits the site, examines it, and conducts a consultation. It takes a considerable amount of time because you have to visit the site in person.

In recent years, due to the development of IT technology, using online services, the customer enters the desired interior options in the information input window provided in the form of an input window through the homepage of the interior company, or requests a consultation through the Internet. Therefore, there is a service to receive recommended interior information from the manager of an interior company.

However, in the case of these services, even if time and space limitations can be overcome, even if the same design estimate is requested, the construction cost required by each contractor is large, and the reliability of the information may be lowered because the contents of the estimate are not specific. However, there is a problem that in the process of collecting opinions on the interior and modifying it, there is a difference from the interior desired by the customer, so that the construction may not be carried out according to the contract or defects may occur due to the poor finishing of the construction.

The present invention relates to a space interior curating technology that can reduce the hassle and waste of time and money for a user to request a quotation from a number of interior companies to perform interior design, and more specifically, to a plurality of interior decoration companies based on artificial neural network. By using a deep learning model to automatically generate 3D spatial information about an indoor space, and using this to provide the user with interior recommendation information recommended for each interior item desired by the user and an interior prediction image accordingly, Users can receive an interior recommendation for a space they want to construct an interior with only by inputting preset data, and when the recommended interior is constructed, the expected result is provided as an image to quickly and intuitively select all interior-related matters and place an order for construction To provide an artificial neural network-based spatial interior curating system and method.

According to an embodiment of the present invention, an artificial neural network-based space interior curating system includes a management server provided to access the interior space curating system, analyze an indoor space, and provide recommended interior information, wherein the management server includes a 3D spatial information generation unit that generates indoor spatial information by analyzing image data or spatial structure data, and generates 3D spatial information implemented by 3D modeling of an interior object using the indoor spatial information; a user information obtaining unit which obtains detailed interior space information, user preference information, desired interior item information, and desired interior cost information from a user; Recommendation for a plurality of interior items so that the interior space detailed information, user preference information, desired interior item information, and desired interior cost information are input to the first deep learning model to enable construction within the desired interior cost range for the interior space an interior concept recommendation information generating unit generating information as interior concept recommendation information; When interior construction is completed according to the interior concept recommendation information by using the interior concept recommendation information and 3D space information, predicted image data that implements an expected result image as a 3D image is generated, and the generated predicted image data is provided to the user. and an interior prediction image providing unit to provide, wherein the first deep learning model receives construction details, construction company information, construction cost information, and construction satisfaction information for a plurality of constructed interiors as input to each input layer of the artificial neural network, It may be a model implemented based on an artificial neural network trained to extract interior concept recommendation information within a specific cost range, and the user information acquisition unit analyzes the interior interest information input from the user and classifies it into direct interest information and indirect interest information. and input the classified direct interest information and indirect interest information into the second deep learning model to determine preference for a plurality of interior items including interior items in which direct interest information exists as well as interior items in which only indirect interest information exists By calculating, user preference information may be generated.

According to an embodiment of the present invention, the 3D spatial information generating unit performs pre-processing by matching a plurality of image data captured in an indoor space according to a preset shooting standard using a depth camera to a reference image, and the pre-processing is performed. Analyze the plurality of image data to set descriptor data for each section, derive the descriptor data shared with each other among image data connected in space among the plurality of image data, and a plurality of images based on the derived descriptor data Creates spatial image information for the entire indoor space by matching the , and 3D modeling by analyzing the spatial image information to calculate the structure of the indoor space, identification of objects included in the indoor space, and shape, area, volume, and distance 3D spatial information implemented with

According to an embodiment of the present invention, the second deep learning model calculates an objective function and a sampling probability distribution for each of a plurality of users, and uses the direct interest information and the sampling probability distribution to obtain the plurality of selects direct interest information and indirect interest information that satisfy a preset criterion for the difference in preference for each user as a set of related items to generate a plurality of item sets, and based on the plurality of item sets, direct interest information or indirect interest information It is a model based on an artificial neural network trained to calculate a preference for a plurality of interior items according to interest information, and is based on the number of words included in text in the direct interest information and indirect interest information and the correlation between the words to calculate a calculated first score, calculate a second score calculated based on the similarity between the image and a pre-stored reference image for similarity calculation, and use the calculated first and second scores to correspond to the user's intention calculate the preference similarities of a plurality of interior items, and select a related item having the highest work similarity among a plurality of related items pre-stored in a preference item pool (POOL) as an item set based on the preference similarities .

According to an embodiment of the present invention, by inputting interior performance information and customer satisfaction information of a plurality of interior companies into a third deep learning model, interior concept information is recommended within the desired interior cost range among the plurality of interior companies with interior image data and Recommendation company that selects at least one interior company that can be constructed together as a construction company and provides it to the user, and transmits the order request data to the received interior company when receiving one of the interior companies among the final construction companies from the user Further comprising a providing unit, the third deep learning model receives the interior performance information and customer satisfaction information of a plurality of interior companies received from the user, desired construction time information, and construction schedule information for each company as input to each input layer of the artificial neural network. , it may be a model based on an artificial neural network learned to select at least one interior company that can be constructed among the at least one interior company selected according to construction time information as a construction company.

According to an embodiment of the present invention, the interior concept recommendation information generating unit may search for information on images, standards, and materials of interior goods included in the desired interior item information through a network, and parse ( parsing) to generate interior goods information, and the interior prediction image providing unit may generate a 3D image in which interior goods are provided indoors by using the generated interior goods information and 3D space information.

Through the artificial neural network-based spatial interior curating system implemented in accordance with an embodiment of the present invention, the user only needs to input the necessary information, and various interior concept information suitable for the user's preference and the projected construction image for the interior space generated accordingly By being provided, you can receive recommendations for interior information that more accurately matches the interior you want, and furthermore, you will be provided with information on the company that can construct the interior according to the interior concept information recommended at the user's desired time and the company's estimate. This has the effect of providing not only user convenience, but also transparent interior service.

1 is a diagram illustrating a connection between an artificial neural network-based spatial interior curating system and a user and an interior company implemented according to an embodiment of the present invention.
2 is a diagram illustrating a management server of a space interior curating system according to an embodiment of the present invention.
3 is a detailed configuration diagram of a management server included in the space interior curating system according to the first embodiment of the present invention.
4 is a detailed configuration diagram of a management server included in the space interior curating system according to the second embodiment of the present invention.
5 is a view showing the first, second, third, and fourth deep learning models included in the spatial interior curating system according to an embodiment of the present invention.
6 is a diagram illustrating the structure of an artificial neural network constituting the first, second, third, and fourth deep learning models shown in FIG. 5 .
7 is a flowchart of a space interior curating method based on an artificial neural network according to an embodiment of the present invention.

Hereinafter, with reference to the accompanying drawings, embodiments of the present invention will be described in detail so that those of ordinary skill in the art to which the present invention pertains can easily implement them. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein.

The terms used in the present invention are only used to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise.

In the present invention, terms such as "comprises" or "have" are intended to designate that the features, numbers, steps, operations, components, parts, or combinations thereof described in the specification exist, but one or more other features It should be understood that this does not preclude the existence or addition of numbers, steps, operations, components, parts, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

Terms such as those defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related art, and unless explicitly defined in the present invention, it should not be interpreted in an ideal or excessively formal meaning. does not

It will also be understood that each block of the drawings and combinations of flowchart diagrams may be implemented by computer program instructions, which may be mounted on a processor of a general purpose computer, special purpose computer, or other programmable data processing equipment. Thus, the instructions, executed by the processor of a computer or other programmable data processing equipment, will create means for performing the functions described in the flowchart block(s).

These computer program instructions may also be stored in a computer-usable or computer-readable memory that may direct a computer or other programmable data processing equipment to implement a function in a particular manner, and thus the computer-usable or computer-readable memory. It is also possible that the instructions stored in the flow chart block(s) produce an article of manufacture containing instruction means for performing the function described in the flowchart block(s).

The computer program instructions may also be mounted on a computer or other programmable data processing equipment, such that a series of operational steps are performed on the computer or other programmable data processing equipment to create a computer-executed process to create a computer or other programmable data processing equipment. It is also possible that instructions for performing the processing equipment provide steps for performing the functions described in the flowchart block(s).

Additionally, each block may represent a module, segment, or portion of code that includes one or more executable instructions for executing specified logical function(s).

And it should be noted that in some alternative embodiments it is also possible for the functions mentioned in the blocks to occur out of order. For example, two blocks shown one after another may in fact be performed substantially simultaneously, or it is possible that the blocks are sometimes performed in the reverse order according to the corresponding function.

At this time, the term '~ unit' used in this embodiment means software or hardware components such as field-programmable gate array (FPGA) or ASIC (Application Specific Integrated Circuit), and '~ unit' refers to what role carry out the

However, '-part' is not limited to software or hardware. The '~ unit' may be configured to reside on an addressable storage medium or may be configured to refresh one or more processors.

Thus, as an example, '~' denotes components such as software components, object-oriented software components, class components, and task components, and processes, functions, properties, and procedures. , subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables. The functions provided in the components and '~ units' may be combined into a smaller number of components and '~ units' or further separated into additional components and '~ units'. In addition, components and '~ units' may be implemented to play one or more CPUs in a device or secure multimedia card.

In describing the embodiments of the present invention in detail, an example of a specific system will be mainly targeted, but the main subject matter to be claimed in the present specification is to extend the scope disclosed herein to other communication systems and services having a similar technical background. It can be applied within a range that does not deviate significantly, and this will be possible at the discretion of a person with technical knowledge skilled in the art.

Hereinafter, an artificial neural network-based spatial interior curating system and method according to an embodiment of the present invention will be described with reference to the drawings.

1 is a diagram illustrating a connection between an artificial neural network-based spatial interior curating system and a user and an interior company implemented according to an embodiment of the present invention.

Referring to FIG. 1, the artificial neural network-based spatial interior curating system 1 according to an embodiment of the present invention may be connected to a plurality of users and terminals of a plurality of interior companies through a network, and receives user information from the user, Based on this, it is possible to provide suitable interior concept recommendation information to the user, and by receiving construction-related information from an interior company, write an order request form according to the user's choice, and provide the function of directly proceeding with the order to the interior company. .

2 is a view showing a management server of a space interior curating system according to an embodiment of the present invention.

Referring to FIG. 2 , the space interior curating system 1 may include a management server 10 , and when there is a large amount of data to be stored in the management server, a separate storage in which a database is built may be further included.

According to an embodiment of the present invention, the management server 10 may be provided to access the space interior curating system 1, analyze the indoor space, and provide recommended interior information, and has an operator resource and a storage space for this. can do.

According to an embodiment of the present invention, the management server 10 may receive information from a plurality of user terminals, and may transmit the interior concept recommendation information generated by processing the input information to the user terminal and provide it to the user. .

3 is a detailed configuration diagram of a management server included in the space interior curating system according to the first embodiment of the present invention.

Referring to FIG. 3 , the management server 10 included in the space interior curating system 1 according to the first embodiment of the present invention includes a 3D spatial information generation unit 100 , a user information acquisition unit 200 , and an interior concept recommendation. It may include an information generating unit 300 and an interior prediction image providing unit 400 .

The 3D spatial information generating unit 100 may generate indoor spatial information by analyzing the collected image data or spatial structure data, and may generate 3D spatial information implemented by 3D modeling of an interior object using the indoor spatial information.

According to an embodiment of the present invention, the image data may be input from a user and may be an image that can recognize the distances of the x, y, and z axes in three dimensions taken using a depth camera, but another According to an embodiment, it may be an image captured by a general camera. In this embodiment, the distances on the x, y, and z axes may be recognized through an algorithm capable of calculating depth information using the distances on the x and y axes.

According to an embodiment of the present invention, the spatial structure data may refer to data on the structure of an indoor space input by a user, and a predetermined type of structure is presented to the user, and a structure selected by the user among the presented structures and Any data that can identify the interior space as the interior object using interview data on the structure can be used without limitation.

According to an embodiment of the present invention, indoor space information may refer to information including information on a structure, an area, a detailed location, and an object located in the indoor space.

According to an embodiment of the present invention, 3D spatial information may refer to information implemented by 3D modeling of an indoor space, which is an interior object, using indoor spatial information. Based on this, interior construction is performed according to future recommended interior concept information. In this case, predicted image data that is an expected 3D image may be generated.

According to an embodiment of the present invention, pre-processing is performed by matching a plurality of image data captured in an indoor space according to a preset shooting standard using a depth camera to a reference image, and a plurality of pre-processed image data are analyzed. Descriptor data can be set for each section.

Here, the preset shooting standard may mean a standard including information on the location and height of the shooting point, the horizontal distance, and the shooting angle of the camera set accordingly, and compares it with the input image to determine whether it meets the criteria can be checked

According to an embodiment of the present invention, the pre-processing is performed by comparing image data taken at the same shooting angle with a reference image according to a preset standard in order to match a plurality of image data captured according to a preset shooting standard to a reference image. It is characterized in that it is determined, and when an error is identified, the corresponding data is corrected by adjusting the photographing angle of the reference image data to the photographing angle of the image data.

According to an embodiment of the present invention, descriptor data shared with each other between image data connected in space among a plurality of image data is derived, and a plurality of images are matched based on the derived descriptor data to form a spatial image for the entire indoor space. information can be generated.

Here, the descriptor data may mean data on numbers that can represent features calculated by converting meaningful features in an image into appropriate numbers in order to measure the similarity between two images.

According to an embodiment of the present invention, a photographing target of a plurality of image data may be an object included in an indoor space or a surrounding area thereof, and may be a part or all of an object having a specific shape and a surrounding area.

According to an embodiment of the present invention, the 3D spatial information generating unit 100 may find a characteristic part (scale-space extrema detection) for each image part for a plurality of acquired image data and classify it as a feature point, The final feature may be selected first, based on the pixel value (intensity) of the core feature and the position or size corresponding to the corner of the object among the key feature points by selecting a reliable final feature from among the feature points.

According to the above embodiment, the inclination of the peripheral area of the final feature point is calculated to obtain the direction indicated by the pixels of the peripheral area as a whole, and the part corresponding to the peripheral area is rotated so that the direction indicated by the pixels of the peripheral area is 0 degrees is set as descriptor data. can

According to an embodiment of the present invention, the descriptor data may include pixel values of a region surrounding the final feature point, and histogram information based on direction information of pixel values of the peripheral region based on the final feature point. By comparing a plurality of image data changed according to a photographing angle and identifying a target point representing the same section, image data adjacent to each other can be accurately matched.

According to an embodiment of the present invention, it is possible to generate 3D spatial information implemented by 3D modeling by analyzing spatial image information to calculate the structure of an indoor space, identification and shape, area, volume, and distance of objects included in the indoor space. have.

According to an embodiment of the present invention, a 3D space implemented by 3D modeling by analyzing 2D spatial image information to recognize a structure or object, and converting the recognized structure and object into 3D according to a preset 3D transformation standard information can be generated.

The user information obtaining unit 200 may obtain detailed interior space information, user preference information, desired interior item information, and desired interior cost information from the user.

Here, the detailed interior space information may refer to information on the detailed location and area of the whole or a specific part for which interior is to be performed among the interior spaces as interior objects.

In addition, user preference information may refer to information that can indicate preference for a specific interior item, and direct interest information indicating an explicit preference for a specific item and information that can infer preference through response, non-response, or relevance It may include indirect interest information including

According to an embodiment of the present invention, user preference information may mean information that can indicate preference for each improvement item, and includes direct interest information indicating an explicit preference for a specific item according to a criterion used in a collaborative filtering technique and It may be classified as indirect interest information including information for inferring preference through response, non-response, or relevance.

Here, the direct interest information may refer to user information directly described by the user or user's preference information for a specific improvement item obtained by analyzing the product evaluation. It may refer to information for predicting a user's preference by using the preference similarity or task similarity of items.

According to an embodiment of the present invention, by performing collaborative filtering on the user preference information included in the user information, it is possible to classify the detailed information included in the user preference information into direct interest information and indirect interest information for each desired improvement item. have.

According to an embodiment of the present invention, the desired interior item information may mean information on an interior item that the user wants to include in the interior, and the desired interior cost information is the interior cost determined to be available for the user to perform the interior. It may mean range information for .

According to an embodiment of the present invention, the user information acquisition unit 200 may classify the interior interest information received from the user into direct interest information and indirect interest information by analyzing the interior interest information, and provide the classified direct interest information and indirect interest information. 2 It is possible to generate user preference information by inputting into the deep learning model and calculating preferences for a plurality of interior items including interior items.

According to an embodiment of the present invention, in the case of an interior item in which direct interest information indicating an explicit preference for an interior item exists, an interior item recommended according to the preference may be presented, and direct interest information and indirect interest information are used. Thus, by calculating the preference for a plurality of interior items, it is possible to calculate the preference for the interior item for which there is no direct interest information, and furthermore, for the interior item for which the direct interest information exists, the weight value is calculated using the indirect interest information. It is possible to calculate a more accurate preference by giving it.

According to an embodiment of the present invention, the second deep learning model calculates an objective function and a sampling probability distribution for each of a plurality of users, and uses the interest information and the sampling probability distribution for each of the plurality of users. A plurality of item sets may be generated by selecting direct interest information and indirect interest information that satisfy a preset criterion for a difference in preference for , as a related item set.

According to an embodiment of the present invention, the objective function of the second deep learning model may be expressed as Equation (1).

는 사용자 u의 직접 관심 정보의 평가 벡터이며,

는 사용자 u의 간접 관심 정보인 평가 벡터이고,

는 사용자 u의 평가 정보 존재 유무에 대한 인디케이터 벡터이며,

는 활성화함수(sigmoid 또는 hyperbolic 탄젠트 함수)이며,

와

는 사용자 u의 인테리어 항목 i,j에 대한 평가 결과 예상치이며,

와

는 정규화항에 대한 중요도를 제어하기 위해 설정하는 파라미터이고

은딥러닝 모델에 대응되는 파라미터를 의미할 수 있다.where u is the user,

is an evaluation vector of user u's direct interest information,

is an evaluation vector that is indirect interest information of user u,

is an indicator vector for the presence or absence of user u's evaluation information,

is the activation function (sigmoid or hyperbolic tangent function),

Wow

is the estimated result of user u's evaluation of interior items i,j,

Wow

is a parameter set to control the importance of the regularization term.

may mean a parameter corresponding to the deep learning model.

According to an embodiment of the present invention, the sampling probability distribution may be configured such that an interior item having a low user's interest is sampled as an interior item having a low preference, and this sampling probability distribution may be expressed as Equation 2 below.

는 소프트맥스(softmax)함수를 의미할 수 있다.in Equation 2

may mean a softmax function.

According to an embodiment of the present invention, the second deep learning model is a model based on an artificial neural network trained to calculate a preference for a plurality of interior items according to direct interest information or indirect interest information based on a plurality of item sets. can

According to an embodiment of the present invention, the second deep learning model may calculate the first score calculated based on the number of words included in the text and the correlation between the words in the direct interest information and the indirect interest information, and the image and a second score calculated based on the degree of similarity with the previously stored reference image for calculating the degree of similarity may be calculated.

According to an embodiment of the present invention, preference similarities of a plurality of interior items corresponding to a user's intention are calculated using the calculated first and second scores, and a plurality of pre-stored items in a preference item pool (POOL) based on the preference similarities are calculated. A related item having the highest work similarity among the related items may be selected as the item set.

According to an embodiment of the present invention, a weight may be set for a plurality of interior tasks according to a preset condition, and a weighted sum (WEIGHTED SUM) of the work similarity calculated from the interior items to which the weight is set may be calculated. A specific item for the interior item may be recommended in consideration of the weighted sum.

According to an embodiment of the present invention, a specific interior item may be recommended through Equation 3 according to the work similarity.

Here, the POOL is a function that can correspond to a set of learning engines capable of performing the objective task that can be created in the present invention, and the Query function is the most similar to the task similarity calculation result with the object tasks stored in the POOL provided by the present invention. It may be a function that recommends a target operation.

The interior concept recommendation information generating unit 300 inputs interior space detailed information, user preference information, desired interior item information, and desired interior cost information to the first deep learning model, and construction is performed within the desired interior cost range for the interior space. Recommendation information for a plurality of interior items may be generated as interior concept recommendation information.

According to an embodiment of the present invention, the first deep learning model may be implemented as a convolutional artificial neural network model, but is not limited thereto. Each input of detailed interior space information, user preference information, desired interior item information, and desired interior cost information Recommendation information for a plurality of interior items can be derived so that construction can be performed within a desired interior cost range for an interior space by performing a convolution operation by inputting it into a floor.

According to an embodiment of the present invention, the interior concept recommendation information generating unit 300 may search for information on images, standards, and materials of interior goods included in desired interior item information through a network, and parse the searched information. (parsing) to generate interior goods information.

According to an embodiment of the present invention, information on images, specifications, and materials for interior goods may be collected in the form of image text, and by categorizing data types through parsing and grouping them into meaningful items, the interior You can create product information.

According to an embodiment of the present invention, a structure and an object recognized using the generated interior product information are converted into 3D according to a preset 3D conversion standard, and the interior product is provided in the room by reflecting this in 3D spatial information. 3D images can be created.

According to an embodiment of the present invention, the interior concept recommendation information generating unit 300 inputs detailed interior space information, 3D spatial information, and interior concept recommendation information to each input layer of the fourth deep learning model and uses it through a convolution operation. Thus, it is possible to calculate the energy efficiency index calculated according to the material and construction method for each interior item according to the structure, area, and interior concept recommendation information of the indoor space.

Here, the energy efficiency index can be calculated according to the average temperature maintained when the same cooling or heating is supplied per predetermined area within the same temperature environment. It can be judged that it can be kept cooler or warmer.

According to an embodiment of the present invention, the energy efficiency index can be provided to the user together with the interior concept recommendation information so that the user can refer to it when selecting one of a plurality of interior concept recommendation information or expected image data according thereto, and According to another embodiment, by setting a reference energy efficiency index, the interior concept recommendation information having a lower energy efficiency index than the reference energy efficiency index may be excluded altogether or provided to the user by specifying that the energy efficiency is poor, thereby helping the user to make an efficient selection.

The interior prediction image providing unit 400 uses the interior concept recommendation information and the 3D space information to generate predicted image data that implements the expected result image as a 3D image when the interior construction is completed according to the interior concept recommendation information, and the generated Expected image data may be provided to the user.

According to an embodiment of the present invention, the interior prediction image providing unit 400 may provide the predicted image data to the user through the user terminal together with the corresponding interior concept recommendation information, and may provide a plurality of interior concept recommendation information and predicted image data accordingly. Among them, the user can select one and select the desired interior concept.

According to an embodiment of the present invention, the interior prediction image providing unit 400 may generate a 3D image in which interior goods are provided indoors by using the generated interior goods information and 3D space information.

4 is a detailed configuration diagram of the management server 11 included in the space interior curating system according to the second embodiment of the present invention.

4, the management server 11 included in the space interior curating system according to the second embodiment of the present invention is shown, and the management server 11 included in the space interior curating system according to the second embodiment is the first The management server 10 included in the space interior curating system according to the first embodiment may further include a recommendation company providing unit 500 .

The recommendation company providing unit 500 inputs the interior performance information and customer satisfaction information of a plurality of interior companies into the third deep learning model, and constructs the interior concept information within the desired interior cost range among the plurality of interior companies together with the recommended interior image data. At least one possible interior company may be selected as a construction company and provided to the user.

According to an embodiment of the present invention, interior performance information and customer satisfaction information of a plurality of interior companies are input to each input layer of the third deep learning modeling, and interior concept information is recommended within a desired interior cost range among a plurality of interior companies. Interior image At least one interior company that can be constructed like the data may be selected, and the selected at least one interior company may be provided to the user in the form of a list.

According to an embodiment of the present invention, the user may select one of the interior companies list and transmit it to the management server 11 .

According to an embodiment of the present invention, the recommendation company providing unit 500 may transmit the order request data to the received interior company when receiving one of the interior companies among the final construction possible companies from the user.

According to an embodiment of the present invention, order request data may be transmitted to a selected interior company, and the order request data may include all of interior concept recommendation information, desired interior cost information, and expected image data.

According to an embodiment of the present invention, the third deep learning model receives interior performance information and customer satisfaction information of a plurality of interior companies received from the user, desired construction time information, and construction schedule information for each company into each input layer of the artificial neural network. , it may be a model based on an artificial neural network trained to select at least one interior company that can be constructed among at least one interior company selected according to the construction time information as a construction company.

5 is a view showing the first, second, third, and fourth deep learning models included in the spatial interior curating system according to an embodiment of the present invention.

Referring to FIG. 5 , first, second, third, and fourth deep learning models that may be included in the space interior curating system 1 according to an embodiment of the present invention are shown.

According to an embodiment of the present invention, in the management server 12 included in the space interior curating system 1, the first deep learning model is the interior concept recommendation information generating unit 300, and the second deep learning model is the user information The acquisition unit 200, the third deep learning model may be included in the recommendation company providing unit 500, and the fourth deep learning model may be included in the interior concept recommendation information generation unit 300, and each deep learning model is a space interior curating system ( It can be driven independently in each configuration of 1), but according to each embodiment, the first, second, third, and fourth deep learning models are organically combined with each other and the information calculated from each deep learning model is input to the input layer. can be used as

6 is a diagram illustrating the structure of an artificial neural network constituting the first, second, third, and fourth deep learning models shown in FIG. 5 .

Referring to FIG. 6 , the structure of the artificial neural network constituting the first, second, third, and fourth deep learning models implemented according to an embodiment of the present invention is shown, and the first, second, third, and fourth deep learning models are It may be implemented as a convolutional artificial neural network model, but is not limited thereto, and may be implemented as a recurrent neural network (RNN).

According to an embodiment of the present invention, 1, 2, 3, and 4 deep learning models may be an artificial neural network composed of an input layer, a hidden layer, and an output layer, and each data input through the input layer goes through a plurality of hidden layers and performs a convolution operation. , and through this, the final data can be output through the output layer.

7 is a flowchart of an artificial neural network-based spatial interior curating method according to an embodiment of the present invention.

By analyzing image data or spatial structure data, indoor spatial information is generated (S10).

According to an embodiment of the present invention, indoor spatial information may be generated by analyzing the collected image data or spatial structure data.

According to an embodiment of the present invention, the image data may be input from a user and may be an image that can recognize the distances of the x, y, and z axes in three dimensions taken using a depth camera, but another According to an embodiment, it may be an image captured by a general camera. In this embodiment, the distances on the x, y, and z axes may be recognized through an algorithm capable of calculating depth information using the distances on the x and y axes.

3D spatial information implemented by 3D modeling of an interior object is generated using the indoor spatial information (S20).

According to an embodiment of the present invention, 3D spatial information implemented by 3D modeling of an interior object may be generated using indoor spatial information, and the 3D spatial information may be implemented by 3D modeling of an indoor space as an interior object using indoor spatial information information, and based on this, when interior construction is performed according to future recommended interior concept information, predicted image data, which is an expected 3D image, can be generated.

According to an embodiment of the present invention, pre-processing is performed by matching a plurality of image data captured in an indoor space according to a preset shooting standard using a depth camera to a reference image, and a plurality of pre-processed image data are analyzed. Descriptor data can be set for each section.

According to an embodiment of the present invention, the pre-processing is performed by comparing image data taken at the same shooting angle with a reference image according to a preset standard in order to match a plurality of image data captured according to a preset shooting standard to a reference image. It is characterized in that it is determined, and when an error is identified, the corresponding data is corrected by adjusting the photographing angle of the reference image data to the photographing angle of the image data.

According to an embodiment of the present invention, descriptor data shared with each other between image data connected in space among a plurality of image data is derived, and a plurality of images are matched based on the derived descriptor data to form a spatial image for the entire indoor space. information can be generated.

According to an embodiment of the present invention, it is possible to find a characteristic part (scale-space extrema detection) for each image part with respect to a plurality of acquired image data and classify it as a characteristic point, and among the classified characteristic points, a core that is a reliable final characteristic point By selecting the feature points, the final feature point may be first selected based on the pixel value (Intensity) of the key feature points and the position or size corresponding to the corner of the object among the key feature points.

According to the above embodiment, the inclination of the peripheral area of the final feature point is calculated to obtain the direction indicated by the pixels of the peripheral area as a whole, and the part corresponding to the peripheral area is rotated so that the direction indicated by the pixels of the peripheral area is 0 degrees is set as descriptor data. can

According to an embodiment of the present invention, the descriptor data may include pixel values of a region surrounding the final feature point, and histogram information based on direction information of pixel values of the peripheral region based on the final feature point. By comparing a plurality of image data changed according to a photographing angle and identifying a target point representing the same section, image data adjacent to each other can be accurately matched.

According to an embodiment of the present invention, it is possible to generate 3D spatial information implemented by 3D modeling by analyzing spatial image information to calculate the structure of an indoor space, identification and shape, area, volume, and distance of objects included in the indoor space. have.

According to an embodiment of the present invention, a 3D space implemented by 3D modeling by analyzing 2D spatial image information to recognize a structure or object, and converting the recognized structure and object into 3D according to a preset 3D transformation standard information can be generated.

It acquires detailed interior space information, user preference information, desired interior item information, and desired interior cost information (S30).

According to an embodiment of the present invention, detailed interior space information, user preference information, desired interior item information, and desired interior cost information may be obtained from a user.

According to an embodiment of the present invention, the desired interior item information may mean information on an interior item that the user wants to include in the interior, and the desired interior cost information is the interior cost determined to be available for the user to perform the interior. It may mean range information for .

According to an embodiment of the present invention, by analyzing the interior interest information input from the user, it is possible to classify the interior interest information into direct interest information and indirect interest information, and input the classified direct interest information and indirect interest information into the second deep learning model. User preference information may be generated by calculating a preference for a plurality of interior items including the item.

According to an embodiment of the present invention, in the case of an interior item in which direct interest information indicating an explicit preference for an interior item exists, an interior item recommended according to the preference may be presented, and direct interest information and indirect interest information are used. Thus, by calculating the preference for a plurality of interior items, it is possible to calculate the preference for the interior item for which there is no direct interest information, and furthermore, for the interior item for which the direct interest information exists, the weight value is calculated using the indirect interest information. It is possible to calculate a more accurate preference by giving it.

According to an embodiment of the present invention, the second deep learning model calculates an objective function and a sampling probability distribution for each of a plurality of users, and uses the interest information and the sampling probability distribution for each of the plurality of users. A plurality of item sets may be generated by selecting direct interest information and indirect interest information that satisfy a preset criterion for a difference in preference for , as a related item set.

According to an embodiment of the present invention, the sampling probability distribution may be configured such that an interior item having a low user's interest is sampled as an interior item having a low preference.

According to an embodiment of the present invention, the second deep learning model is a model based on an artificial neural network trained to calculate a preference for a plurality of interior items according to direct interest information or indirect interest information based on a plurality of item sets. can

According to an embodiment of the present invention, the second deep learning model may calculate the first score calculated based on the number of words included in the text and the correlation between the words in the direct interest information and the indirect interest information, and the image and a second score calculated based on the degree of similarity with the previously stored reference image for calculating the degree of similarity may be calculated.

According to an embodiment of the present invention, preference similarities of a plurality of interior items corresponding to a user's intention are calculated using the calculated first and second scores, and pre-stored in a preference item pool (POOL) based on the preference similarities. A related item having the highest work similarity among a plurality of related items may be selected as the item set.

According to an embodiment of the present invention, a weight may be set for a plurality of interior tasks according to a preset condition, and a weighted sum (WEIGHTED SUM) of the work similarity calculated from the interior items to which the weight is set may be calculated. A specific item for the interior item may be recommended in consideration of the weighted sum.

Recommendation information for a plurality of interior items is generated as interior concept recommendation information through the first deep learning model (S40).

According to an embodiment of the present invention, by inputting detailed interior space information, user preference information, desired interior item information, and desired interior cost information to the first deep learning model, the interior space can be constructed within the desired interior cost range. Recommendation information for a plurality of interior items may be generated as interior concept recommendation information.

According to an embodiment of the present invention, the first deep learning model may be implemented as a convolutional artificial neural network model, but is not limited thereto. Each input of detailed interior space information, user preference information, desired interior item information, and desired interior cost information Recommendation information for a plurality of interior items can be derived so that construction can be performed within a desired interior cost range for an interior space by performing a convolution operation by inputting it into a floor.

According to an embodiment of the present invention, information on images, standards, and materials for interior goods included in desired interior item information can be searched through a network, and interior goods information can be generated by parsing the retrieved information. can

According to an embodiment of the present invention, information on images, specifications, and materials for interior goods may be collected in the form of image text, and by categorizing data types through parsing and grouping them into meaningful items, the interior You can create product information.

According to an embodiment of the present invention, a structure and an object recognized using the generated interior product information are converted into 3D according to a preset 3D conversion standard, and the interior product is provided in the room by reflecting this in 3D spatial information. 3D images can be created.

According to an embodiment of the present invention, interior space detailed information, 3D spatial information, and interior concept recommendation information are input to each input layer of the fourth deep learning model and used through convolution operation to determine the structure, area, and interior concept of an indoor space. The energy efficiency index calculated according to the material and construction method for each interior item according to the recommended information can be calculated.

According to an embodiment of the present invention, the energy efficiency index can be provided to the user together with the interior concept recommendation information so that the user can refer to it when selecting one of a plurality of interior concept recommendation information or expected image data according thereto, and According to another embodiment, by setting a reference energy efficiency index, the interior concept recommendation information having a lower energy efficiency index than the reference energy efficiency index may be excluded altogether or provided to the user by specifying that the energy efficiency is poor, thereby helping the user to make an efficient selection.

When the interior construction is completed, the expected image data obtained by implementing the expected result image as a 3D image is generated (S50).

According to an embodiment of the present invention, when interior construction is completed according to the interior concept recommendation information using the interior concept recommendation information and the 3D space information, it is possible to generate predicted image data in which an expected result image is implemented as a 3D image.

According to an embodiment of the present invention, the interior prediction image providing unit 400 may generate a 3D image in which interior goods are provided indoors by using the generated interior goods information and 3D space information.

The generated predicted image data is provided to the user (S60).

According to an embodiment of the present invention, the predicted image data may be provided to the user through the user terminal, and may be provided to the user through the user terminal together with the corresponding interior concept recommendation information. The user can select one of the image data and select the desired interior concept.

The embodiments of the present invention are not implemented only through the devices and/or methods described above, and although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and the following claims Various modifications and improvements by those skilled in the art using the basic concept of the present invention as defined in are also within the scope of the present invention.

Claims (5)

It includes a management server provided to access the interior space curating system, analyze the interior space, and provide recommended interior information,
The management server,
a 3D spatial information generating unit generating indoor spatial information by analyzing the collected image data or spatial structure data, and generating 3D spatial information in which an interior object is implemented by 3D modeling using the indoor spatial information;
a user information acquisition unit configured to acquire detailed interior space information, user preference information, desired interior item information, and desired interior cost information from a user;
Recommendation for a number of interior items so that the interior space detailed information, user preference information, desired interior item information, and desired interior cost information are input to the first deep learning model to enable construction within the desired interior cost range for the interior space an interior concept recommendation information generating unit generating information as interior concept recommendation information;
When interior construction is completed according to the interior concept recommendation information by using the interior concept recommendation information and 3D space information, predicted image data that implements an expected result image as a 3D image is generated, and the generated predicted image data is provided to the user. Including an interior prediction image providing unit to provide,
The first deep learning model receives construction details, construction company information, construction cost information, and construction satisfaction information for a plurality of constructed interiors into each input layer of the artificial neural network, and extracts interior concept recommendation information within a specific cost range It is a model implemented based on an artificial neural network trained to
The user information obtaining unit,
Direct interest information indicating an explicit preference for a specific item by analyzing the interior interest information input from the user, or non-response to a specific item and the direct interest information to calculate a preference for a plurality of specific items Classified as indirect interest information, and input the classified direct interest information and indirect interest information into the second deep learning model, as well as interior items in which direct interest information exists, as well as interior items and direct interest information in which only indirect interest information exists generating user preference information by calculating preferences for a plurality of interior items including interior items in which all indirect interest information exists;
The direct interest information is an artificial neural network-based space interior curating system, characterized in that it includes user preference information for a specific improvement item obtained by analyzing user information or product evaluation directly described by the user. According to claim 1, wherein the 3D spatial information generating unit,
Pre-processing is performed by matching a plurality of image data captured in an indoor space according to a preset shooting standard using a depth camera to a reference image, and descriptor data is set for each section by analyzing the plurality of image data on which the pre-processing has been performed, ,
Deriving the descriptor data shared with each other among the image data connected in space among the plurality of image data, and matching a plurality of images based on the derived descriptor data to generate spatial image information for the entire indoor space,
Artificial neural network-based, characterized in that by analyzing the spatial image information to generate 3D spatial information implemented by 3D modeling by calculating the structure of the indoor space, identification of objects included in the indoor space, and shape, area, volume, and distance of space interior curating system. According to claim 1, wherein the user information acquisition unit,
The second deep learning model calculates an objective function and a sampling probability distribution for each of the plurality of users, and uses the direct interest information and the sampling probability distribution to create a difference in preference for each of the plurality of users. selects direct interest information and indirect interest information satisfying a preset criterion as a related item set to generate a plurality of item sets, and a plurality of interior items according to direct interest information or indirect interest information based on the plurality of item sets It is a model based on an artificial neural network trained to calculate a preference for
A first score calculated based on the number of words included in the text in the direct interest information and the indirect interest information and a correlation between the words is calculated, and based on the similarity between the image and a pre-stored reference image for similarity calculation calculates the calculated second score, calculates preference similarities of a plurality of interior items corresponding to the user's intention using the calculated first and second scores, and based on the preference similarities, a preference item pool (POOL) An artificial neural network-based spatial interior curating system, characterized in that a related item having the highest work similarity among a plurality of related items previously stored in the . The method of claim 1,
At least one interior that can be constructed like interior image data by inputting interior performance information and customer satisfaction information of a plurality of interior companies into the third deep learning model, and recommending interior concept information within the desired interior cost range among the plurality of interior companies Further comprising a recommendation company providing unit that selects a company as a construction company and provides it to the user, and transmits the order request data to the received interior company when receiving one of the interior companies among the final construction companies from the user,
The third deep learning model receives interior performance information, customer satisfaction information, desired construction time information, and construction schedule information for each company received from the user into each input layer of the artificial neural network, and according to the construction time information An artificial neural network-based spatial interior curating system, characterized in that it is a model based on an artificial neural network trained to select at least one interior company that can be constructed among the selected at least one interior company as a construction company. The method of claim 1 .
The interior concept recommendation information generating unit may search for information on images, specifications, and materials of interior goods included in the desired interior item information through a network, and parses the searched information to generate interior goods information can do,
The interior prediction image providing unit uses the generated interior product information and 3D spatial information to generate a 3D image in which interior products are provided indoors.

Images