KR20190093813A - Method and system for recommending product based on artificial intelligence - Google Patents

Abstract

Disclosed are a method for recommending a product based on artificial intelligence and a system thereof. The method for recommending a product and the system thereof can recommend a product matching the given product based on a degree of matching between the products through a deep neural network (DNN) learning model using a matching product set and a non-matching product set or recommend a product of a real coordination set and a similar coordination set for the given product based on the generation model using GAN.

Description

AI-based product recommendation method and system {METHOD AND SYSTEM FOR RECOMMENDING PRODUCT BASED ON ARTIFICIAL INTELLIGENCE}

The description below relates to a technique for automatically recommending a suitable product for a given product.

The fashion coordinator system is a system for recommending goods or services when a user selects one garment to be worn and selects another garment suitable for it.

The system for recommending a product or service is largely divided into a collaborative filtering method using similarity between users and a content-based filtering method using characteristic values of the product and the user.

For example, Korean Patent Registration No. 10-0882716 (Registration Date 02.02.2009) uses the product code, product name or model name of the product recognized by the information recommendation agent installed in the user terminal automatically product information about the product Recommended techniques are disclosed.

Provides a method and system that can provide artificial intelligence-based product coordinator capabilities.

Through a deep neural network (DNN) learning model using a matching product set and a matching product set, it provides a method and system for recommending a product that matches a given product based on the degree of matching between the products.

It provides a method and system for recommending a product in a coordinating set similar to a real coordinating set for a given product based on a generative model.

A computer-implemented product recommendation method comprising the steps of: collecting a set of correct image images of matching product images; Collecting a wrong answer image set consisting of mismatched product images based on the correct answer image set; Constructing a product recommendation model by performing deep neural network (DNN) based machine learning on the correct answer image set and the incorrect answer image set; And recommending a product matching with the corresponding product through the product recommendation model with respect to a given product, wherein the DNN is a model that outputs the degree of matching by receiving two product images as an input. to provide.

According to an aspect, the collecting of the correct answer image set may be configured as a matching product image set by recognizing individual product images from an image including a plurality of products.

According to another aspect, the collecting of the incorrect answer image set based on the correct answer image set may include, for an arbitrary product image, another product image in the correct answer image set including the random product image, and the other product image. At least some of the product images except the similar image of the correct answer image set including the similar product image, the similar product image of the random product image, and the similar product image of the another product image may be configured as a mismatched product image set. have.

According to another aspect, the step of constructing, for the DNN model, the feature extracted from the image is vectorized and mapped to the multi-dimensional vector space, but in the case of the correct answer image set closer to the distance between the image vectors, the incorrect answer image set In this case, the distance between the image vectors can be learned to be far.

A computer-implemented product recommendation method comprising the steps of: collecting a set of correct image images of matching product images; And building a product recommendation model based on a generative model by learning the correct answer image set, wherein the product recommendation model receives a product image as an input and compares characteristics of other products that match the input product. It provides a product recommendation method, characterized in that to generate information, and to determine a product similar to the generated characteristics as a recommended product.

According to an aspect, the collecting may be configured as a set of product images matching and recognizing individual product images from an image including a plurality of products.

According to another aspect, the building may include learning the correct image set using an encoder-decoder model as the generation model.

According to another aspect, the step of constructing, learning the discrimination model for learning the authenticity of the generation result in order to advance the generation model together, the discrimination model is a given image set using a DB where the coordinate sets are accumulated It may be determined whether is generated as a result of the generation model or the actual coordination set on the DB.

According to another aspect, the constructing may include learning the correct image set using a variational auto-encoder (VAE) which is a model having a conditional variational distribution as the generation model. have.

In combination with a computer, a computer program stored in a computer readable recording medium for executing the product recommendation method on a computer is provided.

Provided is a computer-readable recording medium, in which a program for executing the product recommendation method on a computer is recorded.

A computer-implemented product recommendation system, comprising: at least one processor configured to execute computer readable instructions, the at least one processor collecting a set of correct image images of matching product images, An image collector configured to collect an incorrect image set including an image based on the correct image set; A learning processor configured to construct a product recommendation model by performing machine learning based on deep neural network (DNN) on the correct image set and the incorrect image set; And a product recommendation unit for recommending a product matching the corresponding product through the product recommendation model with respect to a given product, wherein the DNN is a model that outputs a degree of matching by receiving two product images as inputs. To provide.

A computer-implemented product recommendation system, comprising: at least one processor implemented to execute instructions readable by a computer, the at least one processor comprising: an image collection unit configured to collect a correct image set of matching product images; And a learning processor configured to construct a product recommendation model based on a generative model by learning the correct answer image set, wherein the product recommendation model receives a product image as an input and applies characteristics of other products that match the input product. It provides a product recommendation system, characterized in that for generating information about, and to determine a product similar to the generated characteristics as a recommended product.

According to embodiments of the present invention, a DNN learning model using a matching product set and a matching product set may recommend a product that matches a given product based on the degree of matching between the products.

According to embodiments of the present invention, in the recommendation of coordination using a generation model, a product of a coordination set similar to the actual coordination set may be recommended for a given product through the enhancement of the generation model using the GAN.

1 is a diagram illustrating an example of a network environment according to an embodiment of the present invention.
2 is a block diagram illustrating an internal configuration of an electronic device and a server according to an embodiment of the present invention.
3 is a diagram illustrating an example of components that may be included in a processor of a server according to an embodiment of the present invention.
4 is a diagram illustrating a product recommendation method that may be performed by a processor of a server according to the first exemplary embodiment of the present invention.
5 is an exemplary diagram for explaining a process of collecting a set of correct answer images.
6 is an exemplary diagram for explaining a process of collecting an incorrect image set.
7 is a diagram illustrating a DNN-based product recommendation model according to a first embodiment of the present invention.
8 is a diagram illustrating a product recommendation method that can be performed by a processor of a server according to a second embodiment of the present invention.
9 is an exemplary diagram illustrating a product recommendation model based on a generation model according to a second embodiment of the present invention.
10 is an exemplary diagram for explaining a process of upgrading a generation model according to a second embodiment of the present invention through a GAN learning technique.
FIG. 11 is an exemplary view for explaining a process of upgrading a generation model through layer refinement according to the second embodiment of the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Embodiments of the present invention relate to a technique for automatically recommending a suitable product for a given product.

The product recommendation method according to embodiments of the present invention may be performed through a computer device, such as an electronic device or a server, to be described later. In this case, a computer program according to an embodiment of the present invention may be installed and run on a computer device, and the computer device may perform a product recommendation method according to an embodiment of the present invention under control of the driven computer program. . The above-described computer program may be stored in a computer-readable recording medium in combination with a computer device to cause a computer to execute a product recommendation method. For example, the server serves as a server for providing product information, and in particular, as an artificial intelligence-based product coordinator, it can automatically recommend a product that matches the product for a given product. According to an embodiment, it may also be considered that the electronic device performs a product coordinator function.

Hereinafter, specific embodiments of the fashion coordination recommendation model will be described, but are not limited thereto. Embodiments of the present invention may be applied to various coordination recommendation models such as space coordination, home coordination, food coordination, and color coordination in addition to fashion coordination.

1 is a diagram illustrating an example of a network environment according to an embodiment of the present invention. The network environment of FIG. 1 illustrates an example including a plurality of electronic devices 110, 120, 130, and 140, a plurality of servers 150 and 160, and a network 170. 1 is an example for describing the present invention, and the number of electronic devices or the number of servers is not limited as shown in FIG. 1.

The plurality of electronic devices 110, 120, 130, and 140 may be fixed terminals or mobile terminals implemented as computer devices. Examples of the plurality of electronic devices 110, 120, 130, and 140 include smart phones, mobile phones, navigation systems, computers, notebook computers, digital broadcasting terminals, personal digital assistants (PDAs), and portable multimedia players (PMPs). ), Tablet PCs, game consoles, wearable devices, wearable devices, Internet of things (IoT) devices, virtual reality (VR) devices, augmented reality (AR) devices, and the like. For example, although FIG. 1 illustrates the shape of a smart phone as an example of the electronic device 1 110, in the embodiments of the present invention, the electronic device 1 110 may use a wireless or wired communication method to substantially connect the network 170. It may mean one of various physical computer devices that can communicate with other electronic devices 120, 130, 140 and / or servers 150, 160.

The communication method is not limited and includes not only a communication method using a communication network (for example, a mobile communication network, a wired internet, a wireless internet, a broadcasting network, a satellite network, etc.) that the network 170 may include, but also a short range wireless communication between devices. Can be. For example, the network 170 may include a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), and a broadband network (BBN). And one or more of networks such as the Internet. The network 170 may also include any one or more of network topologies, including bus networks, star networks, ring networks, mesh networks, star-bus networks, trees, or hierarchical networks, but It is not limited.

Each of the servers 150 and 160 communicates with the plurality of electronic devices 110, 120, 130, and 140 through the network 170 to provide a command, code, file, content, service, or the like. It may be implemented in devices. For example, the server 150 may be a system that provides a first service to a plurality of electronic devices 110, 120, 130, and 140 connected through the network 170, and the server 160 may also have a network ( It may be a system that provides a second service to the plurality of electronic devices 110, 120, 130, and 140 connected through the 170. As a more specific example, the server 150 may provide a service (for example, a product recommendation service, etc.) for the corresponding application through an application as a computer program installed and driven in the plurality of electronic devices 110, 120, 130, and 140. May be provided to the plurality of electronic devices 110, 120, 130, and 140 as a first service. As another example, the server 160 may provide a service for distributing the file for installing and driving the above application to the plurality of electronic devices 110, 120, 130, and 140 as a second service.

2 is a block diagram illustrating an internal configuration of an electronic device and a server according to an embodiment of the present invention. 2 illustrates an internal configuration of the electronic device 1 110 and the server 150 as an example of the electronic device. In addition, the other electronic devices 120, 130, 140, or the server 160 may also have the same or similar internal configuration as the aforementioned electronic device 1 110 or the server 150.

The electronic device 1 110 and the server 150 may include memories 211 and 221, processors 212 and 222, communication modules 213 and 223, and input / output interfaces 214 and 224. The memories 211 and 221 are non-transitory computer-readable recording media. The memories 211 and 221 are non-transitory computer-readable recording media. The memories 211 and 221 may be non-transitory computer-readable recording media, such as random access memory (RAM), read only memory (ROM), disk drive, solid state drive (SSD), flash memory, or the like. Permanent mass storage devices. In this case, the non-volatile mass storage device such as a ROM, an SSD, a flash memory, a disk drive, or the like may be included in the electronic device 1 110 or the server 150 as a separate persistent storage device that is distinct from the memories 211 and 221. In addition, the memory 211, 221 includes an operating system and at least one program code (for example, a browser installed and driven in the electronic device 1 110 or an application installed in the electronic device 1 110 to provide a specific service). Code) can be stored. These software components may be loaded from a computer readable recording medium separate from the memories 211 and 221. Such a separate computer-readable recording medium may include a computer-readable recording medium such as a floppy drive, disk, tape, DVD / CD-ROM drive, memory card, and the like. In other embodiments, software components may be loaded into the memory 211, 221 through a communication module 213, 223 that is not a computer readable recording medium. For example, at least one program is a computer program that is installed by files provided by a file distribution system (for example, the server 160 described above) through the network 170 to distribute installation files of developers or applications. It may be loaded into the memories 211 and 221 based on (for example, the above-described application).

Processors 212 and 222 may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input / output operations. Instructions may be provided to the processors 212, 222 by the memory 211, 221 or the communication modules 213, 223. For example, the processors 212 and 222 may be configured to execute a command received according to a program code stored in a recording device such as the memory 211 and 221.

The communication modules 213 and 223 may provide a function for the electronic device 1 110 and the server 150 to communicate with each other through the network 170, and the electronic device 1 110 and / or the server 150 may communicate with each other. May provide a function for communicating with another electronic device (eg, electronic device 2 120) or another server (eg, server 160). For example, a request generated by the processor 212 of the electronic device 1 110 according to a program code stored in a recording device such as the memory 211 may be controlled by the server 170 through the network 170 under the control of the communication module 213. 150). Conversely, control signals, commands, contents, files, and the like provided according to the control of the processor 222 of the server 150 are transmitted to the communication module of the electronic device 1 110 via the communication module 223 and the network 170 ( It may be received by the electronic device 1110 through 213. For example, the control signal, command, content, file, etc. of the server 150 received through the communication module 213 may be transmitted to the processor 212 or the memory 211, and the content, file, etc. may be transferred to the electronic device 1. 110 may be stored as a storage medium (permanent storage described above) that may further include.

The input / output interface 214 may be a means for interfacing with the input / output device 215. For example, an input device may include a device such as a keyboard, a mouse, a microphone, a camera, and the like, and an output device may include a device such as a display, a speaker, a haptic feedback device, and the like. As another example, the input / output interface 214 may be a means for interfacing with a device in which functions for input and output are integrated into one, such as a touch screen. The input / output device 215 may be configured as one device with the electronic device 1110. In addition, the input / output interface 224 of the server 150 may be a means for interfacing with an apparatus (not shown) for input or output that may be connected to or included in the server 150. As a more specific example, the processor 212 of the electronic device 1110 uses data provided by the server 150 or the electronic device 2 120 in processing a command of a computer program loaded in the memory 211. The service screen or the content may be displayed on the display through the input / output interface 214.

Also, in other embodiments, the electronic device 1 110 and the server 150 may include more components than those of FIG. 2. However, it is not necessary to clearly show most of the prior art components. For example, the electronic device 1 110 may be implemented to include at least some of the above-described input / output devices 215 or other components such as a transceiver, a global positioning system (GPS) module, a camera, various sensors, a database, and the like. It may further include elements. More specifically, when the electronic device 1 110 is a smartphone, an acceleration sensor, a gyro sensor, a camera module, various physical buttons, a button using a touch panel, an input / output port, and vibration for a smartphone generally include Various components such as a vibrator may be implemented to be further included in the electronic device 1 110.

Hereinafter, a detailed embodiment of an artificial intelligence-based product recommendation method and system will be described.

3 is a block diagram illustrating an example of components that may be included in a processor of a server according to an embodiment of the present invention.

A product recommendation system may be configured in the server 150 according to the present embodiment. The product recommendation system can automatically find suitable products for a given product through image learning using DNN and GAN.

In order to perform the product recommendation method which will be described below, the processor 222 of the server 150 is a component as shown in FIG. 3, the image collector 310, the learning processor 320, and the product recommendation. It may include a portion 330. In some embodiments, the components of the processor 222 may be optionally included in or excluded from the processor 222. In addition, according to an embodiment, the components of the processor 222 may be separated or merged to represent the functions of the processor 222.

The processor 222 and the components of the processor 222 may control the server 150 to perform the steps included in the product recommendation method which will be described below. For example, the processor 222 and the components of the processor 222 may be implemented to execute instructions according to code of an operating system included in the memory 221 and code of at least one program.

Here, the components of the processor 222 may be representations of different functions of the processor 222 performed by the processor 222 according to an instruction provided by the program code stored in the server 150. . For example, the image collector 310 may be used as a functional representation of the processor 222 controlling the server 150 according to the above-described command so that the server 150 collects an image available as learning data.

First Example

4 is a flowchart illustrating a product recommendation method that may be performed by a processor of a server according to the first exemplary embodiment of the present invention.

In operation S410, the processor 222 may read a command required from the memory 221 loaded with a command related to the control of the server 150. In this case, the read command may include a command for controlling the processor 222 to execute steps S420 to S440 to be described later.

In step S420, the image collecting unit 310 collects a product image set (hereinafter, referred to as a 'correct answer image set') that is suitable as correct learning data, and a product image set (hereinafter, referred to as 'incorrect image set') that is incorrect as learning data. Can be collected. First, the image collector 310 may collect images in which a plurality of products are recognized in one image by using object detection technology. For example, product images coordinated by MDs (merchandisers) or sellers such as fashion pictorials, magazines, and shopping malls, and coordinated product images having a predetermined level of coordination scores by general users may be collected. It can be used as an answer coordination image to create correct learning data. The image collector 310 may make a plurality of product images by copying or cutting images of individual products from the corresponding image for each of the coordinating product images collected above, and may use them as a correct image set. In order for machines to learn the harmony between products, images of 'suitable goods' and 'mismatched goods' are needed like humans. If this work is not done logically, the machine can be judged as 'not suitable' between 'goods'. Therefore, the image collecting unit 310 may collect a wrong answer image set together with a correct answer image set as data for learning matching between products. For example, the image collector 310 may collect the incorrect image set through negative sampling excluding the correct image set and the similar product image using the correct image set.

5 is an exemplary diagram for explaining a process of collecting a set of correct answer images.

Referring to FIG. 5, the image collecting unit 310 obtains a product image 500 in which a plurality of products are coordinated, wherein the two objects recognized in the product image 500, that is, the top and the bottom are matched with each other. Product '. In other words, the image collecting unit 310 recognizes the upper image 501 and the lower image 502 from the plurality of products, for example, the product image 500 including the upper and lower, and configures the correct answer image set. Using one coordinating product image, one correct answer image set consisting of a plurality of product images may be created. In this manner, the image collection unit 310 may accumulate correct image sets for use as learning data by generating a product image set suitable for each coordinate product image.

6 is an exemplary diagram for explaining a process of collecting an incorrect image set. For example, as the answer image sets accumulate, it has lower B image sets that match upper A. In the following description, suitable products are marked with '>' and similar products are marked with '~'.

6 shows a process of examining the bottoms that do not suit the top A0.

First, in step S61, the image collecting unit 310 finds goods A1 and A2 similar to the image A0 (A0 to A1 and A2). Similar products can be found based on the similarity between product images of the correct answer image sets. For example, image analysis extracts features related to color, texture, and other attributes. The similarity between the images can be determined. This is exemplary and the method for calculating the image similarity may use at least one of a variety of known algorithms.

Next, in step S62, the image collecting unit 310 checks the bottom B0 in the correct answer image set including the top A0 (A0> B0), and includes similar products for each of the similar items A1 and A2 in the top A0. Check B1 and B2 in the correct answer image set (A1> B1, A2> B2).

Subsequently, in step S63, the image collecting unit 310 searches for similar products for each of B0, B1, and B2 below ([B0 to B0 '], [B1 to B1'], and [B2 to B2 ', B2 "). ]).

Finally, the image collection unit 310 defines that the bottoms that match the top A0, as well as the bottoms that match the top A0 and other bottoms similar to the top A0, and the bottoms that match the bottoms A0 all match the top A0 (A0). > B0, B0 ', B1, B1', B2, B2 ', B2 "), except that the products are defined as not matching A0.

As such, the image collector 310 may sample at least a portion of the remaining product images except the correct answer image set and the similar product image to create an image set that does not match the image A0 (negative sampling).

In FIG. 4, in step S430, the learning processor 320 may build a product recommendation model including matching results between images by performing machine learning using the correct answer image set and the incorrect answer image set. At this time, the machine learning uses the DNN, the input of the model is two product images to determine the degree of matching, and the output outputs the degree of matching of the two product images. For example, referring to FIG. 7, the learning processor 320 receives the upper image 701 and the lower image 702 as input and vectorizes the features extracted from the respective images 701 and 702 to obtain the upper image image. The DNN model 700 can be trained by mapping the image images of the lower and lower images into a vector of a multidimensional vector space, but closer to the distance between the two image vectors in the case of the correct answer image set, and the distance between the two image vectors in the case of the incorrect answer image set. .

4 again, in step S440, the product recommendation unit 330 may automatically recommend a product that matches the corresponding product through the DNN-based product recommendation model for the given product. When a new product image is input, the product recommendation unit 330 calculates the degree of matching between the corresponding product image and the candidate product image by passing the product recommendation model, and recommends the product with the highest degree of matching or plural in the order of the highest degree of matching. Products can be recommended. In other words, the product recommendation unit 330 may vectorize the features extracted from a given product image, map them to a vector in a multidimensional vector space, and calculate a matching degree based on a distance from another image vector in the vector space. Recommended products can be provided according to.

Therefore, the product recommendation method according to the present invention recognizes a plurality of products in one image under the assumption that the coordinated product is made of matching products, and learns the matching product set. In particular, by performing machine learning on a product image set that does not match with a matching product image set using a DNN method, more accurate and effective product recommendation is possible through a model that learns not only suitable products but also unsuitable products.

2nd Example

8 is a flowchart illustrating a product recommendation method that may be performed by a processor of a server according to the second exemplary embodiment of the present invention.

In operation S810, the processor 222 may read a necessary command from the memory 221 loaded with a command related to the control of the server 150. In this case, the read command may include a command for controlling the processor 222 to execute steps S820 to S840 to be described later.

In operation S820, the image collecting unit 310 may collect a matching product image set as data for learning matching between products. The image collecting unit 310 may collect images in which a plurality of products are recognized in one image by using an object recognition technology. For example, product images coordinated by MDs or sellers such as fashion pictorials, magazines, and shopping malls, and coordinated product images having a predetermined coordination score of a certain level may be collected by general users, and these images are matched between products. Can be used as training data for learning. The image collector 310 may accumulate image sets for use as learning data by generating a product image set suitable for each coordinate product image. A detailed collection process of the learning data is as described with reference to FIG. 5.

In operation S830, the learning processor 320 may build a product recommendation model based on a generation model by learning a product image set collected as learning data. In this case, the product recommendation model may receive a product image as input and generate information on characteristics of another product that matches the input product, and then determine a product similar to the generated characteristic as a recommendation product. The generation model vectorizes the features extracted from a given product image and sets it as an input vector. The image feature vector may be expressed in the form of an embedding vector through a pre-trained image learning model and may be used as a feature of a product recommendation model. In addition, the generation model derives an output vector expressing characteristics of suitable products and outputs the output vector. The learning direction of the product recommendation model extracts a feature vector for each product from a matching product image set to learn a relationship between the matching products.

For example, as shown in FIG. 9, an encoder-decoder model used in language processing such as machine translation may be used as the generation model 900. For example, the learning processor 320 receives a vector corresponding to the image 901 of the image and generates a vector of the image 902 to be matched through the generation model 900 so that these vectors are the closest distances in the vector space. Learn to have

Referring back to FIG. 8, in step S840, the product recommendation unit 330 may automatically recommend a product that matches the corresponding product through a product recommendation model based on a generation model for a given product. In the product recommendation model based on the generation model, the output vector expressing the characteristics of the product suitable for the product is derived from the feature vector of the given product image and based on the distance between the derived output vector and the feature vector extracted from the candidate product images. For example, the product image having the closest distance between the vectors may be provided as the recommended product.

In addition, the learning processor 320 may learn a discriminant model (discriminator) for determining the authenticity of the generation result using the GAN to advance the generation model 900, which is also referred to as adversarial training. do. GAN is mainly used in the field of image processing such as image synthesis or text-to-image synthesis, and in the field of artificial intelligence dialogue which improves conversation performance by determining whether it is a machine language or a human language. In the present invention, it is used to upgrade the generation model for product recommendation. The discrimination model may determine whether a given image set is generated as a result of the generation model or corresponds to an actual coordination set on the DB using the DB where the coordination sets are accumulated. Referring to FIG. 10, the learning processor 320 generates a given product image 901 and a generation model 900 using a matching product image set, that is, a DB 1010 in which real data sets are accumulated. The discriminant model 1000 for determining whether or not the recommended product image 902 is an actual coordinate set or generated data generated by a machine is authentic.

By conducting such an advanced generation model using GAN, artificial mass production of a discordant set is unnecessary, and a small number of disparity sets are required to learn a characteristic that tries to resemble a trend of a coordinator set that a real human being suits. Even with a set of images, you can expect high product recommendation performance in a semi-supervised manner.

In addition, it is also possible to upgrade the structure of the generation model 900 itself in addition to the alternative training for learning the discriminant model 1000. As an example, there is a method of elaborating a layer of the generation model 900, and it is possible to compressively utilize signals such as color, pattern, and style of the product through the convolution layer and the Max pooling layer. As another example, a highway layer may be added to the generation model 900 for learning performance or speed improvement, or a variable auto-encoder (VAE) which is a model having a conditional variational distribution may be used. Referring to FIG. 11, a sophisticated estimation is possible by giving a generation model 900 including an encoder-decoder model, a latent space Z 1100 along a specific distribution as random noise. In other words, by indicating the information indicating the product recommendation range in the latent space 1100 of the generation model 900, it is possible to recommend a product within the corresponding range. For example, when a top is given, at least one category information to be recommended among product classification categories such as bottoms, bags, shoes, and accessories may be specified to enable sophisticated product recommendation within a specified category. Through hybrid modeling using deep learning and probabilistic machine learning, simple DNNs can easily handle difficult probability distribution spaces.

As described above, according to embodiments of the present invention, a DNN learning model using a matching product set and a matching product set may recommend a product that matches a given product based on the degree of matching between the products. In addition, according to the embodiments of the present invention, in the recommendation of coordination using a generation model, a product of a coordination set similar to the actual coordination set may be recommended for a given product through the enhancement of the generation model using the GAN.

The apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the devices and components described in the embodiments may include a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable PLU (programmable). It can be implemented using one or more general purpose or special purpose computers, such as logic units, microprocessors, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to the execution of the software. For convenience of explanation, one processing device may be described as being used, but one of ordinary skill in the art will appreciate that the processing device includes a plurality of processing elements and / or a plurality of types of processing elements. It can be seen that it may include. For example, the processing device may include a plurality of processors or one processor and one controller. In addition, other processing configurations are possible, such as parallel processors.

The software may include a computer program, code, instructions, or a combination of one or more of the above, and configure the processing device to operate as desired, or process it independently or collectively. You can command the device. The software and / or data may be embodied in any type of machine, component, physical device, computer storage medium or device in order to be interpreted by or provided to the processing device or to provide instructions or data. have. The software may be distributed over networked computer systems so that they may be stored or executed in a distributed manner. Software and data may be stored on one or more computer readable recording media.

The method according to the embodiment may be embodied in the form of program instructions that can be executed by various computer means and recorded in a computer readable medium. In this case, the medium may be to continuously store a program executable by the computer, or to temporarily store for execution or download. In addition, the medium may be a variety of recording means or storage means in the form of a single or several hardware combined, not limited to a medium directly connected to any computer system, it may be distributed on the network. Examples of the medium include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, And ROM, RAM, flash memory, and the like, configured to store program instructions. In addition, examples of another medium may include a recording medium or a storage medium managed by an app store that distributes an application, a site that supplies or distributes various software, a server, or the like.

Although the embodiments have been described by the limited embodiments and the drawings as described above, various modifications and variations are possible to those skilled in the art from the above description. For example, the described techniques may be performed in a different order than the described method, and / or components of the described systems, structures, devices, circuits, etc. may be combined or combined in a different form than the described method, or other components. Or even if replaced or substituted by equivalents, an appropriate result can be achieved.

Therefore, other implementations, other embodiments, and equivalents to the claims are within the scope of the claims that follow.

Claims (18)

In the computer-implemented product recommendation method,
Collecting a set of correct answer images consisting of matching product images;
Collecting a wrong answer image set consisting of mismatched product images based on the correct answer image set;
Constructing a product recommendation model by performing deep neural network (DNN) based machine learning on the correct answer image set and the incorrect answer image set; And
Recommending a product that matches the product through the product recommendation model for a given product;
Including,
The DNN is a product recommendation method, characterized in that the model receives the two product images as input and outputs the degree of matching. The method of claim 1,
Collecting the correct answer image set,
Recognizing individual product images from images containing multiple products and composing them into a set of matching product images
Product recommendation method characterized by. The method of claim 1,
Collecting the incorrect image set based on the correct image set,
For an arbitrary product image, another product image in the correct answer image set including the random product image, a similar product image of the other product image, and another product in the correct answer image set including the similar product image of the random product image Composing at least some of the product images except the similar product images of the another product image into a mismatched product image set
Product recommendation method characterized by. The method of claim 1,
The building step,
For the DNN model, the feature extracted from the image is vectorized and mapped into a multidimensional vector space, but the distance between the image vectors is closer to the correct image set, and the distance between the image vectors is farther from the correct image set.
Product recommendation method characterized by. In the computer-implemented product recommendation method,
Collecting a set of correct answer images consisting of matching product images; And
Building a product recommendation model based on a generative model by learning the correct answer image set
Including,
The product recommendation model receives a product image as an input, generates information on characteristics of another product matching the input product, and determines a product similar to the generated characteristic as a recommendation product. The method of claim 5,
The collecting step,
Recognizing individual product images from images containing multiple products and composing them into a set of matching product images
Product recommendation method characterized by. The method of claim 5,
The building step,
Training the correct answer image set using an encoder-decoder model as the generation model
Product recommendation method comprising a. The method of claim 5,
The building step,
In order to advance the generation model, a discrimination model for learning the authenticity of the generation result is trained together. To determine if the actual coordination set
Product recommendation method characterized by. The method of claim 5,
The building step,
Learning the correct image set using a variational auto-encoder (VAE) which is a model having a conditional variational distribution as the generation model
Product recommendation method comprising a. A computer program stored in a computer-readable recording medium in combination with a computer for causing the computer to execute the method of recommending a product according to any one of claims 1 to 9. A computer-readable recording medium in which a program for causing a computer to execute the product recommendation method according to any one of claims 1 to 9 is recorded. In the computer-based product recommendation system,
At least one processor implemented to execute computer-readable instructions
Including,
The at least one processor,
An image collection unit for collecting a correct answer image set composed of matching product images and collecting an incorrect answer image set composed of mismatched product images based on the correct answer image set;
A learning processor configured to construct a product recommendation model by performing machine learning based on deep neural network (DNN) on the correct image set and the incorrect image set; And
Product recommendation unit for recommending a product that matches the product through the product recommendation model for a given product
Including,
The DNN is a product recommendation system, characterized in that the model for outputting the degree of matching takes two product images as input. In the computer-based product recommendation system,
At least one processor implemented to execute computer-readable instructions
Including,
The at least one processor,
An image collecting unit for collecting a set of correct answer images including matching product images; And
Learning processing unit to build a product recommendation model based on a generation model (generative model) by learning the correct answer image set
Including,
The product recommendation model receives a product image as an input, generates information on characteristics of another product that matches the input product, and determines a product similar to the generated characteristic as a recommendation product. The method according to claim 12 or 13,
The image collection unit,
Recognizing an individual product image from an image containing a plurality of products and constructing a product image set that matches the correct answer image set
Product recommendation system characterized in that. The method of claim 12,
The learning processing unit,
For the DNN model, the feature extracted from the image is vectorized and mapped into a multidimensional vector space, but the distance between the image vectors is closer to the correct image set, and the distance between the image vectors is farther from the correct image set.
Product recommendation system characterized in that. The method of claim 13,
The learning processing unit,
Learning the correct image set using an encoder-decoder model as the generation model
Product recommendation system characterized in that. The method of claim 13,
The learning processing unit,
In order to advance the generation model, a discrimination model for learning the authenticity of the generation result is trained together. To determine if the actual coordination set
Product recommendation system characterized in that. The method of claim 13,
The learning processing unit,
Learning the correct image set using a variational auto-encoder (VAE) which is a model having a conditional variational distribution as the generation model
Product recommendation system characterized in that.

Images