KR20180066643A - Product searching system using deep learning based on image comparison, and method thereof - Google Patents

Abstract

The present invention relates to a product search system using image comparison based on deep learning and a method thereof. The product search system comprises: a user terminal (1); and a visual filter (120). Accordingly, similar product search can be performed by using the visual filter (120) using deep learning to search for a product based on the similarity of an external form.

Description

{PRODUCT SEARCHING SYSTEM USING DEEP LEARNING BASED ON IMAGE COMPARISON, AND METHOD THEREOF}

More particularly, the present invention relates to a system and method for searching for a similar product using a visual filter using deep-running searching for a product based on an external form of similarity. And more particularly, to a system and method for searching for a product using deep-running-based image comparison.

In the flood of goods, brands, and information, consumers are hesitating and worried. With the obligation to 'find the right answer' And many are making decisions based on a number of recommendations to minimize uncertainty and anxiety. As a result, the number of people suffering from Hamlet syndrome, a type of decision-making disorder that makes large and small choices difficult, has increased.

Online shopping is no exception. In general, online shopping malls provide users with two product search (category, keyword search) items.

However, even if the online shopping mall is not only offline, but it is a product of online shopping mall, it takes more time to find the desired product because there are 1,000 brands and 500,000 products from luxury goods to food.

In fact, when searching for 'women's pants' in 'keyword search' of an existing online shopping mall, an average of 10,000 items are searched and it takes more than 10 minutes to find the desired goods. This has the disadvantage of minimizing the reflection of the customer's needs and maximizing the circulation by sorting all the products for the category regardless of the customer's taste.

In order to prevent a lot of options from increasing due to information overload, 'customized sorting according to customer age' has already occurred. However, there are too many products to be searched, so it is hard to find products that suit the taste of users.

In the related technology field, it is required to develop a technique for increasing the satisfaction of mobile shoppers by making the basic usability more convenient than the search method of the online shopping mall and making it easy to find products that are difficult to find through the keyword search.

Korean Patent Application No. 10-2004-0052898 "Product search method and system capable of supporting purchase decision support in an Internet shopping mall" Korean Patent Application No. 10-2007-0104742 "Method of Displaying Product Search Results and a Product Search Server for the Same" Korean Patent Registration No. 10-1094204 "Method of searching for products by category and method and server for searching < RTI ID = 0.0 > commodity & Korean Patent Registration No. 10-1671449 entitled " Method and apparatus for searching for a product based on location "

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and it is an object of the present invention to provide a product search system using deep-run based image comparison for performing similar product search using a visual filter using deep- And a method.

In addition, if the characteristic vectors calculated by the convolutional neural network (CNN) are similar, the external shape of the object in the image is also generally similar to each other, so that the characteristic vector values It is determined that the two images are externally similar as the difference between the two feature vectors is smaller, so that the user can quickly obtain a product To provide a product search system and method using deep-run-based image comparison.

In addition, the present invention proposes a training algorithm based on a visual filter that is more usable than an existing search method, so as to sort and provide products similar to a user-selected product through the simulation results of the training algorithm and the screen applied to the web And to provide a product retrieval system and method using deep-running-based image comparison.

However, the objects of the present invention are not limited to the above-mentioned objects, and other objects not mentioned can be clearly understood by those skilled in the art from the following description.

In order to accomplish the above object, a product search system using deep-learning-based image comparison according to an embodiment of the present invention includes: a user terminal (1) performing product search using deep-run based image comparison; And an image of a selected commodity received from the user terminal 1 are input to one CNN 110 and then the calculated output vector is compared with output vectors of images of other commodities of the database 130 The product having the smallest distance between the output vectors is realized on the web of the user terminal 1 and a dip that receives the selection signal of the most favorite product among the products newly displayed again from the user terminal 1 A visual filter 120 using running; . ≪ / RTI >

In order to achieve the above object, a product search method using a deep learning based image comparison according to an embodiment of the present invention is a method in which a visual filter-based training algorithm 120a acquires one image (image T) A first step of implementing at least two comparative images (images A and B) and waiting for a selection of images closer to the image T; A second step of updating parameters of the CNN 120 to the database 130 so as to recognize that the visual filter-based training algorithm 120a is an image closer to the image T with respect to the selected image; . ≪ / RTI >

(max(a,b) 함수는 a > b일 때 a라는 값을 획득하고, a < b이면 b라는 값을 획득)을 제공하는 제 2-2 단계; 를 포함할 수 있다. In the second step, the visual filter-based training algorithm 120a generates the image T as a y _T vector, and the image _T A ₂ and y _B vectors are generated as y _A and y _B vectors, respectively, and stored in the database 130; And a visual filter-based training algorithm 120a are used to minimize the vector-based alignment change stored in the database 130 to provide products similar to the selected product using the generated vectors

(max (a, b) function obtains a value when a > b and obtains a value when b < . &Lt; / RTI &gt;

In the method for searching for products using deep learning based image comparison according to another embodiment of the present invention, after the step 2-2, a visual filter-based training algorithm 120a is transmitted from the user terminal 1 to a comparison product A second step of setting a value of 1 or -1 with respect to s according to the selection signal; Can be performed.

Further, in the method of searching a product using deep learning based image comparison according to yet another embodiment of the present invention, after the step 2-3, a visual filter-based training algorithm 120a is performed from the user terminal 1 to the image A A second step of determining whether a selection signal is selected for the image A after receiving a selection signal for the image A; Can be performed.

In addition, in the method of searching a product using the deep learning based image comparison according to another embodiment of the present invention, if the selection signal is a selection for the image A after step 2-4, The algorithm 120a determines that the bag A is more similar to the given test bag T and sets s = 1; And the visual filter-based training algorithm 120a becomes s = 1 in the energy equation according to the choice for bag A,

로 변환되어,

인 경우만 E(s)가 값을 갖고, 그렇지 않으면 E(s)=0으로 설정하는 제 2-6 단계; 및 비주얼 필터 기반의 훈련 알고리즘(120a)이 가방 A의 특징 벡터와 테스트 가방의 특징 벡터 거리가 가방 B의 특징 벡터보다 먼 경우만 CNN(120)의 데이터베이스(130)를 업데이트 하여, 오류 교정을 수행하는 제 2-7 단계; 를 더 포함할 수 있다.

Lt; / RTI &gt;

(S) has a value only if E (s) = 0; otherwise, E (s) = 0; And the visual filter based training algorithm 120a updates the database 130 of the CNN 120 only if the feature vector of the bag A and the feature vector distance of the test bag are greater than the feature vector of the bag B to perform error correction Step 2-7 is performed; As shown in FIG.

The system and method for searching for products using deep learning based image comparison according to an embodiment of the present invention can search similar products using a visual filter using deep learning for searching for products based on an external form similarity Lt; / RTI &gt;

In addition, according to another embodiment of the present invention, a system and method for searching for products using deep-learning-based image comparison, when the feature vectors calculated by a convolutional neural network (CNN) are similar, The similarity between images is compared using the fact that each image passed through CNN has a unique feature vector value so that the smaller the difference between the feature vector values of the two images is, It is possible to provide a product to the user on the basis of a search for a similar product quickly.

In addition, according to another embodiment of the present invention, a product search system and method using a deep learning based image comparison proposes a visual filter-based training algorithm that is more usable than an existing search method, It is possible to sort and provide products similar to the product selected by the user through the screen applied to the web.

1 is a diagram illustrating a product retrieval system using a deep learning based image comparison according to an embodiment of the present invention. FIG. 2 is a diagram illustrating a concept of searching for a product based on a visual filter using deep learning used in a product search system using a deep-running-based image comparison according to an exemplary embodiment of the present invention.
FIG. 3 is a diagram illustrating a training algorithm based on a visual filter 120 based on deep learning used in a product search method using a deep learning-based image comparison according to an embodiment of the present invention. 4 is a flowchart illustrating a method of searching for a product using a deep learning based image comparison according to an embodiment of the present invention. FIG. 5 is a diagram specifically illustrating a process of step S20 of a product search method using the deep learning-based image comparison of FIG.
FIG. 6 and FIG. 7 are diagrams showing a result screen after a simulation by a product search system and method using a deep learning-based image comparison according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a detailed description of preferred embodiments of the present invention will be given with reference to the accompanying drawings. In the following description of the present invention, detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

In the present specification, when any one element 'transmits' data or signals to another element, the element can transmit the data or signal directly to the other element, and through at least one other element Data or signal can be transmitted to another component.

1 is a diagram illustrating a product retrieval system using a deep learning based image comparison according to an embodiment of the present invention. FIG. 2 is a diagram illustrating a concept of searching for a product based on a visual filter using deep learning used in a product search system using a deep-running-based image comparison according to an exemplary embodiment of the present invention.

Referring to FIG. 1, a product search system using deep learning based image comparison includes a user terminal 1, a Convolutional Neural Network (CNN) 110, a visual filter 120 using deep running, A filter based training algorithm 120a, a database 130, and an autoencoder 140. [

As shown in FIG. 2, the visual filter 120 using Deep Learning in this configuration inputs an image of a product selected by the user into one CNN (Convolutional Neural Network) 110, With the output vectors of the images of the different products of the product (130).

At this time, the visual filter 120 using the deep learning allows the products having the smallest distance between the output vectors to be realized on the web of the user terminal 1, and the most favorite product among the products newly displayed again from the user terminal 1 Lt; / RTI &gt;

A visual filter (110) based search method searches through the process several times and searches for the product of the most favorite.

Meanwhile, the auto encoder 140 is a neural network that transforms input information so that there is no incomplete part, and provides an algorithm using a neural network that efficiently compresses information. The CNN 120 is pre-trained (pre-training) results. In contrast, the CNN 110 abstracts local information or provides an algorithm using a neural network that does not change its position.

FIG. 3 is a diagram illustrating a training algorithm based on a visual filter 120 based on deep learning used in a product search method using a deep learning-based image comparison according to an embodiment of the present invention. 4 is a flowchart illustrating a method of searching for a product using a deep learning based image comparison according to an embodiment of the present invention. FIG. 5 is a diagram specifically illustrating a process of step S20 of a product search method using the deep learning-based image comparison of FIG.

Referring to FIG. 3, a product search system using deep learning based image comparison is a program for training a visual filter 120 using deep learning, and includes a training algorithm based visual filter 120a as a separate program .

That is, the visual filter-based training algorithm 120a first implements one image (image T) to the user terminal 1 and then implements two comparison images (image A and image B) And waits for the user to select which image is closer (S10). After step S10, the visual filter-based training algorithm 120a updates the parameters of the CNN 120 to the database 130 so as to recognize that the image is closer to the image T with respect to the selected image (S20 ). The modeling of this process corresponds to Fig.

Next, 3 and 5 to look at the process of the step (S20) in detail, the visual filter based training algorithm (120a) generates an image T as y _T vector, and each y the image A and the image B _A and y _B vectors and stores them in the database 130 (S21).

In accordance with step S21, the visual filter-based training algorithm 120a sets the vector-based sorting change stored in the database 130 as energy knowledge to minimize the variation of the vector-based sorting order in order to provide products similar to the selected product using the generated vectors (S22). &Lt; / RTI &gt;

The function max (a, b) in Equation 1 has a value a when a> b and a value b when a <b. That is, it has a larger value of the two.

값이 0보다 큰 경우

이고, 그렇지 않은 경우에는 0이라는 값을 갖는다. Therefore, Equation (1)

If the value is greater than 0

, And has a value of 0 otherwise.

After step S22, the visual filter-based training algorithm 120a sets a value of 1 or -1 for s according to the selection signal for the comparison product from the user terminal 1 (S23). More specifically, according to the user's choice, s has a value of 1 or -1. If the bag A is considered to be more similar to the bag T for a given test bag T, then s = 1, If you think it is similar to bag T, you have s = -1.

After step S23, the visual filter-based training algorithm 120a receives a selection signal for the images A and B from the user terminal 1 and determines whether the selection signal is a selection for the image A (S24 ).

If it is determined in step S24 that the selection signal is a selection for image A, then the visual filter-based training algorithm 120a determines that bag A is more similar for a given test bag T, (S25).

After step S25, the visual filter-based training algorithm 120a becomes s = 1 in equation (1) according to the selection for bag A,

로 변환되어,

인 경우만 E(s)가 값을 갖고, 그렇지 않으면 E(s)=0으로 설정한다(S26).

Lt; / RTI &gt;

E (s) is set to 0 (S26).

After step S26, the visual filter-based training algorithm 120a determines whether the feature vector of the bag A and the feature vector of the test bag are greater than the feature vector of the bag B, according to the selection signal of the bag A, The error correction is performed by updating the database 130 of the database 130 (S27).

On the other hand, if it is determined in step S24 that the selection signal is the selection for the image B, the visual filter-based training algorithm 120a determines that the bag B is more similar to the given test bag T, -1 &quot; (S25a).

After step S25a, the visual filter-based training algorithm 120a becomes s = -1 in equation (1) according to the selection for bag B,

로 변환되어,

인 경우만 E(s)가 값을 갖고, 그렇지 않으면 E(s)=0으로 설정한다(S26a).

Lt; / RTI &gt;

E (s) is set to 0 (S26a).

After step S26a, the visual filter-based training algorithm 120a determines whether the feature vector of the bag B and the feature vector distance of the test bag are greater than the feature vector of the bag A, according to the selection signal of the bag B, The error correction is performed by updating the database 130 of the database 130 (S27a).

FIG. 6 and FIG. 7 are diagrams showing a result screen after a simulation by a product search system and method using a deep learning-based image comparison according to an embodiment of the present invention. 6 is a diagram for explaining an activation of a visual filter based training algorithm according to an embodiment of the present invention. Referring to FIG. 6, a result of pre-training CNN 120 with an autoencoder 140 with 1,200 bag images is shown. Here, the distance between the feature vectors of the bags is represented by a two-dimensional image. The autoencoder 140 is often used for pre-training of the CNN 120 with a deep-running structure in which the input images are again output as output images. The sizes of the images of all the bags may be adjusted to 32 * 32 pixels, and then put into the CNN 120 to obtain the prior parameter values of the CNN 120. [

Accordingly, FIG. 6A is a picture before alignment, which is scattered regardless of the degree of similarity of images. As time elapses, similarities in similarities of visual aspects of the bag can be aligned with each other as shown in FIG. 6B.

Next, FIG. 7 shows a result screen in which the above-described visual filter-based training algorithm 120a is applied. When a bag suitable for a user's taste is selected (FIG. 7B), other bags similar to the selected bag are aligned, and the result is similar to a bag to be gradually found (see FIG.

The present invention provides a personalized personalized service 'product search using a deep learning-based image comparison' as consumers who want to receive a recommendation of content and products that meet the user's interests and tastes increase. Accordingly, the present invention can be utilized to sort and purchase items similar to individual tastes without the need to enter information (sex, age, taste, etc.) of the user other than the existing online shopping mall that provides indiscriminate product information There will be. In order to provide such a service, a visual filter-based training algorithm 120a may be applied to determine the degree of similarity with the output vector value when passing through the CNN 120, thereby sorting the products.

As described above, a service that allows a user to easily shop for desired items in a short time by simply selecting (clicking) goods can be the biggest problem of 'selecting' meaningful information in a myriad of options living in a flood of information Provides an effect.

The present invention can also be embodied as computer-readable codes on a computer-readable recording medium. A computer-readable recording medium includes all kinds of recording apparatuses in which data that can be read by a computer system is stored.

Examples of the computer-readable recording medium include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device and the like, and also implemented in the form of a carrier wave (for example, transmission over the Internet) .

The computer readable recording medium may also be distributed over a networked computer system so that computer readable code can be stored and executed in a distributed manner. And functional programs, codes, and code segments for implementing the present invention can be easily inferred by programmers skilled in the art to which the present invention pertains.

As described above, preferred embodiments of the present invention have been disclosed in the present specification and drawings, and although specific terms have been used, they have been used only in a general sense to easily describe the technical contents of the present invention and to facilitate understanding of the invention , And are not intended to limit the scope of the present invention. It is to be understood by those skilled in the art that other modifications based on the technical idea of the present invention are possible in addition to the embodiments disclosed herein.

1: User terminal
110: Convolutional Neural Network (CNN)
120: Visual filter using deep running
120a: Visual filter-based training algorithm
130: Database
140: autoencoder

Claims (6)

A user terminal (1) for performing a product search using a deep learning based image comparison; And
After the image of the selected commodity received from the user terminal 1 enters the one CNN (Convolutional Neural Network) 110, the calculated output vector is compared with the output vectors of images of other commodities of the database 130 Deep running, which allows products with the smallest distance between the output vectors to be implemented on the web of the user terminal 1 and receiving a selection signal for the most favorite product among the newly launched products from the user terminal 1 A visual filter 120 using the filter 120; Based on the comparison of the deep learning based image comparison.
The visual filter based training algorithm 120a first implements one image (image T) to the user terminal 1 and then implements at least two comparative images (image A and image B) A first step of waiting for a selection of what the image is; And
A second step of updating the parameters of the CNN 120 to the database 130 so as to recognize that the visual filter-based training algorithm 120a is an image closer to the image T with respect to the selected image; Based on image comparison based on the deep learning based image comparison.
The method according to claim 2,
A second step of the visual filter-based training algorithm 120a generating the image T as a y _T vector, the image A and the image B as y _A and y _B vectors, respectively, and storing the y _A and y _B vectors in the database 130; And
The training algorithm 120a based on the visual filter may be used to minimize the vector-based alignment change stored in the database 130 to provide products similar to the selected product using the generated vectors.

(max (a, b) function obtains a value when a > b and obtains a value when b < Based on image comparison based on the deep learning based image comparison.
4. The method of claim 3,
Step 2-3, in which the visual filter-based training algorithm 120a sets a value of 1 or -1 for s according to the selection signal for the comparison product from the user terminal 1; The method of claim 1, further comprising:
5. The method of claim 4,
2-4, after the visual filter-based training algorithm 120a receives a selection signal for the images A and B from the user terminal 1, and then determines whether the selection signal is a selection for the image A; The method of claim 1, further comprising:
[6] The method of claim 5,
If the selection signal is a selection for the image A, it is determined that the visual filter-based training algorithm 120a has selected the bag A more similar to the given test bag T, and sets the value of s = Step 5; And
Since the visual filter-based training algorithm 120a becomes s = 1 in the energy equation according to the choice for bag A,

Lt; / RTI &gt;

(S) has a value only if E (s) = 0; otherwise, E (s) = 0; And
The visual filter based training algorithm 120a updates the database 130 of the CNN 120 only if the feature vector of the bag A and the feature vector distance of the test bag are greater than the feature vector of the bag B to perform error correction Step 2-7; The method of claim 1, further comprising:

Images