KR20210037485A - Method of searching and providing data of similar fashion goods and computing device therefor - Google Patents

Abstract

According to an embodiment of the present invention, a similar product searching and providing method in a computing device comprises the following steps of: receiving an image, which is a search target, from a terminal; recognizing an object including a fashion item in the received image and cropping the fashion item from the recognized object; estimating image similarity by calculating the cropped image data and fashion product image data stored in a database using sub-networks included in a pre-learned network function; and providing similar product data to the terminal based on the estimated image similarity.

Description

A method of searching and providing similar fashion product data, and a computing device therefor

The present invention relates to searching for and providing similar fashion products, and more particularly, a method of searching and providing similar fashion products by comparing an image received using an image similarity estimation model in an image-to-image comparison method. And a computing device for it.

Traditional consumption methods for fashion products through offline stores have been changed and expanded in various forms without limitation of time, place, etc. through mobile terminals due to the recent development and activation of technology.

In particular, in a search method, image-based or image-based search has been proposed in a search centered on text such as an existing keyword.

However, as the types of fashion products diversify and the number of fashion products increases, it is becoming difficult to quickly and accurately search for a desired fashion product even by such a search operation.

On the other hand, with the recent development of artificial intelligence technology, attempts have been made to improve the search method using this, but still learn the data tagged with the attribute on the image to build a classification model, and if an image is given as an input, the tagged attribute (text) is It is an image-to-text method that returns and returns the search result of the image with the corresponding attribute.The seller manually enters the attribute value and returns the search result from the database, but there are still many targets for matching products. Is becoming.

Accordingly, there are needs for fast and accurate fashion product search and results provision from images.

An object of the present invention is to provide a method for estimating image similarity specialized for a fashion item.

Another object of the present invention is to provide a method of adjusting the number of similar products in a search result provided according to estimation of image similarity and increasing the accuracy of the search result.

Another object of the present invention is to provide a computing device and a system including the same for the above-described method.

The technical problems to be achieved in the present invention are not limited to the above technical problems, and other technical problems not mentioned 　 will be clearly understood by those of ordinary skill in the technical field to which the present invention belongs from the following description.

A method for searching and providing similar products in a computing device according to an embodiment of the present invention for solving the above-described problem includes: receiving an image as a search target from a terminal; Recognizing an object including a fashion item from the received image and cropping the fashion item from the recognized object; Estimating image similarity by calculating the cropped image data and fashion product image data stored in a database using sub-networks included in a pre-learned network function; And providing similar product data to the terminal based on the estimated image similarity.

In a method for searching and providing similar products in a computing device according to an embodiment of the present invention, the sub-networks are characterized in that they include a convolutional neural network (CNN) structure among deep neural network structures.

In a method for searching and providing similar products in a computing device according to an embodiment of the present invention, the sub-networks are characterized in that they constitute at least part of a Siamese network.

A method for searching and providing similar products in a computing device according to an embodiment of the present invention further comprises: setting a threshold value for returning a search result in relation to the estimated image similarity.

A method for searching and providing similar products in a computing device according to an embodiment of the present invention is characterized in that data on an image to be returned as a search result among images whose image similarity is estimated based on the set threshold value is adjusted.

In a method for searching and providing similar products in a computing device according to an embodiment of the present invention, the step of recognizing an object including a fashion item from the received image and cropping a fashion item from the recognized object comprises: It is characterized in that the object recognition and region of interest (ROI) are automatically cropped using a specialized and learned R-CNN series model.

A computing device for searching and providing similar products according to an embodiment of the present invention includes: a receiver configured to receive an image to be searched from a terminal; An object recognition/crop unit that recognizes an object including a fashion item from the received image and crops the fashion item from the recognized object; An image similarity estimation unit for estimating image similarity by calculating the cropped image data and fashion product image data stored in a database using sub-networks included in a pre-learned network function; And a control unit that controls to provide similar product data to the terminal based on the estimated image similarity.

In the computing device for searching and providing similar products according to an embodiment of the present invention, the sub-networks are characterized in that they include a convolutional neural network (CNN) structure among deep neural network structures.

In the computing device for searching and providing similar products according to an embodiment of the present invention, the sub-networks are characterized in that they constitute at least part of a Siamese network.

In the computing device for searching and providing similar products according to an embodiment of the present invention, the control unit may set a threshold value for returning a search result in relation to the estimated image similarity.

In the computing device for searching and providing similar products according to an embodiment of the present invention, the control unit is configured to adjust data for an image to be returned as a search result among images whose image similarity is estimated based on the set threshold. It is characterized.

In the computing device for searching and providing similar products according to an embodiment of the present invention, the object recognition/crop unit recognizes and interests the object by using a model of the Paster R-CNN series learned by specializing in the fashion item. It features automatic cropping of a region (ROI: Region of Interest).

According to the present invention as described above, the effects described below can be obtained. However, the effect that can be obtained through the present invention is not limited thereto.

First, there is an effect of providing a method of estimating image similarity specialized for 　 fashion items.

Second, there is an effect of providing a method of adjusting the number of similar products in search results provided according to estimation of image similarity and increasing the accuracy of search results.

Third, there is an effect of providing a computing device and a system including the same for the above-described method.

1 is a diagram showing a system for searching and providing similar products according to an embodiment of the present invention.
2 is a block diagram illustrating a configuration of a computing device according to an embodiment of the present invention.
3 is a diagram illustrating a process of recognizing an object in an image and cropping an ROI in a computing device according to an embodiment of the present invention.
4 is a diagram illustrating a similarity estimation model for searching for similar fashion products in an image-to-image method according to an embodiment of the present invention.
5 is a diagram illustrating a learning model according to an embodiment of the present invention.
6 is a diagram illustrating an attribute extracted from a fashion product object according to an embodiment of the present invention.
7 is a diagram illustrating a method of providing a similar fashion product search result according to an embodiment of the present invention.
8 is a flowchart illustrating a method of searching for and providing similar fashion products according to an embodiment of the present invention.

Hereinafter, with reference to the accompanying drawings, preferred embodiments according to the present invention will be described in detail. The detailed description to be disclosed hereinafter together with the accompanying drawings is intended to describe exemplary embodiments of the present invention, and is not intended to represent the only embodiments in which the present invention may be practiced.

Only these embodiments are provided to make the posting of the present invention complete, 　 to fully inform the scope of the invention to those of ordinary skill in the art to which the present invention belongs, and 　 the present invention to be defined by the scope of the claims. Only.

In some cases, in order to avoid obscuring the concept of the present invention, well-known structures and devices may be omitted, or may be illustrated in block diagram form centering on the core functions of each structure and device. In addition, the same components will be described using the same reference numerals throughout this specification.

Throughout the specification, 　when a certain part 　"comprising 　 or 　including", 　 means that other components may be further included rather than excluding other components unless otherwise stated. do.

In addition, the term "?? unit" described in the specification refers to a unit that processes at least one function or operation, which may be implemented by hardware or software or a combination of hardware and software. Furthermore, "one (a 　 or 　an)", "one", 　 and similar related words are all in the singular and plural unless otherwise indicated in the specification or clearly contradicted by the context in the context describing the present invention. It can be used in a sense including.

In addition, 　specific terms used in the embodiments of the present invention are provided to aid understanding of the present invention, and 　unless defined otherwise, 　all terms used herein including technical or scientific terms are It has the same meaning as commonly understood by those of ordinary skill in the art to which it belongs. The use of these specific terms may be changed in other forms without departing from the technical spirit of the present invention.

Hereinafter, various embodiments according to the present invention will be described with reference to the accompanying drawings, as follows.

1 is a diagram showing a system for searching and providing similar products according to an embodiment of the present invention.

Referring to FIG. 1, in the system 100 for searching and providing similar products, one terminal 110 and one computing device 120 are illustrated, and both devices are connected through a network to communicate data with each other. You can do it.

Hereinafter, in the present specification, for convenience, the terminal 110 and the computing device 120 are illustrated and described in FIG. 1 in a singular number, but the present invention is not necessarily limited thereto. For example, the terminal 110 and the computing device 120 may be implemented in a plurality of each, and the system 100 for searching and providing similar products according to the present invention may be implemented.

The terminal 110 may be a fixed terminal such as a TV or a PC, or a mobile terminal such as a smart phone, a mobile phone, a navigation system, a notebook computer, or a tablet PC.

The terminal 110 may perform data communication with another terminal (not shown) or the computing device 120 through a wired/wireless communication network.

Meanwhile, the wired/wireless communication network supports a communication protocol so that various data can be exchanged between components of the system 100 for searching and providing similar products according to the present invention, and the type is not particularly limited. If necessary, it can also be supported in the case of collecting data on the web through the terminal 110, the computing device 120, or the like.

According to one embodiment, 　the wired/wireless communication network is, 　Internet Protocol (IP) network supporting large-capacity data communication through Internet Protocol (IP), or All IP integrating different 　IP　 networks. I can.

According to another embodiment, the wired/wireless communication network is a wired network, a mobile communication network including WCDMA, a high speed downlink packet access (HSDPA), and a mobile communication network including Long Term Evolution (LTE), LTE advanced (LTE-A). ), a mobile communication network, a satellite communication network, and a Wi-Fi network, or may be formed by a combination of at least two or more of them. Bluetooth communication can also be supported here.

The computing device 120 is a component that substantially searches for and provides similar products according to the request of the terminal 110, and may be a remotely located server, but is not limited thereto.

Such a computing device 120 refers to a hardware/software (Hardware/Software) that processes or supports various data generation, 　 processing, 　 storage, etc. according to the present invention. It may be named and described as controller)　, but it should be stated in advance that it should not be limited and interpreted by the name.

On the other hand, according to an embodiment, 　 computing device 120, 　 Cloud 　 server, IMS (IP Multimedia Subsystem) 　 server, 　 Telephony Application (Telephony Application) 　 server, IM (Instant Messaging) 　 Server, MGCF (Media Gateway Control Function) It may be implemented in any one of a server, MSG (Messaging Gateway) 　 server, CSCF (Call Session Control Function) 　 server.

Meanwhile, the computing device 120 may provide a file for installing an application to the terminal 110 accessed through a wired/wireless communication network. In this case, the terminal 110 may install an application using a file provided from the computing device 120. In addition, the terminal 110 may be connected to the computing device 120 through its operating system (OS) or program to receive or use its service or fashion product content.

In particular, the computing device 120 receives an image from the terminal 110, searches for a product similar to the received image according to the present invention, and returns the search result to the terminal 110. This will be described in more detail below.

Meanwhile, although not shown, in FIG. 1, the computing device 120 may be implemented as an internal component of the terminal 110 in the form of a single module, or any one of the internal components of the terminal 110 is the computing device. Some or all of the functions of the device 120 may be performed. For example, when searching for and providing similar products according to the present invention is performed through an application installed in the terminal 110, an internal control configuration such as a central processing unit (CPU) of the terminal 110 is applied to the computing device ( 120) can be replaced or partially performed.

FIG. 2 is a block diagram illustrating a configuration of a computing device according to an embodiment of the present invention, and FIG. 3 is a diagram illustrating a process of recognizing an object in an image and cropping an ROI in a computing device according to an embodiment of the present invention. 4 is a diagram illustrating a similarity estimation model for searching for similar fashion products in an image-to-image method according to an embodiment of the present invention, and FIG. 5 is a learning model according to an embodiment of the present invention. 6 is a diagram illustrating an attribute extracted from a fashion product object according to an embodiment of the present invention, and FIG. 7 is a similar fashion product search result according to an embodiment of the present invention. A diagram illustrating a provision method, and FIG. 8 is a flowchart illustrating a method for searching and providing similar fashion products according to an embodiment of the present invention.

As described above, in FIG. 1, one terminal 110 and one computing device 120 are disclosed. In the case of a plurality of devices, each may be implemented by including the same or different components.

Hereinafter, various embodiments will be described based on the flow chart of FIG. 8 to aid in understanding of the present invention and for convenience of description. In this case, the flowchart of FIG. 8 is not necessarily limited to the illustrated order, and some steps may be performed simultaneously or in a different step from the illustrated order.

According to the present invention, a method for searching and providing similar fashion products is described in terms of the computing device 120 for convenience. However, it is not limited thereto. -A search target image is received from the terminal 110 (S810).

The computing device 120 classifies the image to be searched for the received terminal 110 (S820).

The computing device 120 recognizes a fashion product-related object from a search target image of the classified terminal 110, sets a fashion product-related part in the object as a region of interest (ROI), and then a set ROI part. Crop (S830).

The computing device 120 estimates the image similarity by calculating the cropped image data and the fashion product image data stored in the database using sub-networks included in the pre-learned network function (S840). For example, the computing device 120 performs a search through an image-to-image method estimation of a portion of an ROI cropped from an image to be searched using a similarity estimation model.

The computing device 120 provides similar product data to the terminal as a result of the search based on the estimated image similarity (S850).

2 is a block diagram showing an internal configuration of the computing device 120, and the present invention is not limited thereto. For example, although not shown, some components may be added or some of the components illustrated in FIG. 2 may be omitted depending on embodiments. In addition, although each component of FIG. 2 is illustrated and described as an individual component, some components may be implemented as a single module or, conversely, some components may be further subdivided into two or more components.

2, the computing device 120 includes a communication interface unit 210, an image collection unit 220, an image classification unit 230, an object recognition/crop unit 240, an image similarity estimation unit 250, It is configured to include a database (DB: database) 260, a control unit 270, and the like.

First, referring to FIG. 2, the components of the computing device 120 will be briefly described as follows.

The communication interface unit 210 communicates with the terminal 110 shown in FIG. 1 to receive various data including image data to be searched from the terminal 110. Here, the image data to be searched received from the terminal 110 may correspond to at least one of a captured or snapshot image, a captured image, a downloaded image, an image pre-stored in the terminal 110, and the like.

The image collection unit 220 receives image data to be searched for the terminal 110 received through the communication interface unit 210.

The image classification unit 230 classifies the image data to be searched for the terminal 110 received by the image collection unit 220. Images classified in this way can be classified as fashion image wearing shots, fashion image product shots, and non-fashion images. Meanwhile, as described above, the image classification unit 230 uses a deep learning-based multi-class classification model to classify the type of image in order to classify the image to be searched by the received terminal 110. I can. To this end, the image classification unit 230 may receive various types of fashion image data collected from the image collection unit 220 through the web and product images of manufacturers, and learn in advance through the deep learning model. have. The learned result may be stored in the database 260. Meanwhile, the deep learning-based multi-class classification model used in the image classification unit 230 may be stored in the database 260 in advance, and may be in the form of a class (bag), etc., as shown in FIG. 6. have.

The object recognition/crop unit 240 detects and crops an object using a model based on a Faster R-CNN (Convolutional Neural Network Features) series. FIG. 3(a) shows a feature map pyramid extracted from FPN, and FIG. 3(b) shows an example of a structure of an R-CNN series using a feature map pyramid extracted from FPN. A detailed description of the model of the Faster R-CNN series and the description of (a) and (b) of Figs. 3 to be described later refer to well-known contents, and a detailed description thereof will be omitted. However, in the case of R-CNN series models that are not only currently published R-CNN series models but also R-CNN series models that are developed and known in the future, in relation to the present invention, object detection including fashion items and automatic cropping of corresponding fashion items from detection objects It can be used in the process of doing.

First, the object recognition/crop unit 240 is identified as a fashion image as a result of classification by the image classification unit 230 described above, and when the class is confirmed, the identification in the search target image previously received from the terminal 110 Recognizes the objects related to the fashion image of the class.

In other words, in particular, a Faster R-CNN series model that has been trained in advance by specializing in fashion items is constructed, and by using the constructed model, among at least one or more objects in the image received from the terminal 110, particularly related to the fashion item. Objects can be detected. In the case of the conventional object detection model, it was common to detect all objects included in the received image, but the present invention determines and detects only the objects necessary from the image as meaningful objects through an object detection model that is specialized for fashion items and learned in advance. By doing so, it is possible to increase the overall efficiency of the system by reducing the object detection time and computation time.

On the other hand, although not shown, for example, depending on the user's selection or setting, for example, except for items that are not handled among fashion items, a corresponding object detection model that has been trained by specializing only on the specific item(s) handled is used, The efficiency can be further increased.

Alternatively, fashion items may be randomly classified and multiple object detection models corresponding to the classified items may be used. In this case, the object detection model is applied sequentially or in parallel according to a criterion such as high user interest, high sales, or high priority due to the high level of sales or the type or number of products possessed to detect objects related to fashion items. Can be detected.

In addition, in the case of object detection, since the cropping process described later remains, it is not necessary to perform a complicated process such as identifying an object with a fashion item critically, but to detect an object according to a predetermined criterion, etc., and perform the cropping process. I can.

Meanwhile, when the object recognition/crop unit 240 detects a plurality of objects related to the fashion item as a result of object detection, it may process according to a setting. Here, according to the setting, for example, all objects detected to select a specific object among a plurality of objects in the cropping process are transferred to the cropping process, or a plurality of detection objects are determined in advance for convenience in the cropping process. Among them, only the object containing the fashion item related to the desired fashion item may be identified and transferred to the cropping process. On the other hand, it is preferable to group objects including fashion items belonging to the same classification among the detected objects and transmit them through a cropping process.

The object recognition/crop unit 240 determines the fashion image part of the actually identified class from the recognized object as a region of interest, that is, a Region of Interest (ROI), and crops the corresponding part, that is, an ROI part, from the object. do. As the object detection and cropping model used in the present invention, through the R-CNN series model, the ROI can be automatically cropped and used directly for similar product search described later.

In this way, through the object recognition/crop unit 240 according to the present invention, a learning model specialized for fashion items is used, through which recognition accuracy is increased, and a number of cropped images for each object are manually selected in the image search step in the terminal 110. It is possible to search for similar fashion products by putting an integrated coordination image instead of individually inputting them, thereby improving user convenience.

The above is a process of performing object detection and cropping a part of an actual fashion item to be searched from the detected object from the ROI. However, the present invention is not necessarily limited thereto. For example, in the case of object detection, a fashion item to be searched from an image is cropped and used for search. Although not shown, detection of an object including the fashion item according to learning or through a pre-input fashion item The process can be omitted. That is, since the fashion item to be used for search is extracted from the image in the end, the detection process of the object including the object may be removed and replaced with a process of detecting the fashion item. By doing this, the detected fashion item can be cropped immediately and the processing speed or the amount of association can be reduced to increase the system efficiency.

The image similarity estimation unit 250 calculates the similarity by comparing the fashion image cropped by the object recognition/crop unit 240 with the product image stored in the database 260 and image-to-image. The image similarity estimating unit 250 may set a ranking of similar product candidates returned as a search result based on the calculated similarity.

According to the present invention, the image similarity estimating unit 250 calculates the cropped image data and fashion product image data stored in a database using sub-networks included in a pre-learned network function for image similarity estimation and retrieval. Similarity is estimated and similar products are searched through it. Here, the sub-networks may include a convolutional neural network (CNN) structure among deep neural network structures. As shown in FIG. 4, the sub-networks may constitute at least a part of a Siamese network. However, the present invention is not necessarily limited thereto. Meanwhile, as described above, a detailed description of a convolutional neural network (CNN), a Siamese network, and the like among the deep neural network structures will be omitted herein with reference to known techniques.

The conventional method of searching for similar products is mainly an image-to-text search method, i.e., a classification model is established by learning data tagged with an attribute on an image in advance. When an image to be searched is input, the tagged attribute, which is a text, is returned. This was how images with attributes were presented as search results. For example, when a clothing image is requested to be searched, attribute values such as ??white", ??long sleeve", ??round neck", etc. are returned, and the attributes of the returned clothing image are manually entered in the database. Matching and returning the search result, in this case, it was difficult to meet the intention of the search requester because there were many targets of similar products to be matched.

On the other hand, according to the present invention, when the image-to-image search method using the Siamese network shown in FIG. 4 is used, an index of similarity can be calculated numerically. Therefore, by arbitrarily adjusting or setting the calculated similarity index, the similar product search result value to be returned may vary, and through this adjustment, it is possible to more meet the intention or needs of the search requester.

For example, if the information of the terminal requesting the search (e.g., the screen size of the terminal) is known, the search result is returned in relation to the image similarity by estimating the calculated similarity index in order to return the search result optimized for the terminal. It is also possible to adjust this by setting a threshold value for. In other words, if the terminal is a smartphone with a relatively small screen size, rather than providing to view many similar products through cumbersome scrolling, search results for only similar products with the highest similarity so that they can be viewed on one screen at a time without scrolling. Can be provided as. In addition, the feedback of the terminal can be used for the next learning to reduce the error.

In relation to this, in the present invention, a fashion ontology is constructed and used to extract attributes included in an image, but the product candidate group to be input to the similarity estimation model may be reduced by limiting the attributes. This may consist of, for example, 13 upper obedience classes and attribute information such as lower colors, styles, materials, patterns, and shapes. In relation to this, FIG. 6 shows an example of attributes extracted from a fashion object. However, the fashion ontology according to the present invention is not necessarily limited thereto, and may be implemented with different configurations and definitions. In FIG. 5, a learning model is shown, and an attribute value of an object detected through the learning model can be extracted. As an example, Google's Inception-V3 learning model is shown in FIG. 5, and related information is referred to, and a detailed description of the learning model is omitted herein.

For example, images having the same attribute as the detected attribute may be read from the database 260 and used as a search candidate group. Thereafter, the ranking of the candidate group may be determined in connection with the similarity estimation model based on the Siamese network, and the result may be returned as a search result as shown in FIG. 7.

In addition, it is possible to use the product returned as a search result according to the detected attribute as a filter function that allows a user to perform a detailed search by excluding/including.

That is, in the past, there has been a hassle in requiring a retailer or a service provider to directly input product attribute information, but in the present invention, a fashion ontology is built in advance and the attribute is automatically extracted/tagging from the input image to the seller. In terms of not only shortening the service procedure, but also convenience can be increased in terms of consumers.

On the other hand, the above-described process between the terminal (consumer) and the computing device (seller), that is, the similar product search request-the return of the similar product search result and the selection of the terminal according to the provision of the search result are reflected in learning and updated to further increase efficiency. I can.

If the computing device can have terminal information, a personalized similar product search and provision service customized to the terminal user through learning becomes possible. In this way, when personalization is performed, the factor values of the above-described various models can be more simply configured and efficiency can be further improved through learning, as well as results for a desired similar fashion product can be quickly provided as shown in FIG. 7.

On the other hand, the database 260 stores various data described above, such as received images, product information, and fashion ontology described above in connection with the present invention. Meanwhile, although the database 260 is illustrated and described as one in FIG. 2, it is not necessarily limited thereto. Meanwhile, the database 260 does not necessarily have to be built inside the computing device 120 (or the terminal 110). This database 260 also stores data in the form of a cloud and, if necessary, returns a uniform resource locator (url) to access the url returned from the computing device 120 to the terminal 110 to obtain related data. It can also be built to be able to.

The control unit 270 controls the entire process of the computing device 120 shown in FIG. 2 as a whole, and may appropriately control each configuration as necessary. The control unit 270 may replace the above-described image similarity estimation unit 250 or directly control its operation. Meanwhile, the above-described method may be written as a program that can be executed on a computer, and a computer-readable medium is used. It can be implemented in a general-purpose digital computer that operates the program using. In addition, the structure of the data used in the above-described method can be recorded on a computer-readable medium through various means. Computer-readable media storing executable computer code for performing various methods of the present invention include magnetic storage media (e.g., 　 ROM, 　 floppy disk, 　 hard disk, etc.), 　 optical reading media (e.g., CD-ROM, DVD Etc.).

Those of ordinary skill in the technical field related to the embodiments of the present invention will appreciate that it may be implemented in a modified form without departing from the essential characteristics of the above description. Therefore, the disclosed methods should be considered from a descriptive point of view rather than a limiting point of view. The scope of the present invention appears in the claims rather than the detailed description of the invention, and all differences within the scope equivalent thereto should be construed as being included in the scope of the present invention.

Claims (10)

In the method for searching and providing similar products in a computing device,
Receiving an image to be searched from a terminal;
Recognizing an object including a fashion item from the received image and cropping the fashion item from the recognized object;
Estimating image similarity by calculating the cropped image data and fashion product image data stored in a database using sub-networks included in a pre-learned network function; And
Providing similar product data to the terminal based on the estimated image similarity;
A method for searching and providing similar products in a computing device, comprising: a. The method of claim 1,
The sub-networks,
A method for searching and providing similar products in a computing device, comprising a convolutional neural network (CNN) structure among deep neural network structures. The method of claim 1,
The sub-networks,
A method for searching and providing similar products in a computing device, characterized in that it constitutes at least a part of a Siamese network. The method of claim 1,
Setting a threshold value for returning a search result in relation to the estimated image similarity;
A method for searching and providing similar products in a computing device, characterized in that it further comprises a. The method of claim 4,
A method for searching and providing similar products in a computing device, comprising adjusting data on an image to be returned as a search result among images whose image similarity is estimated based on the set threshold. The method of claim 1,
Recognizing an object including a fashion item from the received image, and cropping a fashion item from the recognized object,
A method for searching and providing similar products in a computing device, characterized in that the object recognition and region of interest (ROI) are automatically cropped by using the model of the pattern R-CNN series learned by specializing in the fashion item. In a computing device for searching and providing similar products,
A receiving unit for receiving an image to be searched from a terminal;
An object recognition/crop unit that recognizes an object including a fashion item from the received image and crops the fashion item from the recognized object;
An image similarity estimation unit for estimating image similarity by calculating the cropped image data and fashion product image data stored in a database using sub-networks included in a pre-learned network function; And
A control unit that controls to provide similar product data to the terminal based on the estimated image similarity;
Computing device comprising a. The method of claim 7,
The sub-networks,
A computing device comprising a convolutional neural network (CNN) structure among deep neural network structures. The method of claim 7,
The sub-networks,
A computing device comprising at least a portion of a Siamese network. The method of claim 7,
The control unit,
And setting a threshold value for returning a search result in relation to the estimated image similarity.

Images