KR20200023093A - A method for recommendation of style coordination and an apparatus therefor - Google Patents

Abstract

The present invention relates to a style coordination recommendation method which can provide information on a coordination item which can be styled with a corresponding item as well as an item desired by a user when searching for an image, and a device therefor. When receiving image data including a search target item from a user device, after searching for an image similar to the image data based on the search target item, one or more coordination items corresponding to the search target item are extracted from the searched images, and recommendation information for the search target item is provided by using the extracted one or more coordination items.

Description

A method for recommendation of style coordination and an apparatus therefor}

The present invention relates to an image retrieval technique, and more particularly, to a style coordination recommendation method for recommending a coordinated item that can be combined with a searched item, based on a search result of a similar image with respect to the searched item, and It relates to a device for this.

Image search can be divided into two types according to the method and purpose of finding an image. The first is to use a text search term to find related images. For example, if you enter the keyword 'flowers', you can see different types of flowers. This type is a traditional search method from the very beginning of the Internet. The second is to enter an image search term, which is an image file, and analyze the image file to find the same or similar image. It's the way computer analysis and internet technology have developed. Compared to the keyword method, the user can intuitively search for an image.

Recently, with the development of artificial intelligence (AI) technology, interest in the second image retrieval method that queries the image data and receives a search result is increasing.

Since the image retrieval technology can intuitively search for a product desired by a user, the image retrieval technology can be used in an online shopping mall or the like that trades a product through a network.

In addition, as the interest of the fashion of modern people is increasing, not only the search for a desired product by the user, but also the demand for styling information of other coordination items that can be appropriately matched with the corresponding product is increasing.

Korean Patent Publication No. 2009-0077032, published July 14, 2009 (Name: Style Coordination recommendation method using wireless recognition)

Accordingly, an aspect of the present invention is to provide a style coordination recommendation method and an apparatus therefor that can provide not only an item desired by a user when searching for an image, but also information about a coordinated item that can be styled with the corresponding item.

In particular, the present invention is to provide a style coordination recommendation method and apparatus for recommending a coordination item that can be combined with a search object item based on a search result of a similar image in relation to the search object item.

The present invention provides a style coordination recommendation method as a means for solving the above-mentioned problems. According to an embodiment of the present invention, there is provided a style coordination recommendation method comprising: receiving, by an apparatus for searching for an image, image data including a search target item from a user device; Searching for an image similar to the image data based on the search target item; Extracting one or more coordination items corresponding to the search target item from the searched images; And providing recommendation information on the search target item by using the extracted one or more coordination items.

In one embodiment of the present invention, the style coordination recommendation method further includes calculating a preference for the extracted one or more coordination items, wherein the providing comprises: the one or more coordination items based on the preferences. May be provided as recommendation information.

In the style coordination recommendation method, the calculating may include the preference in proportion to one or more of a frequency of appearing each coordination item from the searched images, and a frequency of a user's selection of an image including each coordination item. Can be calculated.

The calculating may include checking a category of the search target item; Extracting one or more items within the retrieved image; And extracting, as a coordinating item, an item belonging to a category that can be coordinated with the search target item based on a category of the search target item among the one or more items.

In the style coordination recommendation method, the receiving of the image data may include receiving an image search request including the image data, and further, before extracting the coordination item, After providing to the user device, receiving the selection information for one image from the one or more images included in the search results, and extracting the coordination item, the selected image of the search results The coordination item may be extracted from an image including an included search target item.

The present invention also provides an apparatus for recommending style coordination as another means for solving the above problems. Apparatus for recommending a style coordination according to the present invention, the storage module for storing a plurality of images to be used in the image search; A communication module for transmitting and receiving data with a user device through a network; An image retrieval module for retrieving an image similar to the image data based on the search target item upon receiving image data including a search target item from the user device; And a style recommendation module that extracts one or more coordination items corresponding to the search target item from the searched images, and provides recommendation information for the search target item by using the extracted one or more coordination items.

The apparatus for recommending style coordination may include extracting one or more searchable items from image data received from the user device, and extracting the search target item from the one or more searchable items based on a user context. The module may further include.

In addition, in the apparatus for recommending style coordination according to the present invention, the style recommendation module calculates a preference for the extracted one or more coordination items, and then combines the one or more coordination items based on the preferences to search for the search. Style coordination recommendation information may be provided for a target item, and the preference may include a frequency of occurrence of each coordination item from the retrieved images; And a frequency of the user's selection of an image including each coordination item.

The style recommendation module may identify a category of the search target item, extract one or more items from the searched image, and coordinate with the search target item based on a category of the search target item among the one or more items. An item belonging to a possible category may be extracted as a coordination item.

Further, in the device for recommending style coordination according to the present invention, the image search module performs an image search on the image data requested to be searched from the user device, and provides the search result to the user device. When the selection information on one image of the search result is received from the user device, the coordination item may be extracted from an image including the search target item included in the selected image.

The present invention extracts a coordination item that can be styled with a search target item from a plurality of images including items identical or similar to the search target item extracted as an image search result, and then, the frequency of the coordination item that appears in each search result. Or, by calculating the preference for the coordinated item based on the frequency of the user's selection, based on this recommendation of coordinated items that can be appropriately combined with the search target item, through the image search, the user can intuitively search for the desired item In addition, by providing the user with recommended information as a coordinated item that can be styled together with the item desired by the user, the user's convenience can be achieved.

1 is a view for explaining a style coordination recommendation system according to an embodiment of the present invention.
2 is a block diagram showing the configuration of an apparatus for recommending style coordination according to the present invention.
3 is a block diagram illustrating a configuration of a user device for recommending style coordination according to the present invention.
4 is a flowchart illustrating a general image search process.
5 is a flowchart illustrating a style coordination recommendation method according to an embodiment of the present invention.
6 is a flowchart illustrating an image search process to which a style coordination recommendation function is applied according to an embodiment of the present invention.
7 is a flowchart illustrating a style coordination recommendation process according to another embodiment of the present invention.
8 and 9 illustrate examples of a style coordination recommendation process according to an exemplary embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS To make the features and advantages of the present invention more clear, the present invention will be described in more detail with reference to specific embodiments shown in the accompanying drawings.

However, in the following description and the accompanying drawings, detailed descriptions of well-known functions or configurations that may obscure the subject matter of the present invention will be omitted. In addition, it should be noted that like elements are denoted by the same reference numerals as much as possible throughout the drawings.

The terms or words used in the following description and drawings should not be construed as being limited to the common or dictionary meanings, and the inventors may appropriately define the concept of terms for describing their own invention in the best way. It should be interpreted as meanings and concepts corresponding to the technical idea of the present invention based on the principle that the present invention. Therefore, the embodiments described in the specification and the drawings shown in the drawings are only the most preferred embodiments of the present invention, and do not represent all of the technical idea of the present invention, various modifications that can be substituted for them at the time of the present application It should be understood that there may be equivalents and variations.

In addition, terms including ordinal numbers, such as first and second, are used to describe various components, and are used only to distinguish one component from another component, and to limit the components. Not used. For example, without departing from the scope of the present invention, the second component may be referred to as the first component, and similarly, the first component may also be referred to as the second component.

In addition, when a component is referred to as being "connected" or "connected" to another component, it means that it may be connected or connected logically or physically. In other words, although a component may be directly connected or connected to other components, it should be understood that other components may exist in the middle and may be connected or connected indirectly.

In addition, the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In addition, the terms "comprises" or "having" described herein are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or the same. It is to be understood that the present invention does not exclude in advance the possibility of the presence or the addition of other features, numbers, steps, operations, components, parts, or a combination thereof.

In addition, the terms "~", "~", "module", etc. described in the specification mean a unit for processing at least one function or operation, which may be implemented by hardware or software or a combination of hardware and software. have.

In addition, "a" or "an", "one", "the" and the like shall also be indicated otherwise in the context of the present invention (particularly in the context of the following claims). Unless otherwise expressly contradicted by context, it may be used in the sense including both singular and plural.

In addition, embodiments within the scope of the present invention include computer readable media having or conveying computer executable instructions or data structures stored on computer readable media. Such computer readable media can be any available media that can be accessed by a general purpose or special purpose computer system. By way of example, such computer readable media may be in the form of RAM, ROM, EPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage, or computer executable instructions, computer readable instructions or data structures. May be used to store or deliver any program coordination means, and may include, but is not limited to, any other medium that can be accessed by a general purpose or special purpose computer system. .

In the following description and claims, “network” is defined as one or more data links that enable the transfer of electronic data between computer systems and / or modules. When information is transmitted or provided to a computer system via a network or other (wired, wireless, or a combination of wired or wireless) communication connections, this connection can be understood as a computer-readable medium. Computer-readable instructions include, for example, instructions and data that cause a general purpose or special purpose computer system to perform a particular function or group of functions. The computer executable instructions may be binary, intermediate format instructions such as, for example, assembly language, or even source code.

In addition, the present invention relates to personal computers, laptop computers, handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, mobile phones, PDAs, pagers In a network computing environment having various types of computer system configurations including pagers, etc., the present invention may be applied to provide an image retrieval service to the computer systems. The invention may also be practiced in distributed system environments where both local and remote computer systems that are linked via wired networks, wireless data links, or combinations of wired and wireless data links perform tasks. In a distributed system environment, program modules may be located in both local and remote memory storage devices.

First, a style coordination recommendation system according to an embodiment of the present invention will be described.

1 is a view showing a style coordination recommendation system according to an embodiment of the present invention. Referring to FIG. 1, a system according to the present invention includes a search server 100 and a user device 200 connected through a network NW.

The search server 100 and the user device 200 may transmit and receive data to and from each other through the network NW. For example, the communication network (NW) may include a wireless communication network such as a wireless LAN (WLAN), Wi-Fi, Wibro, Wimax, High Speed Downlink Packet Access (HSDPA), etc. Depending on how the system is implemented, it may also include wired communication networks such as Ethernet, xDSL (ADSL, VDSL), Hybrid Fiber Coaxial Cable (HFC), Fiber to The Curb (FTTC), and Fiber To The Home (FTTH).

In addition, the network NW used in the present invention may include, for example, a mobile communication network including a plurality of access networks (not shown) and a core network (not shown) connecting them. Here, the access network is a network that performs wireless communication by directly connecting to a terminal, for example, a plurality of base stations such as a base station (BS), a base transceiver station (BTS), a NodeB, an eNodeB, and a base station controller (BSC). It may be implemented as a base station controller such as a Radio Network Controller (RNC). In addition, as described above, the digital signal processing unit and the wireless signal processing unit, which are integrally implemented in the base station, are divided into digital units (hereinafter referred to as DUs and radio units (hereinafter referred to as RUs)). In addition, a plurality of RUs (not shown) may be installed in each of the areas, and a plurality of RUs may be connected to a centralized DU, and a core network (not shown) constituting a mobile network together with an access network may be connected to an external network. It serves to connect networks, for example other communication networks, such as the Internet.

As described above, the core network is a network system that performs main functions for mobile communication services such as mobility control and switching between access networks, and performs circuit switching or packet switching, and performs mobile network. Manage and control the flow of packets within In addition, the core network may manage inter-frequency mobility and may play a role for interworking with traffic in the access network and the core network and other networks such as the Internet network. The core network may further include a serving gateway (SGW), a PDN gateway (PGW), a mobile switching center (MSC), a home location register (HLR), a mobile mobility entity (MME), and a home subscriber server (HSS). It may be.

In addition, the network NW according to the present invention may include an internet network. The Internet network refers to a common public communication network, that is, a public network, through which information is exchanged according to the TCP / IP protocol. Through the network NW, the search server 100 and the user device 200 interoperate with each other to form an image search system according to the present invention.

The user device 200 requests an image search by transmitting an image search word in the form of an image file to the search server 100 according to a user request. In this case, the image search word may include image data, and the image data may include one or more searchable items. Here, an item means an object included in an image and having an independent meaning or name. The item may be, for example, a product sold in an online shopping mall, or may be nature or a person such as an animal or a plant. When two or more items are included in such image data, a target that a user wants to search may be any one of them.

The search server 100 performs an image search service and collects a plurality of image sources for image search to provide an image database. When the search server 100 receives an image search request including image data of the user device 200, the search server 100 searches for similar images by comparing images of the prepared image database with the searched image. In this case, when the search includes two or more searchable items in the requested image, the search server 100 may extract a search target item desired by the user from among a plurality of items included in the image data. Extraction of the item may be performed through prediction based on a correlation between a user context and a category.

The search server 100 searches for the same or similar image as the searched image from the pre-stored image database based on the search target item, and provides the image search result to the user device 200.

In particular, the search server 100 according to the present invention may use the search results to extract one or more coordination items that can be styled with the search target item to recommend to the user. In this case, one or more coordination items recommended to the user may be provided together with the search result.

The search server 100 is an entity existing on the network, and serves as a web server, a database server, and an application server. According to an exemplary embodiment, the search server 100 may be a server supporting a portal site including a search engine, and may also be a server supporting an online shopping mall site selling various products.

In addition, the user device 200 will be described as a representative example of the mobile communication terminal, but the terminal is not limited to the mobile communication terminal, but a variety of information communication devices, multimedia terminals, wired terminals, fixed terminals and IP (Internet Protocol) terminals, etc. Applicable to the terminal. For example, the terminal may be a mobile terminal having various mobile communication specifications such as a mobile phone, a portable multimedia player (PMP), a mobile internet device (MID), a smart phone, a tablet PC, a tablet PC, and an information communication device. Can be advantageously used. As another example, the terminal may be a notebook, a personal computer, or the like.

Next, the configuration of the search server 100 and the user device 200 for providing the style coordination recommendation service according to the present invention will be described in detail. First, the configuration of the search server 100 according to an embodiment of the present invention will be described.

2 is a block diagram illustrating a configuration of a search server according to an exemplary embodiment of the present invention. Referring to FIG. 2, the search server 100 includes a communication module 110, a storage module 120, and a control module 130.

The communication module 110 is for communication with the user device 200. The communication module 110 connects to the network NW, and when the user device 200 connects to the search server 100 through the network NW, exchanging data including necessary information with the user device 200. Perform communication for When the communication module 110 receives data for transmission from the control module 130 to the user device 200, for example, a search result including a plurality of images and recommendation information about the coordination item corresponding to the search target item, The received data is composed of a packet and transmitted. In addition, the communication module 110 extracts the data, that is, the image search word from the packet received from the user device 200 and transmits it to the control module 130. The image search word includes image data.

The storage module 120 stores a program and data necessary for the operation of the search server 100 and may be divided into a program area and a data area. The program area may store a program for controlling the overall operation of the search server 100, an operating system (OS) for booting the search server 100, an application for image search, and the like. The data area is an area in which data generated according to the operation of the search server 100 and data necessary for the operation of the search server 100 are stored. For example, the storage module 120 may store images for a plurality of items. Various data stored in the storage module 120 may be deleted, changed, or added according to a user's manipulation. In particular, in the present invention, the storage module 120 stores a plurality of image sources to be used for image search.

The control module 130 may control a general operation of the search server 100 and a signal flow between internal blocks of the search server 100 and perform a data processing function for processing data. The control module 130 is preferably a microprocessor or a central processing unit (CPU). The control module 130 includes an image processing module 131, an image retrieval module 133, and a style recommendation module 135.

The image processing module 131 performs preprocessing for image retrieval. In detail, the image processing module 131 extracts one or more searchable items from image data included in the image search request. The extraction of the search target item does not recognize and identify the search target item, but refers to identifying the searchable item among the objects after classifying the objects in the image by regions. For example, the image processing module 131 may extract an item of a category sold on an online shopping mall site as an object to be searched from among objects included in the image.

In particular, when there are a plurality of searchable items in the image data, the image processing module 131 according to the present invention may select one of them as a search target item. In detail, the image processing module 131 transmits the information about the searchable item to the user device 200 and is then selected by the user or based on a user context such as a user's purchase history and a user's selection of a search result. One of the items may be selected as a search target item.

The image search for the selected search target item is performed by the image search module 133.

In detail, the image search module 133 searches for similar images by comparing image data with a plurality of images stored in the storage module 120 based on the search target item selected by the image processing module 133, and uses the search result as a user. To the device 200.

The style recommendation module 135 extracts one or more coordination items that match the search target item from the search result of the image search module 133, and provides the extracted coordination items to the user device 300 as style coordination recommendation information. . In this case, the coordination item may identify a category of the search target item, extract one or more items from the searched image, and then coordinate with the search target item based on a category of the search target item among the extracted one or more items. It may be extracted as an item belonging to a category.

In addition, the style recommendation module 135 extracts a coordination item from a search result, wherein the style recommendation module 135 selects at least one of a frequency of each coordination item from the searched images and a user's selection frequency for an image including each coordination item. In proportion, the preference of each coordination item may be calculated, and an optimal coordination item may be recommended to the user based on the calculated preference.

The operation of the control module 130 including the image processing module 131, the image retrieval module 133, and the style recommendation module 135 will be described in more detail below.

Next, the user device 200 according to an embodiment of the present invention will be described. 3 is a block diagram illustrating a configuration of a user device according to an exemplary embodiment of the present invention. Referring to FIG. 3, a user device 200 according to an exemplary embodiment of the present invention may include a communication unit 210, an input unit 220, a display unit 230, a storage unit 240, a controller 250, and a camera unit 260. It includes.

The communication unit 210 is for communication with other devices including the search server 100 and performs communication through various communication connection methods. The communication unit 210 may communicate by selecting any one of various communication connection type communication functions. The communication unit 210 may be implemented by one module or a plurality of modules. For example, the communication unit 210 according to the WCDMA, LTE, LTE-A, etc. according to the broadband mobile communication method to access the network through the base station, according to the Wireless Local Area Network (WLAN) method using Wi-Fi (wireless fidelity) A wireless near field communication method for accessing a network through an access point (AP) and performing communication may be used. The communication unit 210 may include an RF transmitter for up-converting and amplifying a frequency of a transmitted signal, and an RF receiver for low noise amplifying and down-converting a received signal. An image search request including data received from the controller 250, for example, image data, may be converted into a wireless signal and transmitted through a wireless channel. In addition, the communication unit 210 may receive and transmit a wireless signal including data, for example, an image search result and / or style recommendation information, to the control unit 250 through a wireless channel.

The input unit 220 receives a key operation of the user for controlling the user device 200, generates an input signal, and transmits the input signal to the controller 250. The input unit 220 may include any one of a power key, a numeric key, and a direction key for power on / off, and may be formed as a predetermined function key on one surface of the user device 200. When the display unit 230 is formed of a touch screen, functions of various keys of the input unit 220 may be performed in the display unit 230. When the display unit 230 may perform all functions using only the touch screen, the input unit 220 may be omitted. It may be.

The display unit 230 visually provides a menu, input data, function setting information, and various other information of the user device 200 to the user. The display unit 230 outputs various UI (User Interface) screens such as a boot screen, a standby screen, and a menu screen of the user device 200. The display unit 230 may be formed of a liquid crystal display (LCD), organic light emitting diodes (OLEDs), active matrix organic light emitting diodes (AMOLEDs), and the like. Meanwhile, the display unit 230 may be implemented as a touch screen. In this case, the display unit 230 may include a touch sensor, and the controller 250 may detect a user's touch input through the touch sensor. The touch sensor may be configured as a touch sensor such as capacitive overlay, pressure, resistive overlay, infrared beam, or a pressure sensor. . In addition to the above sensors, all kinds of sensor devices capable of detecting contact or pressure of an object may be used as the touch sensor of the present invention. The touch sensor detects a user's touch input, generates a detection signal, and transmits the detected signal to the controller 250. The sensing signal may include coordinate data input by the user. When the user inputs the touch position movement operation, the touch sensor may generate a detection signal including coordinate data of the touch position movement path and transmit the detected signal to the controller 250.

The storage unit 240 stores a program and data necessary for the operation of the user device 200, and may be divided into a program area and a data area. The program area may store a program for controlling the overall operation of the user device 200, an operating system (OS) for booting the user device 200, an application program, and the like. The data area is an area in which data generated according to the use of the user device 200 is stored. In addition, the storage unit 240 may store various data generated according to the operation of the user device 200. The controller 250 may control an overall operation of the user device 200, a signal flow between internal blocks of the user device 200, and may perform a data processing function of processing data. The controller 250 may be a central processing unit (CPU), an application processor, a graphic processor (GPU), or the like. When the image search word, for example, image data is input, the controller 250 transmits the input image data to the search server 100 through the communication unit 210, and the search result and / or the search target item from the search server 100. When the recommendation information of the appropriate coordination item corresponding to is received, the received search result and / or the recommendation information of the appropriate coordination item corresponding to the search target item may be displayed on the display unit 130. In addition, when one image of the search result is selected through the input unit 220, the corresponding selection information is transmitted to the search server 100, and the coordination item related to the search target item included in the image selected from the search server 100 is selected. The recommendation information may be received.

In addition, the camera unit 260 is a means for capturing an image, through which the image data can be obtained. For example, the user may select the image data photographed through the camera unit 260 and request a search or style coordination recommendation.

4 is a flowchart illustrating a typical image retrieval process performed in the above-described system.

Referring to FIG. 4, when a user selects a query image to be used for image search through the input unit 220 (S105), the user device 200 transmits an image search request including image data corresponding to the query image. The transmission is transmitted to the search server 100 through 210 (S110).

When the control module 130 of the search server 100 receives an image search request from any user device 200 through the communication module 110, the control module 130 compares the image data with a plurality of images stored in the storage module 120. Search for a similar image (S115). In this case, the image search may be performed through a learning algorithm such as deep learning. In addition, the image search may extract a plurality of images having a similarity or higher than a predetermined level with the searched image data.

In this case, when the image data includes two or more searchable items, one search target item may be selected from among them, and the image search may be performed based on the selected search target item.

The search target item may be extracted as a searchable object among a plurality of objects included in the image. The search server 100 may select one of a plurality of items as a search target item. The selection may be made according to a user input, or may be predicted based on a user context, and may be made based on the predicted category.

The search server 100 extracts a user context such as a user's search history, and accumulates and stores the category of the searched image and the extracted user context in the storage module 120 as one set of training data, and stores the user's interest. The relationship between the category of the item and the category of the item intended by the user at the time of performing the search may be analyzed, and a search target item may be selected based on the correlation.

In addition, the search server 100 provides a search result including a plurality of searched images corresponding to the image search request to the user device 200 (S120), and the control unit 250 of the user device 200 receiving the search result. ) Displays a plurality of images that are search results through the display unit 230 (S125). In this case, the user may select an image desired by the user from among a plurality of images, and the image selected by the user is likely to be a user's preferred or desired image. Accordingly, when one image is selected, the controller 250 detects the image through the input unit 220 or the display unit 230, and retrieves information about the image selected by the user through the communication unit 210. Can be delivered to. The image processing module 133 of the search server 100 classifies, accumulates and stores the images selected by the user among the plurality of images included in the search result, and groups the images searched by category according to the design classification. After creating a plurality of groups, a user's design preference can be determined according to the number of elements in each group. Based on these preferences, a user can select an item with high preference among a plurality of items included in the image data. Can be.

When the search target item is selected as described above, the search server 100 searches for an image similar to the search target item by performing an image search based on the search target item.

The present invention recommends a coordination item that can be styled together with the search target item based on the search result extracted during the image search process.

5 is a flowchart illustrating a style coordination recommendation method according to an embodiment of the present invention.

Referring to FIG. 5, when the image data including the search target item is received from the user device 100 (S205), the search server 100 according to the present invention extracts a search target item from the image data (S210). ). The extraction of the search target item separates each object in the image through image processing such as edge extraction, recognizes a category of each separated object, and then sets a predetermined category, for example, a category sold in an online shopping mall server. It may be made by checking whether or not included in the search target item stored in the storage module 120. In addition, when there are a plurality of search target items, one of the plurality of items may be extracted as a search target item based on a user context such as purchase history and user selection.

The search server 100 searches for similar images among the image data stored in the storage module 120 based on the extracted search target item (S215). In this case, the image search may be performed through a learning algorithm such as deep learning.

The search server 100 extracts one or more coordination items corresponding to the search target item from the searched images (S220). In more detail, in step S220, after checking the category of the search target item, extracting one or more items from the searched image, the search based on the category of the search target item among the extracted one or more items An item belonging to a category that can be coordinated with the target item may be extracted as the coordinate item. For example, when the search target item belongs to the 'top' category of the clothing category, an item belonging to 'bottom', 'miscellaneous goods', or 'accessories' within the clothing category may be selected as a coordinating item. When the target item belongs to the 'blouse' in the 'top' category, the category other than 'blouse' in the 'top' category may be further included in the coordination item.

Through the step S220, a plurality of coordination items may be extracted for each of the found plurality of images, and thus, an optimal coordination item should be selected among the plurality of coordination items.

To this end, the search server 100 according to the present invention calculates a preference for the extracted one or more coordination items (S225). In this case, the preference represents an unspecified number of preferences for styling each coordination item on a search target item, and the frequency of each coordination item appearing from the searched images, and a user's selection frequency for an image including each coordination item. It can be calculated in proportion to one or more of the above. In other words, the searched images are all images that include the same or similar items as the search target item, and the specific coordination items appear together when the search target item and the coordination item are styled together. Therefore, the frequency at which each of the coordination items appears may be represented by the styling frequency of the corresponding coordination item and the search target item, and the higher the frequency, the more people can use the corresponding coordination items.

When the preference is calculated as described above, the search server 100 provides the at least one coordination item as styling code recommendation information for the search target item based on the calculated preference of each coordination item (S230). In more detail, the coordination items may be recommended to the user in the order of high preference.

6 is a flowchart illustrating an image search process to which a style coordination recommendation function is applied according to an embodiment of the present invention.

Referring to FIG. 6, in the style coordination recommendation system according to the present invention, the user device 200 receives or selects a query image to be used for image search from the user through the input unit 220 (S305).

Accordingly, the user device 200 transmits an image search request including image data corresponding to the query image to the search server 100 through the communication unit 210 (S310).

When the search server 100 receives an image search request from the user device 200 through the communication module 110, the search server 100 compares the image data included in the image search request with a plurality of images stored in the storage module 120 to obtain a similar image. The image is searched for (S315). Through the image search, a plurality of images having a degree of similarity with a predetermined level or more may be extracted.

In this case, when the image data includes two or more searchable items, one search target item may be selected from among them, and the image search may be performed based on the selected search target item.

The search target item may be extracted as a searchable object among a plurality of objects included in the image. The search server 100 may select one of a plurality of items as a search target item. The selection may be made according to a user input, or may be predicted based on a user context, and may be made based on the predicted category.

The search server 100 extracts a user context such as a user's search history, and accumulates and stores the category of the searched image and the extracted user context in the storage module 120 as one set of training data, and stores the user's interest. The relationship between the category of the item and the category of the item intended by the user at the time of performing the search may be analyzed, and a search target item may be selected based on the correlation.

Next, the search server 100 provides a search result including a plurality of searched images corresponding to the image search request to the user device 200 (S320), and controls the control unit of the user device 200 receiving the search result ( 250 displays a plurality of images that are search results through the display unit 230 (S325).

In this case, the user may select an image desired by the user from among a plurality of images.

As such, when any one image is selected from the search results, the user device 200 detects the image through the input unit 220 or the display unit 230, and retrieves information about the image selected by the user through the communication unit 210. Transfer to step 100 (S330).

The search server 100 extracts a coordination item corresponding to the search target item included in the selected image or the same item as the search target item in operation S335. To this end, the search server 100 may perform a search corresponding to the selected image again, and use the search result to extract a coordination item that matches the search target item of the selected image. Here, the process of extracting the coordination item may be performed as shown in FIG. 5.

The search server 100 transmits the extracted coordination item to the user device 200 as style coordination recommendation information of the search target item (S340), and the user device 200 displays the coordination item on the display unit 230 to display the user. To provide.

The style coordination recommendation method according to the present invention may be performed separately from the image search. 7 is a flowchart illustrating a style coordination recommendation process according to another embodiment of the present invention.

Referring to FIG. 7, in another embodiment of the present disclosure, when the user selects image data for querying (S405), the user device 200 transmits the selected image data to the search server 100 to recommend style coordination. Request (S410).

The search server 100 searches for an image similar to the image data included in the request (S415). In this case, as described above, the search server 100 may extract a search target item from the image data, and search for an image including similar items based on the extracted search target item.

The search server 100 extracts a plurality of coordination items that are styled together with the search target item from the images extracted as the search result (S420). Extraction of the coordination item may be performed as shown in FIG. 5.

In addition, the search server 100 transmits the extracted coordination item to the user device 200 as style coordination recommendation information for the search target item (S425), and the user device 200 displays it on the display unit 230 ( S430).

8 and 9 illustrate examples of a style coordination recommendation process according to an exemplary embodiment of the present invention.

Fig. 8 belongs to the lower "blouse" category of the "upper" category as a search target item from the image data for which the user requests a search.

When the search is performed based on the image data, as illustrated in FIG. 9, a plurality of images including the blouse may be searched.

In this case, the search server 100 according to the present invention extracts the coordinated items that are styled together with the search target item (blouse) from the searched image as shown in FIG. 9.

For example, in the first image of FIG. 9, an item may be extracted from a blouse and a skirt. Among them, a skirt except for a blouse, which is a search target item, is extracted as a coordinated item.

When one or more coordination items are extracted for each image, duplicate coordination items may exist among them, and the coordination items may be calculated as a preference, and the coordination items having a high preference may be provided as style coordination recommendation information for the search target item.

As described above, the specification includes the details of a number of specific implementations, but these should not be understood as being limited to the scope of any invention or claimable, but rather may be specific to a particular embodiment of a particular invention. It should be understood as a description of the features. Certain features that are described in this specification in the context of separate embodiments may be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable subcombination. Furthermore, while the features operate in a specific combination and may be depicted as such initially claimed, one or more features from the claimed combination may in some cases be excluded from the combination, the claimed combination being a subcombination. Or a combination of subcombinations.

Likewise, although the operations are depicted in the drawings in a specific order, it should not be understood that such operations must be performed in the specific order or sequential order shown in order to obtain desirable results or that all illustrated operations must be performed. In certain cases, multitasking and parallel processing may be advantageous. In addition, the separation of the various system components of the above-described embodiments should not be understood as requiring such separation in all embodiments, and the described program components and systems will generally be integrated together into a single software product or packaged into multiple software products. It should be understood that it can.

Specific embodiments of the subject matter described in this specification have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order but still achieve desirable results. As an example, the process depicted in the accompanying drawings does not necessarily require that particular illustrated or sequential order to obtain the desired results. In certain implementations, multitasking and parallel processing may be advantageous.

The foregoing description presents the best mode of the invention, and provides examples to illustrate the invention and to enable those skilled in the art to make and use the invention. The specification thus produced is not intended to limit the invention to the specific terms presented. Thus, while the present invention has been described in detail with reference to the examples described above, those skilled in the art can make modifications, changes and variations to the examples without departing from the scope of the invention.

Therefore, the scope of the present invention should be determined by the claims rather than by the described embodiments.

The present invention extracts a coordination item that can be styled with a search target item from a plurality of images including items identical or similar to the search target item extracted as an image search result, and then, the frequency of the coordination item that appears in each search result. Or, by calculating the preference for the coordination item based on the frequency of the user's selection, based on this recommendation of the coordination item that can be appropriately combined with the search target item, through the image search, the user can intuitively search for the desired item In addition, by providing the user with recommended information as a coordinated item that can be styled together with the item desired by the user, the user's convenience can be achieved.

100: search server 110: communication module
120: storage module 130: control module
131: image processing module 133: image search module
135: style recommendation module 200: user device
210: communication unit 220: input unit
230: display unit 240: storage unit
250: control unit 260: camera unit

Claims (12)

The device for image search,
Receiving image data including a search target item from a user device;
Searching for an image similar to the image data based on the search target item;
Extracting one or more coordination items corresponding to the search target item from the searched images; And
And providing recommendation information on the search target item using the extracted one or more coordination items. The method of claim 1,
Calculating preferences for the extracted one or more coordination items,
The providing step, the style coordination recommendation method, characterized in that to provide the at least one coordination item based on the preference information. The method of claim 1, wherein the calculating
How often each coordination item appears from the retrieved images; And
The method of claim 1, wherein the preference is calculated in proportion to one or more of a user's selection frequency for an image including each coordination item. The method of claim 1, wherein the extracting the coordination item comprises:
Identifying a category of the search target item;
Extracting one or more items within the retrieved image; And
And extracting, as a coordination item, an item belonging to a category that can be coordinated with the search target item based on a category of the search target item among the one or more items. The method of claim 1, wherein receiving the image data
And receiving an image search request including the image data. The method of claim 1,
Before the step of extracting the coordination item, after providing a search result to the user device, and further comprising the step of receiving selection information for one image from one or more images included in the search results,
The extracting of the coordination item comprises: extracting the coordination item from an image including a search target item included in the selected image among the search results. A storage module for storing a plurality of images to be used for image retrieval;
A communication module for transmitting and receiving data with a user device through a network; And
An image retrieval module for retrieving an image similar to the image data based on the search target item upon receiving image data including a search target item from the user device; And
And a style recommendation module for extracting one or more coordination items corresponding to the search target item from the searched images, and providing recommendation information for the search target item by using the extracted one or more coordination items. Device for recommending style coordination. The method of claim 7, wherein
And an image processing module for extracting one or more searchable items from the image data received from the user device, and extracting the search target item from the one or more searchable items based on a user context. Device for recommendation. The method of claim 7, wherein the style recommendation module
The apparatus for recommending style coordination, after calculating a preference for the extracted one or more coordination items, combining the one or more coordination items based on the preferences and providing style coordination information on the search target item. . The method of claim 9,
The preference may be determined by how often each coordination item appears from the retrieved images; And a user's selection frequency for an image including each coordination item, which is calculated in proportion to one or more. The method of claim 7, wherein the style recommendation module
Identify a category of the search target item, extract one or more items from the searched image, and coordinate an item belonging to a category that can be coordinated with the search target item based on a category of the search target item among the one or more items Apparatus for recommending style coordination, characterized in that the extraction. The method of claim 7, wherein
The image search module performs an image search on the image data requested to be searched from the user device, and provides the search result to the user device.
The style recommendation module, when receiving selection information on one image of a search result from the user device, extracts the coordination item from an image including a search target item included in the selected image. Device for recommendation.

Images