KR20200005060A - A method for image searching using an captured image based on guidelines and an apparatus therefor - Google Patents

Abstract

The present invention relates to a method for searching an image using a photographing image based on guidelines and a device therefor. The present invention provides a method for searching an image of a user device and a user device therefor, the method comprising the steps of: allowing a photographing unit to obtain photographing composition information which guides photographing composition with regard to products to be searched by a user; allowing the photographing unit to display guidelines on a screen based on the photographing composition information; allowing the photographing unit to photograph the products in accordance with the guidelines; allowing a searching unit to generate an image of the photographed product by an image searching word; and allowing the searching unit to transmit an image searching request including the image searching word to a searching server.

Description

A method for image searching using an captured image based on guidelines and an apparatus therefor}

The present invention relates to an image retrieval technique, and more particularly, when performing an image retrieval stored in a database using a user-photographed image, the image retrieval according to a guideline for guiding the same composition of the image stored in the database. The present invention relates to an image retrieval method and an apparatus using a captured image.

Image search can be classified into two types according to the method and purpose of finding an image. The first is to search for related images using text search terms. For example, if you enter the keyword 'flowers', you can see different types of flowers. This type is a traditional search method that has been around since the beginning of the Internet. The second is to enter an image search term, which is an image file, and analyze the image file to find the same or similar image. This is how computer analysis and the Internet have developed. Compared with the keyword method, the user can intuitively search for an image.

Korean Patent Publication No. 2012-0001847, published Jan. 05, 2012 (Name: Image Search System and Image Search Method)

Disclosure of the Invention An object of the present invention is to provide an image retrieval method using a guideline-based photographed image and a device therefor, which induces to capture an image according to the same composition as the search target image, and performs an image retrieval through the captured image as derived. In providing.

The image search method of the user device according to an embodiment of the present invention for achieving the above object is a step of obtaining the photographing composition information for guiding the photographing composition for the product to be searched by the user, and the photographing Displaying the guideline on the screen according to the photographing composition information; photographing the product according to the guideline by the photographing unit; generating a image search word of the photographed product as an image search term; The search unit transmits an image search request including the image search word to a search server.

The photographing composition information includes a reference distance representing a reference distance, which is the shortest distance from a reference point of a camera photographing a reference image, a reference Euler angle including yaw, roll, and pitch measured at the reference point, and an outline of the reference image. It is characterized by including.

The displaying of the guideline on the screen may include measuring a distance from a focal point of the camera unit of the user device to a product, calculating a distance difference between the reference distance and the measured distance, and calculating the distance difference. And adjusting and displaying the size of the guide line.

After adjusting and displaying the size, the method may further include outputting a distance adjustment guide message guiding to adjust the distance between the user device and the product according to the calculated distance difference.

The displaying of the guideline on the screen may include measuring an Euler angle at the focal point, calculating an angle difference between the reference Euler angle and the measured Euler angle, x-axis according to the calculated angle difference, and rotating the guide line based on at least one of y and z axes.

The method outputs an angle adjustment guide message for guiding the user device to rotate the axis about at least one of the x-axis, the y-axis, and the z-axis according to the calculated angle difference. It further comprises the step.

The photographing unit photographing the product according to the guideline may include detecting a photographing input of the user, and a distance difference between the measured distance from the focus of the camera unit of the user device to the product and the reference distance is less than a predetermined distance. Simultaneously determining whether an angle difference between the Euler angle measured at the focus point and the reference Euler angle satisfies a shooting condition that is less than a predetermined angle; and if the shooting condition is satisfied, shooting is performed, and the determination result And if the condition is not satisfied, outputting a guide message indicating that the photographing condition is not satisfied.

According to an aspect of the present disclosure, the acquiring of the photographing composition information may include transmitting a dictionary image photographing the product to the search server irrespective of the composition, and corresponding to a kind of the product of the dictionary image from the search server. Receiving the photographing composition information.

According to another aspect, the acquiring the photographing composition information includes transmitting the kind of the product to the search server and receiving the photographing composition information corresponding to the kind of the product from the search server. .

A user device for image retrieval according to a preferred embodiment of the present invention for achieving the above object includes a communication unit for communication with a search server, and a distance sensor for taking an image and measuring the image and the distance. The camera unit, a display unit for displaying a screen, and photographing composition information for guiding a photographing composition of a product to be searched by a user, the guide line is displayed on the screen according to the photographing composition information. And a photographing unit for photographing a product according to the guidelines through the camera unit, an image of the photographed product as an image search word, and an image search request including the image search word to the search server through the communication unit. It includes a search unit for transmitting.

The photographing composition information includes a reference distance representing a reference distance, which is the shortest distance from a reference point of a camera photographing a reference image, a reference Euler angle including yaw, roll, and pitch measured at the reference point, and an outline of the reference image. It is characterized by including.

The camera unit includes a distance sensor for measuring the distance from the focus of the camera unit to the goods. Accordingly, the photographing unit calculates a distance difference between the distance measured by the distance sensor and the reference distance, and adjusts the size of the guideline according to the calculated distance difference to display the distance through the display unit. .

The photographing unit may adjust the size of the guideline and display the guideline, and then output a distance adjustment guide message to guide the user device to adjust the distance between the product and the product according to the calculated distance difference.

The user device further comprises a sensor unit for measuring the Euler angle at the focus. Accordingly, the photographing unit calculates an angle difference between the reference Euler angle and the measured Euler angle, and rotates the guideline based on at least one axis among x, y, and z axes according to the calculated angle difference. It is characterized by displaying through the display unit.

The photographing unit rotates and displays the guide line, and then guides the user device to rotate the user device based on at least one axis among x, y, and z axes according to the calculated angle difference. It is characterized by outputting a message.

When the photographing unit detects an input requesting photographing, a distance difference between the measured distance from the focus of the camera unit to the product and the reference distance is less than a predetermined distance and at the same time the difference between the angle of the Euler angle and the reference Euler angle measured at the focus Determines whether the photographing condition is less than the predetermined angle.

Then, the photographing unit photographs the product if the photographing condition is satisfied as a result of the determination, and outputs a guide message indicating that the photographing condition is not satisfied if the photographing condition is not satisfied. It features.

According to the present invention, an image search is performed using an image photographed according to the composition of a product image stored in a database to be searched as a search word, thereby improving the accuracy of the image search.

1 is a view for explaining an image retrieval system using a guideline-based photographed image according to an embodiment of the present invention.
2 is a block diagram illustrating a configuration of a search server according to an embodiment of the present invention.
3 is a block diagram illustrating a configuration of a user device according to an exemplary embodiment of the present invention.
4 is a flowchart illustrating a method for image retrieval using guideline-based captured images according to an exemplary embodiment of the present invention.
5 is a screen example for explaining a method for image search using a guideline-based photographed image according to an exemplary embodiment of the present invention.
6 is a flowchart illustrating a method of displaying a guideline according to an embodiment of the present invention.
7 to 10 are views for explaining a method for displaying a guideline according to an embodiment of the present invention.
11 is a flowchart illustrating a method of capturing an image based on a guideline according to an embodiment of the present invention.
12 is a diagram for describing a method of photographing an image based on a guideline according to an embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS To make the features and advantages of the present invention more clear, the present invention will be described in more detail with reference to specific embodiments shown in the accompanying drawings.

However, in the following description and the accompanying drawings, detailed descriptions of well-known functions or configurations that may obscure the subject matter of the present invention will be omitted. In addition, it should be noted that like elements are denoted by the same reference numerals as much as possible throughout the drawings.

The terms or words used in the following description and drawings should not be construed as being limited to the ordinary or dictionary meanings, and the inventors may appropriately define the concept of terms for explaining their own invention in the best way. It should be interpreted as meaning and concept corresponding to the technical idea of the present invention based on the principle that the present invention. Therefore, the embodiments described in the present specification and the configuration shown in the drawings are only the most preferred embodiments of the present invention, and do not represent all of the technical ideas of the present invention, and various alternatives may be substituted at the time of the present application. It should be understood that there may be equivalents and variations.

In addition, terms including ordinal numbers, such as first and second, are used to describe various components, and are used only to distinguish one component from another component, and to limit the components. Not used. For example, without departing from the scope of the present invention, the second component may be referred to as the first component, and similarly, the first component may also be referred to as the second component.

In addition, when a component is referred to as being "connected" or "connected" to another component, it means that it may be connected or connected logically or physically. In other words, although a component may be directly connected or connected to other components, it should be understood that other components may exist in the middle, and may be connected or connected indirectly.

In addition, the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In addition, the terms "comprises" or "having" described herein are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or the same. It is to be understood that the present invention does not exclude in advance the possibility of the presence or the addition of other features, numbers, steps, operations, components, parts, or a combination thereof.

In addition, terms such as "?", "?", And "module" described in the specification refer to at least one function or operation.

Means a unit to be processed, which may be implemented in hardware or software, or a combination of hardware and software.

Also, "a or an", "one", "the" and the like are also indicated differently herein in the context of describing the present invention (especially in the context of the following claims). Unless otherwise expressly contradicted by context, it may be used in the sense including both the singular and the plural.

In addition, embodiments within the scope of the present invention include computer readable media having or conveying computer executable instructions or data structures stored on the computer readable medium. Such computer readable media can be any available media that can be accessed by a general purpose or special purpose computer system. By way of example, such computer readable media may be in the form of RAM, ROM, EPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage, or computer executable instructions, computer readable instructions or data structures. And may include, but are not limited to, any other medium that can be used to store or deliver certain program code means, and can be accessed by a general purpose or special purpose computer system. .

In the following description and claims, “network” is defined as one or more data links that enable the transfer of electronic data between computer systems and / or modules. When information is transmitted or provided to a computer system via a network or other (wired, wireless, or a combination of wired or wireless) communication connections, this connection can be understood as a computer-readable medium. Computer-readable instructions include, for example, instructions and data that cause a general purpose or special purpose computer system to perform a particular function or group of functions. The computer executable instructions may be, for example, binary, intermediate format instructions such as assembly language, or even source code.

In addition, the present invention relates to personal computers, laptop computers, handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, mobile phones, PDAs, pagers In a network computing environment having various types of computer system configurations, including (pager) and the like, it may be applied to providing advertisements to the computer systems. The invention may also be practiced in distributed system environments where both local and remote computer systems that are linked via wired networks, wireless data links, or combinations of wired and wireless data links perform tasks. In a distributed system environment, program modules may be located in both local and remote memory storage devices.

First, an image retrieval system using guideline-based photographed images according to an embodiment of the present invention will be described. 1 is a view for explaining an image retrieval system using a guideline-based photographed image according to an embodiment of the present invention. Referring to FIG. 1, an image search system includes a search server 100 and a user device 200.

The search server 100 and the user device 200 may transmit and receive data to and from each other through a communication network (NW). For example, the communication network NW may include a wireless communication network such as a wireless LAN (WLAN), Wi-Fi, Wibro, Wimax, High Speed Downlink Packet Access (HSDPA), Depending on the system implementation method, wired communication networks such as Ethernet, xDSL (ADSL, VDSL), Hybrid Fiber Coaxial Cable (HFC), Fiber to The Curb (FTTC), and Fiber To The Home (FTTH) may be included.

In addition, the communication network NW of the present invention may include, for example, a mobile communication network including a plurality of access networks (not shown) and a core network (not shown) connecting them. Here, the access network is a network that performs wireless communication by directly connecting to a terminal, for example, a plurality of base stations such as a base station (BS), a base transceiver station (BTS), a NodeB, an eNodeB, and a base station controller (BSC). It may be implemented as a base station controller such as a Radio Network Controller (RNC). In addition, as described above, the digital signal processing unit and the wireless signal processing unit, which are integrally implemented in the base station, are divided into digital units (hereinafter referred to as DUs and radio units (hereinafter referred to as RUs)). In addition, a plurality of RUs (not shown) may be installed in each of the areas, and a plurality of RUs may be connected to a centralized DU, and a core network (not shown) constituting a mobile network together with an access network may be connected to an external network. It serves to connect networks, for example, other communication networks such as the Internet network.

As described above, the core network is a network system that performs main functions for mobile communication services such as mobility control and switching between access networks, and performs circuit switching or packet switching. Manage and control the flow of packets within In addition, the core network may manage inter-frequency mobility and may play a role for interworking with traffic in the access network and the core network and other networks such as the Internet network. The core network may further include a serving gateway (SGW), a PDN gateway (PGW), a mobile switching center (MSC), a home location register (HLR), a mobile mobility entity (MME), and a home subscriber server (HSS). It may be.

In addition, the communication network (NW) according to the present invention may include an Internet network. The Internet network refers to a general public communication network, that is, a public network, through which information is exchanged according to the TCP / IP protocol. The search server 100 and the user device 200 interwork with each other through the communication network NW to form an image search system according to the present invention.

Here, the search server 100 is an entity existing on the network, and serves as a web server, a database server, and an application server. According to a preferred embodiment, the search server 100 searches for and provides a web server that provides a shopping mall web site, a database server that stores images of items sold on the web site, and an image that is the same as or similar to an image search term. It can act as an application server. In the following embodiment, it is assumed that the search server 100 is a server that provides an Internet shopping mall service that sells various products. Accordingly, the search server 100 may search for images in the plurality of images stored in the database. Such an image sold in an internet shopping mall may be an image of a product. Meanwhile, according to another exemplary embodiment, the search server 100 may search for images from a plurality of web pages by accessing another server, that is, a web server, through a network.

In addition, the user device 200 will be described as a representative mobile communication terminal, but the terminal is not limited to the mobile communication terminal, but a variety of information communication devices, multimedia terminals, wired terminals, fixed terminals and IP (Internet Protocol) terminal, etc. Applicable to the terminal. For example, the terminal may be a mobile terminal having various mobile communication specifications such as a mobile phone, a portable multimedia player (PMP), a mobile internet device (MID), a smart phone, a tablet PC, a tablet PC, and an information communication device. Can be advantageously used. As another example, the terminal may be a notebook, a personal computer, or the like.

According to an embodiment of the present invention, the search server 100 provides the user device 200 with photographing composition information corresponding to the type of goods to be searched by the user. The photographing composition information guides the photographing composition of the product to be searched by the user according to the type of the product. In particular, the photographing composition information is guided to the same composition of the product image stored in the search server (100).

The user device 200 may display the guideline based on the photographing composition information. In addition, the user device 200 may take an image according to the displayed guideline, and transmit the captured image as a search word to the search server 100 to request an image search. Accordingly, the search server 100 receives an image photographed with the same composition as the stored product image, and performs an image search on the stored product image through the received image. Therefore, since the image search is performed on the image captured with the same composition as the image search word, the accuracy of the image search may be improved.

Then, the configuration of the search server 100 and the user device 200 according to an embodiment of the present invention in more detail. First, the configuration of the search server 100 according to an embodiment of the present invention will be described. 2 is a block diagram illustrating a configuration of a search server according to an embodiment of the present invention. Referring to FIG. 2, the search server 100 includes a communication module 110, a storage module 120, and a control module 130.

The communication module 110 is for communication with the user device 200. The communication module 110 performs communication for exchanging data including necessary information with the user device 200 when the user device 200 accesses the search server 100. When the communication module 110 receives data for transmission from the control module 130 to the user device 200, for example, a search result including a plurality of images, the communication module 110 configures the received data into a packet and transmits the received data. In addition, the communication module 110 extracts data, that is, an image from a packet received from the user device 200, and transmits the extracted data to the control module 130.

The storage module 120 stores a program and data necessary for the operation of the search server 100 and may be divided into a program area and a data area. The program area may store a program for controlling the overall operation of the search server 100, an operating system (OS) for booting the search server 100, an application for image search, and the like. The data area is an area in which data generated according to the operation of the search server 100 and data necessary for the operation of the search server 100 are stored. The data stored in the storage module 120 includes a plurality of product images for each kind of goods and photographing composition information for each kind of goods. Here, the photographing composition information includes a parameter for displaying a guideline for guiding a composition of photographing the product when photographing to generate an image search word. Various data stored in the storage module 120 may be deleted, changed, or added according to a user's manipulation.

The control module 130 may control a general operation of the search server 100 and a signal flow between internal blocks of the search server 100 and may perform a data processing function for processing data. The control module 130 is preferably a central processing unit (CPU). The control module 130 includes an information providing module 131 and an image search module 133.

The information providing module 131 serves to provide shooting composition information requested by the user device 200. The information providing module 131 may receive an image or a product type from the user device 200 through the communication module 110. When the image is received, after identifying the product type of the image, the photographing composition information corresponding to the corresponding product type may be extracted from the storage module 120 and transmitted to the user device 200 through the communication module 110. Similarly, when receiving the product type, the photographing composition information corresponding to the product type may be extracted from the storage module 120 and transmitted to the user device 200 through the communication module 110. When the image search module 133 receives the image search word, the image search module 133 searches for an image stored in the storage module 120 through the received image search word. The operation of the control module 130 including the information providing module 131 and the image search module 133 will be described in more detail below.

Next, the user device 200 according to an embodiment of the present invention will be described. 3 is a block diagram illustrating a configuration of a user device according to an exemplary embodiment of the present invention. Referring to FIG. 3, a user device 200 according to an embodiment of the present invention includes a communication unit 210, an input unit 220, a display unit 250, a storage unit 260, and a control unit 270.

The communication unit 210 is for communication with other devices including the search server 100 and performs communication through various communication connection methods. The communication unit 210 may communicate by selecting any one of various communication connection type communication functions. The communication unit 210 may be implemented by one module or a plurality of modules. For example, the communication unit 210 according to the WCDMA, LTE, LTE-A, etc. according to the broadband mobile communication method for accessing the network through the base station, according to the Wireless Local Area Network (WLAN) method using Wi-Fi (wireless fidelity) A wireless near field communication method for accessing a network through an access point (AP) and performing communication may be used. The communication unit 210 may include an RF transmitter for upconverting and amplifying a frequency of a transmitted signal, and an RF receiver for low noise amplifying and downconverting a received signal. Data received from the controller 270, for example, an image search word may be converted into a wireless signal and transmitted through a wireless channel. In addition, the communication unit 210 may receive a wireless signal including data, for example, an image search result, through a wireless channel, and transmit the data to the controller 270.

The camera unit 220 is for photographing or capturing an image or an image. To this end, the camera unit 220 includes at least an image sensor. In the embodiment of the present invention, 'shooting' means generating an image generated by the image sensor of the camera unit 220 as a digital file according to a specific image file format (for example, JPG, PNG, etc.), and 'shooting' Means simply displaying an image generated by the image sensor through the display unit 250.

The image sensor receives light reflected from a subject and converts the light into an electrical signal, and may be implemented based on a Charged Coupled Device (CCD), a Complementary Metal-Oxide Semiconductor (CMOS), or the like. The camera unit 220 may further include an analog-to-digital converter, and may convert the analog signal output from the image sensor into a digital signal and output the digital signal to the controller 270. In particular, the camera unit 220 includes a distance sensor 221. The distance sensor 221 is a reference point, for example, the focus of the camera unit 200, that is, the three-dimensional coordinates (x, y, z of each pixel of the image photographed using the focal point of the image sensor as the origin of the three-dimensional coordinates). ) Is output to the control unit 270. Accordingly, the controller 270 may acquire the shortest distance from the focus to the photographed object. The distance sensor 221 may use various types of sensors using a laser, infrared light, visible light, and the like. The distance sensor 221 is a laser type 3D scanner using any one of TOP (Phase of Time), Phase Shift and Online Waveform Analysis, a laser type 3D scanner using optical triangulation, white light or modulated light. Optical three-dimensional scanner, Photoheld Real Time photo, Optical three-dimensional scanner, Optical method using pattern projection or line scanning, Laser full body scanner, Photo scanner using photogrammetry, Kinect Real time scanner using Fusion).

The sensor unit 230 includes a plurality of inertial sensors. Here, the inertial sensor may exemplify an acceleration sensor, a gyro sensor, and the like. Accordingly, the sensor unit 230 may measure the Euler angle of the user device 200 through the plurality of inertial sensors. Such Euler angles include yaw, roll and pitch. The sensor unit 230 continuously measures the Euler angle of the user device 200 and transmits the measured Euler angle to the controller 270.

The input unit 240 receives a key operation of a user for controlling the user device 200, generates an input signal, and transmits the generated input signal to the controller 270. The input unit 220 may include any one of a power key, a numeric key, and a direction key for power on / off, and may be formed as a predetermined function key on one surface of the user device 200. When the display unit 250 is a touch screen, the functions of the various keys of the input unit 220 may be performed on the display unit 250. When the display unit 250 may perform all functions using only the touch screen, the input unit 220 may be omitted. It may be.

The display unit 250 visually provides a menu, input data, function setting information, and various other information of the user device 200 to the user. In particular, the display unit 250 may continuously display an image captured by the controller 270 through the camera unit 220. The display unit 250 outputs various screens such as a boot screen, a standby screen, a menu screen, and the like of the user device 200. The display unit 250 may be formed of a liquid crystal display (LCD), organic light emitting diodes (OLEDs), active matrix organic light emitting diodes (AMOLEDs), or the like. Meanwhile, the display unit 250 may be implemented as a touch screen. In this case, the display unit 250 may include a touch sensor, and the controller 270 may detect a user's touch input through the touch sensor. The touch sensor may be configured as a touch sensing sensor such as capacitive overlay, pressure type, resistive overlay, infrared beam, or pressure sensor. . In addition to the above sensors, all kinds of sensor devices capable of detecting contact or pressure of an object may be used as the touch sensor of the present invention. The touch sensor detects a user's touch input, generates a detection signal, and transmits the detected signal to the controller 270. The sensing signal may include coordinate data input by the user. When the user inputs the touch position movement operation, the touch sensor may generate a detection signal including coordinate data of the touch position movement path and transmit the detected signal to the controller 270.

The storage unit 260 stores a program and data necessary for the operation of the user device 200, and may be divided into a program area and a data area. The program area may store a program for controlling the overall operation of the user device 200, an operating system (OS) for booting the user device 200, an application program, and the like. The data area is an area in which data generated according to the use of the user device 200 is stored. In addition, the storage unit 260 may store various types of data generated according to the operation of the user device 200.

The controller 270 may control an overall operation of the user device 200 and a signal flow between internal blocks of the user device 200, and may perform a data processing function for processing data. The controller 270 may be a central processing unit (CPU), an application processor, a graphic processor (GPU), or the like. The controller 270 includes a photographing unit 271 and a searching unit 273.

The photographing unit 271 requests the search server 100 to obtain photographing composition information for guiding a photographing composition of a product to be searched by a user. Then, the photographing unit 271 displays the guideline through the display unit 250 according to the photographing composition information, and then photographs the product according to the guideline through the camera unit 220. The search unit 273 generates an image search word of an image of the product photographed by the photographing unit 271. Then, the search unit 270 requests an image search by transmitting an image search request including the image search word to the search server 100 through the communication unit 210. The operation of the controller 270 including the photographing unit 271 and the searching unit 273 will be described in more detail below.

Next, a method for image retrieval using guideline-based photographed images according to an embodiment of the present invention will be described. 4 is a flowchart illustrating a method for image retrieval using guideline-based captured images according to an exemplary embodiment of the present invention. 5 is a screen example for explaining a method for image search using a guideline-based photographed image according to an exemplary embodiment of the present invention.

4, the control unit 270 of the user device 200 transmits a composition information request message to the search server 100 through the communication unit 210 in step S110. The composition information request message is a request for photographing composition information about a product to be searched by a user. The composition information request message includes a dictionary image of a product, which the user wants to search through the camera unit 220, regardless of the photographing composition information, or a kind of the product that the user wants to search.

When the information providing module 131 of the search server 100 receives the composition information request message through the communication module 110, it identifies a product type according to the composition information request message received in step S120, and stores the storage module 120. Extracting photographing composition information corresponding to the product type identified in FIG. For example, if the preliminary image is included in the composition information request message, the information providing module 131 identifies a kind of product from the preliminary image and extracts photographing composition information corresponding to the identified kind of product. Also, if the type of product is included in the composition information request message, the information providing module 131 extracts photographing composition information corresponding to the type of the product. Then, the information providing module 131 transmits the photographing composition information to the user device 200 through the communication module 110 in step S130.

As shown in FIG. 5, the photographing composition information includes a reference point S of the camera photographing the reference image according to the type of product, that is, the reference distance Ds and the reference point (the shortest distance from the focus to the product C). Reference Euler angles Ys, Rs, Ps including yaw, roll and pitch measured in S) and a guideline L indicating the outline of the reference image. Euler angle means yaw means the angle rotated about the Z axis, roll means the angle rotated about the X axis, pitch is the angle rotated about the Y axis Means.

When the photographing unit 271 of the user device 200 receives the photographing composition information through the communication unit 110, the photographing unit 271 displays the guideline L on the screen through the display unit 250 based on the photographing composition information in step S140. . These guidelines are indicated according to the reference distance Ds and Euler angles Ys, Rs, Ps. This step S140 will be described in more detail below.

Next, the photographing unit 271 photographs the product by adjusting the photographing composition according to the guideline L displayed on the display unit 250 according to the user's input in step S150. This step S150 will also be described in more detail below.

Subsequently, the searcher 273 generates an image searched image according to the guideline L in step S160, and requests an image search by transmitting the image search word to the search server 100 in step S170.

The image search module 133 of the search server 100 receives the image search word through the communication module 110, and the image of the image search word among the plurality of product images stored in the storage module 120 based on the image search word in step S170. Search for images that are the same or similar to. Then, in operation S180, the image search result is transmitted to the user device 200 through the communication module 110. Then, the search unit 273 receives the search result through the communication unit 210 and displays the search result through the display unit 250.

As described above, according to an embodiment of the present invention, by photographing an image with the same composition as the type of goods stored in the storage module 120 and performing an image search using the image as an image search term, the accuracy of the image search may be improved. Can be.

Next, it will be described in more detail in the above-described step S140. 6 is a flowchart illustrating a method of displaying a guideline according to an embodiment of the present invention. 7 to 10 are views for explaining a method for displaying a guideline according to an embodiment of the present invention.

It is assumed that the photographing unit 271 of the user device 200 receives the photographing composition information from the search server 100 through the communication unit 210 in step S210. As described above, the photographing composition information includes a reference distance Ds, reference Euler angles Ys, Rs, and Ps, and a guideline L. It is assumed that the user focuses the product through the lens of the camera unit 220 to photograph the product.

In this case, as shown in FIG. 7, the photographing unit 271 may move from the focus S of the camera unit 220 to the target product G through the distance sensor 221 of the camera unit 220 in step S220. Measure the distance Dr. In operation S230, the photographing unit 271 calculates a distance difference Dd between the reference distance Ds and the measured distance Dr.

Subsequently, as shown in FIG. 8, the photographing unit 271 adjusts the size of the guideline L according to the distance difference Dd calculated together with the product captured by the camera unit 250. The display unit 250 displays a screen.

In addition, the photographing unit 271 guides to adjust the distance between the user device 200 and the product G according to the distance difference Dd previously calculated in step S250 (10a, 10b, 10c). Outputs The distance adjustment guide message may be at least one of the icon image 10a, the text 10b, and the voice 10c.

Meanwhile, as illustrated in FIG. 9, the photographing unit 271 measures the Euler angles Yr, Rr, and Pr at the focus of the camera unit 220 through the sensor unit 230 in step S260. Then, the photographing unit 271 calculates the angular difference (Yd, Rd, Pd) between the reference Euler angles (Ys, Rs, Ps) and the measured Euler angles (Yr, Rr, Pr) in step S270.

Then, the photographing unit 271 is the x-axis, the y-axis according to the angle difference (Yd, Rd, Pd) calculated with the goods captured by the camera unit 250, as shown in Figure 10 in step S280 And rotate the guide line L based on at least one axis of the z-axis to display the screen through the display unit 250. For example, assume that the reference Euler angles (Ys, Rs, Ps) are (-0.05, 0.05, -0.25) rad and the measured Euler angles (Yr, Rr, Pr) are (-0.05, -0.10, -0.25). . Accordingly, the calculated angle differences Yd, Rd, and Pd are (0, -0.15, 0) rad. That is, it is assumed that there is no difference in the case of yaw and pitch, and only 0.15rad difference occurs in the roll. Then, the photographing unit 271 displays the guide line L by rotating about 0.15 rad about the X axis.

In addition, the photographing unit 271 rotates based on at least one axis among the x-axis, the y-axis, and the z-axis of the user device 200 according to the distance difference Yd, Rd, and Pd previously calculated in operation S250. Outputs guide angle adjustment guide messages 20a, 20b, and 20c. The distance adjustment guide message may be at least one of the icon image 20a, the text 20b, and the voice 20c.

Next, the above-described step S150 will be described in more detail. 11 is a flowchart illustrating a method of capturing an image based on a guideline according to an embodiment of the present invention. 12 is a diagram for describing a method of photographing an image based on a guideline according to an embodiment of the present invention.

Referring to FIG. 11, the photographing unit 271 continuously measures the reference distance Ds and the reference Euler angles Ys, Rs, and Ps as described with reference to FIGS. 6 to 10 described above in operation S310. The distance Dr and the measured Euler angles Yr, Rr and Pr are compared and the guideline L is displayed according to the distance difference and the angle difference. The user will move the user device 200 to align the goods G displayed on the display unit 250 with the guidelines L. FIG.

In addition, the user may input to request photographing of the product G through the input unit 240 or the display unit 250.

Accordingly, when the photographing unit 271 detects a photographing request through the input unit 240 or the display unit 250 in step S320, it is determined whether the photographing condition is satisfied in step S330. Here, the shooting conditions are the distance difference (Dd) between the measured distance (Dr) and the reference distance (Ds) from the focus of the camera unit to the target product is less than a predetermined distance and at the same time the Euler angles (Yr, Rr, Pr) measured at the focus and This means that the angle difference Yd, Rd, Pd of the reference Euler angles Ys, Rs, and Ps is less than a predetermined angle. In other words, when the product G captured by the camera unit 220 is displayed on the display unit 250, the guide line L overlaps with a predetermined area or more.

As a result of the determination in step S330, if the shooting condition is not satisfied as shown in FIG. 12A, the photographing unit 271 indicates guide messages 30a, 30b, 30c, and 30d indicating that the shooting condition is not satisfied in step S340. ) For example, the guidance message 30a, 30b, 30c, 30d may be an icon image 30a, a background color 30b, or a text that is induced to be displayed according to the guideline L as shown in FIG. It may be at least one of 30c and voice 30d.

In addition, as a result of the determination in step S330, as shown in FIG. Accordingly, according to an exemplary embodiment of the present invention, the image of the product G may be photographed in a composition corresponding to the image of the type of the corresponding product stored in the storage module 120 of the search server 100.

As described above, the specification includes the details of a number of specific implementations, but these should not be understood as being limited to the scope of any invention or claimable, but rather may be specific to a particular embodiment of a particular invention. It should be understood as a description of the features. Certain features that are described in this specification in the context of separate embodiments may be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable subcombination. Furthermore, while the features may operate in a particular combination and may be initially depicted as so claimed, one or more features from the claimed combination may in some cases be excluded from the combination, the claimed combination being a subcombination Or a combination of subcombinations.

Likewise, although the operations are depicted in the drawings in a specific order, it should not be understood that such operations must be performed in the specific order or sequential order shown in order to obtain desirable results or that all illustrated operations must be performed. In certain cases, multitasking and parallel processing may be advantageous. Moreover, the separation of the various system components of the above-described embodiments should not be understood as requiring such separation in all embodiments, and the described program components and systems will generally be integrated together into a single software product or packaged into multiple software products. It should be understood that it can.

Specific embodiments of the subject matter described in this specification have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order but still achieve desirable results. As an example, the process depicted in the accompanying drawings does not necessarily require that particular illustrated or sequential order to obtain desirable results. In certain implementations, multitasking and parallel processing may be advantageous.

The foregoing description presents the best mode of the invention, and provides examples to illustrate the invention and to enable those skilled in the art to make and use the invention. The specification thus produced is not intended to limit the invention to the specific terms presented. Thus, while the present invention has been described in detail with reference to the above examples, those skilled in the art can make modifications, changes, and variations to the examples without departing from the scope of the invention.

Therefore, the scope of the present invention should be determined by the claims rather than by the described embodiments.

The present invention relates to a method for an image retrieval using a guideline-based photographed image and an apparatus therefor, and according to the present invention, induces photographing with the same composition as a product image stored in a search server, and the image search word photographed according to the derivation. By performing an image search using, the accuracy and reliability of the image search may be improved. Therefore, the present invention has industrial applicability because the present invention is not only sufficiently commercially available or commercially viable, but also practically clearly implemented.

100: search server 110: communication module
120: storage module 130: control module
131: information providing module 133: image search module
200: user device 210: communication unit
220: camera unit 221: distance sensor
230: sensor unit 240: input unit
250: display unit 260: storage unit
270: control unit 271: the recording unit
273: search unit

Claims (12)

Acquiring photographing composition information for guiding a photographing composition of a product to be searched by a user;
Displaying, by the photographing unit, a guideline on a screen according to the photographing composition information;
Photographing a product by the photographing unit in accordance with the guidelines;
A search unit generating an image of the photographed product as an image search word; And
And transmitting, by the search unit, an image search request including the image search word to a search server. The method of claim 1,
The photographing composition information
And a guideline for indicating a reference distance, which is the shortest distance from the reference point of the camera photographing the reference image, and a reference Euler angle including the yaw, roll, and pitch measured at the reference point, and an outline of the reference image. Image retrieval method of the user device. The method of claim 2,
Displaying the guidelines on the screen
Measuring a distance from a focal point of the camera unit of the user device to a product;
Calculating a distance difference between the reference distance and the measured distance; And
And adjusting and displaying the size of the guide line according to the calculated distance difference. The method of claim 3,
After adjusting and displaying the size,
And outputting a distance adjustment guide message for guiding adjustment of a distance between the user device and the product according to the calculated distance difference. The method of claim 2,
Displaying the guidelines on the screen
Measuring an Euler angle at the focus;
Calculating an angle difference between the reference Euler angle and the measured Euler angle; And
And rotating and displaying the guide line based on at least one of x, y, and z axes according to the calculated angle difference. The method of claim 5,
After rotating and displaying the axis,
And outputting an angle adjustment guide message for guiding the user device to rotate about the at least one axis among x, y and z axes according to the calculated angle difference. How to retrieve images from your device. The method of claim 2,
The photographing unit photographing the product in accordance with the guidelines
Detecting a photographing input of a user;
The difference between the measured distance from the focus of the camera unit of the user device to the product and the reference distance is less than a predetermined distance, and at the same time the difference between the angle of the Euler angle and the reference Euler angle measured at the focus satisfies the shooting condition. Determining whether or not;
And if the photographing condition is satisfied, photographing the photographing condition and if the photographing condition is not satisfied, outputting a guide message indicating that the photographing condition is not satisfied. How to search for images on your device. A communication unit for communicating with a search server;
A camera unit which photographs an image and includes a distance sensor measuring a distance from the image;
A display unit for displaying a screen;
When the user acquires photographing composition information for guiding a photographing composition of a product to be searched for, the guideline is displayed on the screen through the display unit according to the photographing composition information, and the product is matched to the guideline through the camera unit. Shooting unit for photographing; And
And a search unit which generates the image of the photographed product as an image search word and transmits an image search request including the image search word to the search server through the communication unit. The method of claim 8,
The photographing composition information
And a guideline for indicating a reference distance, which is the shortest distance from the reference point of the camera photographing the reference image, and a reference Euler angle including the yaw, roll, and pitch measured at the reference point, and an outline of the reference image. User device for image search. The method of claim 9,
The camera unit
It includes a distance sensor for measuring the distance from the focus of the camera unit to the product,
The photographing unit
Calculate a distance difference between the distance measured by the distance sensor and the reference distance,
And adjust the size of the guideline according to the calculated distance to display the guideline through the display unit. The method of claim 9,
The user device is
Further comprising a sensor unit for measuring the Euler angle at the focus,
The photographing unit calculates an angle difference between the reference Euler angle and the measured Euler angle, and rotates the guide line based on at least one axis among x, y, and z axes according to the calculated angle difference. User device for image search, characterized in that displayed through. The method of claim 9,
The photographing unit
When we detect an input requesting a shot,
It is determined whether the distance difference between the measured distance from the focus point of the camera unit to the product and the reference distance is less than a predetermined distance, and at the same time the angle difference between the Euler angle and the reference Euler angle measured at the focus is less than a predetermined angle. and,
If the photographing condition is satisfied, the product is photographed.
And if the photographing condition is not satisfied, outputting a guide message indicating that the photographing condition is not satisfied.

Images