KR101701952B1 - Method, apparatus and computer program for displaying serch information - Google Patents

Abstract

The present invention relates to a method for enabling an electronic device to display shopping information, which includes the steps of: displaying a shopping information search menu by fixing the shopping information search menu in a notification list; executing a camera function and displaying a camera preview image if a user request for a camera search in the shopping information search menu is received; transmitting query by extracting an image which can be searched in the preview image; and displaying the shopping information by changing a screen or overlaying a link of the shopping information on the preview image if the shopping information matching the image is received. Accordingly, the present invention can improve the shopping convenience of a buyer and an advertisement effect for a seller.

Description

[0001] METHOD, APPARATUS AND COMPUTER PROGRAM FOR DISPLAYING SERCH INFORMATION [0002]

The present invention is directed to a method and apparatus for providing search information in an electronic device.

More specifically, the present invention relates to a method and an apparatus for acquiring a query for shopping information search by a user's intuitive operation in a portable terminal, and displaying shopping information such as related goods, services, advertisements, and the like.

In recent years, commerce such as promotion and marketing using online has been activated in the background of the increased wired and wireless Internet environment. In this regard, buyers searching for magazines, blogs, videos on YouTube, etc. from a desktop or mobile terminal connected to the Internet finds a product they like and searches for a product name and the like to purchase. Famous actresses list the name of the bags they listened to at the airport and the names of the childcare items in the entertainment program that rises above the ranking of real-time search terms on the portal site. However, at this time, the user has to separately open a web page for searching and search for a product name and a selling place, and if the name is not known, the user can not easily search.

Meanwhile, sellers spend a lot of money to promote their products in addition to commercial advertisements, media sponsorship, and recruitment of online applications. In recent years, online word of mouth is an important variable for product sales. However, in spite of the expenditure of publicity, it is frequent that the shopping information such as the name of the product and the selling place can not be disclosed. Indirect advertising issues can arise because individual media viewers' prior approval for product name exposure can not be received individually.

As such, there is a need for both users and sellers to provide shopping information in a more intuitive user interface (UI) environment for product images on-line.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems. In particular, the present invention provides an intuitive user interface for searching shopping information in an electronic device and provides a method and apparatus for efficiently providing shopping information such as related price information, online sales site, advertisement, coupon, and promotion information .

A method of displaying shopping information by an electronic device according to an embodiment of the present invention includes: displaying a shopping information search menu on a notification list by fixing the shopping information; Performing a camera function and displaying a camera preview image upon receiving a user request for camera search among the shopping information search menu; Extracting a searchable image from the preview image and transmitting a query; And displaying the link of the shopping information on the preview image when the shopping information matching with the image is received.

Further, a method of displaying shopping information by an electronic device according to another embodiment of the present invention includes: displaying and fixing a shopping information search menu including at least one search option in a notification list; Receiving a user request for text search among the shopping information search menu, executing a text input function and transmitting a shopping information search request query for the received text; Receiving a user request for voice search in the shopping information search menu, executing a voice input function, converting the received voice into text, and transmitting a shopping information search request query for the converted text; And displaying the matching shopping information.

According to another aspect of the present invention, there is provided a method of displaying shopping information in an electronic device, comprising: transmitting a screen shot to a shopping information search request query when a screen shot is generated; And displaying the link of the shopping information overlaid on the screen shot or switching the screen shot to the matching shopping information display screen upon receiving the shopping information on the screen shot.

Further, the application stored in the medium for performing the process of displaying the shopping information in the electronic device according to the embodiment of the present invention may include a function of displaying the shopping information search menu fixed on the notification list when the application is executed; A function of executing a camera function and displaying a camera preview image upon receiving a user request for camera search among the shopping information search menu; Extracting a searchable image from the preview image and transmitting a query; And a function of overlaying a link of the shopping information on the preview image when the shopping information matching the image is received.

Meanwhile, an electronic device for displaying shopping information according to an embodiment of the present invention includes: a display unit for displaying a user interface; And a shopping information search menu is fixedly displayed on the notification list, and upon receipt of a user request for camera search among the shopping information search menu, a camera function is executed, a camera preview image is displayed, And transmitting the query, and upon receiving the shopping information matching the image, controlling the display unit to display a link of the shopping information overlaid on the preview image.

Further, a method for providing shopping information to a client according to an embodiment of the present invention includes: generating a database for a plurality of products and extracting feature information of an image included in the database; Receiving a query from the client; Separating a character or a numeric object in the query and converting the object into text; Determining a product or service matching the query using the text; And providing search information for the matching goods or services to the client.

According to the present invention, the shopping information search menu can be fixedly displayed on the notification list of the electronic device, and even when any function of the electronic device is used, the user can receive related shopping information through the search menu . Therefore, the purchaser increases the convenience of shopping, and the seller has an effect of increasing the advertisement effect.

1 is a flowchart for explaining an example of displaying search information in an electronic device according to an embodiment of the present invention;
2 is a diagram for explaining a search method according to an embodiment of the present invention;
3 is a view for explaining an example of a user interface for displaying a search icon according to an embodiment of the present invention;
4 is a view for explaining a search method according to another embodiment of the present invention;
5 is a view for explaining an example of a user interface displaying a search icon according to another embodiment of the present invention;
6 is a diagram for explaining a search method according to another embodiment of the present invention;
7 is a view for explaining an example of providing search information in a service server according to an embodiment of the present invention;
8 is a diagram for explaining a search method according to another embodiment of the present invention;
9 is a view for explaining an example of a user interface for displaying a search menu according to another embodiment of the present invention;

It is to be understood that the present invention is not limited to the description of the embodiments described below, and that various modifications may be made without departing from the technical scope of the present invention. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail.

In the drawings, the same components are denoted by the same reference numerals. And in the accompanying drawings, some of the elements may be exaggerated, omitted or schematically illustrated. It is intended to clearly illustrate the gist of the present invention by omitting unnecessary explanations not related to the gist of the present invention. Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a flowchart for illustrating an example of displaying search information in an electronic device according to an embodiment of the present invention.

Hereinafter, it is assumed that the electronic device is a mobile device, but the present invention is not limited thereto. That is, the electronic device in the present invention should be understood as a concept including all types of electronic devices capable of requesting a product search, such as a desktop, a smart phone, a tablet PC, and displaying search information.

Meanwhile, in the IT industry, the term background can be used in various meanings, but in the present specification, the meaning of background can be interpreted as including a state in which an application is driven without displaying an execution screen. Further, the term foreground can be understood as a concept including a state in which an application is driven while an execution screen is displayed.

Further, the term " displayed page in an electronic device " in this specification includes a screen loaded on an electronic device and / or contents inside the screen so that the page can be immediately displayed on the screen according to a scroll of a user As shown in Fig.

For example, in the display of the mobile device, the entire execution screen of the application extended in the horizontal or vertical direction and displayed according to the scrolling of the user can be included in the concept of the page, and the screen during the camera roll is also included in the concept of the page . Furthermore, a screen or an application screen displayed on a web browser can be displayed on the page.

On the other hand, a list view that sequentially displays items related to a corresponding operation such as web search, e-book, text chatting service, and mail list search on a mobile device through a plurality of lines of text, bar, May be included in the concept of the page.

In step 110 of Figure 1, the electronic device may execute an information retrieval program according to an embodiment of the present invention. The search program may be in the form of an application or plug-in installed in a mobile device, an ActiveX, an advertising SDK (module, API), or the like, installed in a desktop or an extension program installed in a web browsing tool.

The program may be downloaded and installed via a service server and / or application open marketplace according to an embodiment of the present invention. Further, the program may be executed in the form of an advertisement SDK (API, module) loaded in another application.

In step 110, the electronic device can perform processing for user authentication and security together with the installation of the application. The electronic device can also proceed to provide a user agreement on providing information to a service server or a separate web site server.

In step 120, the electronic device may provide at least one search option through the settings menu of the application execution screen. The search option is for setting the search method.

According to the embodiment of the present invention, when arbitrary contents are displayed on the electronic device as a search option, a method of automatically searching for related goods and / or services even when there is no separate user search request, A method of receiving a user search request through a search icon only when possible, and / or a method of searching for a product only when there is a search request of an explicit user.

In particular, the information retrieval according to the embodiment of the present invention can be performed without separately executing an application for retrieving, separately opening a page for retrieval, or receiving no additional query input. That is, according to the present invention, it is possible to perform the search while the specific contents are displayed without any separate action for searching. Various retrieval schemes according to embodiments of the present invention will be described below with reference to the accompanying drawings.

If an optional search option is selected at step 130, the information retrieval application according to an embodiment of the present invention is driven in the background to perform the search. For example, even if a user's application termination command is received (step 135), the application according to an embodiment of the present invention can be executed in the background without being completely terminated. (Step 140) This is for proceeding the search without a separate action for search application execution, search page display, or the like.

If, on the other hand, no search option is selected in step 130, it is interpreted that the user does not want to retrieve information using the application (step 132) and execution of the application is terminated according to the termination instruction. (Step 133)

In step 150, the electronic device may execute a search application in the background while simultaneously executing various other applications in the foreground. In this case, the execution screen of the foreground application can be displayed as single or multi-layer.

In step 155, the electronic device may proceed to retrieve information about the associated goods and / or services while displaying any content. If the resources of the electronic device are sufficient, the directly matching goods and / or services can be determined, but if not, the electronic device can perform the search by transmitting the query to the service server performing the information search.

For example, when there is a search request for a web page (that is, when a search request is received while the web browser is running), the electronic device transmits the URL information of the displayed web page to the search request query, Image, and moving image data, and transmits the extracted data to the query. At this time, if the URL information can not be obtained, or if text, image, or moving image data can not be extracted from the URL, a screen shot of the web page can be acquired and transmitted as a query.

According to another embodiment of the present invention, when a search request is received while an application other than a web browser is being executed, the electronic device acquires a screen shot and / or data that can be uploaded from the page, Lt; / RTI > For example, if a photo viewer application is running, the electronic device can send the photo file as a query. As another example, if a video playback application is running, the electronic device may send a screen shot at a particular point in time with the query.

Thereafter, in step 160, the electronic device receives the matching search information from the service server. For example, the service server extracts an image, a moving image, or a text included in a query to search for matching items in a database, and can provide related shopping information. Specific examples of the service server determining whether or not the goods are matched will be described later with reference to the accompanying drawings.

On the other hand, the electronic device can switch to the foreground and display it when receiving the matching shopping information. (Step 165) The shopping information may include relevant advertisement information, price information of matching goods / services, lowest price information, lowest price online merchant, similar merchandise, coupon, and / or promotion information. Further, information on other goods / services that can be recommended using the search history log of the user may also be included.

For example, if the user is browsing a web page containing a flower image, a flower delivery advertisement list, flower flower lesson promotion information, and / or rose festival discount coupon information may be displayed as shopping information according to the embodiment of the present invention .

As another example, if the user is browsing the SNS including the shoe image, according to the embodiment of the present invention, the shopping information may be written in different shoes of the same brand, same or similar color, shape and / Suggested shopping information for shoes can be displayed.

As another example, according to the embodiment of the present invention, when a user browses a picture including a toy image, the shopping information includes a toy name, a manufacturer, a manufacturing year, a lowest price, price fluctuation information, And / or how to use it.

Furthermore, the information may be displayed in text, image and / or action script form, each of which may be associated with an online merchant. Further, the information may be displayed as a separate page, or may be displayed as an icon including a shopping information link near the item on the page that the user is viewing. Furthermore, the information may be displayed overlaid on the page being viewed.

2 is a diagram for explaining a search method according to an embodiment of the present invention. In the description of FIG. 2, the parts overlapping with FIG. 1 described above will be omitted or briefly described.

According to the embodiment of FIG. 2, when there is a search request of a user, a query requesting a search is transmitted. When the search application is executed in the background, the search icon is displayed on the notification list and / or the shared menu list. The user browses the specific page, selects the search icon of the notification list and / or the shared menu list, The shopping information of the goods included in the shopping list can be searched.

More specifically, in step 210, when the above-described method is set as the search option, the application may be switched to the background (step 220) and a shopping information search menu may be created on the notification list and / or the shared menu list.

In particular, when included in the notification list, the shopping information search menu is preferably fixed to the notification list for the purpose of the present technology. That is, the shopping information search menu may not be deleted while the application is running in the background despite the user's request to cancel the advertisement. (Step 230)

The user may view a plurality of pages, content, and / or applications while executing the application of the electronic device at step 240.

If the user wants to search related shopping information while browsing a specific page, the user can select a search icon included in the shared list by calling a notification window or selecting a search icon included in the notification list or calling a sharing menu have. According to an embodiment of the present invention, a notification window or a sharing menu containing a search icon may be called according to the prior art. This is to increase the intuitiveness of the product search by using a user-friendly UI. A detailed description of the user interface in which the search icon is displayed will be described later with reference to the accompanying drawings.

If the application running in foreground is a web browser (step 260), then the electronic device may send a query to the service server, including a history log of the web browser. (Step 270) The history log may include a browsing execution history of the web browser. In particular, the URL information of the web page that has been most recently executed, that is, the URL information of the web page viewed immediately before the user selects the search icon, .

On the other hand, in step 270, the electronic device extracts the image, moving image, and text data included in the URL of the web page, and transmits the extracted data to the query. Furthermore, if the URL, text, image, or video data can not be extracted, the screen shot may be extracted and transmitted to the query.

According to another embodiment of the present invention, the electronic device may transmit a query if a screen shot is created in the foreground (step 261) or if the application running in foreground is a photo viewer (step 262). In particular, in the case of a screen shot, according to an embodiment of the present invention, a screen shot itself may be transmitted as a query. However, if the web browser is running in the foreground, a record log may be extracted instead of a screen shot.

For example, when a user viewing a moving image frequently wonders about a car model name appearing on the screen, the user can generate a screen shot at the time the car is displayed and call a notification window to create a product search icon. The screen shot may be generated as a user gesture such as a hardware key input, an icon touch, or the like matching the function, or a user action that rubs the screen or shakes the electronic device.

As another example, when the user is inquiring about the lowest price of a telephone on display on which a user is shopping on a large-sized shopping mart, the user takes a photograph of the telephone using the camera application of the electronic device, calls a notification window The user can request the product search by selecting the icon. Alternatively, the user may request a product search by inputting a sharing button, a photographing button, and a screen shot generating gesture in a state in which a camera preview screen is displayed without photographing the phone.

(Step 275). According to another embodiment of the present invention, the electronic device may send an embedded character and / or numerical object of the image (OCR) (step 272) and convert the text into text to transmit the text query. (Step 273)

Meanwhile, according to another embodiment of the present invention, the electronic device may first determine whether the target screen shot or photo is image searchable (step 274), and transmit the image query only when searchable. (Step 275)

For example, the electronic device can determine that image retrieval is possible when the level at which the feature information can be extracted from the image is greater than a predetermined range. More specifically, the electronic device extracts a feature descriptor that expresses features of the image and / or features of the image, which can be used to determine whether there is the same similarity between images in the image, as a vector value, It can be determined that the image search is possible.

In Figure 2, steps 272 through 275 are intended to reduce the load the server takes to retrieve queries, and may be omitted as optional steps. In this case, the electronic device may extract a screenshot of the foreground application and transmit it to the query regardless of whether the user separately generates the screenshot. (Step 268)

If shopping information on the matching object is received from the service server in response to the query, the electronic device switches the shopping information searching application to foreground in step 280, and displays the shopping information in step 290.

3 is a diagram for explaining an example of a user interface displaying a search icon according to an embodiment of the present invention.

3A shows an example in which a search icon 310 according to an embodiment of the present invention is displayed in a notification list of a notification window.

For example, in a state where a user is viewing an arbitrary page, a drag starting from the top edge of the display may be input to call an alert window as shown in FIG. 3A, and a search icon 310 displayed in the alert list may be selected The user can request a search for the page being viewed.

In particular, when the search icon is included in the notification list, it is preferable that the shopping information search menu is fixed to the notification list. That is, the shopping information search menu may not be deleted while the application is running in the background despite the user's request to cancel the advertisement.

FIG. 3B is an example in which a search icon 320 is displayed in the shared list according to an embodiment of the present invention.

For example, the user can select a shared menu in a state of browsing an arbitrary page to invoke a shared list as shown in FIG. 3B. In this way, a search icon displayed on the shared list is selected, You can request a search.

4 is a diagram for explaining a search method according to another embodiment of the present invention. In the description of FIG. 4, the parts that are the same as those of FIG. 2 described above will be omitted or briefly explained.

In the example of FIG. 4, the electronic device first determines whether a search for a specific page is possible, and displays the search icon only when search is possible. According to the above embodiment, when a product can be searched, the user is notified of the fact by using a search icon or the like, and the query is transmitted only when the user selects an icon. Therefore, resources required for transmitting and processing a query in the server are alleviated .

If the method is set as a search option (step 405), the search application can be switched to the background (step 410) and the user can browse several pages while executing another application at step 420.

According to an embodiment of the present invention, a search icon may be displayed only when a search for a page is possible. (Steps 440 and 450)

For example, the electronic device displays the number of images included in the displayed page by using the number of images included in the displayed page, the size of each of the images, the shape of each of the images, the text of the page, It is possible to determine whether or not the content included in the page can be searched and display the search icon on the page only when it is possible.

For example, if the number of pieces of feature information that can be extracted from the image is within a predetermined range, the electronic device can determine that the image search is possible and display the search icon. More specifically, the electronic device extracts a feature descriptor that expresses features of the image and / or features of the image, which can be used to determine whether there is the same similarity between images in the image, as a vector value, It is judged that image search is possible.

As another example, when the application running on the foreground is a web browser, the electronic device checks the history log and if one or more of the parameters of the main URL, the sub-URL, and the sub-URL of the web page being browsed is included in the preset target It can be judged as searchable. For example, when the web browser accesses a predetermined target page, the electronic device can determine that the search is possible and display the search icon. The target page may be a product information detailed page of an online shopping site.

A description of a specific user interface for displaying the search icon will be described later with reference to the accompanying drawings.

When the user selects the displayed search icon, a query for that page may be sent. (Steps 460 and 470). Further, when the matching shopping information is received in response to the query, the goods searching application can be switched to the foreground and display the received matching shopping information. (Steps 480 and 490) Steps 470 to 490 are briefly described because they are substantially the same as Steps 260 to 290 of FIG. 2 described above.

On the other hand, in the above-described embodiment, it is necessary to check whether or not a product included in a page can be searched. In order to reduce the resources required for the above procedure, the electronic device periodically checks whether or not the web browser is executed. When the web browser is being executed, the electronic device periodically checks the log of the web browser and only when the web browser is updated You can check whether you can search. (Step 425)

More specifically, a search application according to an embodiment of the present invention periodically checks whether a web browser is executed in the background, and periodically checks a log record of the web browser when the web browser is running. In this way, it is possible to check whether or not the search can be performed only when the web browser is updated without confirming the searchability of all the web pages.

FIG. 5 is a diagram for explaining an example of a user interface displaying a search icon according to the search method described with reference to FIG.

As shown in FIG. 5A, according to the embodiment of the present invention, even if a large number of products are displayed on a certain page, the retrieval icon is not displayed unless the format of the page is suitable for product retrieval.

However, if the user selects a specific object in FIG. 5A and the relevant item detail description page as shown in FIG. 5B is displayed, the search icon 510 may be displayed when the format of the page is suitable for the search.

According to this method, when a search icon is displayed on a page that is being viewed in a user's view, it can be recognized that a search for the corresponding article is possible, and when shopping information is displayed only by selecting a search icon, It may be recognized as a link for a call.

6 is a diagram for explaining a search method according to another embodiment of the present invention. In the description of FIG. 6, the parts overlapping with those of FIGS. 2 to 4 described above will be omitted or briefly explained.

According to the search method of FIG. 6, a query for shopping information search for the displayed page can be automatically transmitted without any type of user input requesting a search.

If the method is set as a search option (step 610), the search application can be switched to the background (step 620) and the user can browse several pages while executing another application at step 630.

For example, if the application running in the foreground is a web browser, the URL to the web page is sent to the query (step 650) and the service server determines the matching product and / or service using the data provided in the URL can do.

At this time, if the service server provides shopping information, the search application is switched to foreground in step 660 and shopping information can be displayed in step 670. Further, according to the embodiment of the present invention, the shopping information may be displayed overlaid on a page to which the query is transmitted, without switching the screen.

At this time, even if the shopping information is received from the service server, the user can set to display the shopping information only when he or she wants to display the shopping information. Steps 650 to 670 are briefly described because they are substantially the same as steps 260 to 290 of FIG. 2 described above.

On the other hand, in the above embodiment, since the query is automatically transmitted, the query can be set to be transmitted only when the web browser is updated.

More specifically, a search application according to an embodiment of the present invention periodically checks whether a web browser is executed in the background, and periodically checks a log record of the web browser when the web browser is running. (Step 640)

In this manner, the query can be transmitted only when the web browser is updated without confirming the searchability of all the web pages. (Step 650), and if the matching shopping information is received and displayed in response to the query (step 670), then the browsing history information is changed or the web page from which the query is sent is moved to the previous page, .

7 is a diagram for explaining an example of providing search information in a service server according to an embodiment of the present invention.

In step 710, the shopping information providing server can establish a shopping information database. The shopping information may include a product name, a product image of various angles and sizes, a usage method, a review of a purchase, a manufacturer, a vendor, price information, advertisement information, coupon information and / or promotion information, May also be included in the connection with the recommended product using. The method of collecting such shopping information in various paths and designing and storing the database may follow the conventional technology.

Furthermore, the service server can generate characteristic information of product images stored in the database and construct the database as a database. For example, the service server may extract the feature region of the product image, calculate the feature descriptor, and index the feature information of the images for at least one category for efficiency of search.

In step 720, the service server receives the query from the electronic device and, in step 730, can convert the data contained in the query.

For example, when the URL information is included in the query, the service server can extract image, moving image, and / or text data provided by the URL and select only the goods.

Or if the query includes image data, the service server extracts the eigenvalues (hash information, header information, etc.) of the image from the query image and / or extracts feature information of the image and processes it into a form suitable for image retrieval . For example, the service server can transform the feature descriptor of the query image on the same basis as indexing the feature descriptor of the database image. This is to perform the subsequent search processing quickly and efficiently.

Thereafter, in step 740, the service server may determine whether the query data matches the goods and / or services included in the database.

For example, the service server may search the set of feature descriptors of the database image to determine whether the distance between the feature descriptor vector value of the database image and the feature descriptor vector value of the query image is within a preset threshold value. Furthermore, the degree of coincidence between the query image and the database image is judged (Geometric Consistency) by using the characteristic information of the image or the attribute of the feature information such as the position information, scale, rotation degree and distribution information of the feature descriptor, Value, and search for matching goods and / or services.

Although not separately shown in FIG. 7, the service server may apply deep running or machine learning to the series of processes of FIG.

For example, the service server learns feature information extracted from arbitrary images to generate the category, and when it is determined that the feature of the query image corresponds to the generated specific category, it is determined that the information matches the information belonging to the category You may.

Thereafter, in step 750, the service server may provide shopping information for the matching goods and / or services.

8 is a diagram for explaining a search method according to another embodiment of the present invention. According to the embodiment of FIG. 8, the electronic device can provide a variety of options such as text search, voice search, camera search, etc. in the user interface for shopping information search.

If the embodiment of FIGS. 1 to 7 described above is implemented in such a manner that the electronic device requests the search for the displayed content while displaying the content, FIG. 8 shows a method of retrieving the search target content after the user requests the search It can be said that it is about the example to be implemented.

In step 810, the electronic device may execute a shopping information retrieval application according to an embodiment of the present invention, and may be executed in the background despite the user's termination command.

In step 820, the electronic device may generate a shopping information search menu on the notification list and / or the shared menu list. In particular, when the information is included in the notification list, the shopping information search menu is preferably fixed to the notification list for the purpose of the present technology. That is, the shopping information search menu may not be deleted while the application is running in the background despite the user's request to cancel the advertisement.

The shopping information search menu may include a text search, a voice search, and / or a camera search option. A specific example of a user interface for a search menu according to an embodiment of the present invention will be described below with reference to the accompanying drawings.

If the user selects text search in step 830, the electronic device may execute a text input function to receive the text and transmit it to the query. (Step 847)

For example, if the user directly inputs text into the text search window among the shopping information search menus fixed in the notification list, the shopping information search application can transmit the text input in the background to the query.

As another example, if the user selects a text search window from the shopping information search menu, the shopping information application according to the embodiment of the present invention may be switched from background to foreground execution, and a text input window of the application may be displayed. Then, when the user inputs text into the text input window of the application, the application can transmit the inputted text to the query.

If, on the other hand, the user selects voice search at step 840, the electronic device may execute a voice input function to receive the voice and convert it to text (step 845) to transmit the text query. (Step 847)

For example, when the user selects the voice search window from the shopping information search menu fixed in the notification list, the microphone function is activated, and the user can input voice to the microphone. The shopping information search application running in the background can convert the input voice to text and transmit the converted text to the query.

As another example, if the user selects a voice search window from the shopping information search menu, the shopping information application according to the embodiment of the present invention may be switched to foreground execution in the background and the microphone function may be activated. The shopping information search application running in the foreground can then convert the input voice to text and transmit the converted text to the query.

On the other hand, when the user selects the camera search in step 850, the electronic device can execute the camera function to display the preview. (Step 853)

At this time, the electronic device can determine whether an image can be searched from the preview image (step 856) and specify an object to be searched. (Step 858) In particular, according to the embodiment of the present invention, the electronic device can specify the object to be searched regardless of whether or not the captured image is generated for the preview screen.

FIG. 8 illustrates that step 858 is performed after step 856, but the present invention is not limited thereto. That is, the steps 856 and 588 may proceed each time or proceed in reverse order.

For example, the electronic device refers to a sensor value such as an acceleration sensor, an illuminance sensor, and the like to determine that a stable shooting environment can be predicted, so that an image search is possible. Instead of generating a preview image separately, You can also submit queries.

As another example, the electronic device may transmit an image query only when a shooting button is input or when a preview screen at a predetermined time is captured at a predetermined cycle, and the image search possibility of the captured screen is determined.

For example, the electronic device can determine that image retrieval is possible when the number of feature information extractable from the preview screen is within a preset range. More specifically, the electronic device extracts a feature descriptor representing a characteristic region and / or an image feature, which can be used to determine whether there is a similarity between images on a preview screen at a specific point in time or a captured preview screen, If the extracted feature information is within the preset range, it can be determined that the image search is possible.

On the other hand, the electronic device may convert a text and / or numerical object included in the captured image into a text and transmit the text query. (Steps 860, 866, 868), as well as an image query with the text query.

Steps 856 through 866 in Figure 8 are intended to reduce the load the server takes to retrieve queries, and may be omitted as optional steps.

If shopping information on the matching object is received from the service server in response to the query, the electronic device can overlay the icon linked with the shopping information on the preview screen in step 870. [ The link icon is a concept that includes an image and / or a keyword form for a matching product, service or advertisement.

At this time, when the user selects the link icon, the application can switch to the foreground and display the matching shopping information. (Steps 880 and 890) The shopping information may include relevant advertisement information, price information of matching goods / services, lowest price information, lowest price online retailer, similar merchandise, coupon, and / or promotion information. Further, information on other goods / services that can be recommended using the search history log of the user may also be included.

FIG. 9 is a diagram for explaining an example of a user interface displaying a search menu on a notification list and overlaying a link icon on a preview screen according to the embodiment of FIG. 8 described above.

For example, the user can input a drag starting from the top edge of the display to call the notification window as shown in FIG. 9A, and select a search method by selecting a search menu 905 displayed in the notification list have. The shopping information search menu may include a text search 910, a voice search 920, and / or a camera search 930 option.

For example, when the user selects the camera option, the electronic device can invoke the camera function and display a preview image as shown in FIG. 9B. 9B is an example of a camera photographing a specific page of a fashion magazine book. At this time, when a stable shooting environment can be predicted by referring to sensor values such as an acceleration sensor and an illuminance sensor, the electronic device can transmit a query for the preview screen.

In the example of FIG. 9B, the preview image can be identified by 936 and 932 portions. In particular, if the portion 932 is an object of a character and / or numeric shape, the electronic device may extract the text by OCR processing the portion 932 according to an embodiment of the present invention. In this case, the electronic device may transmit an image query for the 936 portion and a text query for the 932 portion, respectively, or each.

When shopping information on the matching object is received from the service server in response to the query, the icon linked with the matching shopping information may be overlaid on the preview screen as shown in FIG. 9C. In the example of FIG. 9C, the link icon is represented by an alias of a matching product such as 940, fashion information such as 960, discount information such as 970, similar product information such as 980, and related advertising information such as 950.

If the user subsequently selects the link icon, the application will switch to foreground to display more specific matching shopping information.

.

The embodiments of the present invention disclosed in the present specification and drawings are merely illustrative of specific embodiments of the present invention and are not intended to limit the scope of the present invention in order to facilitate understanding of the present invention. It is to be understood by those skilled in the art that other modifications based on the technical idea of the present invention are possible in addition to the embodiments disclosed herein.

Claims (9)

A method of displaying shopping information in an electronic device,
The electronic device fixing the shopping information search menu on the notification list and displaying the shopping information search menu;
Performing a camera function and displaying a camera preview image when the electronic device receives a user request for a camera search among the shopping information search menu;
Wherein when the sensor value of at least one of the acceleration sensor and the illumination sensor of the electronic device is included in a preset range for extracting a searchable image from the preview image, Extracting an image;
Transmitting, by the electronic device, the captured image to a query when the number of pieces of feature information that can be used to judge whether the images are similar to each other in the captured image is included in a predetermined range; And
When the electronic device receives the shopping information for the query, switching the screen or overlaying the preview information on the preview image to display the shopping information. delete 2. The method of claim 1, wherein after extracting the capture image
The electronic device distinguishing characters or numerical objects included in the extracted captured image and converting the objects into text; And
Wherein the electronic device further comprises transmitting at least one of the converted text and the extracted captured image to the query. delete The method according to claim 1,
Wherein the step of displaying the shopping information search menu comprises:
Wherein the electronic device fixes and displays a shopping information search menu including at least one search option in the notification list,
Executing a text input function when the electronic device receives a user request for text search in the shopping information search menu and transmitting a shopping information search request query for the received text; And
When the electronic device receives a user request for voice search in the shopping information search menu, the electronic device executes a voice input function, converts the received voice into text, and transmits a shopping information search request query for the converted text Further comprising the steps of: The method according to claim 1,
The electronic device transmitting the screen shot to a shopping information search request query when a screen shot is generated; And
Further comprising the step of, when the electronic device receives the shopping information on the screen shot, displaying the link of the shopping information overlaid on the screen shot, or switching the screen shot to the matching shopping information display screen And displaying the shopping information. In the electronic device, a shopping information search menu is fixedly displayed on a notification list;
Performing a camera function and displaying a camera preview image upon receiving a user request for a camera search among the shopping information search menu in the electronic device;
In the electronic device, if the sensor value of at least one of the acceleration sensor and the illuminance sensor is included in a predetermined range for extracting a searchable image from the preview image, a capture image for the preview image at that point of time is extracted step;
Transmitting, in the electronic device, the captured image to a query if the number of pieces of feature information that can be used to determine whether or not the images are the same is included in the captured image is included in a predetermined range; And
And upon receiving the shopping information for the query from the electronic device, overlaying and displaying the link of the shopping information on the preview image. An electronic device for displaying shopping information,
A display unit for displaying a user interface; And
The shopping information search menu is fixedly displayed on the notification list, and when the user request for the camera search is received from the shopping information search menu, the camera function is executed, the camera preview image is displayed, and at least one of the acceleration sensor and the illuminance sensor The sensor value extracting unit extracts a captured image of the preview image at the time when the sensor value is included in a preset range so as to extract a searchable image from the preview image, When the number of pieces of feature information that can be used for the query is included in a preset range, the control unit transmits the captured image to the query, and when receiving the shopping information for the query, To be displayed on the preview image And an electronic component. In a method for a server to provide shopping information to a client,
The server creating a database for a plurality of goods;
The server receiving a query image from the client;
The server distinguishing characters or numeric objects from the query image and converting the objects into text;
Determining, by the server, a goods or service matching the query using the text; And
And the server providing search information for the matching goods or services to the client.

Images