WO2017143575A1

WO2017143575A1 - Method for retrieving content of image, portable electronic device, and graphical user interface

Info

Publication number: WO2017143575A1
Application number: PCT/CN2016/074597
Authority: WO
Inventors: 郜文美
Original assignee: 华为技术有限公司
Priority date: 2016-02-25
Filing date: 2016-02-25
Publication date: 2017-08-31
Also published as: CN107533566A

Abstract

A method for retrieving content of an image, a portable electronic device, and a graphical user interface. The method comprises: detecting a first operation applied to an image, the image being displayed on a screen of a portable electronic device (203); in response to the first operation, obtaining whole content of the image as a first retrieval result of a retrieval key element, and displaying the first retrieval result (204); detecting a second operation applied to the image (205); in response to the second operation, determining a recognition region in which recognition is performed on the image (206); obtaining content of the image coinciding with the recognition region, as a second retrieval result of the retrieval key element (207); and displaying the second retrieval result (208). The method can simplify operation steps when a user retrieves content of an image.

Description

Method for retrieving the content of a picture, portable electronic device and graphical user interface

Technical field

Embodiments of the present invention relate to mobile communication technologies, and in particular, to a method for retrieving content of a picture and a portable electronic device.

Background technique

At present, many portable electronic devices (for example, mobile phones, tablets, etc.) have the function of recognizing specific content (such as characters, products, plants, barcodes, two-dimensional codes, characters, etc.) in a picture, and can also recognize The content is retrieved and the search results are displayed. Specifically, in the prior art, the specific process of recognizing an item in a photo by using a portable electronic device is: opening a camera -> entering an option -> clicking an intelligent object -> taking a photo -> finger drawing a target object - >Click the search button -> to get the search results. The operation steps of the prior art are relatively cumbersome, and it takes a long time from obtaining the picture to obtaining the corresponding search result, the real-time performance is poor, and the user experience is not good.

Summary of the invention

This document provides a method for retrieving the content of a picture, a portable electronic device, and a graphical user interface to simplify the steps and improve the user experience.

In a first aspect, a method of retrieving content of a picture, the method being applied to a portable electronic device, comprising: detecting a first operation acting on a picture, the picture being displayed on a screen of the portable electronic device Retrieving, in response to the first operation, acquiring a first search result that uses the entire content of the picture as a search element, displaying the first search result; detecting a second operation acting on the picture; a second operation, determining an identification area for identifying the picture, acquiring a second search result that uses the content of the picture and the recognition area as a search element, and displaying the second search result, wherein the The content of the picture that coincides with the identification area is part of the overall content of the picture, the first operation being different from the second operation.

After the portable electronic device detects the first operation of the user on the screen displayed on the screen, the first retrieval result of the overall content of the image as the retrieval element may be acquired and the first retrieval result is displayed, and the user is detected to the image. After the second operation, the second search result with the partial content of the picture as the search element may be acquired and the second search result may be displayed, thereby, the portable electronic device may realize the opposite picture when detecting different operations. Overall content or Local content automatically performs retrieval, and in the prior art, it is necessary to first open the intelligent object recognition option, obtain a photo containing the target object, and then select an area containing the target object in the photo, and finally click on the search to obtain the search result, which is simplified. The user steps to retrieve the content of the image enhances the user experience.

With reference to the first aspect, in a first implementation manner of the first aspect, after the displaying the second search result, the method further includes: detecting a third operation acting on the picture, the third operation Including dragging, enlarging or reducing, the third operation is different from the first operation; in response to the third operation, acquiring a third search result that the content of the picture coincides with the identification area as a retrieval element And displaying a third retrieval result, wherein the identification area is a preset area on the screen.

With reference to the first aspect, in a second implementation manner of the first aspect, after the displaying the second retrieval result, the method further includes: detecting a fourth operation acting on the identification frame displayed on the screen, The fourth operation includes dragging or moving, the identification frame coincides with a boundary line of the identification area, the fourth operation is different from the first operation; in response to the fourth operation, acquiring the picture The content that coincides with the recognition area is used as a fourth search result of the search element, and the fourth search result is displayed.

With reference to the first aspect, in a third implementation manner of the first aspect, determining the identification area that identifies the picture includes: using the content that overlaps the preset area on the screen with the picture as the picture The identified recognition area is displayed, or the content circled on the picture by the recognition frame displayed on the screen is used as an identification area for identifying the picture.

With reference to the first aspect, in a fourth implementation manner of the first aspect, determining, by the second operation, the identification area that identifies the picture, the second operation includes a re-press operation, determining The area enclosed by the closed curve centered on the action point on the screen is the identification area, and the size of the closed curve is preset.

With reference to the first aspect, in a fifth implementation manner of the first aspect, determining, in response to the second operation, an identification area that identifies the picture, the second operation comprises: double-clicking or knuckle sniping And determining, by the double-click or the closed curve centered on the touch point of the knuckle, the area enclosed by the closed curve is the identification area, wherein the size of the closed curve is preset.

It should be understood that the above-mentioned knuckle sniper may be that the user's knuckles tap the screen of the portable electronic device, and the number of slamming may be one time or multiple times, and may be set as needed.

In conjunction with the first aspect, in a sixth implementation of the first aspect, in response to the second operation, Determining an identification area for identifying the picture, comprising: the second operation comprises two-finger expansion, and determining to enclose the closed curve centered on a midpoint of the line segment formed by the initial touch point connection of the two fingers on the screen The area is the identification area, and the size of the closed curve is preset.

With reference to the first aspect, in a seventh implementation manner of the first aspect, determining, in response to the second operation, the identification area that identifies the picture, the second operation includes delimiting or sliding, determining The area enclosed by the circled track or the sliding track is the identification area.

When the second operation is a circled or sliding operation, the recognition area may be determined according to a curve trajectory generated on the screen by the circle or slide operation, specifically, the user may slide on the screen through the finger or knuckle of one finger or The generated curve trajectory is determined to determine the recognition area, and the user may determine the recognition area by sliding or enclosing the generated curve trajectory on the screen by the finger or knuckles of the plurality of fingers. In addition, the above-mentioned circled trajectory and the trajectory of the sliding may be a closed curve or a non-closed curve, and the curve may be a solid line or a broken line. When the circled or slid track is a non-closed curve, the non-closed curve may be used. The area enclosed by the closed curve formed by the two end points of the closed curve connected by line segments or curves is the above-mentioned identification area. Among them, the finger's finger or knuckle sliding on the screen can be understood as the finger's finger or knuckle continuously sliding on the screen. The fingertip or knuckle of the finger is defined on the screen as a fingertip or knuckle of the finger to draw a circle on the screen or to draw other closed curves (also can be a non-closed curve).

With reference to the first aspect, and any one of the first to seventh implementation manners of the first aspect, in an eighth implementation manner of the first aspect, determining, in response to the second operation, Before the image identifies the identified area, the method further includes determining that the picture is in an enlarged state and the duration of the picture is in an enlarged state exceeds a first threshold.

With reference to the eighth implementation manner of the first aspect, in the ninth implementation manner of the first aspect, the determining that the picture is in an enlarged state comprises: determining an enlarged size of the picture and an initial display of the picture The ratio of the size exceeds the second threshold.

In a second aspect, a portable electronic device is provided, the portable electronic device comprising means for performing the method provided by the first aspect or any of the possible implementations of the first aspect.

In a third aspect, a portable electronic device is provided, the portable electronic device comprising: a processor, a detector, a memory, a bus system; the processor, the detector, and the memory are connected by the system bus; The memory is for one or more programs, the one or more programs comprising instructions that, when executed by the portable electronic device, cause the portable electronic device to perform any of the first aspect or the first aspect Possible implementations provide the methods.

In a fourth aspect, a computer readable storage medium storing one or more programs, the one or more programs being executed by the portable electronic device, the portable electronic device performing the first aspect or the first aspect The method provided by any of the possible implementations.

In a fifth aspect, a graphical user interface is provided on a portable electronic device, the portable electronic device comprising a display, a memory, a plurality of applications, and one or one of executing one or more programs stored in the memory A plurality of processors, the graphical user interface comprising a user interface displayed in accordance with the method provided by the first aspect or any one of the possible implementations of the first aspect, wherein the display comprises a touch-sensitive surface and a display screen.

In some implementations, the overall content refers to all of the content contained in the picture or the entire content of the picture displayed on the screen.

In some implementations, the closed curve includes a rectangular curve, a circular curve, and an elliptic curve or other closed curve when the identification region is determined according to the second operation.

In some implementations, the identification area is an area enclosed by a preset size closed curve preset at the preset position of the screen, and the closed curve includes a circular curve, a rectangular curve, an elliptic curve, and other closures. curve.

In some implementations, the closed curve shows a curve on the screen.

In some implementations, the closed curve is a transparent curve that is not displayed on the screen.

In some implementations, the identification area is a longitudinal middle strip area or a horizontal middle strip area of the screen or a central block area of the entire screen, wherein the longitudinal middle strip area or the horizontal middle strip shape The size of the area and the central block area is preset.

In some implementations, after detecting the second operation, and before acquiring the second search result that the content of the picture and the recognition area coincides as the search element according to the second operation, the method further includes: Detecting whether there are other operations acting on the picture, if the other operations are not detected within the preset time, then the content that coincides with the identification area is used as the second retrieval result of the retrieval element; if it is preset If the other operations are detected in the time, then, in response to the other operations, and correspondingly processing the image according to the other operations, detecting whether the device still exists in the preset time after the corresponding processing of the image Another operation of the picture, if the other operation is not detected within a preset time, the content that coincides with the identification area is used as a second retrieval result of the retrieval element, wherein the other operations include amplification, At least one of zooming in, dragging, or moving.

In some implementations, the first operation includes clicking, long pressing, double clicking, double pressing, double clicking, Knuckle slamming, two-finger extension, two-finger contraction, delineation, and sliding.

In some implementations, the second operation includes a click, a long press, a double tap, a re-press, a double tap, an knuckle tap, a two-finger extension, a two-finger contraction, a circle, and a slide. It should be understood that the first operation is different from the second operation in a particular implementation. For example, the first operation is a re-press and the second operation is a deduction.

In some implementations, the third operation is an operation subsequent to the second operation, the third operation being the same as or different from the second operation.

In some implementations, the fourth operation is different from the second operation, the second operation is for a picture, and the fourth operation is for an identification frame on the screen.

In some implementations, when the operation is a two-finger contraction, a circular or rectangular area centered on a line segment formed by the initial finger-point connection of the two fingers on the screen is determined as the identification area.

In some implementations, an initial display size of the picture is a size that the picture is displayed on the screen after the portable electronic device opens the picture.

In some implementations, before the detecting the operation of the picture, the method further includes: the portable electronic device automatically identifying and retrieving all valid objects or objects in the picture, obtaining a retrieval result, and Store the search results.

In some implementations, before the operation of detecting the picture, the method further includes: the portable electronic device automatically identifying and retrieving all valid objects or objects in the picture, and acquiring the retrieval result specifically includes The portable electronic device searches for the overall content of the picture to obtain an overall retrieval result, and the portable electronic device searches for each object in the picture to obtain a retrieval result of each object, and the overall retrieval result and each object The search results are stored.

In some implementations, the image in the identification area is identified by using at least one of the following technologies: optical character recognition (OCR) based text recognition technology, barcode/two-dimensional code recognition technology , face detection technology, article detection technology.

In some implementations, performing the search according to the search element to obtain the search result includes: searching for a search result according to the search element in the portable electronic device, or sending the search element to a server on the network side, by the server The search is performed and the search result is obtained, and the server transmits the search result to the portable electronic device, and the search result is displayed by the portable electronic device.

In some implementations, the picture displayed on the screen of the portable electronic device is a picture obtained from an album, a webpage, or an application.

In some implementations, the portable electronic device includes a mobile phone, a mobile computer, and a flat Board computers, personal digital assistants, media players, smart TVs.

After the operation is detected, the recognition area can be determined and the image in the identification area is recognized, thereby obtaining the retrieval result, thereby simplifying the operation steps of retrieving the content of the picture, and further, implementing the image in the picture according to different operations Images from different regions are retrieved to improve the user experience.

DRAWINGS

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings to be used in the embodiments of the present invention will be briefly described below. It is obvious that the drawings described below are only some embodiments of the present invention, Those skilled in the art can also obtain other drawings based on these drawings without paying any creative work.

1 is a flow chart of a method for retrieving content of a picture according to an embodiment of the present invention.

2 is a schematic diagram of a central area in a method of retrieving content of a picture according to an embodiment of the present invention.

3 is a flow chart of a method for retrieving content of a picture according to an embodiment of the present invention.

4 is a flow chart of a method for retrieving content of a picture according to an embodiment of the present invention.

FIG. 5 is a schematic diagram of a method for searching content of a picture to retrieve content of a picture according to an embodiment of the present invention.

FIG. 6 is a schematic diagram of searching the content of a picture according to an embodiment of the present invention.

FIG. 7 is a schematic diagram of searching a content of a picture according to an embodiment of the present invention.

FIG. 8 is a schematic diagram of searching the content of a picture according to an embodiment of the present invention.

FIG. 9 is a schematic diagram of searching a content of a picture according to an embodiment of the present invention.

FIG. 10 is a schematic diagram of searching a content of a picture according to an embodiment of the present invention.

FIG. 11 is a schematic diagram of searching the content of a picture according to an embodiment of the present invention.

FIG. 12 is a schematic diagram of searching a content of a picture according to an embodiment of the present invention.

FIG. 13 is a schematic diagram of searching the content of a picture according to an embodiment of the present invention.

FIG. 14 is a schematic diagram of searching a content of a picture according to an embodiment of the present invention.

Figure 15 is a diagram showing the retrieval of the content of a picture in accordance with an embodiment of the present invention.

16 is a schematic diagram of searching a content of a picture according to an embodiment of the present invention.

FIG. 17 is a schematic diagram of searching the content of a picture according to an embodiment of the present invention.

FIG. 18 is a schematic diagram of searching the content of a picture according to an embodiment of the present invention.

19 is a schematic block diagram of a portable electronic device in accordance with an embodiment of the present invention.

20 is a schematic block diagram of a portable electronic device in accordance with an embodiment of the present invention.

21 is a schematic block diagram of a portable electronic device in accordance with an embodiment of the present invention.

detailed description

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are a part of the embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

Embodiments of the present invention provide a method for retrieving content of a picture, which can automatically implement retrieval of content, when a user browses a picture through a portable electronic device (a picture in an album or a picture in a web page), when the portable electronic After the device detects the first operation of the image displayed on the screen by the user, the first retrieval result that takes the overall content of the image as the retrieval element is automatically acquired, and the first retrieval result is displayed, and the second detection of the user by the user is detected. After the operation, the second retrieval result with the partial content of the picture as the retrieval element may be automatically acquired, and the second retrieval result is displayed, so that the user can retrieve the content in the image through a simpler operation, and can also Different gestures retrieve different areas of the picture. The method for searching the content of the picture in the embodiment of the present invention is described in detail below with reference to the specific embodiments.

FIG. 1 is a flow chart showing a method of retrieving the content of a picture according to an embodiment of the present invention. The method can be performed by a portable electronic device (eg, a cell phone, a tablet, etc.). The specific process of searching the content of the picture is as follows:

101. A portable electronic device displays a picture.

102. The portable electronic device captures and judges the user's operation in real time.

It should be understood that the portable electronic device herein may capture the user's operation by displaying a screen of the picture, and some operations of the user may be set to trigger the recognition of the picture, and other gestures of the user may not trigger the recognition of the picture. The above operations may include, but are not limited to, double tapping, long press, zoom in, re-press, zoom out, move, hover, circle, slide, knuckle tap and knuckle slip. It should be understood that a long press refers to a user's finger pressing time on the screen for more than a certain time threshold.

103. The portable electronic device detects the first operation.

The first operation includes a click, a long press, a double tap, a double press, a double tap, an knuckle tap, a two-finger extension, a two-finger contraction, a circle, and a slide.

104. The portable electronic device displays the first search result.

The first search result is a search result obtained as a search element for the entire content of the picture.

105. The portable electronic device detects that the user's operation is the second operation.

The second operations include click, long press, double click, double press, double click, knuckle tap, two finger extension, two finger contraction, circle, and slide. It should be understood that the first operation is different from the second operation in a particular implementation. For example, the first operation is a re-press and the second operation is a deduction. It should be understood that after the user performs the second operation on the picture, there may be operations such as moving or dragging the picture.

106. The portable electronic device determines an identification area that identifies the picture.

The covered picture content of the identification area is a part of the overall content of the above picture, and the identification area may be an area enclosed by a closed curve of a preset size preset at a preset position of the screen. Specifically, the identification area may be preset in a central area of the screen, and the size and specific shape of the central area may be preset, as shown in FIG. 2, the central area may be a longitudinal strip middle section of the screen, and a horizontal direction. The center strip area or the central block area of the screen as a whole.

107: Acquire a second search result that the content that overlaps with the identification area is the search element.

108. Display a second search result. After the second search result is displayed, the process returns to step 102 to continue the loop process.

Optionally, after the picture is displayed, the recognition process for the picture can be started. Then, after detecting the first operation of the picture by the user, the search result can be displayed, and the step of searching the content of the picture is simplified, and the detection is performed. When the second operation of the user is performed, part of the picture content covered by the recognition area is retrieved, and the retrieval result is displayed.

Optionally, in order to enable the existing operation to trigger the retrieval in combination with a specific state of the picture (for example, an enlarged state or a reduced state), the method for identifying the content shown in FIG. 1 may also be between step 103 and step 104. Or, between steps 105 and 106, whether the picture is in an enlarged or reduced state is determined, and whether the content of the picture is retrieved is determined according to the result of the determination.

Specifically, after detecting the first operation of the user, determining whether the picture is in an enlarged or reduced state, and if the picture displayed on the screen is in an enlarged or reduced state, performing the entire content of the picture as a retrieval element, obtaining the first A search result, if the picture is in a non-zoomed or non-reduced state (it can be considered that the picture opened in step 101 is in the initial display state, if the currently displayed picture is not scaled relative to the initial display state of the picture, then the current picture is considered to be not The zoom state), then the retrieval of the overall content of the picture is not performed to obtain the first search result.

In addition, after detecting the second operation of the user, it may also be judged whether the picture is in an enlarged state, if the picture is in an enlarged state, and the picture state stabilization time exceeds the time t ₀ (the size of the time t ₀ may be preset, for example , you can set t ₀ to 0.5 seconds) and then execute 106 to 108. If the picture is not in the enlarged state, then 106 to 108 are not executed.

It should be understood that the zoom-in state herein may mean that the display ratio of the picture relative to the initial state is greater than 100%, or may be that the picture is enlarged and enlarged to a certain multiple with respect to the original picture to be in an enlarged state, for example, when the picture is enlarged to the original picture. The picture is considered to be magnified when it is twice as large as the original picture.

It can be seen that the method for retrieving the content of the picture in the embodiment of the present invention can determine whether the picture is in an enlarged or reduced state after detecting the first operation or the second operation of the user.

FIG. 3 is a flow chart showing a method of retrieving the content of a picture according to an embodiment of the present invention. The method for searching the content of the picture in the embodiment of the present invention will be described in detail below with reference to FIG. As shown in FIG. 3, steps 201-205 are the same as steps 101-105 in the method shown in FIG. 1 (for details of steps 201-205, refer to steps 101-105 in FIG. 1), to avoid repetition, here No longer described in detail. Step 206 is highlighted below.

206. Determine, according to an action area of the second operation, an identification area that identifies the picture;

The specific implementation manner of step 206 includes the following:

When the user's second operation is to double-click the picture, then the circular or rectangular area circled on the original picture centered on the double-click touch point is determined as the identification area, wherein the size of the circular area or the rectangular area is It is pre-set.

When the second operation of the user is two-finger expansion, it is determined that the area surrounded by the closed curve centered on the midpoint of the line segment formed by the initial touch point connection of the two fingers on the screen is the identification area. Specifically, a circular or rectangular area centered at a midpoint of the line segment may be determined as an identification area. Specifically, a center line of the rectangular area may coincide with the line segment, and a diameter of the circular area may coincide with the line segment. Or the length of the center line of the rectangular area may be greater than the line segment, and the diameter of the circular area may be greater than the length of the line segment, wherein the size of the circular area or the rectangular area is preset.

When the second operation gesture of the user is to press (press hard), the area enclosed by the closed curve centered on the action point of the re-press operation is determined as an identification area, wherein the size of the closed curve is preset. . Specifically, a circular or rectangular area centered on the touch point of the operation gesture may be determined as the identification area, wherein the size of the circular area or the rectangular area is Set.

After the identification area for identifying the picture is determined via step 206, steps 207 and 208 are continued to obtain the search result. Since steps 207 to 208 are the same as steps 107 to 108 in the method shown in FIG. 1, both of the second retrieval results are obtained and the second retrieval result is displayed. Therefore, in order to avoid repetition, steps 207 and 208 are not described in detail herein. .

Optionally, in order to enable the existing operation to trigger the retrieval in conjunction with the specific state of the picture, after the first operation of the user is detected in step 203, and before the first retrieval result is displayed in step 204, the method may further include: The electronic device determines whether the picture is in an enlarged or reduced state. If the picture displayed on the screen is in an enlarged or reduced state, then the first operation is performed in response to the first operation, and the entire content of the image is used as a retrieval element to obtain the first retrieval result, if the image is in a non- The enlarged or non-reduced state can then not respond to the first operation, nor retrieve the entire content of the picture to obtain the first search result.

In addition, after the second operation of the user is detected in step 205, and before the identification area of the picture is determined in step 206, the method may further include: determining whether the picture is in an enlarged state, if the picture is in an enlarged state, and The picture state stabilization time exceeds the time t ₀ (the size of the time t ₀ can be set in advance, for example, t ₀ can be set to 0.5 seconds), and then steps 206 to 208 are performed. It should be understood that the zoom-in state herein may mean that the display ratio of the picture relative to the initial state is greater than 100%, or may be that the picture is enlarged and enlarged to a certain multiple with respect to the original picture to be in an enlarged state, for example, when the picture is enlarged to the original picture. The picture is considered to be magnified when it is twice as large as the original picture.

That is to say, the method for retrieving the content of the picture in the embodiment of the present invention may determine whether the picture is in a zoom state after detecting the first operation or the second operation of the user.

In the embodiment of the present invention, after the picture is displayed, the retrieval of the content of the picture can be initiated. Then, after detecting the operation gesture of the user on the picture, the corresponding retrieval result can be displayed, which simplifies the retrieval of the content of the picture. a step, and enabling the user to retrieve content of different areas in the picture through different gestures (eg, touching the user when detecting that the user's operation gesture is "double click", "two finger expansion" or "force pressing" The content in the area is retrieved), which improves the user experience.

Searching the Content of the Picture As can be seen from Figures 1 and 3, the identification boxes of the methods for retrieving the content of the pictures in both embodiments are fixed. In fact, when the content of the picture is recognized, the recognition frame can also be moved, and the user can control the movement of the recognition frame through a series of gestures. The content in the area covered by the identification frame can be identified when the recognition frame is moved to the screen. The case where the identification frame can be moved will be described in detail below with reference to FIG.

4 is a flow chart showing a method of retrieving the content of a picture in accordance with an embodiment of the present invention. The method can be performed by the portable electronic device described above. As shown in FIG. 4, steps 301-302 are the same as steps 101-102 in the method shown in FIG. 1. To avoid repetition, details are not described herein again. The following focuses on the steps that are different from the method shown in FIG. 1.

303. The portable electronic device detects that the user's operation is the second operation. The second operation includes at least one of double tapping, double pressing, double tapping, knuckle tapping, two-finger expansion, two-finger contraction, enclosing, and sliding.

304. Generate an identification frame in the action area of the second operation on the screen, and use the content circled on the picture as the recognition area for identifying the picture. The identification frame is movable, and its specific shape may be a rectangle, a circle, an ellipse or other irregular figure. When the user clicks on a screen area other than the identification box, the recognition frame can automatically disappear.

305. Acquire a second search result that is a search element by using the content that overlaps the identification area with the image.

306. Display a second search result.

307. Detecting the movement of the identification frame, and continuing to identify the content circled on the picture by the identification frame. Specifically, if the second operation of the user includes re-pressing, the identification frame may be dragged, and the dragging at this time may be, after lifting the finger, pressing the identification frame again to drag, or not lifting the finger. Continue to hold down the identification box to drag. If the user's second operation is floating, the dragging at this time may be to continue to maintain the hovering gesture and perform the floating mode to achieve the dragging of the recognition frame; or the finger may be away from the screen, and then click the screen button. Live the identification box and drag it.

It should be understood that, in the embodiment shown in FIG. 1 to FIG. 4, the picture displayed by the portable electronic device may be a picture opened by the user from an album in the portable electronic device, or may be obtained by a camera that the user activates the portable electronic device. A picture, or a picture obtained when a user browses a web page with a portable electronic device. In addition, before detecting the operation on the picture, the portable electronic device can automatically identify and retrieve all valid objects or objects in the picture, obtain the retrieval result, and store the retrieval result. More specifically, the portable electronic device retrieves the overall content of the picture to obtain an overall retrieval result, and retrieves each object in the picture to obtain a retrieval result of each object, and then stores the overall retrieval result and the retrieval result of each object. In this way, when the operation on the picture is detected, the search result that has been stored can be quickly called and the search result is displayed. In addition, portable electronic devices can be retrieved from within the portable electronic device based on search elements The search result is obtained, or the portable electronic device sends the search element to the server on the network side, and the server searches for the search result, and the server transmits the search result to the portable electronic device, and the search result is displayed by the portable electronic device.

In addition, in the embodiment of FIG. 1 to FIG. 4, the barcode/two-dimensional code recognition technology, the face detection technology, the item detection technology, and the like may be used to acquire the first search result or the second search result. Specifically, the retrieval result may be a technique such as face retrieval and recognition, item retrieval and recognition, single number search, and item number search. After the first search result or the second search result is obtained, the first search result or the second search result may be displayed on a picture or a blank area or a screen around the picture, and the search result may be specifically a link or a link icon. Users can get more detailed information by clicking on the link or the link icon.

The entire flow of the method for retrieving the content of the picture according to the embodiment of the present invention is described in detail with reference to FIG. 1 to FIG. 4 . The following will be combined with FIG. 5 to FIG. A detailed description of the method of retrieving the content of the picture.

The initial display interface after the portable electronic device opens the picture is shown in FIG. 5. In FIG. 5, the picture is located in the middle of the screen of the portable electronic device, and there are three objects in the picture (here, the picture is represented by a circle, a pentagon and a triangle). The three objects), if you want to implement the retrieval of the content in the picture, you can operate the picture on the displayed interface to achieve the retrieval of the content in the picture.

As shown in FIG. 6, when the portable electronic device detects the first operation on the picture, for example, the first operation is a click, the portable electronic device acquires the first search with the overall content of the picture as a search element. As a result, that is, the retrieval elements are the three objects in FIG. 6, and the first retrieval result is displayed on the screen or on the picture or around the picture.

As shown in FIG. 7, when the portable electronic device detects the second operation on the picture, for example, the second operation is a double click, the portable electronic device will acquire the content in which the picture coincides with the preset area in the center of the screen as Retrieve the second search result of the feature, that is, the search element is a pentagon object, and the second search result is displayed on the screen or on the picture or around the pentagon object. In addition, as shown in FIG. 8, the portable electronic device detects a third operation acting on the picture. For example, the third operation is to move the picture such that different objects are in a preset area in the center of the screen, so that the portable electronic device is The retrieval result of the new object moved to the preset area is automatically acquired, and the obtained retrieval result is displayed on the screen or on the picture or near the new object (in FIG. 8 is a circle moving to the pre-area) Objects are retrieved).

As shown in FIG. 9, when the portable electronic device detects a second operation acting on the picture, the second operation is two-finger expansion or two-finger contraction, etc., the portable electronic device automatically forms a diameter of two contacts. A circular recognition area, and the content in the identification area is retrieved, and the obtained retrieval result is displayed on the screen or on the picture or near the target object.

As shown in FIG. 10, the portable electronic device can also generate a movable identification frame, and the user can retrieve different objects in the picture by moving the identification frame, as shown in FIG. 10, when the portable electronic device detects After the second operation of the user, an identification frame may be generated in the active area on the screen, and the content bounded by the identification frame on the picture is used as an identification area for identifying the picture, and the identification frame in FIG. 10 is a A rectangular frame that the user can move down or to the right to move the recognition area to retrieve different objects in the picture.

When the second operation of the user is circled or slid, the recognition area can be determined according to the curve trajectory generated on the screen by the circle or slide operation. Specifically, the user can determine the recognition by sliding or delineating the generated curve trajectory by one finger. In the area, the user can also determine the recognition area by sliding or enlarging the generated curve trajectory on the screen by a plurality of fingers. The above-mentioned circled or slid track is an identification area. Specifically, as shown in FIG. 11, the user draws a closed curve on the screen by sliding or ringing operation, and the closed curve encloses a five in the middle of the picture. The area of the edge object is the recognition area. It should be understood that, as shown in FIG. 12, the user can also determine a recognition area by drawing a non-closed curve on the screen by sliding or delineating operation. When the user draws a non-closed curve, the two non-closed curves can be The end points are connected together to form a closed curve, and the area enclosed by the closed curve is the above identified area. It should also be understood that the trajectory circled or slid in Figures 11 and 12 can be either a broken line or a solid line, or a transparent curve (the slide or circled trajectory will not be displayed on the screen, but will still follow the trajectory of the slide. Determine the recognition area), or it can be other types of curves.

In order to more intuitively and intuitively display the retrieval effect of the method for searching the content of the image in the embodiment of the present invention, the method for searching the content of the image in the embodiment of the present invention is combined with the specific recognition result in FIG. 13 to FIG. Carry out a detailed description.

As shown in FIG. 13, after the user opens the picture stored by the portable electronic device (the picture can be considered to have not been subjected to the scaling process at this time), the portable electronic device detects the first operation of the user, and then the portable electronic device selects the picture. The overall content is searched as a search element, and the search result "NBA All-Star Lineup" is displayed on the picture. After performing the overall search, if the user wants to know more about each character in the picture, as shown in FIG. 14, when the user takes the second operation After operating the picture, the portable electronic device automatically retrieves the image in the central area of the screen and displays the search result "O'Neill" on the picture. If the user wants to know more about other characters in the picture, the third operation can be operated to operate the picture. As shown in FIG. 15, after the user moves the picture to the right, the portable electronic device automatically performs the screen. The content of the central area is retrieved and the search result "Duncan" is displayed on the picture. It can be seen from the retrieval process of FIG. 13 to FIG. 15 that the user only needs to perform the first operation on the image when searching for the overall content of the image, and can perform the second operation on the image when the user wants to retrieve the image. At this time, the portable electronic device can automatically retrieve the content under the central area, and display the search result. If the user searches for other content in the picture, the content to be retrieved can be moved to the central area of the screen by moving the picture. It is convenient for the user's operation.

As shown in FIG. 16, after the user opens the picture stored by the portable electronic device, the user performs a first operation on the picture, and the portable electronic device searches the entire content of the picture, and displays the search result “Funny Dessert” in the picture. on. If the user wants to know more about each target in the picture, the user can double-click the picture, as shown in FIG. 17, the area double-clicked by the user is the target in the upper left corner, and then the portable electronic device acquires the target search in the upper left corner. As a result, the search result "fruit mousse cup" is displayed in the vicinity of the target. As shown in FIG. 18, the user continues to double-click the picture, and the area that the user double-clicks is the target object in the lower right corner. At this time, the portable electronic device acquires the target search result in the lower right corner, and displays the search result “strawberry cake” on the target. Near the object. 16 to FIG. 18, the touch area of the user's operation gesture can automatically retrieve the content of the area, and display the search result on the screen or on the image or near the image, which is convenient for the user. operating.

It should be understood that, in the method for retrieving the content of the image in the embodiment of the present invention, the portable electronic device may automatically identify and retrieve all valid objects or objects in the image in advance before detecting the operation on the image, and obtain the retrieval result. And store the results of the search. In addition, the portable electronic device can also retrieve the overall content of the image to obtain an overall retrieval result, and retrieve each object in the image to obtain a retrieval result of each object, and store the overall retrieval result and the retrieval result of each object. Thus, when the portable electronic device detects the user's operation, the previously stored search result can be quickly called and the search result displayed.

The method for retrieving the content of a picture according to an embodiment of the present invention is described in detail above with reference to FIG. 1 to FIG. 18. Referring to FIG. 19 to FIG. 21, a device for searching content of a picture according to an embodiment of the present invention will be described below. . It should be understood that the content of the picture is retrieved as described in FIGS. 19 to 21 The apparatus can implement the various steps of the method of retrieving the content of the picture described in FIGS. 1 to 18, and the repeated description is omitted as appropriate for brevity.

FIG. 19 is a schematic block diagram of an apparatus for retrieving content of a picture according to an embodiment of the present invention. The portable electronic device 400 includes:

The detecting unit 410 is configured to detect a first operation applied to a picture, where the picture is displayed on a screen of the portable electronic device;

The processing unit 420 is configured to: in response to the first operation, acquire a first search result that uses the entire content of the picture as a search element, and display the first search result;

The detecting unit 410 is further configured to detect a second operation that is applied to the picture;

The processing unit 420 is further configured to: in response to the second operation, determine an identification area that identifies the picture, and acquire a second search result that uses the content that overlaps the picture and the identification area as a search element, and displays the The second search result, wherein the content of the picture that coincides with the identification area is a part of the overall content of the picture, and the first operation is different from the second operation.

The portable electronic device can recognize the whole or partial content of the picture when detecting different operations, and the prior art needs to first open the intelligent object recognition option, obtain the photo containing the target object, and then select the target object in the photo. The recognition area, and finally click on the search to obtain the search result, simplifies the user's operation of searching the content of the picture.

Optionally, as an embodiment, the detecting unit 410 is further configured to detect a third operation that is applied to the picture, where the third operation includes dragging, zooming in, or zooming out, the third operation and the third The processing unit 420 is further configured to: in response to the third operation, acquire a third search result that uses the content that overlaps the image and the identification area as a search element, and display a third search result, where The identification area is a preset area on the screen.

Optionally, as an embodiment, the detecting unit 410 is further configured to detect a fourth operation that is applied to the identification frame displayed on the screen, where the fourth operation includes dragging or moving, the identification frame is The boundary line of the identification area coincides, the fourth operation is different from the first operation; the processing unit 420 is further configured to: in response to the fourth operation, acquire content that overlaps the image and the identification area as a retrieval The fourth search result of the element displays the fourth search result.

Optionally, as an embodiment, the processing unit 420 is specifically configured to: in response to the second operation, content that overlaps the preset area on the screen and the picture as an identification area that identifies the picture. Or, the identification box displayed on the screen is circled on the picture The content serves as an identification area for identifying the picture.

Optionally, as an embodiment, the second operation includes a re-press operation, and the processing unit 420 is specifically configured to: determine that the re-press operation is surrounded by a closed curve centered on an action point on the screen. The area is the identification area, and the size of the closed curve is preset.

Optionally, as an embodiment, the second operation includes a double tap or an knuckle tap, and the processing unit 420 is specifically configured to: determine, by using the double tap operation or the touch point of the knuckle tap as a center The area enclosed by the closed curve is the identification area, wherein the size of the closed curve is preset.

Optionally, as an embodiment, the second operation includes two-finger expansion, and the processing unit 420 is specifically configured to: determine, centered on a midpoint of the line segment formed by the initial touch point connection of the two fingers on the screen. The area enclosed by the closed curve is the identification area, and the size of the closed curve is preset.

Optionally, as an embodiment, the second operation includes delimiting or sliding, and the processing unit is specifically configured to: determine an area enclosed by the circled track or the sliding track as the identification area.

Optionally, as an embodiment, the portable electronic device further includes: a determining unit 430, configured to determine, before the processing unit is responsive to the second operation and determining an identification area that identifies the picture The picture is in an enlarged state and the picture is in an enlarged state for a duration exceeding a first threshold.

FIG. 20 is a schematic block diagram of an apparatus for retrieving content of a picture according to an embodiment of the present invention. The portable electronic device 500 includes:

Processor 510, detector 520, memory 530, bus system 540;

The processor 510, the memory 530, and the detection 520 are connected by the bus;

The memory 530 is configured to store one or more programs, the one or more programs including instructions;

The detector 520 is configured to detect a first operation applied to a picture when the instruction is executed by the portable electronic device, wherein the picture is displayed on a screen of the portable electronic device; the processor 510, in response to the first operation, acquiring a first search result that uses the entire content of the picture as a search element, and displaying the first search result;

The detector 520 is further configured to detect a second operation that acts on the picture;

The processor 510 is further configured to: in response to the second operation, determine an identification area that identifies the picture, and acquire a second search result that uses the content that overlaps the picture and the identification area as a search element, and displays the The second search result, wherein the content of the picture that coincides with the identification area is a part of the overall content of the picture, and the first operation is different from the second operation.

In the embodiment of the present invention, after the detector 520 of the portable electronic device detects the first operation of the user on the screen displayed on the screen, the processor 510 can directly retrieve the entire content of the image and retrieve the result. Displayed, and after the detector 520 detects the user's second operation on the picture, the processor 510 can retrieve part of the content of the picture and display the retrieved result, thereby indicating that the portable electronic device is detecting different The operation can realize the retrieval of the whole or partial content of the picture. In the prior art, it is necessary to first open the intelligent object recognition option, obtain the photo containing the target object, and then select the identification area containing the target object in the photo, and then click Compared with the retrieval result, the retrieval operation step of the user is simplified, and the user can recognize the images of different regions in the image through different gestures, thereby enhancing the retrieval effect and improving the user experience.

Optionally, as an embodiment, the detector 520 is further configured to detect a third operation that is applied to the picture, where the third operation includes dragging, zooming in, or zooming out, the third operation and the third An operation is different. The processor 510 is further configured to: in response to the third operation, acquire a third search result that is a content of the search area and the content of the recognition area, and display a third search result, where The identification area is a preset area on the screen.

Optionally, as an embodiment, the detector 520 is further configured to detect a fourth operation that is applied to the identification frame displayed on the screen, the fourth operation includes dragging or moving, the identification frame and the The boundary line of the identification area coincides, the fourth operation is different from the first operation; the processor 510 is further configured to: in response to the fourth operation, acquire content that overlaps the image and the identification area as a retrieval The fourth search result of the element displays the fourth search result.

Optionally, as an embodiment, the processor 510 is specifically configured to: in response to the second operation, content that overlaps the preset area on the screen and the picture as an identification area that identifies the picture. Or, the content circled on the picture by the recognition frame displayed on the screen is used as an identification area for identifying the picture.

Optionally, as an embodiment, the second operation includes a re-press operation, and the processor 510 is specifically configured to: determine a closed curve centered on a point of action of the re-press operation on the screen. The area is the identification area, and the size of the closed curve is preset.

Optionally, as an embodiment, the second operation includes a double tap or an knuckle tap, and the processor 510 is specifically configured to: determine, by using the double tap or the touch point of the knuckle The area enclosed by the closed curve is the identification area, wherein the size of the closed curve is preset.

Optionally, as an embodiment, the second operation includes two-finger expansion, and the processor 510 is specifically configured to: determine, centered on a midpoint of the line segment formed by the initial touch point connection of the two fingers on the screen. The area enclosed by the closed curve is the identification area, and the size of the closed curve is preset.

Optionally, as an embodiment, the second operation includes delimiting or sliding, and the processor 510 is specifically configured to: determine an area enclosed by the circled track or the sliding track as the identification area. .

Optionally, as an embodiment, the processor 510 is specifically configured to: before the second operation and determining the identification area that identifies the picture, determine that the picture is in an enlarged state and the picture is in The length of the amplified state exceeds the first threshold.

The present invention also includes a computer readable storage medium storing one or more programs that, when executed by the portable electronic device, can perform the operations as shown in FIGS. 1 through 18 A method of retrieving the content of a picture.

The present invention also includes a graphical user interface on a portable electronic device, the portable electronic device including a display, a memory, a plurality of applications, and one or more for executing one or more programs stored in the memory A processor, the graphical user interface comprising a user interface displayed in accordance with a method of retrieving content of a picture as shown in FIGS. 1 through 18, wherein the display comprises a touch-sensitive surface and a display screen.

The specific structure of the portable electronic device according to the embodiment of the present invention will be described below with reference to FIG. 21. It should be understood that the structural knowledge shown in FIG. 21 is the most specific case in the portable electronic device of the embodiment of the present invention, and the embodiment of the present invention is not The portable electronic device is limited.

21 is a schematic block diagram of a portable electronic device that retrieves content of a picture in accordance with an embodiment of the present invention. The apparatus includes various modules for performing the method of searching for the content of the picture in the embodiment of the present invention. The portable electronic device will be described in detail below with reference to FIG. As shown in FIG. 21, the portable electronic device includes a processor unit, an input unit, a storage unit, a communication unit, a peripheral interface, an output unit, and a power source. These units or components communicate over one or more buses. It should be understood that the portable electronic device shown in FIG. 21 is only the most specific one. In the case, it does not constitute a limitation on the portable electronic device of the embodiment of the present invention. It may be a bus-shaped structure or a star-shaped structure, and may include more or less components than those illustrated, or may combine some components. , or different parts layout. The portable electronic device of the embodiments of the present invention may be any mobile or portable electronic device, including but not limited to a mobile phone, a mobile computer, a tablet computer, a Personal Digital Assistant (PDA), a media player, a smart TV, and Combination of two or more of the above. The respective modules or units of the portable electronic device of the embodiments of the present invention are described in detail below.

An input unit for implementing user interaction with an electronic device and/or information input into the electronic device. For example, the input unit can receive numeric or character information input by the user to generate a signal input related to user settings or function control. In an embodiment of the present invention, the input unit may be a touch panel, or may be other human-computer interaction interfaces, such as a physical input key, a microphone, etc., and may be other external information capture devices, such as a camera. A touch panel, also known as a touch screen or touch screen, collects operational actions that the user touches or approaches on. For example, the user uses an action of any suitable object or accessory such as a finger or a stylus on or near the touch panel, and drives the corresponding connecting device according to a preset program. Optionally, the touch panel may include two parts: a touch detection device and a touch controller. Wherein the touch detection device detects a touch operation of the user, converts the detected touch operation into an electrical signal, and transmits the electrical signal to the touch controller; the touch controller receives the electrical signal from the touch detection device, and Convert it to contact coordinates and send it to the processing unit. The touch controller can also receive and execute commands from the processing unit. In addition, touch panels can be implemented in various types such as resistive, capacitive, capacitive + pressure sensitive, infrared (infrared) and surface acoustic waves. In other embodiments of the present invention, the physical input keys used by the input unit may include, but are not limited to, a physical keyboard, function keys (such as a volume control button, a switch button, etc.), a trackball, a mouse, a joystick, and the like. Or a variety. An input unit in the form of a microphone can collect the voice input by the user or the environment and convert it into a command executable by the processing unit in the form of an electrical signal.

Specifically, the input unit of the portable electronic device of the embodiment of the present invention may be various types of sensor components, such as a Hall device, which can be used to detect physical quantities of electronic devices, such as force, moment, pressure, stress, position, displacement. Speed, acceleration, angle, angular velocity, number of revolutions, speed, and time when the working state changes, etc., are converted into electricity for detection and control. Other sensor components may also include gravity sensors, three-axis accelerometers, gyroscopes, electronic compasses, ambient light sensors, proximity sensors, temperature sensors, humidity sensors, pressure (pressure) sensors, piezoelectric Sensors, heart rate sensors, fingerprint readers, etc.

A processor unit, which is the control center of the portable electronic device, which can connect various parts of the entire portable electronic device using various interfaces and lines, by running or executing software programs and/or modules stored in the storage unit, and calling storage Data within the storage unit to perform various functions of the electronic device and/or process data. The processor unit may be composed of an integrated circuit (IC), for example, may be composed of a single packaged IC, or may be composed of a plurality of packaged ICs that have the same function or different functions. For example, the processor unit may include only a central processing unit (CPU), or may be a GPU, a digital signal processor (DSP), and a control chip (for example, a baseband) in the communication unit. A combination of chips). In the embodiment of the present invention, the CPU may be a single operation core, and may also include multiple operation cores.

The communication unit is configured to establish a communication channel, and enable the portable electronic device to perform voice communication, text communication, and data communication with the remote electronic device or the server through the communication channel. The communication unit may include a wireless local area network (WLAN) module, a Bluetooth module, a baseband module, and the like, and a radio frequency (RF) circuit corresponding to the communication module. Perform wireless local area network communication, Bluetooth communication, infrared communication, and/or cellular communication system communication, such as Wideband Code Division Multiple Access (WCDMA) and/or High Speed Downlink Packet Access (High Speed Downlink Packet) Access, referred to as HSDPA). The communication module is used to control communication of components in the electronic device, and can support Direct Memory Access.

Specifically, various communication modules in the communication unit of the portable electronic device of the embodiment of the present invention may be in the form of an integrated circuit chip, and may be selectively combined without including all communication modules and corresponding Antenna group. For example, the above communication unit may include only a baseband chip, a radio frequency chip, and a corresponding antenna to provide a communication function in a cellular communication system. The portable electronic device can be connected to a cellular network (Cellular Network) or the Internet (Internet) via a wireless communication connection established by the communication unit, such as wireless local area network access or WCDMA access. The radio frequency chip can form a radio frequency circuit, and the radio frequency circuit can be used to receive and transmit signals during information transmission or reception. For example, the radio frequency circuit may process the downlink information of the base station and then process it to the processing unit; in addition, transmit the designed uplink data to the base station. Typically, the radio frequency circuit includes well-known circuits for performing these functions, including but not limited to antennas System, radio frequency transceiver, one or more amplifiers, tuner, one or more oscillators, digital signal processor, Codec chipset, Subscriber Identity Module (SIM) card, memory, and the like. In addition, the RF circuit can communicate with the network and other devices through wireless communication. The wireless communication may use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication), GPRS (General Packet Radio Service), CDMA (Code Division Multiple Access), WCDMA (Wideband Code Division Multiple Access). HSDPA (High Speed Downlink Packet Access), LTE (Long Term Evolution), email (electronic mail), SMS (Short Messaging Service), and the like.

An output unit that is not limited to the image output unit and the sound output unit. The image output unit is used to output text, pictures, and/or video. The image output unit may include a display panel, such as a display panel configured in the form of an LCD (Liquid Crystal Display), an OLED (Organic Light-Emitting Diode), or an FED (field emission display). Alternatively, the image output unit may comprise a reflective display, such as an electrophoretic display, or a display utilizing an Interferometric Modulation of Light. The image output unit may comprise a single display or multiple displays of different sizes. In a specific embodiment of the present invention, the touch panel used by the input unit can also serve as a display panel of the output unit. For example, when the touch panel detects a touch or proximity gesture operation thereon, it is transmitted to the processing unit to determine the type of the touch event, and then the processing unit provides a corresponding visual output on the display panel according to the type of the touch event. Although in FIG. 21, the input unit and the output unit are two independent components to implement the input and output functions of the electronic device, in some embodiments, the touch panel and the display panel may be integrated to implement the electronic device. Input and output functions. For example, the image output unit may display various graphical user interfaces (GUIs) as virtual control components, including but not limited to windows, scroll axes, icons, and scrapbooks, for the user to touch. The way to operate.

A storage unit, which can be used to store software programs and modules, and the processing unit executes various functional applications of the electronic device and implements data processing by running software programs and modules stored in the storage unit. The storage unit mainly includes a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function, such as a sound playing program, an image playing program, and the like; and the data storage area can be stored according to the electronic device. Use the created data (such as audio data, phone book, etc.). In a specific embodiment of the present invention, the storage unit may include a volatile memory, such as a non-volatile dynamic random access memory (Nonvolatile). Random Access Memory (NVRAM), Phase Change RAM (PRAM), Magnetoresistive Random Access Memory (MRAM), etc., may also include non-volatile memory, such as at least A disk storage device, an electrically erasable programmable read-only memory (EEPROM), a flash memory device, such as a NOR flash memory or a NAND flash memory. The non-volatile memory stores operating systems and applications executed by the processing unit. The processing unit loads the running program and data from the non-volatile memory into the memory and stores the digital content in a plurality of storage devices. The operating system includes various components and/or drivers for controlling and managing conventional system tasks such as memory management, storage device control, power management, and the like, as well as facilitating communication between various hardware and software. In the embodiment of the present invention, the operating system may be an Android system of Google Inc., an iOS system developed by Apple Corporation, a Windows system/Windows Phone system developed by Microsoft Corporation, or an embedded operating system such as Vxworks.

A power source that supplies power to different components of a portable electronic device to maintain its operation. As a general understanding, the power source may be a built-in battery, such as a common lithium ion battery, a nickel hydride battery, etc., and an external power source that directly supplies power to the electronic device, such as an AC adapter. In some embodiments of the present invention, the power source may also be more widely defined, and may further include, for example, a power management system, a charging system, a power failure detecting circuit, a power converter or an inverter, and a power status indicator ( Such as light emitting diodes, as well as any other components associated with the generation, management and distribution of electrical energy of electronic devices.

It should be understood that the term "and/or" herein is merely an association relationship describing an associated object, indicating that there may be three relationships, for example, A and/or B, which may indicate that A exists separately, and A and B exist simultaneously. There are three cases of B alone. In addition, the character "/" in this article generally indicates that the contextual object is an "or" relationship.

It should be understood that, in various embodiments of the present invention, the size of the sequence numbers of the above processes does not mean the order of execution, and the order of execution of each process should be determined by its function and internal logic, and should not be taken to the embodiments of the present invention. The implementation process constitutes any limitation.

Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the various examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the solution. Professionals can use different parties for each specific application The described functionality is implemented, but such implementation should not be considered to be beyond the scope of the invention.

A person skilled in the art can clearly understand that for the convenience and brevity of the description, the specific working process of the system, the device and the unit described above can refer to the corresponding process in the foregoing method embodiment, and details are not described herein again.

In the several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the device embodiments described above are merely illustrative. For example, the division of the unit is only a logical function division. In actual implementation, there may be another division manner, for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

The functions may be stored in a computer readable storage medium if implemented in the form of a software functional unit and sold or used as a standalone product. Based on such understanding, the technical solution of the present invention, which is essential or contributes to the prior art, or a part of the technical solution, may be embodied in the form of a software product, which is stored in a storage medium, including The instructions are used to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like, and the program code can be stored. Medium.

The above is only a specific embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily think of changes or substitutions within the technical scope of the present invention. It should be covered by the scope of the present invention. Therefore, the scope of the invention should be determined by the scope of the appended claims.

Claims

A method for retrieving content of a picture, the method being applied to a portable electronic device, comprising:

Detecting a first operation acting on a picture, the picture being displayed on a screen of the portable electronic device;

And in response to the first operation, acquiring a first search result that uses the entire content of the picture as a search element, and displaying the first search result;

A second operation acting on the picture is detected;

Determining, in response to the second operation, an identification area that identifies the picture, acquiring a second search result that uses the content of the picture and the identification area to be the search element, and displaying the second search result, wherein The content of the picture that coincides with the identification area is part of the overall content of the picture, and the first operation is different from the second operation.
The method of claim 1, wherein after displaying the second search result, the method further comprises:

A third operation acting on the picture is detected, the third operation comprising dragging, zooming in or zooming out, the third operation being different from the first operation;

And in response to the third operation, acquiring a third search result that the content of the picture coincides with the identification area as a retrieval element, displaying a third retrieval result, wherein the identification area is a preset on the screen region.
The method of claim 1, wherein after displaying the second search result, the method further comprises:

Detecting a fourth operation acting on the identification frame displayed on the screen, the fourth operation including dragging or moving, the identification frame coincides with a boundary line of the identification area, the fourth operation and the fourth One operation is different;

In response to the fourth operation, acquiring a fourth search result in which the content of the picture coincides with the recognition area as a search element displays the fourth search result.
The method of claim 1, wherein determining an identification area for identifying the picture in response to the second operation comprises:

The content that overlaps the preset area on the screen with the picture is used as an identification area for identifying the picture, or

Generating an identification frame in an active area of the second operation on the screen, the identification frame being The content circled on the picture serves as an identification area for identifying the picture, wherein the identification frame is movable.
The method of claim 1, wherein determining an identification area for identifying the picture in response to the second operation comprises:

The second operation includes re-pressing, determining that the area enclosed by the closed curve centered on the action point on the screen is the identification area, and the size of the closed curve is preset.
The method of claim 1, wherein determining an identification area for identifying the picture in response to the second operation comprises:

The second operation includes a double tap or an knuckle tap, and an area enclosed by a closed curve centered on the touch point of the double tap or the knuckle tap is determined as the identification area, wherein the closed curve The size is preset.
The method of claim 1, wherein in response to the second operation, determining an identification region identifying the picture in response to the second operation comprises:

The second operation includes two-finger expansion, and the area enclosed by the closed curve centered on the midpoint of the line segment formed by the initial touch point connection of the two fingers on the screen is the identification area, the closed curve The size is preset.
The method of claim 1, wherein determining an identification area for identifying the picture in response to the second operation comprises:

The second operation includes delineating or sliding to determine an area enclosed by the circled trajectory or the trajectory of the slid as the identification area.
The method of any of claims 1-8, wherein the method further comprises: prior to determining the identified area identifying the picture in response to the second operation, the method further comprising:

Determining that the picture is in an enlarged state and the picture is in an enlarged state exceeds a first threshold.
The method of claim 9, wherein the determining that the picture is in an enlarged state comprises:

It is determined that the ratio of the enlarged size of the picture to the initial display size of the picture exceeds a second threshold.
A portable electronic device, comprising:

a detecting unit, configured to detect a first operation acting on a picture, the picture being displayed on a screen of the portable electronic device;

a processing unit, configured to: in response to the first operation, acquire a first search result that uses the entire content of the picture as a search element, and display the first search result;

The detecting unit is further configured to detect a second operation that is applied to the picture;

The processing unit is further configured to: in response to the second operation, determine an identification area that identifies the picture, acquire a second search result that uses the content of the picture and the identification area to be the search element, and display the a second search result, wherein the content of the picture coincident with the identification area is a part of the overall content of the picture, and the first operation is different from the second operation.
The portable electronic device according to claim 11, wherein said detecting unit is further configured to detect a third operation acting on said picture, said third operation comprising dragging, enlarging or reducing, said third The operation is different from the first operation;

The processing unit is further configured to: in response to the third operation, acquire a third search result that uses the content of the picture and the identification area to be the search element, and display a third search result, where the identification area is The preset area on the screen.
A portable electronic device according to claim 11, wherein said detecting unit is further configured to detect a fourth operation acting on the identification frame displayed on the screen, said fourth operation comprising dragging or moving, said The identification frame coincides with a boundary line of the identification area, and the fourth operation is different from the first operation;

The processing unit is further configured to, in response to the fourth operation, acquire a fourth retrieval result that is a content of the image that overlaps the identification area as a retrieval element, and display the fourth retrieval result.
The portable electronic device of claim 11, wherein the processing unit is specifically configured to:

Responding to the second operation, the content that overlaps the preset area on the screen with the picture is used as an identification area for identifying the picture, or

The content circled on the picture by the recognition frame displayed on the screen is used as an identification area for identifying the picture.
The portable electronic device of claim 11, wherein the second operation comprises re-pressing, and the processing unit is specifically configured to:

It is determined that the area enclosed by the closed curve centered on the action point on the screen is the identification area, and the size of the closed curve is preset.
The portable electronic device of claim 11, wherein the second operation comprises a double tap or an knuckle tap, the processing unit being specifically configured to:

The area enclosed by the closed curve centered on the touch point of the double tap or the knuckle tap is determined as the identification area, wherein the size of the closed curve is preset.
The portable electronic device of claim 11, wherein the second operation comprises two-finger expansion, and the processing unit is specifically configured to:

The area enclosed by the closed curve centered on the midpoint of the line segment formed by the two fingers on the initial touch point connection on the screen is determined as the identification area, and the size of the closed curve is preset.
The portable electronic device of claim 11, wherein the second operation comprises delineation or sliding, and the processing unit is specifically configured to:

An area defined by the circled trajectory or the trajectory of the slid is determined as the identification area.
The portable electronic device according to any one of claims 11 to 18, wherein the portable electronic device further comprises:

And a determining unit, configured to determine that the picture is in an enlarged state and the time in which the picture is in an enlarged state exceeds a first threshold before the processing unit is responsive to the second operation and determining an identification area that identifies the picture.
A portable electronic device, comprising:

Processor, detector, memory, system bus;

The processor, the detector, and the memory are connected by the system bus;

The memory is for storing one or more programs, the one or more programs comprising instructions that, when executed by the portable electronic device, cause the portable electronic device to perform any of claims 1-10 The method described in the item.
A computer readable storage medium storing one or more programs, when the one or more programs are executed by the portable electronic device, the portable electronic device performing the method of any one of claims 1-10 method.
A graphical user interface on a portable electronic device, the portable electronic device including a display, a memory, a plurality of applications, and one or more processors for executing one or more programs stored in the memory, The graphical user interface includes a user interface displayed in accordance with the method of any of claims 1-10, wherein the display includes a touch-sensitive surface and a display screen.