WO2017107855A1

WO2017107855A1 - Picture searching method and device

Info

Publication number: WO2017107855A1
Application number: PCT/CN2016/110193
Authority: WO
Inventors: 康琳; 马士龙
Original assignee: 阿里巴巴集团控股有限公司; 康琳; 马士龙
Priority date: 2015-12-25
Filing date: 2016-12-15
Publication date: 2017-06-29
Also published as: CN106919326A

Abstract

A picture searching method and device. The method comprises: upon detection of a first operation for a first picture, determining a search dimension corresponding to the first picture (202); and searching, according to the search dimension corresponding to the first picture, for a second picture matching the first picture in the search dimension (203), thus realizing an automatic searching process of similar pictures triggered by the first operation. The present invention gives comparatively less user operation complexity, reduced time consumption, and less energy consumption relative to using a manual picture searching method.

Description

Image search method and device

Technical field

The present application relates to the field of communications technologies, and in particular, to a picture search method and apparatus.

Background technique

With the rapid development of mobile communication technology and the advent of the mobile multimedia era, mobile phones, as a necessary mobile communication tool for people, have evolved from simple call tools to intelligent development, and have evolved into a mobile information collection and processing platform.

At present, a large number of pictures can be stored in a mobile phone, such as a picture that can be stored by a mobile phone, or an image that is stored by other means.

When a user needs to search for a picture with a certain similar or similar characteristic from these large numbers of pictures, it is necessary to view a large number of pictures for comparison, which is complicated and time consuming, and correspondingly, the power consumption of the mobile phone is also large. As the storage capacity of mobile phones increases, the amount of image storage increases, so the problem becomes more prominent.

Application content

To solve the above problem, an embodiment of the present application provides a picture search method and apparatus.

The image search method provided by the embodiment of the present application includes:

Obtaining a picture search instruction, the picture search instruction being generated according to the detected first operation for the first picture;

Determining a search dimension corresponding to the first picture;

Searching, according to the search dimension corresponding to the first picture, a second picture that matches the first picture in the search dimension;

Output the search results.

Preferably, the first operation for the first picture comprises:

a first screen touch operation performed in a screen display area of the first picture; or

Obtaining the operation of the first picture by scanning; or

Obtaining the operation of the first picture by shooting; or,

Pasting the first picture in the search box; or,

An operation for selecting an option for triggering a picture search in the picture operation option menu, wherein the picture operation option menu is popped up after the second screen touch operation performed in the screen display area of the first picture is detected; or The operation of the user interface UI component for triggering the image search is selected in the screen display area of the first picture or in the preview view window.

The first screen touch operation performed in the screen display area of the first picture includes: a multi-finger zoom sliding operation performed in a screen display area of the first picture; or, in the A sliding operation of a single finger in a set direction in a screen display area of a picture.

Preferably, the determining the search dimension corresponding to the first picture comprises:

Obtaining a default search dimension, and determining the default search dimension as a search dimension corresponding to the first image, where the default search dimension includes one or more search dimensions; or

Determining, in the pop-up menu of the first picture, an option for selecting a search dimension, the selected search dimension as a search dimension corresponding to the first image, wherein the pop-up menu is in the first A third screen touch operation performed in a screen display area of a picture is detected and popped up, and one search dimension option in the popup menu corresponds to one or more search dimensions; or

Determining, according to the detected type of the first operation, and the corresponding relationship between the type of the first operation and the search dimension, the search dimension corresponding to the detected first operation, where the first operation corresponds to one or more Search for dimensions.

Preferably, the search dimension comprises one or more of the following dimensions:

a search dimension based on an object in the image;

Search dimension based on the hue of the image;

Search dimension based on the shooting time of the picture;

a search dimension based on the location of the image;

The search dimension of a photo-based device or source.

When the search dimension includes a search dimension based on an object in the image, the searching for a second image that matches the first image on the search dimension according to the search dimension corresponding to the first image, including :

Extracting feature information of the first picture;

And searching for the second picture according to the extracted feature information, where the feature information of the second picture matches the feature information of the first picture.

The first operation for the first picture is a first screen touch operation performed in a screen display area of the first picture, and the first picture is extracted according to a pixel of the first picture Characteristic information, including:

Acquiring an action area of the first screen touch operation according to a first screen touch operation performed in a screen display area of the first picture;

Extracting feature information of the first picture in the active area.

Preferably, searching for the second picture that matches the first picture in the search dimension comprises: searching from the locally stored picture library and/or the network side stored picture library on the search dimension and the The second picture matches the first picture.

Preferably, the outputting the search result comprises: displaying the searched second picture in a picture preview manner according to the order of matching degree, the number of the second pictures being one or more.

Preferably, the outputting the search result comprises: displaying the found second image by using N sub-windows, each sub-window is for displaying a preview view of the second image or the second image, which is found based on one search dimension, each sub-window It is allowed to switch to display a preview view of the second picture or the second picture found based on the corresponding search dimension according to the sliding touch operation, where N is an integer greater than or equal to 1.

The image search device provided by the embodiment of the present application includes:

An acquiring module, configured to acquire a picture search instruction, where the picture search instruction is generated according to the detected first operation for the first picture;

a determining module, configured to determine a search dimension corresponding to the first picture;

a searching module, configured to search, according to the search dimension corresponding to the first picture, a second picture that matches the first picture in the search dimension;

Output module for outputting the search results.

Preferably, the first operation for the first picture comprises:

Obtaining the operation of the first picture by scanning; or

Obtaining the operation of the first picture by shooting; or,

Pasting the first picture in the search box; or,

An operation for selecting an option for triggering a picture search in the picture operation option menu, wherein the picture operation option menu is popped up after the second screen touch operation performed in the screen display area of the first picture is detected; or ,

The operation of the user interface UI component for triggering the image search is selected in the screen display area of the first picture or in the preview view window.

Preferably, the determining module is specifically configured to:

Determining, according to the detected type of the first operation, and the corresponding relationship between the type of the first operation and the search dimension, the search dimension corresponding to the detected first operation, where the first The operation corresponds to one or more search dimensions.

a search dimension based on an object in the image;

Search dimension based on the hue of the image;

Search dimension based on the shooting time of the picture;

a search dimension based on the location of the image;

The search dimension of a photo-based device or source.

The searching module is specifically configured to: when the search dimension includes a search dimension based on an object in the image, extract feature information of the first image, and search for a second image according to the extracted feature information, where the second The feature information of the picture matches the feature information of the first picture.

The searching module is specifically configured to: when the first operation for the first picture is a first screen touch operation performed in a screen display area of the first picture, according to a screen in the first picture And performing a first screen touch operation performed in the display area, acquiring an action area of the first screen touch operation, and extracting feature information of the first picture in the active area.

Preferably, the searching module is specifically configured to: search for a second picture that matches the first picture in the search dimension from a locally stored picture library and/or a picture library stored in the network side.

Preferably, the output module is specifically configured to display the second picture that is found in a picture preview manner according to the order of matching degree, the number of the second pictures being one or more.

Preferably, the output module is specifically configured to display the searched second image by using N sub-windows, each sub-window is configured to display a preview view of the second image or the second image that is found based on one search dimension, each sub-window It is allowed to switch to display a preview view of the second picture or the second picture found based on the corresponding search dimension according to the sliding touch operation, where N is an integer greater than or equal to 1.

The terminal provided by the embodiment of the present application includes:

monitor;

a memory for storing computer program instructions;

a processor coupled to the memory for reading computer program instructions stored by the memory and, in response, performing the following operations:

Determining a search dimension corresponding to the first picture;

Output the search results.

Preferably, the first operation for the first picture comprises:

Obtaining the operation of the first picture by scanning; or

Obtaining the operation of the first picture by shooting; or,

Pasting the first picture in the search box; or,

The first screen touch operation performed in the screen display area of the first picture includes:

a multi-finger zoom sliding operation performed in a screen display area of the first picture; or

A sliding operation of the single finger in the set direction in the screen display area of the first picture.

Preferably, the processor is specifically configured to:

The option for selecting the search dimension in the pop-up menu of the first picture is selected The search dimension corresponding to the first picture is determined as a search dimension corresponding to the first picture, wherein the pop-up menu is popped after the third screen touch operation performed in the screen display area of the first picture is detected. One search dimension option in the popup menu corresponds to one or more search dimensions; or

a search dimension based on an object in the image;

Search dimension based on the hue of the image;

Search dimension based on the shooting time of the picture;

a search dimension based on the location of the image;

The search dimension of a photo-based device or source.

The processor is specifically configured to: when the search dimension includes, based on a search dimension of an object in the image, extract feature information of the first image, and search for a second image according to the extracted feature information, where the second The feature information of the picture matches the feature information of the first picture.

The processor is specifically configured to: when the first operation for the first picture is a first screen touch operation performed in a screen display area of the first picture, according to a screen in the first picture And performing a first screen touch operation performed in the display area, acquiring an action area of the first screen touch operation, and extracting feature information of the first picture in the active area.

Preferably, the processor is specifically configured to: search for a second picture that matches the first picture in the search dimension from a locally stored picture library and/or a picture library stored in the network side.

Preferably, the processor is specifically configured to: in a picture preview manner, instruct the display to display the found second picture in an order of matching degree from high to low, the number of the second pictures being one or more.

Preferably, the processor is specifically configured to: instruct the display to display the found second image by using N sub-windows, each sub-window is used to display the second image found based on a search dimension. A preview view of the slice or the second picture, each of the sub-windows being allowed to switch to display a preview view of the second picture or the second picture found based on the corresponding search dimension according to the sliding touch operation, where N is an integer greater than or equal to 1.

In the foregoing embodiment of the present application, after acquiring a picture search instruction generated according to the detected first operation for the first picture, determining a search dimension corresponding to the first picture, and according to the first a search dimension corresponding to the picture, searching for a second picture that matches the first picture on the search dimension, thereby implementing an automatic search process of similar pictures triggered by the first operation, compared with manually performing image search The user operation is low in complexity and short in time, and accordingly, the power consumption is also small.

DRAWINGS

1 is a schematic diagram of a network architecture applicable to an embodiment of the present application;

2 is a schematic diagram of a picture search process according to an embodiment of the present application;

3, FIG. 4, and FIG. 5 are schematic diagrams of screen touch operations for triggering a similar picture search process in an embodiment of the present application;

6 is a schematic diagram showing display of search results in an embodiment of the present application;

FIG. 7 is a schematic structural diagram of a picture search apparatus according to an embodiment of the present application;

FIG. 8 is a schematic structural diagram of a terminal according to an embodiment of the present application.

detailed description

An embodiment of the present application provides an automatic picture search solution implemented on a terminal. The solution provided by the embodiment of the present application can be applied to a mobile terminal, a PC (personal computer), or a smart device, and the mobile terminal can be a mobile phone, a PDA (Personal Digital Assistant), a smart wearable device, or the like. Especially suitable for mobile smart devices with touch screens.

The embodiments of the present application are described in detail below with reference to the accompanying drawings.

FIG. 1 exemplarily shows a network architecture to which the embodiment of the present application is applied. As shown in FIG. 1 , the network architecture may include a terminal and a server on the network side, and the terminal and the server enter through the network. Line communication.

The servers in the above architecture can be used to provide storage or access services to the user's photo library. The user picture library is used to store the picture of the user. For example, the user can take a picture through the mobile phone and upload it to the user picture library for storage. The server can also provide a data synchronization service. Through data synchronization, pictures stored on the terminal can be synchronously stored or updated into the user picture library.

The terminal in the above architecture can implement the image search method provided by the embodiment of the present application. Further, in the picture search process, the picture in the picture library of the terminal may be searched, or the picture of the user of the terminal in the user picture library may be searched by the server, or both the local picture library of the terminal and the network side may be searched. Photo gallery.

In the above network architecture, the terminal and the server can exchange information through the network, and the network can be a wide area network, a local area network or the Internet, or an Internet using mobile communication technology. The terminal can access the Internet wirelessly, and the server is usually connected to the Internet by wire.

Preferably, the terminal and the server can adopt cloud computing technology to implement information processing based on powerful functions of the cloud computing technology. Servers and terminals can use cloud-based technology-based operating systems, such as YunOS, to consolidate cloud and terminal resources and services.

Referring to FIG. 2, it is a schematic diagram of a picture search process provided by an embodiment of the present application, where the process can be implemented on a terminal side. As shown, the process can include the following steps:

Step 201: Acquire a picture search instruction for the first picture, the picture search instruction is generated according to the detected first operation for the first picture. That is, after the first operation for the first picture is detected, the execution of the following steps 202 to 204 may be triggered.

The "first picture" does not specifically refer to a certain type of picture, but is a way of describing a convenient use.

In this step, the first operation for the first picture may include one of the following operations:

Operation 1: A first screen touch operation performed in a screen display area of the first picture. As an example, when the first picture is displayed in the picture display interface, such as being displayed in a normal size (or highlighted) in the picture browsing window, instead of only displaying its preview view, in this case, if the user Performing a screen touch operation in the display area of the first picture triggers a similar picture search process with the first picture as a reference.

The first screen touch operation performed in the screen display area of the first picture may be a single-finger touch operation (also referred to as a single touch operation) performed in the screen display area of the first picture, or may be a multi-finger operation. The touch operation (also referred to as a multi-touch operation), the embodiment of the present application does not limit the type of the touch operation, as long as the touch operation capable of triggering the similar picture search process is defined in advance. Preferably, in order to distinguish from a commonly used screen touch operation for a picture, such as a single-finger horizontal sliding operation for triggering switching of a picture, the embodiment of the present application selects a specific screen touch operation to trigger a similar picture search process. For example, one of the following screen touch operations may be selected to trigger a similar image search process:

- A multi-finger zoom slide operation performed in the screen display area of the first picture. Preferably, the multi-finger zooming operation may be a two-finger pinch sliding operation, which is not limited in this embodiment of the present application. As an example, as shown in FIG. 3, when only the first picture is displayed in the picture browsing window 301, if the user performs the two-finger pinch-sliding operation 303 in the display area 302 of the first picture, this case Next, after the operation is detected, a similar picture search process based on the first picture is triggered.

- a sliding operation of the single finger in the set direction performed in the screen display area of the first picture, preferably the setting direction may be a downward direction. As an example, as shown in FIG. 4, when only the first picture is displayed in the picture browsing window 401, if the user performs a single-pointing down sliding operation 403 in the display area 402 of the first picture, this case Next, after the operation is detected, a similar picture search process based on the first picture is triggered.

Operation 2: The operation of obtaining the first picture by scanning is applicable to the terminal provided with the picture scanning device. As an example, after the terminal scans the first picture by using the scanning device disposed thereon, the similar picture search process using the first picture as a reference may be automatically triggered according to the obtained first picture scan information.

Operation 3: The operation of obtaining the first picture by shooting is applied to a terminal provided with an imaging device such as a camera. As an example, after the terminal performs photographing through the camera set thereon, the similar image search process based on the captured first image may be automatically triggered.

Operation 4: The operation of pasting the first picture in the search box. As an example, when the user opens an application for implementing image search, and pastes the selected first image into the application After the search box in the search interface, a similar image search process based on the first image will be triggered.

Operation 5: an operation for selecting an option for triggering a picture search in a popup menu of the first picture. As an example, when it is detected that the second screen touch operation is performed in the screen display area of the first picture, a picture operation option menu is popped up, and the menu may include one or more picture operation options, at least including An option for triggering an image search, for example, the menu may include the following options: "search for similar pictures", "copy pictures", "share pictures"; when the option for triggering image search (such as "search for similar pictures") is After selection, a similar image search process based on the first picture will be triggered. The second screen touch operation may be a long press or a click operation in the first picture display area, which is not limited in this embodiment of the present application.

Operation 6: The operation of the UI (User Interface) component for triggering the image search is selected in the screen display area of the first picture or in the preview view window. The UI component may be a function button or a text box or other type of UI component, which is not limited in this embodiment of the present application. As an example, in the image preview interface, a UI component can be set in the preview view window of each image, such as a button named "similar image search", when the user triggers a picture in the preview view window by operation " After the similar image search button, a similar image search process based on the image can be triggered. For another example, as shown in FIG. 5, if it is detected that the user performs a single-finger press in the screen display area 501 of the first picture and holds the operation for 3 seconds (ie, long press operation) 502, a UI component is generated (eg, The button named "Find Similar Image" in the figure is displayed in the interface, and if the UI component is triggered (such as being clicked), a similar image search process based on the first image is triggered.

Operation 7: Set the function key operation when the first picture is displayed on the screen. As an example, when the first picture is displayed (or highlighted) in a normal size in the picture browsing window, instead of only displaying its preview view, if it is detected that the user has pressed a particular button of the terminal, A similar picture search process based on the first picture is triggered.

Operation 8: A set gesture operation performed when the first picture is displayed on the screen. As an example, when the first picture is displayed in a normal size in the picture browsing window, instead of only displaying its preview view, if the user is detected to perform a specific gesture operation, for example, The action of the moving arm triggers a similar picture search process based on the first picture.

The above merely exemplarily gives several operations for triggering a similar picture search process, in addition to which other operations may be included, for example, when the first picture is displayed in a normal size in the picture browsing window, instead of only displaying it In the case of the preview view, the similar image search process based on the first picture is triggered by the voice command, and the embodiments of the present application are not enumerated here.

Step 202: Determine a search dimension corresponding to the first picture.

The embodiment of the present application can support image search of multiple dimensions to meet different needs of users. In a similar image search process, you can search based on one dimension or multiple dimensions. For example, the image search dimension provided by the embodiment of the present application may include one or more combinations of the following dimensions:

- Based on the search dimensions of the objects in the image. For example, for a person or building or other object in the first picture, search for other pictures that contain the object. If the image is searched by using the dimension, the feature information of the first image and the feature information of other images in the image library may be extracted; then the feature information of the first image is compared with the feature information of other images, and the similarity measure function is used. Calculating the similarity between the pictures, wherein the similarity can be generally characterized by an Euclidean distance, a Cosine distance, or a Mahalanobis distance; if the similarity between the second picture and the first picture is higher than a set threshold, the second picture is As a searched image similar to the first picture. Among them, feature extraction is a process of converting an image into a vector. The vector is called an image feature. Feature extraction can be done by a feature extraction model. The input of the model is a picture (the value of each pixel in the picture), and the output is an image. feature. There are a variety of feature extraction methods in the field of image analysis, such as color histograms, LBP, Gabor, deep convolutional networks, etc., and embodiments of the present application can use these methods for feature extraction.

- Search dimensions based on the hue of the image. For example, based on the hue of the first picture, other pictures that are consistent or approximately identical to the hue are searched for. Among them, the hue of the picture can be calculated by using an image analysis algorithm.

- Search dimensions based on the time taken for the photo. For example, according to the shooting time of the first picture, other pictures that are the same or similar to the shooting time are searched. The shooting time of the picture can usually be saved in the picture library as metadata or attribute information of the picture. Shooting of two pictures The similar shooting time means that the difference between the shooting times of the two pictures is within the set duration.

- Search dimensions based on the location of the image. For example, according to the shooting location of the first picture, search for other pictures that are the same or similar to the shooting location. The location information of the image can be saved in the image library as metadata or attribute information of the image. For example, after taking a photo with a mobile phone, the current location of the mobile phone can be obtained, and related information (such as latitude and longitude) of the current location is saved as the metadata or attribute information of the image in the image library. The similarity between the two pictures is that the distance between the two pictures is within the set distance.

- Search dimensions for photo-based devices or sources. For example, according to the photographing device used in the first picture, search for other pictures taken using the same photographing device. Information about the shooting device used by the image (such as device type, model number, etc.) can be saved in the image library as metadata or attribute information of the image. For example, after taking a photo with a mobile phone, you can save the phone's related information (phone type, etc.) as metadata or attribute information of the image to the photo library; import photos from the camera's memory card to the phone image In the case of the library, the information (such as type, model, etc.) of the camera that took the photo may be saved as a metadata or attribute information of the image to the photo library; if the image is from WeChat (a social application), etc. The application can save the related information of the application as the metadata or attribute information of the image to the image library when saving the image to the image library.

The above is only exemplarily shown in the image search dimension provided by the embodiment of the present application. The embodiment of the present application is not limited to the above search dimension. For example, the search of the same or similar size may be performed based on the size of the image. List them one by one.

As described above, one or more search dimensions can be used when performing an image search. The embodiment of the present application preferably provides the following search modes (the one of the method 1 to the mode 3) to determine the search dimension to be used in the image search, and specifically includes:

Method 1: Obtain a default search dimension, and determine a default search dimension as a search dimension corresponding to the first picture, and the default search dimension may include one or more search dimensions.

Manner 2: The operation for selecting a search dimension in the pop-up menu of the first picture is selected, and the selected search dimension is determined as the search dimension corresponding to the first picture. Among them, the The pop-up menu pops up after the third screen touch operation performed in the screen display area of the first picture, and one search dimension option in the pop-up menu corresponds to one or more search dimensions. The third screen touch operation may be a one-finger long touch operation in a screen display area of the first picture. As an example, when the user presses the screen for 3 seconds in the display area of the first picture displayed in the picture browsing window, a menu pops up, which includes "search for the same person", "search for the same color", "search for the same source" "Option; if the user selects the "Search for the same person" option in the menu, the trigger uses a similar image search process based on feature extraction; if the user selects the "Search for the same color" option in the menu, the trigger based on the first image is triggered. Tone finds the search process for pictures with the same hue.

Manner 3: determining a search dimension corresponding to the detected first operation according to the detected type of the first operation, and the corresponding relationship between the type of the first operation and the search dimension, where a first operation corresponds to one Or multiple search dimensions. As an example, the correspondence between the type of the touch operation of the screen and the search dimension is set in advance. For example, the one-point swipe operation corresponds to the “search dimension based on the object in the image”, and the single-point swipe operation corresponds to the “image-based hue search”. Dimensions"; thus, when it is detected that the one-point up slide operation is performed in the first picture display area, it is determined to perform "picture search based on the search dimension of the object in the picture"; when it is detected in the first picture display area When the single-point down slide operation is performed, it is determined that the "search dimension based on the hue of the picture" is used for image search.

Step 203: Search for a second picture that matches the first picture on the search dimension according to the search dimension corresponding to the first picture.

The second picture refers to a picture that matches or is similar to the first picture in one or some search dimensions, and the second picture may be one or more.

In a specific implementation, the second picture that matches the first picture in the corresponding search dimension may be searched from the picture library stored locally by the terminal and/or the picture library stored on the network side.

Preferably, in order to further improve the flexibility and meet the requirements of the user, the embodiment of the present application allows the user to select a local area in the first picture, and extract a feature based on the local area, thereby searching for a second picture similar to the feature of the local area. Specifically, the action area of the first screen touch operation may be acquired according to the first screen touch operation performed in the screen display area of the first picture; and then the feature information of the first picture in the active area is extracted, and further Other pictures The operation of feature extraction is performed; and the similarity between the images is calculated by the extracted feature information, thereby determining a second picture similar to the first picture. When feature extraction is performed on other pictures, feature information of the same size area in other pictures may be extracted according to the size of the action area, and the area may be in different positions for one picture (one picture in another picture) The size of the feature extracts feature information of multiple regions, and when the feature information or the similarity comparison is performed, the feature information closest to the feature information extracted in the first image in one image may be retained as the feature information of the image, thereby measuring The similarity of the picture to the first picture.

For example, if the first picture contains multiple characters, the user can select a person's head by a single-finger click operation, thereby triggering feature extraction on the character's head area; for other pictures in the picture library to be compared, for example a second picture, where the different positions in the second picture (preferably including the same position as the selected person's head in the first picture) are feature-extracted by the size of the area, and the feature information of each feature information and the first picture is calculated. The similarity retains the similarity in which the maximum value is the similarity of the second picture, and if the similarity is greater than the threshold, the second picture is output as the found similar picture. In this way, you can find additional images that contain the object for each object.

Step 204: Output the search result.

In this step, the search result can be output to the display device for display. Correspondingly, if the second picture matching the first picture is found, the found picture may be displayed, otherwise the prompt information of the picture that does not find the matching may be given.

Preferably, when the found second picture is displayed, the found second picture may be displayed in a picture preview manner according to the order of matching degree from high to low. Wherein, for the foregoing “based on the search dimension of the object in the picture”, the matching degree refers to the similarity between the pictures determined based on the feature extraction method; for the aforementioned “search dimension based on the tone of the picture”, the matching degree refers to The closeness of the tone value of the picture; for the aforementioned "search dimension based on the picture-taking time", the matching degree refers to the proximity of time; for the aforementioned "search dimension of the picture-based shooting location", the matching degree refers to the location The proximity. If multiple dimensions are used for image search, the matching degree of multiple dimensions (for example, using a weighted summation algorithm) may be combined to determine the total matching degree, and the search result is displayed according to the total matching degree.

In the specific implementation, in order to facilitate the user to view the search result, N sub-windows can be set and used. N sub-windows display the found second picture, each sub-window is used to display a preview view of the second picture or the second picture found based on one search dimension, each sub-window is allowed to switch display according to the corresponding search dimension according to the sliding touch operation A preview view of the second or second picture found, N being an integer greater than or equal to one. 6 exemplarily shows a search result page by taking three sub-windows as an example. As shown in FIG. 6, the area 601 is used to display the first picture, and the sub-window 602 is used to display the "search dimension based on the object in the picture". a preview view of the picture to which the sub-window 603 is used to display a preview view of the "search dimension based on the hue of the picture", and the sub-window 604 is used to display the image "found based on the search dimension of the photographing time of the picture" Preview view. The preview view in the child window 602 can be browsed by a one-way right swipe operation 605, and the preview view in the child window 603 can be browsed by a one-point left swipe operation 606.

Further, the matching degree of the picture may also be displayed, for example, the matching degree is displayed below the found picture, so that the user can browse and select.

As can be seen from the above description, in the embodiment of the present application, after acquiring the image search instruction generated according to the detected first operation for the first picture, determining a search dimension corresponding to the first picture, and And searching, according to the search dimension corresponding to the first picture, a second picture that matches the first picture in the search dimension, thereby implementing an automatic search process of a similar picture triggered by the first operation, and adopting a manual manner Compared with the image search, the user operation is low in complexity and time-consuming, and accordingly, the power consumption is also small.

Based on the same technical concept, the embodiment of the present application further provides a picture search device, which can implement the foregoing picture search process. The device can be a terminal or a component integrated inside the terminal. The device can be implemented by software or by a combination of software and hardware.

FIG. 7 is a schematic structural diagram of a picture search apparatus according to an embodiment of the present disclosure. The picture search apparatus may include: an obtaining module 701, a determining module 702, a searching module 703, and an output module 704, where:

The obtaining module 701 is configured to acquire a picture search instruction, where the picture search instruction is generated according to the detected first operation for the first picture;

a determining module 702, configured to determine a search dimension corresponding to the first picture;

The searching module 703 is configured to search according to the search dimension corresponding to the first picture. Searching for a second picture on the dimension that matches the first picture;

The output module 704 is configured to output a search result.

Preferably, the first operation for the first picture comprises:

Obtaining the operation of the first picture by scanning; or

Obtaining the operation of the first picture by shooting; or,

Pasting the first picture in the search box; or,

Preferably, the determining module 702 is specifically configured to:

a search dimension based on an object in the image;

Search dimension based on the hue of the image;

Search dimension based on the shooting time of the picture;

a search dimension based on the location of the image;

The search dimension of a photo-based device or source.

The searching module 703 may be specifically configured to: when the search dimension includes the search dimension of the object in the image, extract feature information of the first image, and search for the second image according to the extracted feature information, where the second The feature information of the picture matches the feature information of the first picture.

The searching module 703 may be specifically configured to: when the first operation for the first picture is a first screen touch operation performed in a screen display area of the first picture, according to a screen in the first picture And performing a first screen touch operation performed in the display area, acquiring an action area of the first screen touch operation, and extracting feature information of the first picture in the active area.

Preferably, the searching module 703 is specifically configured to: search for a second picture that matches the first picture in the search dimension from a locally stored picture library and/or a picture library stored in the network side.

Preferably, the output module 704 is specifically configured to: display the second picture that is found in a picture preview manner according to the order of matching degree from high to low, the number of the second pictures being one or more.

Preferably, the output module 704 is specifically configured to: display the searched second image by using N sub-windows, each sub-window is configured to display a preview view of the second image or the second image that is found based on one search dimension, each sub-window It is allowed to switch to display a preview view of the second picture or the second picture found based on the corresponding search dimension according to the sliding touch operation, where N is an integer greater than or equal to 1.

Based on the same technical concept, the embodiment of the present application further provides a terminal, where the terminal can implement the foregoing image search process.

FIG. 8 is a schematic structural diagram of a terminal according to an embodiment of the present application. The terminal may generally include a processor 801, a memory 802, and a display 803.

Wherein, the processor 801 can be a general purpose processor (such as a microprocessor or any conventional processor, etc.), a digital signal processor, an application specific integrated circuit, a field programmable gate array, or He has programmable logic devices, discrete gates or transistor logic devices, and discrete hardware components. Memory 802 may specifically include internal memory and/or external memory, such as random access memory, flash memory, read only memory, programmable read only memory or electrically erasable programmable memory, registers, and the like, which are well established in the art. Display 803 can include touch screen control circuitry.

There is a data communication connection between the processor 801 and other modules, such as data communication based on the bus architecture. The bus architecture may include any number of interconnected buses and bridges, specifically linked by one or more processors represented by processor 801 and various circuits of memory represented by memory 802. The bus architecture can also link various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and, therefore, will not be further described herein. The bus interface provides an interface. The processor 801 is responsible for managing the bus architecture and general processing, and the memory 802 can store data used by the processor 801 in performing operations.

The picture search process disclosed in the embodiment of the present application may be applied to the processor 801 or implemented by the processor 801. In the implementation process, each step of the picture search process may be completed by an integrated logic circuit of hardware in the processor 801 or an instruction in the form of software. The methods, steps, and logical block diagrams disclosed in the embodiments of the present application can be implemented or executed. The steps of the method disclosed in the embodiments of the present application may be directly implemented as a hardware processor, or may be performed by a combination of hardware and software modules in the processor. The software module can be located in a conventional storage medium such as random access memory, flash memory, read only memory, programmable read only memory or electrically erasable programmable memory, registers, and the like.

Specifically, the processor 801 is coupled to the memory 802 for reading computer program instructions stored by the memory 802 and, in response, performing the following operations:

Determining a search dimension corresponding to the first picture;

Output the search results.

Preferably, the first operation for the first picture comprises:

Obtaining the operation of the first picture by scanning; or

Obtaining the operation of the first picture by shooting; or,

Pasting the first picture in the search box; or,

In the screen display area of the first picture or in the preview view window, the UI component for triggering the picture search is selected.

Preferably, the processor 801 is specifically configured to:

a search dimension based on an object in the image;

Search dimension based on the hue of the image;

Search dimension based on the shooting time of the picture;

a search dimension based on the location of the image;

The search dimension of a photo-based device or source.

The processor 801 may be specifically configured to: when the search dimension includes, based on a search dimension of an object in the image, extract feature information of the first image, and search for a second image according to the extracted feature information, where the second The feature information of the picture matches the feature information of the first picture.

The processor 801 is specifically configured to: when the first operation for the first picture is a first screen touch operation performed in a screen display area of the first picture, according to a screen in the first picture And performing a first screen touch operation performed in the display area, acquiring an action area of the first screen touch operation, and extracting feature information of the first picture in the active area.

Preferably, the processor 801 is specifically configured to: search for a second picture that matches the first picture in the search dimension from a locally stored picture library and/or a picture library stored in the network side.

Preferably, the processor 801 is specifically configured to: in the order of the matching degree from high to low, instruct the display to display the second picture that is found in a picture preview manner, the number of the second pictures being one or more.

Preferably, the processor 801 is specifically configured to: instruct the display to display the searched second image by using the N sub-windows, each sub-window is configured to display a preview view of the second image or the second image that is found based on one search dimension, each The sub-window is allowed to switch to display a preview view of the second picture or the second picture found based on the corresponding search dimension according to the sliding touch operation, where N is an integer greater than or equal to 1.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (system), and computer program products according to embodiments of the present application. It will be understood that each flow and/or block of the flowchart illustrations and/or FIG. These computer program instructions can be provided to a general purpose computer, A processor of a special purpose computer, embedded processor or other programmable data processing device to produce a machine such that instructions executed by a processor of a computer or other programmable data processing device are generated for implementation in a flow or in a flowchart A flow and/or block diagram of a device in a box or a plurality of functions specified in a plurality of blocks.

The computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device. The apparatus implements the functions specified in one or more blocks of a flow or a flow and/or block diagram of the flowchart.

These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce computer-implemented processing for execution on a computer or other programmable device. The instructions provide steps for implementing the functions specified in one or more of the flow or in a block or blocks of a flow diagram.

While the preferred embodiment of the present application has been described, it will be apparent that those skilled in the art can make further changes and modifications to the embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and the modifications and

It will be apparent to those skilled in the art that various modifications and changes can be made in the present application without departing from the spirit and scope of the application. Thus, it is intended that the present invention cover the modifications and variations of the present invention.

Claims

An image search method, comprising:

Obtaining a picture search instruction, the picture search instruction being generated according to the detected first operation for the first picture;

Determining a search dimension corresponding to the first picture;

Searching, according to the search dimension corresponding to the first picture, a second picture that matches the first picture in the search dimension;

Output the search results.
The method of claim 1, wherein the first operation for the first picture comprises:

a first screen touch operation performed in a screen display area of the first picture; or

Obtaining the operation of the first picture by scanning; or

Obtaining the operation of the first picture by shooting; or,

Pasting the first picture in the search box; or,

An operation for selecting an option for triggering a picture search in the picture operation option menu, wherein the picture operation option menu is popped up after the second screen touch operation performed in the screen display area of the first picture is detected; or ,

The operation of the user interface UI component for triggering the image search is selected in the screen display area of the first picture or in the preview view window.
The method of claim 2, wherein the first screen touch operation performed in a screen display area of the first picture comprises:

a multi-finger zoom sliding operation performed in a screen display area of the first picture; or

A sliding operation of the single finger in the set direction in the screen display area of the first picture.
The method of claim 1, wherein the determining a search dimension corresponding to the first picture comprises:

Obtaining a default search dimension, determining the default search dimension as the first image pair The search dimension that should be included, the default search dimension including one or more search dimensions; or,

Determining, in the pop-up menu of the first picture, an option for selecting a search dimension, the selected search dimension as a search dimension corresponding to the first image, wherein the pop-up menu is in the first A third screen touch operation performed in a screen display area of a picture is detected and popped up, and one search dimension option in the popup menu corresponds to one or more search dimensions; or

Determining, according to the detected type of the first operation, and the corresponding relationship between the type of the first operation and the search dimension, the search dimension corresponding to the detected first operation, where the first operation corresponds to one or more Search for dimensions.
The method of claim 1 or 4, wherein the search dimension comprises one or more of the following dimensions:

a search dimension based on an object in the image;

Search dimension based on the hue of the image;

Search dimension based on the shooting time of the picture;

a search dimension based on the location of the image;

The search dimension of a photo-based device or source.
The method according to claim 5, wherein when the search dimension comprises a search dimension based on an object in a picture, the searching for the search dimension according to the search dimension corresponding to the first picture The second picture matching the first picture includes:

Extracting feature information of the first picture;

And searching for the second picture according to the extracted feature information, where the feature information of the second picture matches the feature information of the first picture.
The method according to claim 6, wherein the first operation for the first picture is a first screen touch operation performed in a screen display area of the first picture, the according to the A pixel of a picture, extracting feature information of the first picture, including:

Acquiring an action area of the first screen touch operation according to a first screen touch operation performed in a screen display area of the first picture;

Extracting feature information of the first picture in the active area.
The method of claim 1, wherein the finding a second picture that matches the first picture in the search dimension comprises:

A second picture matching the first picture on the search dimension is found from a locally stored picture library and/or a picture library stored on the network side.
The method of claim 1 wherein said outputting a search result comprises:

The found second picture is displayed in a picture preview manner according to the order of matching degree from high to low, and the number of the second pictures is one or more.
The method of claim 1 wherein said outputting a search result comprises:

Displaying the found second image by using N sub-windows, each sub-window is for displaying a preview view of the second image or the second image found based on one search dimension, each sub-window being allowed to switch display according to the sliding touch operation based on the corresponding search A preview view of the second picture or the second picture found by the dimension, where N is an integer greater than or equal to 1.
A picture search device, comprising:

An acquiring module, configured to acquire a picture search instruction, where the picture search instruction is generated according to the detected first operation for the first picture;

a determining module, configured to determine a search dimension corresponding to the first picture;

a searching module, configured to search, according to the search dimension corresponding to the first picture, a second picture that matches the first picture in the search dimension;

Output module for outputting the search results.
The apparatus according to claim 11, wherein the first operation for the first picture comprises:

a first screen touch operation performed in a screen display area of the first picture; or

Obtaining the operation of the first picture by scanning; or

Obtaining the operation of the first picture by shooting; or,

Pasting the first picture in the search box; or,

An operation for selecting an option for triggering a picture search in the picture operation option menu, wherein the picture operation option menu is popped up after the second screen touch operation performed in the screen display area of the first picture is detected; or ,

The operation of the user interface UI component for triggering the image search is selected in the screen display area of the first picture or in the preview view window.
The device according to claim 12, wherein the first screen touch operation performed in a screen display area of the first picture comprises:

a multi-finger zoom sliding operation performed in a screen display area of the first picture; or

A sliding operation of the single finger in the set direction in the screen display area of the first picture.
The device according to claim 11, wherein the determining module is specifically configured to:

Obtaining a default search dimension, and determining the default search dimension as a search dimension corresponding to the first image, where the default search dimension includes one or more search dimensions; or

Determining, in the pop-up menu of the first picture, an option for selecting a search dimension, the selected search dimension as a search dimension corresponding to the first image, wherein the pop-up menu is in the first A third screen touch operation performed in a screen display area of a picture is detected and popped up, and one search dimension option in the popup menu corresponds to one or more search dimensions; or

Determining, according to the detected type of the first operation, and the corresponding relationship between the type of the first operation and the search dimension, the search dimension corresponding to the detected first operation, where the first operation corresponds to one or more Search for dimensions.
The apparatus of claim 11 or 14, wherein the search dimension comprises one or more of the following dimensions:

a search dimension based on an object in the image;

Search dimension based on the hue of the image;

Search dimension based on the shooting time of the picture;

a search dimension based on the location of the image;

The search dimension of a photo-based device or source.
The device according to claim 15, wherein the search module is specifically configured to:

Extracting, according to the search dimension of the object in the picture, the feature information of the first picture, and searching for the second picture according to the extracted feature information, the feature information of the second picture and the first picture The feature information matches.
The device according to claim 16, wherein the search module is specifically configured to:

The first operation for the first picture is a first screen touch operation performed in a screen display area of the first picture when the first screen touch operation is performed in a screen display area of the first picture Obtaining an action area of the first screen touch operation, and extracting feature information of the first picture in the active area.
The device according to claim 11, wherein the search module is specifically configured to:

A second picture matching the first picture on the search dimension is found from a locally stored picture library and/or a picture library stored on the network side.
The device according to claim 11, wherein the output module is specifically configured to:

The found second picture is displayed in a picture preview manner according to the order of matching degree from high to low, and the number of the second pictures is one or more.
The device according to claim 11, wherein the output module is specifically configured to:

Displaying the found second image by using N sub-windows, each sub-window is for displaying a preview view of the second image or the second image found based on one search dimension, each sub-window being allowed to switch display according to the sliding touch operation based on the corresponding search A preview view of the second picture or the second picture found by the dimension, where N is an integer greater than or equal to 1.
A terminal, comprising:

Display

a memory for storing computer program instructions;

a processor coupled to the memory for reading computer program instructions stored by the memory and, in response, performing the following operations:

Obtaining a picture search instruction, the picture search instruction being generated according to the detected first operation for the first picture;

Determining a search dimension corresponding to the first picture;

Searching, according to the search dimension corresponding to the first picture, a second picture that matches the first picture in the search dimension;

Output the search results.