CN111506752A

CN111506752A - Search method, search device, electronic equipment and computer storage medium

Info

Publication number: CN111506752A
Application number: CN201910091414.2A
Authority: CN
Inventors: 王晓振
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2019-01-30
Filing date: 2019-01-30
Publication date: 2020-08-07

Abstract

The embodiment of the invention provides a searching method, a searching device, electronic equipment and a computer storage medium, wherein the searching method comprises the following steps: acquiring a marking picture of a target object subjected to marking operation; generating search information according to the marked picture, wherein the search information carries picture information of the marked picture and mark indication information used for indicating the marking operation; and sending the search information to search for the target object. By the embodiment of the invention, the accuracy of picture searching can be improved, and more accurate searching results can be obtained.

Description

Search method, search device, electronic equipment and computer storage medium

Technical Field

The embodiment of the invention relates to the technical field of computers, in particular to a searching method, a searching device, electronic equipment and a computer storage medium.

Background

The picture search is a way to obtain corresponding information through pictures, and with the development of search technology, more and more search tools or applications provide a picture search function.

The conventional picture search usually only uploads one picture, and the server identifies the main content in the picture after receiving the picture and uses the identification result as further information search. However, if the main content of the uploaded picture is large, or the target object that the user wants to search occupies a small amount of the uploaded picture due to various reasons, the server may not be effectively identified in the identification process because the target object that the user wants to search is not obvious in the uploaded picture, and the final search result is affected.

Therefore, how to improve the accuracy of image search is an urgent problem to be solved.

Disclosure of Invention

In view of the above, embodiments of the present invention provide a search scheme to solve the above problems.

According to a first aspect of the embodiments of the present invention, there is provided a search method, including: acquiring a marking picture of a target object subjected to marking operation; generating search information according to the marked picture, wherein the search information carries picture information of the marked picture and mark indication information used for indicating the marking operation; and sending the search information to search for the target object.

According to a second aspect of the embodiments of the present invention, there is provided another search method, including: receiving search information, wherein the search information carries picture information of a picture to be searched and mark indication information used for indicating whether mark operation is carried out on the picture to be searched; if the marking operation is determined to be carried out on the picture to be searched according to the marking indication information, acquiring an image area marked by the marking operation; and determining the information of the target object to be searched according to the image information of the image area, and searching the target object according to the information of the target object.

According to a third aspect of the embodiments of the present invention, there is provided a search apparatus including: the first acquisition module is used for acquiring a marking picture which is used for marking a target object; a generating module, configured to generate search information according to the tagged picture, where the search information carries picture information of the tagged picture and tag indication information used for indicating the tagging operation; and the sending module is used for sending the searching information to search the target object.

According to a fourth aspect of the embodiments of the present invention, there is provided another search apparatus including: the third receiving module is used for receiving search information, wherein the search information carries picture information of a picture to be searched and mark indicating information used for indicating whether a mark operation is carried out on the picture to be searched; the second obtaining module is used for obtaining an image area marked by the marking operation if the marking operation is determined to be carried out on the picture to be searched according to the marking indication information; the determining module is used for determining the information of the target object to be searched according to the image information of the image area; and the searching module is used for searching the target object according to the information of the target object.

According to a fifth aspect of embodiments of the present invention, there is provided an electronic apparatus, including: the system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete mutual communication through the communication bus; the memory is used for storing at least one executable instruction, and the executable instruction causes the processor to execute the operation corresponding to the searching method of the first aspect or the second aspect.

According to a sixth aspect of embodiments of the present invention, there is provided a computer storage medium having stored thereon a computer program which, when executed by a processor, implements the search method according to the first or second aspect.

According to the searching scheme provided by the embodiment of the invention, the picture which marks the target object to be searched is used for searching, and the generated searching information also carries the marking indication information so as to clearly indicate that the picture is the marked picture instead of the original picture, so that the searching executing party can clearly know the target object to be searched in the picture, and further more accurate searching is carried out on the target object according to the image information which is marked by the marking operation and is related to the target object. Therefore, even if the target object occupies a small or not prominent enough area in the picture, the target object can be effectively identified, and then more accurate searching is carried out according to the target object, so that the picture searching accuracy is improved, and more accurate searching results are obtained.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in the embodiments of the present invention, and it is also possible for a person skilled in the art to obtain other drawings based on the drawings.

Fig. 1 is a flowchart illustrating steps of a searching method according to a first embodiment of the present invention;

FIG. 2 is a flowchart illustrating steps of a searching method according to a second embodiment of the present invention;

FIG. 3 is a schematic diagram of an interface including a marking operation option according to the embodiment shown in FIG. 2;

FIG. 4 is a flowchart illustrating steps of a searching method according to a third embodiment of the present invention;

FIG. 5 is a flowchart illustrating steps of a searching method according to a fourth embodiment of the present invention;

FIG. 6 is a flowchart illustrating steps of a searching method according to a fifth embodiment of the present invention;

fig. 7 is a block diagram of a search apparatus according to a sixth embodiment of the present invention;

fig. 8 is a block diagram of a search apparatus according to a seventh embodiment of the present invention;

fig. 9 is a block diagram of a search apparatus according to an eighth embodiment of the present invention;

fig. 10 is a block diagram of a searching apparatus according to a ninth embodiment of the present invention;

fig. 11 is a schematic structural diagram of an electronic device according to a tenth embodiment of the present invention;

fig. 12 is a schematic structural diagram of an electronic device according to an eleventh embodiment of the invention.

Detailed Description

In order to make those skilled in the art better understand the technical solutions in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be described clearly and completely with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all embodiments. All other embodiments obtained by a person skilled in the art based on the embodiments of the present invention shall fall within the scope of the protection of the embodiments of the present invention.

The following further describes specific implementation of the embodiments of the present invention with reference to the drawings.

Example one

Referring to fig. 1, a flowchart illustrating steps of a searching method according to a first embodiment of the present invention is shown.

The present embodiment explains a search method according to an embodiment of the present invention from the perspective of a search information sender such as a client. The searching method of the embodiment comprises the following steps:

step S102: and acquiring a marking picture of the target object subjected to marking operation.

In the embodiment of the invention, the picture for searching is marked so as to mark the target object to be searched in the picture. The target object may be any suitable object to be searched, such as a commodity, an animal, a plant, a person, a tool, and the like. By the method, on one hand, even if the picture has more information of other objects except the target object or the target object has smaller proportion, the target object can be clearly marked, so that the identification efficiency and effect of the target object are improved.

In practical applications, the specific implementation of the marking operation on the target object can be implemented by those skilled in the art in an appropriate manner according to actual needs, including but not limited to, implementing the marking on the target object in the picture by a marking tool such as a graffiti tool, or by specific marking operation options such as lines or shapes with various attributes, and the like.

Step S104: and generating search information according to the marked pictures.

The search information carries picture information for marking pictures and mark indication information for indicating the marking operation.

The marking indication information may be implemented by a person skilled in the art in any appropriate manner according to actual situations, for example, in a manner of indication mark, where 0 indicates that no marking operation has been performed, 1 indicates that a marking operation has been performed, and so on. Of course, other suitable approaches are equally applicable, such as a boolean type approach, and the like.

By the marking indication information, the picture search executing party, such as a server performing picture search, can definitely know that the received picture is subjected to marking operation, and therefore, the search executing party can be triggered to process the mark in the marked picture so as to accurately identify the marked target object.

Step S106: and sending search information to search for the target object.

After the search information is generated, the search information may be sent to the picture search executing party, so that the picture search executing party identifies the target object in the picture according to the search information, and further performs a search operation on the target object.

According to the embodiment, the picture which marks the target object to be searched is used for searching, and the generated search information also carries the mark indication information so as to clearly indicate that the picture is the marked picture instead of the original picture, so that the search executing party can clearly know the target object to be searched in the picture, and further more accurate search can be performed on the target object according to the image information which is marked by the mark operation and is related to the target object. Therefore, even if the target object occupies a small or not prominent enough area in the picture, the target object can be effectively identified, and then more accurate searching is carried out according to the target object, so that the picture searching accuracy is improved, and more accurate searching results are obtained.

The search method of the present embodiment may be performed by any suitable electronic device having data processing capabilities, including but not limited to: mobile terminals (such as mobile phones, PADs, etc.), servers, PCs, etc.

Example two

Referring to fig. 2, a flowchart illustrating steps of a searching method according to a second embodiment of the present invention is shown.

The present embodiment still explains the searching method according to the present invention from the perspective of a sender of search information, such as a client. The searching method of the embodiment comprises the following steps:

step S202: and acquiring a marking picture of the target object subjected to marking operation.

In one possible approach, this step can be implemented as: triggering a display mark operation option according to an original picture input by a user; receiving the selection of a user on a marking operation option; and acquiring a marked picture which marks the target object according to the marking operation of the user on the target object in the original picture by using the selected marking operation option.

In this way, the user may upload an original picture to be searched, or the user may take a picture of a target object to be searched through an image capture device of the electronic device held by the user, such as a camera, to form the original picture. After the user uploads the original picture or shoots the original picture, the display of the marking operation option can be triggered, so that the user can select a corresponding marking tool to mark the target object in the original picture through the marking operation option.

By the method, the operation flow of searching the related pictures is effectively utilized, the operation habit of the user is adapted, the operation burden of the user is reduced, and the efficient marking operation can be carried out on the original pictures.

Optionally, the marking operation option may include: lines of a preset width and/or a preset color. The preset width and the preset color can be set properly by those skilled in the art according to actual needs, and the embodiment of the invention is not limited thereto. The marks generated by marking the target object in the original picture are uniform through lines with preset width and/or preset color, so that the picture searching executive party can conveniently identify and process the marks, the implementation of the searching scheme of the embodiment of the invention is simplified, and the implementation cost is reduced.

An interface including a mark operation option is shown in fig. 3, an upper part of the interface shown in fig. 3 shows an original picture uploaded or taken, and a lower part of the interface shown in fig. 3 shows a mark operation option, which is set to be a red line with a thickness of 5pt in this example. When the user clicks the mark operation option, the original picture on the upper part of the interface can be manually marked, and the result of the mark operation in this example is shown as an irregular circular area in the figure. But not limited thereto, in practical applications, the marking operation option may be any suitable option, such as a set shape, or a pattern with a preset width and/or a preset color (e.g., a circle, an ellipse, a rectangle, a square, etc.).

And after the marking operation option is used for marking the target object in the original picture, the marked picture with the mark can be generated.

But not limited to, the method of performing the target object marking operation on the uploaded or shot picture is not limited to, in practical application, the marking operation on the original picture and the operation of performing the picture search by using the marked picture after the marking operation can be independent of each other, and only the mark generated by the marking operation meets a preset marking rule, such as a preset color and/or thickness. For example, the user may perform a marking operation on a target object in the original picture in advance by using an image processing tool, form a mark with a color of red and a thickness of 5pt, and generate a marked picture with the mark. Then, the user can input the tagged picture into a search box of a certain browser, and generate tag indication information indicating that the tagged picture is a picture subjected to a tagging operation through appropriate setting (such as an option of setting the tag indication information). Then, search information is generated based on the input tagged picture and the generated tag indication information to perform a picture search.

Step S204: and receiving search keywords corresponding to the marked pictures input by the user and/or receiving search voice data corresponding to the marked pictures input by the user.

This step is an optional step.

In this embodiment, in addition to the tagged picture that is subjected to the tagging operation, the user may also input a search keyword and/or search voice data for the tagged picture, so as to further describe a target object to be searched in the tagged picture. The search keywords can be obtained from words, words or sentences input by a user, and if the search keywords are obtained from the sentences, the sentences can be analyzed or the keywords can be extracted from the sentences in any appropriate mode to obtain the search keywords from the sentences; the search speech data may be a speech word, a word or a sentence input by the user, and in this case, a person skilled in the art may convert the search speech data into text data in any appropriate manner, and then perform corresponding processing or sentence analysis or keyword extraction on the text data, so as to provide information for subsequent further search.

In one possible way, a marking operation option, a search keyword input option and a search voice input option can be displayed on the same interface, wherein the marking operation option is a necessary operation option, and the search keyword input option and the search voice input option are selectable operation options. The user's operation of these options may not be in sequential order.

In another feasible mode, after the user inputs the original picture, the marking operation option can be displayed first, and after the user uses the marking operation option to mark the target object in the original picture, and generates the marked picture, the two selectable operation options of the search keyword input option and the search voice input option are displayed on another interface. The user's operations for searching for the keyword input option and searching for the voice input option may not be in order.

In another possible manner, after the user inputs the original picture, a marking operation option may be displayed first, after the user performs a marking operation on the target object in the original picture using the marking operation option to generate the marked picture, then one interface is used to display the search keyword input option, which is a selectable option, the user may select "input" or may select to directly enter "next step", no matter what operation the user performs on the search keyword input option, after the operation is completed, another interface may be used to display the search voice input option, which is also a selectable option, and the user may select "input" or may select to directly enter "next step".

Thereby, input of auxiliary information for target object search is achieved. However, in practical applications, step S206 may be directly executed without executing this step.

Step S206: and generating search information according to the marked pictures.

The search information at least carries picture information for marking pictures and mark indication information for indicating the marking operation.

If the user inputs the search keyword and/or the search speech data as described in step S204, optionally, this step may include: and generating search information according to the marked picture and at least one of the search keyword and the search voice data.

In this case, the search information carries at least one of the following information in addition to the picture information of the marked picture and the mark indication information: the search key words are used for indicating that the search information carries the information of the search key words; and searching voice data and indicating that the search information carries the information of the searched voice data. By carrying the information, a search executing party can definitely acquire the information related to the target object to be searched, and further trigger the operation of acquiring the information, so that the final accurate search of the target object is realized.

Optionally, the search information may further carry information of an original picture corresponding to the marked picture, so that if the mark generated by the marking operation covers part of the image information of the target object, the original picture is obtained according to the information of the original picture, so as to restore the covered part of the image information. Of course, without being limited thereto, the search execution side may perform the image restoration operation directly according to the information of the mark generated by the marking operation, such as the width and/or color of the mark line, to restore the covered partial image information.

Step S208: and sending search information to search for the target object.

Sending the search information to a search executing party, such as a server, so that the server obtains information related to the target object through the search information, including but not limited to: information of a target object, search keywords, search voice data, and the like, to perform accurate target object search according to the information.

Step S210: and receiving a search result obtained according to the information of the target object in the image area marked by the marking operation.

After obtaining the information of the corresponding target object, the search execution party, such as the server, may perform the search operation of the target object according to the information, and after obtaining the search result, return the search result to the sender of the search information, such as the client, where the sender of the search information performs further processing, such as display processing, and so on.

EXAMPLE III

Referring to fig. 4, a flowchart illustrating steps of a searching method according to a third embodiment of the present invention is shown.

The present embodiment explains a search method according to an embodiment of the present invention from the viewpoint of a search executing party such as a server. The searching method of the embodiment comprises the following steps:

step S302: search information is received.

The searching information carries picture information of the picture to be searched and mark indicating information used for indicating whether marking operation is carried out on the picture to be searched. Through the search information, a search for a target object in a picture to be searched can be requested.

Wherein the mark indication information may be indication information in any appropriate form, including but not limited to: a boolean type of indication information, indication information represented using 0 and 1, indication information represented using preset characters, and the like.

Step S304: and if the marking operation of the picture to be searched is determined according to the marking indication information, acquiring an image area marked by the marking operation.

Generally, the marking operation is used for marking a target object in a picture to be searched so as to perform accurate search on the target object. If the marking indication information indicates that the marking operation is performed on the picture to be searched, the search executing party can identify the mark generated by the marking operation and further acquire the marked image area, wherein the image area is usually the area where the target object is located, the target object has a large proportion and is concentrated in information in the image area. The identification of the mark and the information extraction of the marked image area can be realized by those skilled in the art in any appropriate manner according to actual needs, such as a convolutional neural network, and the like, which is not limited in this embodiment of the present invention.

Step S306: and determining the information of the target object to be searched according to the image information of the image area, and searching the target object according to the information of the target object.

After the image information of the image area is acquired, the target object in the image area can be identified, so that the information of the target object is acquired. Further, a search for the target object may be performed based on the information of the target object.

For example, the original picture is a picture of a person wearing a down jacket, if the down jacket is marked by the marking operation, information search can be performed on the down jacket, if the down jacket is a browser application, a brand, a price, a selling website or a merchant of the down jacket can be searched, and if the down jacket is a shopping application, information of the down jacket which is sold can be searched.

According to the embodiment, the marked image area is obtained from the picture to be searched according to the mark indication information, and then the information of the target object to be searched is determined from the image information of the image area, so that the target object can be accurately searched. Therefore, the searching executing party can definitely know the target object to be searched in the picture, and can effectively identify the target object even if the target object occupies a small amount or is not prominent enough in the picture, and then more accurate searching is carried out according to the target object, so that the accuracy of picture searching is improved, and more accurate searching results are obtained.

Example four

Referring to fig. 5, a flowchart illustrating steps of a searching method according to a fourth embodiment of the present invention is shown.

The present embodiment still describes the searching method according to the present invention from the perspective of a search executing party, such as a server. The searching method of the embodiment comprises the following steps:

step S402: search information is received.

The searching information at least carries picture information of the picture to be searched and mark indicating information used for indicating whether marking operation is carried out on the picture to be searched.

Optionally, the search information may further carry at least one of the following information: a search keyword, keyword indication information indicating whether the search keyword exists, search voice data, voice indication information indicating whether the search voice data exists. The search keywords and the search voice data are auxiliary information for further explaining a target object to be searched in the picture to be searched, and the target object can be searched more accurately through the auxiliary information. For example, if the target object to be searched is a down jacket in the picture to be searched, the search keyword may be any appropriate keyword, such as at least one of a brand keyword, a price keyword, a category keyword, and the like; and the search voice data can be a voice description of the down jacket, such as at least one of a brand voice description, a usage voice description, a category voice description, and the like.

Optionally, if the marking indication information indicates that the marking operation is performed on the picture to be searched, the search information may also carry information of an original picture corresponding to the picture to be searched, so that when it is determined that the mark generated by the marking operation covers partial image information of the target object, the original picture is acquired, and the covered partial image information is restored.

Step S404: and if the marking operation of the picture to be searched is determined according to the marking indication information, acquiring the image area marked by the marking operation.

In a feasible mode, if the marking operation is determined to be carried out on the picture to be searched according to the marking indication information, carrying out marking identification on the picture to be searched; and acquiring the image area marked by the marking operation according to the marking identification result. Because the marking operation has a certain regularity, the mark generated by the marking operation also has a certain regularity, such as a line with a certain width and/or color, and so on. Accordingly, the mark in the picture to be searched can be identified to determine the image area marked by the mark, and the main body part in the image area should be the target object.

In addition, when the picture to be searched is a picture subjected to the marking operation, and the search information further carries information of an original picture corresponding to the picture to be searched, the obtaining of the image area marked by the marking operation according to the marking identification result may include: if the mark generated by the marking operation covers partial image information of the target object is determined according to the mark identification result, acquiring a corresponding original image according to the information of the original image; restoring partial image information of the marked and covered target object according to the image information of the original image; and acquiring the image area marked by the marking operation according to the marking identification result and the reduction result. In this way, on one hand, effective processing when partial image information of the target object is accidentally covered is realized; and on the other hand, the operation and implementation complexity of image information restoration are also simplified.

By the mode, effective determination and acquisition of the image area marked by the marking operation are achieved.

Step S406: and determining the information of the target object to be searched according to the image information of the image area, and searching the target object according to the information of the target object.

After the image area marked by the marking operation is determined, the image information of the image area can be acquired, and then the information of the target object in the image area is determined, so that the target object is searched.

As described above, the search information may optionally include at least one of a search keyword, keyword indication information indicating whether the search keyword exists, search voice data, and voice indication information indicating whether the search voice data exists, in addition to the picture information and the mark indication information of the picture to be searched.

Based on this, the target object search according to the information of the target object can be realized in one of the following ways:

in the first mode, if the search information carries the search keyword and/or the search voice data, the target object is searched according to the information of the target object and the search keyword and/or the search voice data. In this case, besides the picture information and the mark indication information of the picture to be searched, the search information also carries at least one of a search keyword and search voice data, and the search keyword and/or the search voice data can be directly obtained from the search information to assist in explaining the target object and promote further accurate search. Search keywords and/or search voice data are directly carried in the search information, and the data acquisition speed and efficiency can be improved.

And secondly, if the search keyword is determined to exist according to the keyword indication information and/or the search voice data is determined to exist according to the voice indication information, acquiring the search keyword and/or the search voice data, and searching the target object according to the information of the target object and the search keyword and/or the search voice data. In this case, besides the picture information and the mark indication information of the picture to be searched, the search information also carries at least one of keyword indication information and voice indication information, and when the search keyword is determined to exist according to the keyword indication information and/or the search voice data is determined to exist according to the voice indication information, the search keyword and/or the search voice data is obtained. If the search keyword and/or the search voice data are determined to exist, a corresponding data request can be sent to request the search information sender to send the search keyword and/or the search voice data again; of course, the search keyword and/or the search speech data may also be directly carried in the search information for direct acquisition. In the mode, whether the subsequent acquisition operation is carried out or not is determined through the corresponding indication information, the search information does not need to be integrally detected, the data processing load is reduced, and the data processing efficiency is improved. In addition, if the search keyword and/or the search voice data are not carried in the search information, the data transmission and interaction burden is also reduced.

And if the search information carries the search keyword and the voice indication information indicates that the search voice data exists, acquiring the search voice data and searching the target object according to the information of the target object, the search keyword and the search voice data. In this case, besides the picture information and the mark indication information of the picture to be searched, the search information also carries search keywords and voice indication information, and when the voice indication information indicates that the search voice data exists, the request is sent again to request the search information sender to acquire the search voice data. The search voice data and the search keywords are transmitted separately, so that effective information can be rapidly acquired, and data transmission and interaction burden can be reduced.

And if the search information carries search voice data and the keyword indication information indicates that the search keyword exists, acquiring the search keyword, and searching for the target object according to the information of the target object, the search voice data and the search keyword. In this case, besides the picture information and the mark indication information of the picture to be searched, the search information also carries search voice data and keyword indication information, and when the keyword indication information indicates that the search keyword exists, the request is sent again to request the search information sender to acquire the search keyword. The search voice data and the search keywords are transmitted separately, so that effective information can be rapidly acquired, and data transmission and interaction burden can be reduced.

Through the multiple modes, the accurate search of the target object can be realized based on the acquired effective information.

Step S408: and returning the search result of the target object search to the sender of the search information.

For example, after obtaining the search result corresponding to the target object, the search execution party, such as the server, may return the search result to the sender of the search information, such as the client.

EXAMPLE five

Referring to fig. 6, a flowchart illustrating steps of a searching method according to a fifth embodiment of the present invention is shown.

In this embodiment, a search execution party is taken as a server, a search information sending party is taken as a client, and a search method according to an embodiment of the present invention is described in a specific example form from the perspective of interaction between a user, the client, and the server. The searching method of the embodiment comprises the following steps:

step S502: the user starts the picture search process and inputs the original picture through the search input box.

Step S504: the client receives the original picture, displays a doodle tool and prompts a user that key articles (namely target objects) to be searched in the original picture can be marked in a doodle mode.

In this embodiment, a doodle tool is used for marking a key item to be searched in an original picture.

Step S506: the user marks the key items in the original picture by using a doodle tool and inputs the doodle picture (namely, the marked picture).

Step S508: the client receives the graffiti picture and prompts the user to input text (i.e., search keywords) and/or voice (i.e., search voice data) to improve the accuracy of the search results.

Step S510: the user enters text and/or speech.

Step S512: and the client submits the scrawled picture, the input characters and/or voice and corresponding various indication information to the server together.

Wherein the various indication information at least comprises: the image processing method comprises the steps of marking indication information for indicating that scrawling is carried out on an original image, keyword indication information for indicating that characters input by a user exist and/or voice indication information for indicating that voice input by the user exists.

Optionally, the client may also submit the original pictures to the server together.

Step S514: the server accurately identifies the key articles to be searched from the pictures after the scrawling according to the scrawling position, further accurately searches the range through the input characters and/or voice, and obtains the search result.

Step S516: and the server returns the search result to the client.

Step S518: and the client displays the search result.

According to the method and the device for searching the image, more user searching conditions can be obtained according to the scrawled image and the characters and/or voice input by the user, the image searching range is reduced, and the searching result is more accurate. Even if the target object to be searched occupies a small or not prominent enough amount in the picture, the target object can be effectively identified and searched, and the accuracy and the efficiency of picture searching are improved.

EXAMPLE six

Referring to fig. 7, a block diagram of a search apparatus according to a sixth embodiment of the present invention is shown.

The search apparatus of this embodiment may be disposed at a client, and the search apparatus may include: a first obtaining module 602, configured to obtain a tagged picture obtained by tagging a target object; a generating module 604, configured to generate search information according to the marked picture, where the search information carries picture information of the marked picture and mark indication information used for indicating the marking operation; a sending module 606, configured to send search information to perform target object search.

The searching apparatus of this embodiment is used to implement the corresponding searching method in the foregoing multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, which are not described herein again. In addition, the functional implementation of each module in the search apparatus of this embodiment can refer to the description of the corresponding part in the foregoing method embodiment, and is not repeated here.

EXAMPLE seven

Referring to fig. 8, a block diagram of a search apparatus according to a seventh embodiment of the present invention is shown.

The search apparatus of this embodiment may be disposed at a client, and the search apparatus may include: a first obtaining module 702, configured to obtain a tagged picture obtained by tagging a target object; a generating module 704, configured to generate search information according to the marked picture, where the search information carries picture information of the marked picture and mark indication information used for indicating the marking operation; and a sending module 706, configured to send search information to perform target object search.

Optionally, the first obtaining module 702 is configured to trigger a display mark operation option according to an original picture input by a user; receiving the selection of the marking operation option by the user; and acquiring a marked image which is marked on a target object according to the marking operation of the user on the target object in the original image by using the selected marking operation option.

Optionally, the tagging operation option includes: lines of a preset width and/or a preset color.

Optionally, the search apparatus of this embodiment further includes: a first receiving module 708, configured to receive a search keyword corresponding to a tagged picture input by a user and/or receive search voice data corresponding to the tagged picture input by the user; the generating module 704 is configured to generate search information according to the tagged picture and at least one of a search keyword and a search voice data.

Optionally, the search information further carries at least one of the following information: the search keyword is used for indicating that the search information carries information of the search keyword; the search voice data is used for indicating that the search information carries the information of the search voice data.

Optionally, the search information further carries information of an original picture corresponding to the marked picture.

Optionally, the search apparatus of this embodiment further includes: a second receiving module 710, configured to receive a search result obtained according to the information of the target object in the image region marked by the marking operation.

Example eight

Referring to fig. 9, a block diagram of a search apparatus according to an eighth embodiment of the present invention is shown.

The search apparatus of this embodiment may be provided in a server, and the search apparatus may include: a third receiving module 802, configured to receive search information, where the search information carries picture information of a picture to be searched and flag indication information used to indicate whether a flag operation is performed on the picture to be searched; a second obtaining module 804, configured to obtain an image area marked by the marking operation if it is determined that the marking operation is performed on the picture to be searched according to the marking indication information; a determining module 806, configured to determine information of a target object to be searched according to the image information of the image area; and the searching module 808 is configured to perform target object searching according to the information of the target object.

Example nine

Referring to fig. 10, a block diagram of a search apparatus according to a ninth embodiment of the present invention is shown.

The search apparatus of this embodiment may be provided in a server, and the search apparatus includes: a third receiving module 902, configured to receive search information, where the search information carries picture information of a picture to be searched and flag indication information used to indicate whether a flag operation is performed on the picture to be searched; a second obtaining module 904, configured to, if it is determined that the marking operation is performed on the picture to be searched according to the marking indication information, obtain an image area marked by the marking operation; a determining module 906, configured to determine information of a target object to be searched according to the image information of the image area; a searching module 908, configured to perform target object search according to the information of the target object.

Optionally, the second obtaining module 904 includes: the identification module 9042 is configured to perform mark identification on the picture to be searched if it is determined that the mark operation is performed on the picture to be searched according to the mark indication information; and the region module 9044 is configured to acquire an image region marked by the marking operation according to a marking identification result.

Optionally, when the picture to be searched is a picture subjected to a marking operation, the search information also carries information of an original picture corresponding to the picture to be searched; the region module 9044 is configured to, if it is determined that the mark generated by the marking operation covers partial image information of the target object according to a mark identification result, obtain a corresponding original image according to information of the original image; restoring partial image information of the target object covered by the mark according to the image information of the original picture; and acquiring the image area marked by the marking operation according to the marking identification result and the reduction result.

Optionally, the search information further carries at least one of the following information: the search keyword, keyword indication information indicating whether the search keyword exists, the search voice data, voice indication information indicating whether the search voice data exists.

Optionally, the search module 908 comprises: a first executing module 9082, configured to, if the search information carries the search keyword and/or the search voice data, perform target object search according to the information of the target object and the search keyword and/or the search voice data; or, the second executing module 9084 is configured to, if it is determined that the search keyword exists according to the keyword indication information and/or it is determined that the search voice data exists according to the voice indication information, acquire the search keyword and/or the search voice data, and perform target object search according to the information of the target object and the search keyword and/or the search voice data; or, the third executing module 9086 is configured to, if the search information carries the search keyword and the voice indication information indicates that the search voice data exists, obtain the search voice data, and perform target object search according to the information of the target object, the search keyword, and the search voice data; or, the fourth executing module 9088 is configured to, if the search information carries the search voice data and the keyword indication information indicates that the search keyword exists, obtain the search keyword, and perform target object search according to the information of the target object, the search voice data, and the search keyword.

Optionally, the search apparatus of this embodiment further includes: a returning module 910, configured to return the search result of the target object search to the sender of the search information.

Example ten

Referring to fig. 11, a schematic structural diagram of an electronic device according to a tenth embodiment of the present invention is shown, and the specific embodiment of the present invention does not limit the specific implementation of the electronic device.

As shown in fig. 11, the electronic device may include: a processor (processor)1002, a Communications Interface 1004, a memory 1006, and a Communications bus 1008.

Wherein:

the processor 1002, communication interface 1004, and memory 1006 communicate with each other via a communication bus 1008.

A communication interface 1004 for communicating with other electronic devices, such as other electronic terminals or servers.

The processor 1002 is configured to execute the program 1010, and may specifically perform relevant steps in the above search method embodiment.

In particular, the program 1010 may include program code that includes computer operating instructions.

The processor 1002 may be a central processing unit CPU, or an application specific Integrated circuit (asic), or one or more Integrated circuits configured to implement an embodiment of the present invention. The intelligent device comprises one or more processors which can be the same type of processor, such as one or more CPUs; or may be different types of processors such as one or more CPUs and one or more ASICs.

The memory 1006 is used for storing the program 1010. The memory 1006 may comprise high-speed RAM memory, and may also include non-volatile memory (non-volatile memory), such as at least one disk memory.

The program 1010 may be specifically configured to cause the processor 1002 to perform the following operations: acquiring a marking picture of a target object subjected to marking operation; generating search information according to the marked picture, wherein the search information carries picture information of the marked picture and mark indication information used for indicating the marking operation; and sending the search information to search for the target object.

In an alternative embodiment, the program 1010 is further configured to enable the processor 1002, when acquiring a tagged picture obtained by tagging a target object, to trigger presentation of a tagging operation option according to an original picture input by a user; receiving the selection of the marking operation option by the user; and acquiring a marked image which is marked on a target object according to the marking operation of the user on the target object in the original image by using the selected marking operation option.

In an alternative embodiment, the tagging operation option comprises: lines of a preset width and/or a preset color.

In an alternative embodiment, the program 1010 is further configured to enable the processor 1002 to receive a search keyword corresponding to the tagged picture input by a user, and/or receive search voice data corresponding to the tagged picture input by the user; the program 1010 is also configured to cause the processor 1002 to generate search information from the tag picture and at least one of the search keyword and the search voice data when generating search information from the tag picture.

In an optional implementation manner, the search information further carries at least one of the following information: the search keyword is used for indicating that the search information carries information of the search keyword; the search voice data is used for indicating that the search information carries the information of the search voice data.

In an optional implementation manner, the search information further carries information of an original picture corresponding to the tagged picture.

In an alternative embodiment, the program 1010 is further configured to enable the processor 1002 to receive a search result obtained according to the information of the target object in the image area marked by the marking operation.

For specific implementation of each step in the program 1010, reference may be made to corresponding steps and corresponding descriptions in units in the foregoing search method embodiments, which are not described herein again. It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described devices and modules may refer to the corresponding process descriptions in the foregoing method embodiments, and are not described herein again.

By the electronic device of the embodiment, a picture obtained by marking a target object to be searched is used for searching, and the generated search information also carries marking indication information to clearly indicate that the picture is a marked picture instead of an original picture, so that a search executing party can clearly know the target object to be searched in the picture, and further more accurate search is performed on the target object according to image information which is marked by the marking operation and is related to the target object. Therefore, even if the target object occupies a small or not prominent enough area in the picture, the target object can be effectively identified, and then more accurate searching is carried out according to the target object, so that the picture searching accuracy is improved, and more accurate searching results are obtained.

EXAMPLE eleven

Referring to fig. 12, a schematic structural diagram of an electronic device according to an eleventh embodiment of the present invention is shown, and the specific embodiment of the present invention does not limit the specific implementation of the electronic device.

As shown in fig. 12, the electronic device may include: a processor (processor)1102, a communication Interface 1104, a memory 1106, and a communication bus 1108.

Wherein:

the processor 1102, communication interface 1104, and memory 1106 communicate with one another via a communication bus 1108.

A communication interface 1104 for communicating with other electronic devices, such as other servers or electronic terminals.

The processor 1102 is configured to execute the program 1110, and may specifically perform relevant steps in the above search method embodiment.

In particular, the program 1110 can include program code that includes computer operating instructions.

The processor 1102 may be a central processing unit CPU, or an Application Specific Integrated Circuit (ASIC), or one or more Integrated circuits configured to implement an embodiment of the present invention. The intelligent device comprises one or more processors which can be the same type of processor, such as one or more CPUs; or may be different types of processors such as one or more CPUs and one or more ASICs.

A memory 1106 for storing a program 1110. Memory 1106 may comprise high-speed RAM memory and may also include non-volatile memory (non-volatile memory), such as at least one disk memory.

The program 1110 may be specifically configured to cause the processor 1102 to perform the following operations: receiving search information, wherein the search information carries picture information of a picture to be searched and mark indication information used for indicating whether mark operation is carried out on the picture to be searched; if the marking operation is determined to be carried out on the picture to be searched according to the marking indication information, acquiring an image area marked by the marking operation; and determining the information of the target object to be searched according to the image information of the image area, and searching the target object according to the information of the target object.

In an optional implementation manner, the program 1110 is further configured to enable the processor 1102 to perform mark identification on the picture to be searched if it is determined that the mark operation is performed on the picture to be searched according to the mark indication information, and when the image area marked by the mark operation is acquired, determine that the mark operation is performed on the picture to be searched according to the mark indication information; and acquiring the image area marked by the marking operation according to the marking identification result.

In an optional implementation manner, when the picture to be searched is a picture subjected to a marking operation, the search information also carries information of an original picture corresponding to the picture to be searched; the program 1110 is further configured to, when acquiring an image region marked by the marking operation according to a mark identification result, if it is determined that a mark generated by the marking operation covers partial image information of the target object according to the mark identification result, the processor 1102 acquires a corresponding original picture according to information of the original picture; restoring partial image information of the target object covered by the mark according to the image information of the original picture; and acquiring the image area marked by the marking operation according to the marking identification result and the reduction result.

In an optional implementation manner, the search information further carries at least one of the following information: the search keyword, keyword indication information indicating whether the search keyword exists, the search voice data, voice indication information indicating whether the search voice data exists.

In an optional implementation manner, the program 1110 is further configured to enable the processor 1102 to perform, when performing target object search according to the information of the target object, if the search information carries the search keyword and/or the search voice data, the target object search according to the information of the target object and the search keyword and/or the search voice data; or if the search keyword is determined to exist according to the keyword indication information and/or the search voice data is determined to exist according to the voice indication information, acquiring the search keyword and/or the search voice data, and searching for the target object according to the information of the target object and the search keyword and/or the search voice data; or if the search information carries the search keyword and the voice indication information indicates that the search voice data exists, acquiring the search voice data and searching for a target object according to the information of the target object, the search keyword and the search voice data; or, if the search information carries the search voice data and the keyword indication information indicates that the search keyword exists, acquiring the search keyword, and searching for the target object according to the information of the target object, the search voice data and the search keyword.

In an alternative embodiment, program 1110 is further configured to cause processor 1102 to return search results of the target object search to the sender of the search information.

For specific implementation of each step in the program 1110, reference may be made to corresponding steps and corresponding descriptions in units in the foregoing search method embodiments, which are not described herein again. It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described devices and modules may refer to the corresponding process descriptions in the foregoing method embodiments, and are not described herein again.

Through the electronic equipment of the embodiment, the marked image area is acquired from the picture to be searched according to the mark indication information, and then the information of the target object to be searched is determined from the image information of the image area, so that the target object is accurately searched. Therefore, the searching executing party can definitely know the target object to be searched in the picture, and can effectively identify the target object even if the target object occupies a small amount or is not prominent enough in the picture, and then more accurate searching is carried out according to the target object, so that the accuracy of picture searching is improved, and more accurate searching results are obtained.

It should be noted that, according to the implementation requirement, each component/step described in the embodiment of the present invention may be divided into more components/steps, and two or more components/steps or partial operations of the components/steps may also be combined into a new component/step to achieve the purpose of the embodiment of the present invention.

The above-described method according to an embodiment of the present invention may be implemented in hardware, firmware, or as software or computer code storable in a recording medium such as a CD ROM, a RAM, a floppy disk, a hard disk, or a magneto-optical disk, or as computer code originally stored in a remote recording medium or a non-transitory machine-readable medium downloaded through a network and to be stored in a local recording medium, so that the method described herein may be stored in such software processing on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware such as an ASIC or FPGA. It will be appreciated that the computer, processor, microprocessor controller or programmable hardware includes memory components (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code that, when accessed and executed by the computer, processor or hardware, implements the search methods described herein. Further, when a general-purpose computer accesses code for implementing the search methods shown herein, execution of the code transforms the general-purpose computer into a special-purpose computer for performing the search methods shown herein.

Those of ordinary skill in the art will appreciate that the various illustrative elements and method steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present embodiments.

The above embodiments are only for illustrating the embodiments of the present invention and not for limiting the embodiments of the present invention, and those skilled in the art can make various changes and modifications without departing from the spirit and scope of the embodiments of the present invention, so that all equivalent technical solutions also belong to the scope of the embodiments of the present invention, and the scope of patent protection of the embodiments of the present invention should be defined by the claims.

Claims

1. A search method, comprising:

acquiring a marking picture of a target object subjected to marking operation;

generating search information according to the marked picture, wherein the search information carries picture information of the marked picture and mark indication information used for indicating the marking operation;

and sending the search information to search for the target object.

2. The method of claim 1, wherein the obtaining of the marking picture of the target object subjected to the marking operation comprises:

triggering a display mark operation option according to an original picture input by a user;

receiving the selection of the marking operation option by the user;

and acquiring a marked image which is marked on a target object according to the marking operation of the user on the target object in the original image by using the selected marking operation option.

3. The method of claim 2, wherein the tagging operation option comprises: lines of a preset width and/or a preset color.

4. The method according to any one of claims 1-3, wherein the method further comprises:

receiving search keywords corresponding to the marked pictures and input by a user, and/or receiving search voice data corresponding to the marked pictures and input by the user;

the generating of the search information according to the marked picture comprises: and generating search information according to the marked picture and at least one of the search keyword and the search voice data.

5. The method of claim 4, wherein the search information further carries at least one of the following information:

the search keyword is used for indicating that the search information carries information of the search keyword; the search voice data is used for indicating that the search information carries the information of the search voice data.

6. The method according to any one of claims 1 to 3, wherein the search information further carries information of an original picture corresponding to the tagged picture.

7. The method according to any one of claims 1-3, wherein the method further comprises:

and receiving a search result obtained according to the information of the target object in the image area marked by the marking operation.

8. A search method, comprising:

receiving search information, wherein the search information carries picture information of a picture to be searched and mark indication information used for indicating whether mark operation is carried out on the picture to be searched;

if the marking operation is determined to be carried out on the picture to be searched according to the marking indication information, acquiring an image area marked by the marking operation;

and determining the information of the target object to be searched according to the image information of the image area, and searching the target object according to the information of the target object.

9. The method according to claim 8, wherein if it is determined that the marking operation is performed on the picture to be searched according to the marking indication information, acquiring an image area marked by the marking operation includes:

if the marking operation of the picture to be searched is determined according to the marking indication information, marking identification is carried out on the picture to be searched;

and acquiring the image area marked by the marking operation according to the marking identification result.

10. The method according to claim 9, wherein when the picture to be searched is a picture subjected to a marking operation, the search information further carries information of an original picture corresponding to the picture to be searched;

the acquiring the image area marked by the marking operation according to the marking identification result comprises: if the mark generated by the marking operation covers partial image information of the target object according to the mark identification result, acquiring a corresponding original picture according to the information of the original picture; restoring partial image information of the target object covered by the mark according to the image information of the original picture; and acquiring the image area marked by the marking operation according to the marking identification result and the reduction result.

11. The method according to any one of claims 8-10, wherein the search information further carries at least one of the following information:

the search keyword, keyword indication information indicating whether the search keyword exists, the search voice data, voice indication information indicating whether the search voice data exists.

12. The method of claim 11, wherein the performing a target object search according to the information of the target object comprises:

if the search information carries the search keyword and/or the search voice data, searching for a target object according to the information of the target object and the search keyword and/or the search voice data;

alternatively, the first and second electrodes may be,

if the search keyword is determined to exist according to the keyword indication information and/or the search voice data is determined to exist according to the voice indication information, obtaining the search keyword and/or the search voice data, and searching for a target object according to the information of the target object and the search keyword and/or the search voice data;

alternatively, the first and second electrodes may be,

if the search information carries the search keyword and the voice indication information indicates that the search voice data exists, acquiring the search voice data and searching for a target object according to the information of the target object, the search keyword and the search voice data;

alternatively, the first and second electrodes may be,

and if the search information carries the search voice data and the keyword indication information indicates that the search keyword exists, acquiring the search keyword, and searching for the target object according to the information of the target object, the search voice data and the search keyword.

13. The method of claim 12, wherein the method further comprises:

and returning the search result of the target object search to the sender of the search information.

14. A search apparatus, comprising:

the first acquisition module is used for acquiring a marking picture which is used for marking a target object;

a generating module, configured to generate search information according to the tagged picture, where the search information carries picture information of the tagged picture and tag indication information used for indicating the tagging operation;

and the sending module is used for sending the searching information to search the target object.

15. The device of claim 14, wherein the first obtaining module is configured to trigger a display mark operation option according to an original picture input by a user; receiving the selection of the marking operation option by the user; and acquiring a marked image which is marked on a target object according to the marking operation of the user on the target object in the original image by using the selected marking operation option.

16. The apparatus of claim 15, wherein the tagging operation option comprises: lines of a preset width and/or a preset color.

17. The apparatus of any one of claims 14-16, wherein the apparatus further comprises:

the first receiving module is used for receiving search keywords which are input by a user and correspond to the marked pictures and/or receiving search voice data which are input by the user and correspond to the marked pictures;

the generating module is used for generating search information according to the marked picture and at least one of the search keyword and the search voice data.

18. The apparatus of claim 17, wherein the search information further carries at least one of the following information:

19. The apparatus according to any one of claims 14 to 16, wherein the search information further carries information of an original picture corresponding to the tagged picture.

20. The apparatus of any one of claims 14-16, wherein the apparatus further comprises:

and the second receiving module is used for receiving a search result obtained according to the information of the target object in the image area marked by the marking operation.

21. A search apparatus, comprising:

the third receiving module is used for receiving search information, wherein the search information carries picture information of a picture to be searched and mark indicating information used for indicating whether a mark operation is carried out on the picture to be searched;

the second obtaining module is used for obtaining an image area marked by the marking operation if the marking operation is determined to be carried out on the picture to be searched according to the marking indication information;

the determining module is used for determining the information of the target object to be searched according to the image information of the image area;

and the searching module is used for searching the target object according to the information of the target object.

22. The apparatus of claim 21, wherein the second obtaining means comprises:

the identification module is used for carrying out mark identification on the picture to be searched if the mark operation on the picture to be searched is determined according to the mark indication information;

and the area module is used for acquiring the image area marked by the marking operation according to the marking identification result.

23. The apparatus according to claim 22, wherein when the picture to be searched is a picture subjected to a marking operation, the search information further carries information of an original picture corresponding to the picture to be searched;

the area module is used for acquiring a corresponding original picture according to the information of the original picture if the mark generated by the marking operation covers the partial image information of the target object according to the mark identification result; restoring partial image information of the target object covered by the mark according to the image information of the original picture; and acquiring the image area marked by the marking operation according to the marking identification result and the reduction result.

24. The apparatus according to any one of claims 21-23, wherein the search information further carries at least one of the following information:

25. The apparatus of claim 24, wherein the search module comprises:

a first execution module, configured to search for a target object according to the information of the target object and the search keyword and/or the search voice data if the search information carries the search keyword and/or the search voice data;

alternatively, the first and second electrodes may be,

a second execution module, configured to, if it is determined that the search keyword exists according to the keyword indication information and/or it is determined that the search voice data exists according to the voice indication information, obtain the search keyword and/or the search voice data, and perform target object search according to the information of the target object and the search keyword and/or the search voice data;

alternatively, the first and second electrodes may be,

a third execution module, configured to, if the search information carries the search keyword and the voice indication information indicates that the search voice data exists, obtain the search voice data, and perform target object search according to the information of the target object, the search keyword, and the search voice data;

alternatively, the first and second electrodes may be,

and the fourth execution module is configured to, if the search information carries the search voice data and the keyword indication information indicates that the search keyword exists, acquire the search keyword and perform target object search according to the information of the target object, the search voice data, and the search keyword.

26. The apparatus of claim 25, wherein the apparatus further comprises:

and the returning module is used for returning the search result of the target object search to the sender of the search information.

27. An electronic device, comprising: the system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete mutual communication through the communication bus;

the memory is used for storing at least one executable instruction which causes the processor to execute the operation corresponding to the search method according to any one of claims 1-7; alternatively, the executable instructions cause the processor to perform operations corresponding to the search method of any one of claims 8-13.

28. A computer storage medium having stored thereon a computer program which, when executed by a processor, implements a search method according to any one of claims 1 to 7; or, when executed by a processor, implement the search method of any one of claims 8-13.