TW201915787A - Search method and processing device - Google Patents

Abstract

A method including extracting an image feature vector of a target image, wherein the image feature vector is used for representing image content of the target image; and determining, in the same vector space, a text corresponding to the target image according to a correlation between the image feature vector and a text feature vector of the text, wherein the text feature vector is used for representing semantics of the text. The method solves the problems of low efficiency and high requirements on the system processing capability in the conventional techniques, thereby achieving a technical effect of easily and accurately implementing image tagging.

Description

Searching method and processing equipment

The invention belongs to the technical field of the Internet, and particularly relates to a search method and a processing device.

With the continuous development of the Internet, e-commerce and other technologies, the demand for image data is increasing. How to analyze and use image data more effectively will have a great impact on e-commerce. In the process of processing image data, recommending labels for images can more effectively implement image aggregation, image classification, image retrieval, and so on. Therefore, the demand for image data recommendation labels is increasing. For example, user A wants to search for products through image search. In this case, if the image can be automatically tagged, then after uploading the image, the user can automatically recommend category keywords and categories related to the image. Attribute keywords. Or in other scenes where image data exists, you can automatically recommend text for the image (for example, tags, etc.), without the need for artificial classification and marking. For how to mark images simply and efficiently, no effective solution has been proposed.

The object of the present invention is to provide a search method and processing device, which can simply and efficiently mark images. The present invention provides a search method and processing device implemented as follows: A search method, the method includes: capturing an image feature vector of a target image, wherein the image feature vector is used to characterize the image content of the target image; In the same vector space, the label corresponding to the target image is determined according to the correlation between the image feature vector and the text feature vector of the label, wherein the text feature vector is used to represent the semantics of the label. A processing device includes a processor and a memory for storing processor-executable instructions. When the processor executes the instructions, the processor implements: capturing an image feature vector of a target image, wherein the image feature vector is used for characterization The image content of the target image; determining the label corresponding to the target image according to the correlation between the image feature vector and the text feature vector of the label in the same vector space, wherein the text feature vector is used for Characterize the semantics of the label. A search method, the method includes: capturing image features of a target image, wherein the image features are used to characterize the image content of the target image; in the same vector space, according to the image features and text features of text The degree of correlation between them determines the text corresponding to the target image, where the text features are used to characterize the semantics of the text. A computer-readable storage medium having computer instructions stored thereon, the steps of the above method being implemented when the instructions are executed. The method and processing device for determining an image label provided by the present invention consider that a text search method based on an input target image can be used to directly search for a recommended text based on an input target image without adding an image matching operation in the matching process. , You can directly get the corresponding text by determining the correlation between the image feature vector and the text feature vector. Through the above methods, the problems of low efficiency and high system processing capability requirements of the existing recommended text methods are solved, and the technical effect of simple and accurate image tagging is achieved.

In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described The embodiments are only a part of the embodiments of the present invention, but not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by a person of ordinary skill in the art without creative efforts should fall within the protection scope of the present invention. At present, there are some methods for recommending text for images. For example: training a model that searches images to generate an image feature vector for each image. For any two images, the greater the similarity between the image feature vectors, That means the two images are more similar. Based on this principle, existing search methods generally collect an image set and control the images in the image set to cover the entire application scene as much as possible. Then, one or more images similar to the image input by the user can be determined from the image set through a search and matching method based on the image feature vector, and then the text of the one or more images is used as a text set, and the text is extracted from the text. One or more confidences are determined collectively as the recommended text for the image. This search method needs to maintain an image set covering the entire application scene. The accuracy of text recommendation depends on the size of the image set and the accuracy of the text in the image set. The text often needs to be manually labeled, which is more complicated to implement. In view of the problems in the text recommendation method of searching by images, considering that the method of searching by images can be used to directly search for the recommended text based on the input target image, without the need to add image matching in the matching process. , You can directly get the corresponding text by matching the target image, that is, you can use text search to recommend text for the target image. The above text can be short tags, long tags, specific text content, etc., which specific text content is not limited in the present invention, and can be selected according to actual needs. For example, when uploading pictures in the e-commerce scene, the text can be short tags. If in a matching system of poetry and pictures, the text can be verses, that is, different text content can be selected according to the actual application scenario. Types of. Consider that you can extract features from the image and feature from the text. Then, calculate the correlation between the image and each text in the label set based on the captured features, and determine the text of the target image according to the correlation. Based on this, a search method is provided in this example. As shown in FIG. 1, by extracting the image feature vector in the target image that is used to represent the image content of the target image, and the text feature vector in the text that is used to represent the semantics of the text To calculate the correlation between the image feature vector and the text feature vector, so as to determine the text corresponding to the target image. That is, the data of the two modalities of text and image can be converted into the feature vector of the feature in the same space through their respective encodings, and then the distance between the features can be used to measure the correlation between the text and the image. Text as the text of the target image. In one embodiment, an image may be uploaded through a client, wherein the client may be a terminal device or software used by the client for operation. Specifically, the client may be a terminal device such as a smart phone, tablet computer, notebook computer, desktop computer, smart watch, or other wearable device. Of course, the client can also be software that can run on the terminal device. For example: mobile Taobao, Alipay or browser and other applications. In one embodiment, considering the processing speed in practical applications, the text feature vector of each text can be retrieved in advance, so that after acquiring the target image, only the image feature vector of the target image needs to be retrieved, and no longer Extract the text feature vector of the text, which can avoid repeated calculations and improve processing speed and efficiency. As shown in FIG. 2, the text determined for the target image can be circled in the following ways, but is not limited to: 1) the correlation between the text feature vector and the image feature vector of the target image is greater than a preset threshold Or more text as the text corresponding to the target image; For example, the preset threshold is 0.7, that is, if the correlation between the text feature vector of one or several characters and the image feature vector of the target image is greater than 0.7, you can use these characters as the characters determined for the target image. (2) The text having the correlation between the text feature vector and the image feature vector of the target image at a previously preset number is used as the text of the target image. For example, if the preset number is 4, you can sort according to the degree of correlation between the text feature vector and the image feature vector of the target image, and use the 4 characters with the correlation degree in the top 4 as the text determined for the target image. However, it is worth noting that the above-mentioned texts that are defined for the target image are only a schematic description. In actual implementation, other determination strategies can also be adopted. For example, the correlation can be set to a preset number. And the text whose correlation exceeds the preset threshold is used as the determined text. Which method is adopted may be selected according to actual needs, which is not specifically limited in the present invention. In order to easily and efficiently obtain the image feature vector of the target image and the text feature vector of the text, the image feature vector and text feature vector can be extracted by training to obtain a coding model. As shown in FIG. 2, a label is used as an example for description. An image coding model and a label coding model can be established. The image feature vector and the text feature vector can be extracted through the established image coding model and the label coding model. In one embodiment, a coding model can be established in the following ways: S1: Obtain user search of target scenes (for example: search engine, e-commerce) and image data based on search text clicks. Based on these behavior data, a large number of images can be obtained -Multi-tag information. Among them, the user search text and the image data based on the search text points can be historical search and access logs derived from the target scene. S2: Perform word segmentation and part-of-speech analysis on the obtained search text; S2: Remove characters such as numbers, punctuation marks, garbled characters, and retain visually separable words (such as nouns, verbs, adjectives, etc.), and use these words as labels; S3: Deduplicate the image data based on search text clicks; S4: Combine tags with similar meanings in the tag set, remove tags that have no practical meaning, and tags that cannot be identified visually (such as development, problems, etc.); S5: Considering that the <Image Single Label> data set is more conducive to network convergence than the <Image Single Label> data set, it is possible to convert the <Image Single Label> into a <Image Single Label> pair. For example, if the multi-tag pair is <image, tag1: tag2: tag3>, then it can be converted into three single-tag pairs: <image tag1>, <image tag2>, and <image tag3>. During the training of each triplet pair, one image corresponds to only one positive sample label. S6: Train the obtained multiple single label pairs to obtain an image coding model for extracting image feature vectors from the image and a label encoding model for extracting text feature vectors from the labels, and try to make the same picture label as possible The image feature vector and text feature vector in the alignment are relatively related. For example, the image coding model can be a neural network model that uses ResNet-152 as the image feature vector extraction, uniformly normalizes the original image to a preset pixel value (for example: 224x224 pixels) as input, and then uses pool5 layer features as Network output. The length of the output feature vector is 2048. Based on this neural network model, transfer learning is performed using non-linear transformation to obtain the final feature vector that can reflect the content of the image. As shown in FIG. 2, the image in FIG. 2 can be converted into a feature vector that can reflect the content of the image. The label encoding model can be one-hot encoding to convert each label to a vector. Considering that one-hot encoding vectors are generally sparse long vectors, for convenience of processing, one-hot encoding can be converted to lower dimensions through the Embedding Layer. For dense vectors, the formed vector sequence is used as the text feature vector corresponding to the label. For text networks, a two-layer fully connected structure can be used, and other non-linear calculation layers can be added to enhance the expression ability of text feature vectors. Get the text feature vector of N labels corresponding to a certain image. That is, the label is finally converted into a fixed-length real number vector. For example, the “dress” in FIG. 2 is converted into a text feature vector through a label coding model, and the text feature vector can reflect the original semantics, thereby facilitating comparison with the image feature vector. In one embodiment, considering that multiple tags are compared at the same time, the processing speed of the computer is required to be relatively fast, and the processing capability of the processor is high. Therefore, as shown in FIG. 3, the images can be determined one by one. The correlation between the feature vector and the text feature vector of each label in multiple labels; after each correlation is determined, the correlation calculation results are stored on the hard disk without having to store them all in memory In the body, after the correlation between the labels in the label set and the image feature vector is calculated, the similarity ranking or similarity judgment may be performed to determine one or more label texts that can be used as target image labels. In order to determine the correlation between the text feature vector and the image feature vector, it can be characterized by Euclidean distance. Specifically, both the text feature vector and the image feature vector can be characterized by a vector, that is, in the same vector space, the correlation between the two feature vectors can be determined by comparing the Euclidean distance between the two feature vectors. Specifically, the image and the text can be mapped into the same feature space, so that the feature vectors of the image and the text are in the same vector space. In this way, the text feature vector with high correlation and the image feature vector can be controlled to be close in the space, and related Keep it low. Therefore, the correlation between the image and the text can be determined by calculating the text feature vector and the image feature vector. Specifically, the matching degree between the text feature vector and the image feature vector can be the Euclidean distance between the two vectors. When the value of the Euclidean distance calculated based on the two vectors is smaller, it can represent the relationship between the two vectors. The better the matching degree is, on the contrary, when the value of the Euclidean distance calculated based on the two vectors is larger, it can indicate that the matching degree between the two vectors is worse. In one embodiment, in the same vector space, the Euclidean distance between the text feature vector and the image feature vector can be calculated. The smaller the Euclidean distance, the higher the correlation between the two, and the greater the Euclidean distance, indicating the The lower the correlation. Therefore, when performing model training, a small Euclidean distance can be used as a training target to obtain the final coding model. Correspondingly, when determining the relevance, the relevance between the image and the text can be determined based on the Euclidean distance, so that the text more relevant to the image is selected. The above only measures the correlation between the image feature vector and the text feature vector based on the Euclidean distance. In actual implementation, the correlation between the image feature vector and the text feature vector can also be determined in other ways. For example, it can also include cosine distance, Manhattan distance, etc. In addition, in some cases, the correlation may be numeric or non-numeric. For example, it may only be a characterization of degree or trend. In this case, you can use The preset rule enables the content of the characterization representation to be quantified to a specific value. Furthermore, the quantized value can be used later to determine the correlation between the two vectors. For example, if the value of a certain dimension is “medium”, the character can be quantified as a binary value or a hexadecimal value of the ASCII code. The matching degree between the two vectors described in the embodiment of the present invention is not the same as the above. Limited. After counting the correlation between the image feature vector and the text feature vector, so as to determine the text corresponding to the target image, taking into account that there may be overlap between the obtained texts or determining completely unrelated texts, in order to improve the text determination Accuracy, can further remove the wrong text or deduplicate the text, so that the final determined text is more accurate. In one embodiment, in the process of determining the labels, sorting according to the similarity, and selecting the first N as the determined labels, it is inevitable that the labels of the same attribute are marked several times, for example: one The pictures of "bowl" may have "bowl" and "pot" in the tags with higher relevance, but the tags about color or style are not ranked high, so there is not one. In this case, in this way, you can directly push the first few tags of the correlation as the determined tags, or you can set rules to determine several tag categories, and select the most relevant tag in each category as the determined tag, for example : Choose one product type, one color, one style, etc. Which strategy is specifically adopted may be selected according to actual needs, which is not limited in the present invention. For example, if it is determined that the first and second correlations are red correlation 0.8 and purple correlation 0.7, then in the setting strategy, the first few tags are recommended as tags, then the red and Purple is recommended as a label. In the setting strategy, only one is selected for each category. For example, when only one color label is selected, the red correlation is greater than the purple correlation. Therefore, red is selected as the recommended label. In the above example, text and image modal data are converted into feature vectors of the same vector space through their respective coding models, and then the distance between the feature vectors is used to measure the correlation between the label and the image. , Use highly relevant tags as the text determined for the image. However, it is worth noting that the method introduced in the above example is to unify the image and the text into the same vector space, so that the correlation between the image and the text can be directly matched. The above example is explained by applying this method to the image search method, that is, given an image, tagging the image or generating descriptive information, or generating related text information, and so on. In actual implementation, it can also be used to search for pictures by text, that is, given text, search for matches to obtain corresponding pictures. The processing method and ideas are similar to the above search of pictures, and will not be repeated here. . The above search method is described below with reference to several specific scenarios. However, it is worth noting that this specific scenario is only for better explanation of the present invention and does not constitute an improper limitation on the present invention. 1) The e-commerce website releases the product as shown in Figure 4. User A intends to sell a second-hand dress of his own. After taking a picture and transmitting the picture to the e-commerce website platform, he usually needs to set a label for the picture himself, for example , Enter: long, red, dress as the label for this image. This will inevitably increase user operations. Through the above-mentioned method for determining an image label of the present invention, automatic labeling can be achieved. After user A uploads the captured picture, the system background can automatically identify and tag the picture. Through the above method, the image feature vector of the uploaded image can be extracted, and then the correlation between the image feature vector and the text feature vector of multiple labels that have been captured in advance is calculated to obtain the image feature vector and each label. Relevance of the text. Then, according to the level of relevance, the tags determined by the uploaded photos are determined and automatically labeled, which reduces user operations and improves the user experience. 2) Photo album The photos taken or downloaded from the Internet are stored in the cloud photo album or mobile photo album. Through the above method, the image feature vector of the uploaded image can be extracted, and then the correlation between the image feature vector and the text feature vector of multiple labels that have been captured in advance is calculated to obtain the image feature vector and each label. Relevance of the text. Then, according to the level of relevance, the tags determined by the uploaded photos are determined and automatically labeled.标记 After marking, it is more convenient to implement photo classification, and you can also quickly locate the target picture when searching for pictures in the album later. 3) Search for products with pictures For example, in search modes such as Polaroid, users need to upload an image, and then search for related or similar products based on this image. In this case, after the user uploads the picture, the above method can be used to extract the image feature vector of the uploaded picture, and then the captured image feature vector and the text feature vector of multiple tags that have been previously extracted are performed. The correlation is calculated to obtain the correlation between the image feature vector and each label text. Then, according to the level of relevance, the tags determined by the uploaded photos are determined. After marking the pictures, the tags can be searched by the tags, which can effectively improve the accuracy of the search and improve the recall rate. 4) Searching for poems with pictures For example: as shown in Figure 5, in some applications or scenes, you need to match the poems with pictures. After the user uploads an image, you can search and match the corresponding poems based on the picture. In this case, after the user uploads the image, the above method can be used to extract the image feature vector of the uploaded image, and then the captured image feature vector and the text feature vector of multiple poems that have been previously extracted The correlation is calculated to obtain the correlation between the image feature vector and the text feature vector of each poem. Then, according to the level of relevance, the content of the poem corresponding to the uploaded photo can be determined, and the content of the poem, or the title and author of the poem can be presented. The above four scenes are used as examples for illustration. In actual implementation, there are other scenes that can use this method. As long as the image tag pairs of the scene are captured based on different scenes, and then trained to obtain an image coding model and a text coding model that fit the scene.实施 The method embodiments provided in the embodiments of the present invention may be executed in a mobile terminal, a computer terminal, a server, or a similar computing device. Taking a server running as an example, FIG. 6 is a block diagram of a hardware structure of a server of a searching method according to an embodiment of the present invention. As shown in FIG. 6, the server 10 may include one or more (only one shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) , A memory 104 for storing data, and a transmission module 106 for communication functions. Persons of ordinary skill in the art can understand that the structure shown in FIG. 6 is only schematic, and it does not limit the structure of the electronic device. For example, the server 10 may further include more or fewer components than those shown in FIG. 6, or have a different configuration from that shown in FIG. 5. The memory 104 may be used to store software programs and modules of application software, such as program instructions / modules corresponding to the search method in the embodiment of the present invention. The processor 102 runs the software programs and modules stored in the memory 104. Thus, various functional applications and data processing are performed, that is, the above search method is implemented. The memory 104 may include high-speed random memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely disposed with respect to the processor 102, and these remote memories may be connected to the computer terminal 10 through a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof. The transmission module 106 is used to receive or send data via a network. Specific examples of the above-mentioned network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission module 106 includes a network interface controller (NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission module 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.参考 Please refer to FIG. 7. In a software implementation, the search device is applied to a server and may include a request initiation unit, a response receiving unit, and a password display unit. Wherein: an acquisition unit for acquiring an image feature vector of a target image, wherein the image feature vector is used for characterizing the image content of the target image; a determination unit is used in the same vector space according to the image The correlation between the feature vector and the text feature vector of the label determines the label corresponding to the target image, wherein the text feature vector is used to represent the semantics of the label. In an implementation manner, the determining unit may be further configured to determine a label corresponding to the target image according to a correlation between the image feature vector and a text feature vector of the label, and determine the label corresponding to the target image. The Euclidean distance between the text feature vectors determines the correlation between the target image and the label. In one embodiment, the determining unit may be specifically configured to use one or more tags whose correlation between the text feature vector and the image feature vector of the target image is greater than a preset threshold as the corresponding tags of the target image Or, using a tag having a correlation between the text feature vector and the image feature vector of the target image at a preset number as the label of the target image. In one embodiment, the determining unit may be specifically configured to determine the correlation between the image feature vector and the text feature vector of each of the multiple tags one by one; after determining the image feature vector and each of the multiple tags, After the similarity between the text feature vectors of the tags, based on the determined similarity between the image feature vector and the text feature vector of each tag in the plurality of tags, a tag corresponding to the target image is determined. In one embodiment, the capturing unit may be further configured to obtain search and click behavior data before capturing the image feature vector of the target image. The search and click behavior data includes search text and image data based on the search text click. Converting the search and click behavior data into a plurality of image tag pairs; and training and obtaining a data model for capturing image feature vectors and tag features according to the plurality of image tag pairs. In one embodiment, converting the search click behavior data into a plurality of image tag pairs may include: performing word segmentation processing and part-of-speech analysis on the search text; determining tags from data obtained from the word segmentation processing and part-of-speech analysis; Performing deduplication processing on the image data clicked based on the search text; and establishing an image tag pair according to the determined label and the image data obtained after the deduplication processing. The method and processing device for determining image tags provided by the present invention, considering that a text search method can be adopted to directly search and determine a recommended tag based on the input target image, without the need to add an image matching operation in the matching process. , You can directly obtain the corresponding label text by determining the correlation between the image feature vector and the text feature vector. Through the above methods, the problems of low efficiency and high system processing capability requirements of the existing recommended labeling methods are solved, and the technical effect of simple and accurate image tagging is achieved. While the present invention provides method operation steps as described in the embodiments or flowcharts, more or less operation steps may be included based on conventional or non-creative labor. The sequence of steps listed in the embodiments is only one way of executing the steps, and does not represent the only sequence of execution. When the actual device or client product is executed, it may be executed sequentially or in parallel according to the method shown in the embodiment or the accompanying drawings (for example, a parallel processor or a multi-threaded environment).装置 The devices or modules described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. For the convenience of description, when describing the above device, the functions are divided into various modules and described separately. When implementing the present invention, the functions of each module can be implemented in the same software or multiple software and / or hardware. Of course, a module that implements a certain function may also be implemented by combining multiple submodules or subunits. The method, device or module described in the present invention may be implemented by a computer-readable program code in a suitable manner by the controller. For example, the controller may adopt, for example, a microprocessor or a processor and the storage may be implemented by the (micro) Computer-readable media, logic gates, switches, application specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers, computer-readable program code (such as software or firmware) executed by a processor Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that, in addition to implementing the controller in pure computer-readable program code, it is entirely possible to program the method logic to make the controller use logic gates, switches, special application integrated circuits, and programmable logic controllers. And embedded microcontroller to achieve the same function. Therefore, such a controller can be considered as a hardware component, and the device included in the controller for implementing various functions can also be considered as a structure within the hardware component. Or even, a device for implementing various functions can be regarded as a structure that can be both a software module implementing the method and a hardware component.部分 Some modules in the device of the present invention can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc. that perform specific tasks or implement specific abstract data types. The invention can also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices. According to the description of the above embodiments, those skilled in the art can clearly understand that the present invention can be implemented by means of software plus necessary hardware. Based on such an understanding, the technical solution of the present invention in essence or a part that contributes to the existing technology can be embodied in the form of a software product, or it can be embodied in the implementation process of data migration. The computer software product can be stored in storage media, such as ROM / RAM, magnetic disks, optical disks, etc., including several instructions to make a computer device (can be a personal computer, mobile terminal, server, or network device, etc.) The method described in each embodiment or part of the embodiment of the present invention is performed.各个 Each embodiment in this specification is described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. All or part of the present invention can be used in many general-purpose or special-purpose computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, mobile communication terminals, multi-processor systems, microprocessor-based systems, programmable electronic devices, network PCs, small computers, Mainframe computers, distributed computing environments including any of the above systems or equipment, and more. Although the present invention is described through the examples, those skilled in the art know that the present invention has many variations and changes without departing from the spirit of the present invention, and it is hoped that the scope of the attached patent application includes these variations and changes without departing from the spirit of the present invention. .

102‧‧‧ processor

106‧‧‧Transmission Module

10‧‧‧Server

104‧‧‧Non-volatile memory

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are only the present invention. For some ordinary people skilled in the art, some embodiments described in the invention can also obtain other drawings according to these drawings without paying creative labor. 1 is a method flowchart of an embodiment of a search method provided by the present invention; FIG. 2 is a schematic diagram of establishment of an image coding model and a tag coding model provided by the present invention; FIG. 3 is a method of another embodiment of the search method provided by the present invention Flow chart; FIG. 4 is a schematic diagram of automatic image tagging provided by the present invention; FIG. 5 is a schematic diagram of searching for poems by map provided by the present invention; FIG. 6 is a schematic diagram of a server provided by the present invention; Block diagram of the structure of the device.

Claims (15)

A search method, characterized in that the method includes: capturing an image feature vector of a target image, wherein the image feature vector is used to characterize the image content of the target image; in the same vector space, according to the image The correlation between the feature vector and the text feature vector of the text determines the text corresponding to the target image, wherein the text feature vector is used to represent the semantics of the text. The method according to claim 1, wherein before determining the text corresponding to the target image based on the correlation between the image feature vector and the text feature vector of the text, the method further comprises: according to the image feature vector and The Euclidean distance between the character feature vectors determines the correlation between the target image and the text. The method according to claim 1, wherein determining the text corresponding to the target image according to the correlation between the image feature vector and the text feature vector of the text includes: combining the text feature vector with the target image One or more characters whose correlation between image feature vectors is greater than a preset threshold value as the text corresponding to the target image; or, the correlation between the text feature vector and the image feature vector of the target image is located at The previously preset amount of text is used as the text of the target image. The method according to claim 1, wherein determining the text corresponding to the target image according to the correlation between the image feature vector and the text feature vector of the text includes: 确定 determining the image feature vector and a plurality of The correlation between the text feature vectors of each text in the text; 确定 after determining the similarity between the image feature vector and the text feature vectors of each text in multiple texts, based on the determined image feature vector and The similarity between the character feature vectors of each character in the plurality of characters determines the character corresponding to the target image. The method according to claim 1, before the capturing of the image feature vector of the target image, further comprising: obtaining search and click behavior data, wherein the search and click behavior data includes: search text and an image clicked based on the search text Data; 转换 converting the search and click behavior data into multiple image text pairs; training to obtain a data model for capturing image feature vectors and text feature vectors based on the multiple image text pairs. The method according to claim 5, wherein converting the search click behavior data into a plurality of image text pairs includes: 进行 performing word segmentation processing and part-of-speech analysis on the search text; 中 from data obtained from word segmentation processing and part-of-speech analysis Determine the text; 进行 Deduplicate the image data based on the search text click; 建立 Create an image text pair based on the determined text and image data obtained after deduplication processing. The method according to claim 6, wherein the image text pair includes a single tag pair, and the single tag pair carries: one image and one text. A processing device includes a processor and a memory for storing processor-executable instructions. When the processor executes the instructions, the method realizes: a method for determining image text, wherein the method includes: capturing a target image Image feature vector, wherein the image feature vector is used to characterize the image content of the target image; in the same vector space, determining the image feature vector according to the correlation between the image feature vector and the text feature vector of the text The text corresponding to the target image, wherein the text feature vector is used to represent the semantics of the text. The processing device according to claim 8, wherein, before determining the text corresponding to the target image according to the correlation between the image feature vector and the text feature vector of the text, the processor is further configured to: The Euclidean distance between the image feature vector and the text feature vector determines the correlation between the target image and the text. The processing device according to claim 8, wherein the processor determines the text corresponding to the target image according to the correlation between the image feature vector and the text feature vector of the text, including: combining the text feature vector with One or more characters whose correlation between image feature vectors of the target image is greater than a preset threshold value as the corresponding text of the target image; or alternatively, one of the text feature vector and the image feature vector of the target image The inter-relative degree is located in a predetermined number of characters as the characters of the target image. The processing device according to claim 8, wherein the processor determines the text corresponding to the target image according to the correlation between the image feature vector and the text feature vector of the text, including: 确定 determining the images one by one The correlation between the feature vector and the character feature vector of each character in the multiple characters; after determining the similarity between the image feature vector and the character feature vector of each character in the multiple characters, based on the determined all The similarity between the image feature vector and the character feature vector of each character in the plurality of characters is used to determine the character corresponding to the target image. The processing device according to claim 8, wherein, before the image feature vector of the target image is captured, the processor is further configured to: obtain search and click behavior data, wherein the search and click behavior data includes: search text and Image data based on search text clicks; 转换 converting the search and click behavior data into multiple image text pairs; training to obtain a data model for capturing image feature vectors and text feature vectors based on the multiple image text pairs. The processing device according to claim 12, wherein the processor converts the search click behavior data into a plurality of image text pairs includes: performing word segmentation processing and part-of-speech analysis on the search text; segmenting processing and part-of-speech analysis Text is determined from the obtained data; deduplication processing is performed on the image data clicked based on the search text click; an image text pair is established according to the determined text and image data obtained after deduplication processing. A search method, characterized in that the method includes: capturing image features of a target image, wherein the image features are used to characterize the image content of the target image; in the same vector space, according to the image features and The correlation between the text features of the text determines the text corresponding to the target image, where the text features are used to characterize the semantics of the text. A computer-readable storage medium has computer instructions stored thereon that, when executed, implement the steps of the method of any one of claims 1 to 7.