TW202004516A - Optimization method for searching exclusive personalized pictures - Google Patents

Abstract

An optimization method for searching exclusive personalized pictures is revealed herein. This method includes steps of obtaining characteristics of at least one user by automatic establishment of users' data or recording users' behavior and machine learning, allowing the users' characteristics and keywords the users input in a search engine to become a search key combination, and comparing the search key combination and a label of a picture to determine whether to keep and manifest the picture or not. The picture may be kept and manifested if the label matches with the search key combination, so as to get search hits for exclusive personalized pictures.

Description

Exclusive personalized image search optimization method

The present invention relates to an exclusive and personalized image search optimization method, in particular to a data searcher who can automatically filter out images with a high degree of preference for the data searcher's own preferences when performing image search, so that the search results A method that better meets the needs of data seekers.

With the vigorous development of information technology, the phenomenon of information explosion is increasing day by day, and at the same time, it has changed the relationship between people looking for information, learning, interacting... and so on. In the massive network data objects, mainly by inputting a keyword in a search engine, the data matching the keyword is selected to obtain a search result.

However, although the search results returned by the search engine include network data objects that match the keywords entered by the user, a large number of network data objects that are not required by the user are still included in the search results. Therefore, the user must still go further after searching Filtering the data in the returned search results, that is, none of the existing search methods select the network data objects according to the user's personal needs, not only causes trouble in search, but also is quite time-consuming.

Please refer to the TW I468956 invention patent case, which relates to a personalized search ranking method including: obtaining several preset attributes; receiving a keyword; searching for several candidate information based on the keyword; receiving a client identification from a client Information; based on the client identification information, obtain one of the favorite attribute weights and a disliked attribute weight for each preset attribute; obtain a correspondence between each candidate information and the default attribute; according to the correspondence relationship, favorite attribute weights and no Love attribute weights, calculate one candidate information weight for each candidate information; sort candidate information according to the candidate information weight of candidate information, and send the sorted candidate information to the client.

Please refer again to the TW I421713 invention patent case, which relates to a system and/or method for providing enhanced content search, mainly based on a matrix indicating the user's preference for information points like a website. The information that a user visits a specific website can be accumulated, for example, it is related to a beacon or other tracker located on the publisher's website. The enhanced content may be provided by the publisher's website or generated in other ways.

Due to the development of network information, publishers hope to provide information to users, but they do not want to cause a negative reaction to users. The aforementioned prior art is to provide information review on the premise of avoiding disturbing users, and the present invention also It is developed in this direction.

The present invention proposes that when performing a picture search, it can automatically filter out the pictures with high preferences according to the preferences of the data searcher, so as to further make the search results more meet the needs of the data searcher, which is the main purpose of the present invention .

An exclusive personalized image search optimization method, including the following steps:

At least one user characteristic is obtained through the user's independent data creation or through the user's behavior records and machine learning;

The user characteristics and keywords entered by the user in a search engine are organized into a search key combination;

Parse the tag content in the target picture, and compare the search key combination with the tag content, and when the tag content matches the search key combination, retain the target picture and display it on a display device .

The exclusive personalized image search optimization method as described above, in which the tag information in the target image is accessed by linking the Google Cloud Vision API during the process of parsing the tag content of the target image.

The exclusive personalized image search optimization method as described above, wherein when comparing the search key combination with the label content, the text of the search key combination is directly compared with the text of the label content .

The exclusive personalized image search optimization method as described above, wherein when comparing the search key combination with the tag content, the text of the search key combination is first converted into a search key combination vector, and the The text of the label content is also converted into a label content vector, and then the search key combination vector is compared with the label content vector.

An exclusive personalized image search optimization method, the steps include:

At least one user characteristic is obtained through the user's independent data creation or through the user's behavior records and machine learning;

Build a semantic library;

Select words in the semantic database that have all synonyms that are included in the keyword;

The user characteristics and the words in the semantic database that have all synonyms that are included in the keyword are organized into a search key combination;

Parse the tag content in the target picture, and compare the search key combination with the tag content, and when the tag content matches the search key combination, retain the target picture and display it on a display device .

The exclusive personalized image search optimization method as described above, wherein the steps of creating a semantic database include: a step of importing a training gallery, a step of parsing the tag content that may appear in each picture in the gallery, and collecting the possible occurrences And store the content of the tag in the semantic database.

The exclusive personalized image search optimization method as described above, wherein, in the step of parsing the label content in the target image, the suitability S _{m of the} plurality of images in the gallery to generate a total of k labels is calculated first, And m = 1,...,k, then the level of S _j decides whether to keep the label (S _j represents the suitability of image j); suppose each image gets q label content and its corresponding score (q appears in the range of j Middle), the corresponding score is represented by Y _ij , where i is the picture number (i = 1,...,n and j = 0,...,q), when j is 0 means that the picture does not produce any labels, when calculating Summing up the total score of all k tags, if there are many occurrences and the quality score is high, the calculated S _m will be relatively high, so the tag represents that this can be retained to the semantic database, and its calculation formula is as follows:

。

.

The exclusive personalized image search optimization method as described above, wherein the step of creating a semantic database includes the steps of inputting Chinese keywords, and translating the input Chinese keywords into English with a translation application and storing them in Semantic steps.

The exclusive personalized image search optimization method as described above, wherein the step of creating a semantic database further includes a step of expanding the semantic database, including a step of finding synonyms through an existing semantic database, and adding the synonyms to the Stored in the semantic database.

The exclusive personalized image search optimization method as described above, wherein, in the process of analyzing the tag content of the target image, the tag information in the image is accessed by connecting to the Google Cloud Vision API.

In order to make the technical content, the purpose of the invention and the effect achieved by the invention more complete and clear disclosure, it is described in detail below, and please refer to the drawings and figures disclosed:

Please refer to the first figure, which is a flow chart of a preferred embodiment of the method for searching and optimizing the personalized image of the present invention.

The exclusive personalized picture search optimization method of the present invention includes the following steps:

Step one (S11): Create user characteristics, obtain at least one user characteristic through the user's independent data creation or through the user's behavior records and machine learning; that is, the user autonomously establishes at least one behavior through an interface, or is used by the user The habit of the machine knows the user's behavior through machine learning; the aforementioned behavior can be a preference, such as: color, sports news, entertainment news, public people, or art groups... etc. people, things, times, places, things Item of information

Step 2 (S12): Create a search key combination, and organize the user characteristics obtained in step 1 (S11) and the keywords entered by the user in a search engine into a search key combination;

Step three (S13): analyze the label content of the target picture and compare it with the search key combination, analyze the label content in the target picture, and compare the search key combination obtained in step two (S12) with the label content in the target picture Yes, when the tag content in the target picture matches the search key combination, the target picture is retained and displayed on a display device. If there is a picture that cannot be parsed, it means that the picture has not been trained in the past, or It is more complicated and cannot generate the corresponding label, so it cannot be parsed. The data content in this respect will not be displayed in the search list.

As a preferred solution of the exclusive personalized picture search optimization method of the present invention, the target pictures retained and displayed on the display device can be further filtered or sorted by a date filter.

As a preferred solution of the exclusive personalized image search optimization method of the present invention, in the process of parsing the label content of the target image, the label information in the image is accessed through the link to the Google Cloud Vision API.

As a preferred solution of the exclusive personalized image search optimization method of the present invention, when comparing the search key combination with the label content, the text of the search key combination is directly compared with the text of the label content; The text of the search key combination is converted into the search key combination vector, and the text of the label content is also converted into the label content vector. The converted search key combination vector is then compared with the converted label content vector.

Since the annotations of the images generated by the Google Cloud Vision API have many possibilities of change, and each person’s text describing the image is not the same, resulting in a semantic gap phenomenon. To solve this problem, the present invention further provides a semantic database Steps to solve this problem by creating a semantic database before performing an exclusive personalized image search.

The step of creating a semantic database is performed before the step of creating a user feature in step one or the step of creating a search key combination in step two; wherein, the second figure shows the step of creating the semantic database before the step of creating a search key combination in step two. Steps; specifically, the second preferred embodiment of the method for optimizing the search for a personalized image of the present invention, the steps include

Step one (S21): Create user characteristics, obtain at least one user characteristic through the user's independent data creation or through the user's behavior records and machine learning; that is, the user autonomously establishes at least one behavior through an interface, or is used by the user The habit of the machine knows the user's behavior through machine learning; the aforementioned behavior can be a preference, such as: color, sports news, entertainment news, public people, or art groups... etc. people, things, times, places, things Item of information

Step 2 (S22): Establish a semantic database to collect and integrate all words with the same meaning, so as to avoid the semantic gap caused by the difference in the description of the target picture from affecting the accuracy of subsequent target picture retrieval Sex

Step 3 (S23): Select the synonyms of the words in the semantic database and the keywords, and select all the words of the same meaning in the semantic database with the keywords entered by the user in a search engine;

Step 4 (S24): Create a search key combination, organize the user characteristics obtained in step 1 (S21) and all the synonyms with the keywords selected in step 3 (S23) and included in the keywords entered by the user in the search engine Search key combinations into one;

Step 5 (S25): Analyze the label content of the target image and compare it with the search key combination, analyze the label content in the target image, and compare the search key combination obtained in step 4 (S24) with the label content in the target image Yes, when the content of the tag in the target picture matches the search key combination, the target picture is retained and displayed on a display device.

The method of creating a semantic database can be divided into two types, namely: (1) using a small amount of imported pictures to create a desired semantic database and (2) directly entering keywords to create a desired semantic database.

Among them, please refer to the third figure for the first establishment method. The steps include:

Step one (S31): import training library;

Step two (S32): Analyze the label content that may appear in each picture in the gallery; in this step, first calculate the suitability S _m of the multiple pictures in the gallery to generate a total of k tags, and m = 1, …,K, and then the level of S _j decides whether to keep the label (S _j represents the suitability of figure j); suppose each image gets q label content and its corresponding score (q appears in the range of j), the corresponding score Represented by Y _ij , where i is the picture number (i = 1,...,n and j = 0,...,q), when j is 0 means that the picture does not produce any labels, and all k are added in the calculation The total score of the label, if there are many occurrences and the quality score is high, the calculated S _m will be relatively high, so the label represents that it can be retained to the semantic database, and its calculation formula is as follows:

。

.

Step three (S33): Collect the content of the possible tags and store them in the semantic database.

For the second way to build a semantic database, please refer to the fourth figure. The steps include:

Step one (S41): Enter Chinese keywords;

Step 2 (S42): Translate the entered Chinese keywords into English with a translation application;

Step three (S43): Collect the translated English and store it in the semantic database.

In addition, the data of the existing semantic database can also be used to expand the semantic database of the present invention. Please refer to the fifth figure. The steps include:

Step one (S51): Find synonymous words through the existing semantic database (for example: WordNet);

Step two (S52): add synonyms to the semantic database for storage.

In addition, the purpose of obtaining image annotations or labeling is to compare with the establishment of a search key combination to identify whether the target image meets the user's specific preference of the desired image, thereby enhancing the exclusive Search efficiency of personalized pictures.

Tagging is an important model for building machine learning. A large number of training pictures need to be imported to provide machine learning algorithms (Machine Learning algorithms) for learning. Especially if the pictures are not trained out of date, the pictures cannot be identified and provide tags. Before the conversion, a large amount of training data is required. However, because the training machine is quite time-consuming and costly, in order to effectively solve this problem, the present invention adopts Google Cloud Vision API1, which provides image labeling (Label Detection). Features.

The sixth picture is from ImageNet [J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, "Imagenet: A large-scale hierarchical image database," in Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on. IEEE, 2009, pp. 248–255.] The picture of the polar bear provided in the test case, to be uploaded to the Google Cloud Vision API for identification, and return 3 strokes The result of the tagging operation, the API will return a specified number of JSON data.

According to the test results of the sixth picture, the Google Cloud Vision API can correctly parse out that this picture is a polar bear, and brings 98% compliance. In addition, the picture may also be a mammal or an animal, 96% respectively With a confidence level of 95%, these trust levels can be used to distinguish the trust weight of the label source.

That is, the present invention can access the tag information in its pictures by using the Google Cloud Vision API based on artificial intelligence, and achieve the purpose of accurate search by establishing a search key combination and tag keywords in the pictures for screening and comparison.

The above mentioned are only some of the embodiments of the present invention, and are not intended to limit the present invention. Therefore, those who have been modified slightly according to the creative spirit and features of the present invention should also be included in the scope of this patent.

In summary, the embodiments of the present invention can indeed achieve the expected use effect, and the specific technical means disclosed by it have not only not been seen in similar products, nor have they been disclosed before application, and have fully complied with the patent law. Regulations and requirements, I filed an application for an invention patent in accordance with the law, pleaded for the examination, and granted the patent.

(S11)‧‧‧Step 1

(S12)‧‧‧Step 2

(S13)‧‧‧Step 3

(S21)‧‧‧Step 1

(S22)‧‧‧Step 2

(S23)‧‧‧Step 3

(S24)‧‧‧Step 4

(S25)‧‧‧Step 5

(S31)‧‧‧Step 1

(S32)‧‧‧Step 2

(S33)‧‧‧Step 3

(S41)‧‧‧Step 1

(S42)‧‧‧Step 2

(S43)‧‧‧Step 3

(S51)‧‧‧Step 1

(S52)‧‧‧Step 2

Figure 1: Step flow chart of a preferred embodiment of a method for searching and optimizing a personalized image of the present invention

Figure 2: The flow chart of the second preferred embodiment of the method for optimizing the search for an exclusive personalized picture of the present invention

Figure 3: The flow chart of the first step to build a semantic database

Figure 4: Flow chart of the second step to build a semantic database

Figure 5: Steps to expand the semantic database of the existing semantic database

Figure 6: Upload a polar bear image to the Google Cloud Vision API to identify the results of the tagging operation

(S11)‧‧‧Step 1

(S12)‧‧‧Step 2

(S13)‧‧‧Step 3

Claims (6)

An exclusive personalized image search optimization method, which includes the following steps: 　　Create at least one user characteristic by user's independent data or through user's behavior record and machine learning; Words are organized into a search key combination; 　　 Analyze the tag content in the target picture, and compare the search key combination with the tag content, and when the tag content matches the search key combination, the target picture Reserved and displayed on a display device; wherein, when comparing the search key combination with the label content, the text of the search key combination is directly compared with the text of the label content; or When the search key combination is compared with the label content, the text of the search key combination is first converted into a search key combination vector, and the text of the label content is also converted into a label content vector, and then the The search key combination vector is compared with the label content vector. The exclusive personalized image search optimization method as described in item 1 of the patent application scope, wherein in the process of parsing the label content of the target image, the label information in the target image is accessed by linking the Google Cloud Vision API. An exclusive personalized image search optimization method, the steps of which include: creating at least one user feature through user-independent data creation or through user behavior records and machine learning; creating a semantic database; selecting the semantic database that has Vocabulary of all synonymous words contained in the keywords; the user characteristics and the vocabulary in the semantic database with all the synonyms of the keywords are organized into a search key combination; parsing the label content in the target picture, Comparing the search key combination with the tag content, and when the tag content matches the search key combination, the target picture is retained and displayed on a display device; wherein, the creation of a semantic The step of the library may be performed before the step of obtaining at least one user feature; the step of creating a semantic database includes the steps of importing a training gallery, analyzing the tag content that may appear in each picture in the gallery, collecting sites The step of describing the content of tags that may appear and storing in the semantic database; in the step of parsing the content of tags in the target picture, the suitability S _{m of the} k tags generated by multiple pictures in the gallery is calculated first, And m = 1,...,k, then the level of S _j decides whether to keep the label (S _j represents the suitability of figure j); suppose each image gets q label content and its corresponding score (q appears in the range of j Middle), the corresponding score is represented by Y _ij , where i is the picture number (i = 1,...,n and j = 0,...,q), when j is 0 means that the picture does not produce any labels, when calculating Summing up the total score of all k tags, if there are many occurrences and the quality score is high, the calculated S _m will be relatively high, so the tag represents that this can be retained to the semantic database, and its calculation formula is as follows:

. The automatic image classification method as described in item 3 of the patent application scope, wherein the step of creating a semantic database includes the steps of inputting Chinese keywords and translating the input Chinese keywords into a translation application Steps of storing English in the semantic database. The exclusive personalized image search optimization method as described in item 4 of the patent application scope, wherein the step of creating a semantic database also includes a step of expanding the semantic database, including the step of finding synonyms through the existing semantic database The synonymous words are added to the semantic database for storage. The exclusive personalized image search optimization method as described in items 3, 4 or 5 of the patent application scope, wherein in the process of parsing the tag content of the target image, the image is accessed through the link Google Cloud Vision API Tag information.

Images