WO2018119593A1 - 一种语句推荐方法及装置 - Google Patents

一种语句推荐方法及装置 Download PDF

Info

Publication number
WO2018119593A1
WO2018119593A1 PCT/CN2016/112163 CN2016112163W WO2018119593A1 WO 2018119593 A1 WO2018119593 A1 WO 2018119593A1 CN 2016112163 W CN2016112163 W CN 2016112163W WO 2018119593 A1 WO2018119593 A1 WO 2018119593A1
Authority
WO
WIPO (PCT)
Prior art keywords
keyword
sentence
image
keywords
statement
Prior art date
Application number
PCT/CN2016/112163
Other languages
English (en)
French (fr)
Inventor
胡慧
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to CN201680088593.9A priority Critical patent/CN109643332B/zh
Priority to PCT/CN2016/112163 priority patent/WO2018119593A1/zh
Publication of WO2018119593A1 publication Critical patent/WO2018119593A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/40—Processing or translation of natural language

Definitions

  • the present application relates to the field of image processing and sentence processing technologies, and in particular, to a statement recommendation method and apparatus.
  • Another implementation in the prior art is to identify the image content using image recognition technology and generate a corresponding description. For example, for a picture of a big girl playing with a little girl, the description is "two young girls are playing with lego toy"; for a picture of a salad, the resulting description is "the salad has many different types of Vegetables in it".
  • the description generated in this way is an objective statement of the connections between objects in the picture and their attributes and activities involved. However, these objectivity statements do not have a positive impact on the user's photo sharing when sharing images.
  • the present application provides a sentence recommendation method and apparatus for recommending a sentence that is more in line with the artistic conception of the image and the subjective thought of the user, and improves user satisfaction.
  • an embodiment of the present application provides a method for recommending a sentence, including:
  • N keywords of the target image include M direct keywords obtained by parsing the target image and indirect keywords associated with the M direct keywords; N, M are positive integers, And N>M;
  • the first statement is any statement in the statement library
  • a statement that the degree of matching between the statement library and the target image is greater than or equal to the first threshold is recommended to the user.
  • the indirect keywords associated with the M direct keywords may be extended according to M direct keywords. Keywords that reflect the user's subjective thoughts and image moods. Therefore, recommending statements based on direct keywords and indirect keywords can make the recommended statements more consistent with the image mood and the user's subjective thoughts, and improve user satisfaction.
  • the method further includes:
  • Determining a theme to which the first statement belongs determining a weight value corresponding to the topic to which the first sentence belongs according to the topic to which the first statement belongs and the topic and the weight value correspondence table;
  • the method further includes:
  • the weight value corresponding to the topic to which the target sentence belongs is adjusted according to the selected target sentence, so that the weight value can be fully updated according to the feedback information, which is advantageous for recommending a sentence more suitable for the user.
  • acquiring the N keywords of the target image including:
  • a keyword having a degree of association with each of the direct keywords is greater than or equal to a second threshold as each of the direct keys
  • An indirect keyword associated with the word the keyword association table includes a degree of association between the plurality of keywords
  • the indirect keyword of the direct keyword is obtained according to the keyword association table, thereby expanding the range of the keyword of the target image, and the indirect keyword can be a key that reflects the user's subjective thought and image mood based on the direct keyword. Words, which can make the recommended statement more in line with the image mood and the subjective thinking of the user, and improve user satisfaction.
  • parsing the target image to obtain the M direct keywords including:
  • Parsing the target image to obtain a feature of the target image if it is determined that the feature of the target image includes a face feature, determining the face expression according to the face feature to obtain a face keyword Obtaining a scene keyword according to a feature other than the face feature in the feature of the target image; and obtaining the M directly according to the face keyword and the scene keyword Key words.
  • the keyword association table is obtained by:
  • the training set includes a plurality of first sentence image pairs, and each of the plurality of first sentence image pairs includes a statement and an image corresponding to the statement;
  • the keyword association table is constructed according to the obtained training set, so that the keyword association table has a theoretical basis, and lays a foundation for subsequent user recommendation sentences.
  • the training set further includes a plurality of unpaired sentences and a plurality of unpaired images
  • Keyword association table Calculating between each keyword in the first keyword set and each keyword in the second keyword set according to a set of items of each of the plurality of first sentence image pairs Correlation degree, get the keyword association table, including:
  • the method further includes:
  • Determining a target statement selected in the recommended sentence composing the target sentence and the target image into a third sentence image pair; and obtaining, according to the third sentence image pair, a set of items of the third sentence image pair;
  • the embodiment of the present application provides a statement recommendation device, which is used to implement any one of the foregoing first aspects, and includes a corresponding function module, which is used to implement the steps in the foregoing method.
  • the embodiment of the present application provides another statement recommending apparatus, which is used to implement any one of the foregoing first aspects, including a communication interface, a processor, a memory, and a bus system, respectively, for implementing the above method. step.
  • N keywords of the target image are obtained, and the N keywords include M direct keywords obtained by parsing the target image and indirect keywords associated with the M direct keywords; N, M a positive integer, and N>M; for the first statement in the statement library, respectively calculating the similarity between the N keywords and the keywords included in the first sentence, and according to the N key a degree of similarity between the word and the keyword included in the first sentence, obtaining a matching degree between the target image and the first sentence; the first statement is any statement in the statement library; A statement that the degree of matching between the statement library and the target image is greater than or equal to the first threshold is recommended to the user.
  • the indirect keywords associated with the M direct keywords may be extended according to M direct keywords. Key words that reflect the user's subjective thoughts and image moods. Therefore, recommending sentences based on direct keywords and indirect keywords can make the recommended sentences more consistent with the image mood and the user's subjective thoughts, and improve user satisfaction.
  • FIG. 1 is a schematic diagram of a system architecture applicable to the present application
  • FIG. 2a is a schematic flowchart of a method for recommending a sentence provided by the present application
  • 2b is a schematic flow chart of obtaining direct keywords of a target image
  • Figure 2c is a schematic diagram of the effect of the target sentence matching the target image
  • 2d is a schematic overall flow chart of a method for recommending a sentence in an embodiment of the present application
  • FIG. 3 is a schematic structural diagram of a sentence recommendation device provided by the present application.
  • FIG. 4 is a schematic structural diagram of another sentence recommendation device provided by the present application.
  • the statement library can include multiple types of statements, such as verses, famous words, beautiful descriptive statements, etc.; or, you can filter the statements in the statement library as needed. Specifically, if you only want to recommend verses, Only verses can be included in the statement library.
  • the generated image tag may be "sea” or "ocean”.
  • the statement library takes a variety of types of statements, including verses, famous words, and beautiful descriptive statements, in the statement library as an example, matching the words “sea” or “ocean” with the words included in the statement library, and finally recommending the words for the user. It may be "the sea is beautiful” and other statements including the "sea”.
  • the image includes the sea. If you want to recommend a verse for the user, the words used in the verse to describe the sea or the ocean are mostly “Bohai” and " At the sea, “Jianghai”, etc., the keywords “Bohai”, “Sea”, “Jianghai”, etc.
  • the final recommended sentence for the user may be “When the wind and the waves break, there will be a time when the cloud sails to the sea”, “the sea rises in the bright moon, the horizon ends at this time”, “the boat passes away, the river and the sea send the rest of life”, etc. .
  • the image tags in the prior art are generated directly according to the image content.
  • These image tags are often words that directly describe the image content, such as the above-mentioned “sea”, “Bohai”, “sea”. Etc. Therefore, subsequent statements that are recommended by the user based on these words that directly describe the content of the image are also often statements that include those words that directly describe the content of the image.
  • the recommended sentences in the above way can no longer meet people's needs.
  • the image content includes the sea
  • the subjective emotions of the user that the sea can represent are associated with “inclusiveness” or “mystery”
  • the user may also wish to obtain a statement about the subjective idea of “inclusive” or “mystery”.
  • a statement that includes an image tag does not reflect this subjective idea of the user; for example, if the figure If the content includes Chaoyang, because the user's subjective emotions can be described as "struggling" or "hope", the user may also wish to get a statement about the subjective thinking of "spitgle” or "hope”.
  • the sun will be Chaoyang As an image tag, it is difficult to get a statement that reflects the user's subjective thinking.
  • the present application provides a sentence recommendation method, which is based on the M direct keywords of the image and the indirect keywords associated with the M direct keywords, and the indirect keywords associated with the M direct keywords may be based on
  • the keywords that can be reflected by the M direct keywords can reflect the user's subjective thoughts and image mood. Therefore, according to the direct keywords and the indirect keywords, the recommended sentences can make the recommended sentences more in line with the image mood and the user's subjective thoughts. To improve user satisfaction.
  • the statement recommendation method in the present application can be applied to various scenarios.
  • the first possible scenario is that after the user selects a picture requiring a recommendation statement on the terminal used, the terminal performs the statement recommendation method in the present application as the user recommendation statement. And presented to the user;
  • the second possible scenario is that after the user selects the picture requiring the recommended statement on the terminal used, the server connected to the terminal executes the statement recommendation method in the present application as the user recommendation statement, and passes the terminal.
  • the first possible scenario is that after the user selects a picture requiring a recommendation statement on the terminal used, the terminal performs the statement recommendation method in the present application as the user recommendation statement. And presented to the user;
  • the second possible scenario is that after the user selects the picture requiring the recommended statement on the terminal used, the server connected to the terminal executes the statement recommendation method in the present application as the user recommendation statement, and passes the terminal.
  • the statement recommendation method in the present application can be applied to various scenarios.
  • the first possible scenario is that after the user selects a picture
  • FIG. 1 is a schematic diagram of a system architecture applicable to the present application.
  • the system architecture includes a server 101, one or more terminals, such as the first terminal 1021, the second terminal 1022, and the third terminal 1023 shown in FIG. 1.
  • the first terminal 1021, the second terminal 1022, and the third terminal 1023 can each communicate with the server 103 via a network (eg, a wireless network).
  • a network eg, a wireless network
  • the statement recommendation method is performed by the first terminal 1021, the second terminal 1022, or the third terminal 1023, and the statement is recommended for the user using the first terminal 1021, the second terminal 1022, or the third terminal 1023.
  • the server 101 stores a keyword association table and a statement library, and sends the keyword association table and the statement library to the terminal, and the terminal receives and stores the keyword association table and the statement library, so that the keyword association table can be based on the keyword association table.
  • the statement library is recommended for the user.
  • the server 101 may update the keyword association table and the statement library according to the set period, and send the updated keyword association table and the statement library to the terminal, so that the terminal timely updates the keyword association table and the statement library, thereby The user recommends a more appropriate statement; or, the terminal can also follow the set period. New keyword association table and statement library.
  • the statement recommendation method is executed by the server 101 to recommend a sentence for the user using the first terminal 1021, the second terminal 1022, or the third terminal 1023.
  • the server 101 stores the keyword association table and the statement library, and the first terminal 1021, the second terminal 1022, or the third terminal 1023 does not need to store the keyword association table and the statement library.
  • the user using the first terminal 1021, the second terminal 1022, or the third terminal 1023 selects a picture requiring a recommendation sentence in the first terminal 1021, the second terminal 1022, or the third terminal 1023, the first terminal 1021 and the second terminal 1022 or The third terminal 1023 may send the picture to the server 101.
  • the server 101 executes the sentence recommendation method according to the received picture, the recommended statement is sent to the first terminal 1021, the second terminal 1022, or the third terminal 1023. Similarly, the server 101 can update the keyword association table and the statement library according to the set period in order to recommend a more appropriate sentence for the user.
  • the first terminal 1021, the second terminal 1022, or the third terminal 1023 sends a picture to the server, which causes a large pressure on the network bandwidth. Therefore, the first terminal 1021 and the second terminal The terminal 1022 or the third terminal 1023 can parse the picture that needs the recommendation sentence, obtain the direct keyword of the picture and send it to the server, and the server recommends the statement according to the direct keyword of the picture, thereby effectively reducing the amount of data that needs to be transmitted. .
  • the period for updating the keyword association table and the statement library may be the same or different, and is not limited.
  • the keyword association table may also be updated under the trigger of other conditions (for example, the user selects the target statement from the recommended statement), and is specifically described later.
  • the terminal may be a device having a picture display function and a statement presentation function, and may specifically be a handheld device with a wireless connection function or another processing device connected to the wireless modem, via the wireless access network and one or more A mobile terminal that communicates with the core network.
  • the terminal can be a mobile phone, a computer, a tablet, or the like.
  • the terminal can also be a portable, pocket, handheld, computer built-in or in-vehicle mobile device.
  • the terminal may be part of a user equipment (UE).
  • the server can be a computer device with processing capabilities, and the like.
  • FIG. 2a is a schematic flowchart of a method for recommending a sentence provided by the present application. As shown in FIG. 2a, the method includes:
  • Step 201 Acquire N keywords of the target image; the N keywords include M direct keywords obtained by parsing the target image and indirect keywords associated with the M direct keywords; N, M are a positive integer, and N>M;
  • Step 202 Calculate, for the first sentence in the statement library, a similarity between the N keywords and a keyword included in the first sentence, and according to the N keywords and the first The degree of similarity between the keywords included in the statement, the degree of matching between the target image and the statement is obtained; the first statement is any statement in the statement library;
  • Step 203 Recommend, to the user, a statement that the matching degree between the statement library and the target image is greater than or equal to the first threshold.
  • the target image may be an image of a sentence to be recommended, and may be a picture in an album stored by the terminal, and the image may be a picture taken by the user, or may be a picture drawn by the user, which is not limited.
  • the N keywords of the target image include both direct keywords and indirect keywords associated with direct keywords
  • the indirect keywords associated with the M direct keywords may be extended according to M direct keywords. Key words that reflect the user's subjective thoughts and image moods. Therefore, recommending sentences based on direct keywords and indirect keywords can make the recommended sentences more consistent with the image mood and the user's subjective thoughts, and improve user satisfaction.
  • the target image is parsed to obtain a feature of the target image, and if it is determined that the feature of the target image includes a face feature, the face is determined according to the face feature.
  • a facial expression obtaining a face keyword according to a feature other than the face feature in the feature of the target image; obtaining the scene keyword according to the face keyword and the scene keyword Direct keywords.
  • the target image may be analyzed by using a convolutional neural network for feature extraction.
  • the feature of the target image is sent to the target detector to obtain an object in the target image.
  • an object in the target image For example: coffee cups, coffee beans, people, stones, etc. It is also possible to determine the category to which the object belongs, such as fruits, flowers, animals, plants, household items, and the like.
  • the face detection determines the face region, and the face is input into the expression classifier to obtain whether the face is a smile face, wherein the face detection can adopt the existing harr feature-based adaboost classifier method
  • a three-layer convolutional neural network can be used to train the smile classifier. Among them, if the face is a smiling face, the obtained face keyword can be a smile, a smile, and the like.
  • the scene classifier includes multiple, for example, an event scene classifier, a location scene classifier, and other scene classifiers, etc., and may also be implemented by a multi-scene recognition method based on a convolution network, and correspondingly, the scene Key words may refer to various types of keywords, for example, keywords used to characterize a place such as a stadium, outdoor, etc., keywords used to characterize events such as eating, sleeping, playing soccer, and other keywords such as adventure, Journey, challenge, city, forest, grassland, sky, stream, seaside, rocks, woods, ornaments, decoration, nature, sea of clouds, streets, buildings, lakes, rice fields, animals, plants, flowers, fireworks, villages, coast, ecology , ruins, ruins, sunsets, sunrises, etc.
  • keywords used to characterize a place such as a stadium, outdoor, etc.
  • keywords used to characterize events such as eating, sleeping, playing soccer
  • other keywords such as adventure, Journey, challenge, city, forest, grassland, sky, stream, seaside, rocks,
  • FIG. 2b is a schematic flowchart of acquiring direct keywords of a target image.
  • feature 1 and feature 2 are obtained, and feature 1 and feature 2 are input to the target detection classifier.
  • the face detection is performed, and input to the expression classifier to obtain a face keyword, for example, laughing and happy; and after the feature 2 is determined to be a feature other than the person, the character is sent to
  • the scene classifier gets the scene keywords, for example, the stadium, eating.
  • the direct keywords of the target target image are laugh, happy, stadium, and meal.
  • M direct keywords are obtained, which are respectively keyword 1, keyword 2, keyword 3, ..., keyword M, for each of the M direct keywords, according to a keyword association table, wherein a keyword whose degree of association with each of the direct keywords is greater than or equal to a second threshold is used as an indirect keyword associated with each of the direct keywords; according to each of the direct keys An indirect keyword associated with the word, and an indirect keyword associated with the M direct keywords.
  • the keyword association table includes the degree of association between a plurality of keywords, as shown in Table 1,
  • Table 1 Examples of some of the contents in the keyword association table
  • keywords having a degree of association with the keyword 1, keyword 2, keyword 3, ..., and keyword M greater than or equal to the second threshold are obtained, thereby obtaining indirect keywords associated with M direct keywords.
  • the second threshold can be set by a person skilled in the art according to experience and actual situation.
  • the keyword 1 is “sunrise”. After querying Table 1, the degree of association between “hope” and “sunrise” is greater than the second threshold, then “hope” is the indirect key of “sunrise”. word.
  • the keyword association table may be obtained through offline training, and the following describes the generation process of the keyword association table.
  • a training set is obtained, where the training set includes a plurality of first sentence image pairs, and each first sentence image pair of the plurality of first sentence image pairs includes a statement and an image corresponding to the statement Obtaining, according to the direct keyword of the image in each of the first sentence image pairs and the keyword of the statement in each of the first sentence image pairs, obtaining a set of items of each first sentence image pair; extracting the Deriving a direct keyword of each of the plurality of first sentence image pairs to obtain a first keyword set; extracting keywords of each of the plurality of first sentence image pairs to obtain a second keyword set; Calculating the degree of association between each keyword in the first keyword set and each keyword in the second keyword set in a set of items of each first sentence image pair in the first sentence image pair, Get the keyword association table.
  • various sentences are collected from the Internet (including celebrity quotes, mood sentences, and The sentence of the theme such as food and life) and the image, the statement and the image corresponding to the statement are used as the first sentence image pair to obtain a training set, and the training set further includes a plurality of unpaired sentences and a plurality of unpaired images.
  • the statement and the image corresponding to the statement refer to an image that has been matched with the statement when the statement is collected, or a statement that has been matched with the image when the image is acquired, such that the already matched statement and image constitute the first statement Image pair.
  • the sentences may be filtered, for example, by using an sentiment classifier to filter out negative emotion sentences; similarly, after the images are collected, the images may also be filtered to ensure training.
  • the rationality of the collection is preferably, by using an sentiment classifier to filter out negative emotion sentences; similarly, after the images are collected, the images may also be filtered to ensure training. The rationality of the collection.
  • image recognition technology is used to obtain direct keywords of the image to form a first keyword set; and natural language understanding related techniques are used to obtain keywords of the sentence to form a second keyword set; Shown is a direct keyword of the first sentence image alignment and an example of the keyword portion content of the sentence.
  • the image A and the statement a constitute the first sentence image pair, and the direct keyword of the image A obtained by the analysis image A is "sunrise” and “the sea”, and the statement a is "repenting the past, it is better to struggle for the future".
  • an association analysis algorithm such as a frequent item set, is used to obtain a degree of association between each keyword in the first keyword set and each keyword in the second keyword set. (Ai
  • the first unpaired image of the plurality of unpaired images acquiring the first unpaired image from the initial keyword association table according to the direct keyword of the first unpaired image An indirect keyword associated with a direct keyword; calculating a similarity between an indirect keyword associated with a keyword in each of the plurality of unpaired sentences and a direct keyword of the first unpaired image And an unpaired statement having a similarity greater than or equal to a third threshold and the first unpaired image composing a second sentence image pair; the first unpaired image being any unpaired image of the plurality of unpaired images .
  • the indirect keyword associated with “sunrise” may be obtained from the initial keyword association table, for example, “ Struggle; calculate the similarity between the keywords included in each unpaired statement and the struggle, and obtain the unpaired statement with the similarity greater than or equal to the third threshold: "We should work hard, have some As "so, the unpaired statement and the first unpaired image can be combined into a second sentence image pair.
  • the third threshold can be set by a person skilled in the art based on experience and actual conditions.
  • the item set of each of the plurality of second sentence image pairs may be obtained, and further, the item set and the plurality of the first sentence image pair may be obtained according to The item set of the second sentence image pair calculates a degree of association between each keyword in the first keyword set and each keyword in the second keyword set to obtain a keyword association table.
  • step 202 if the keyword included in the first sentence is the keyword 1a and the N keywords of the target image are the keyword 1b, the keyword 2b, ..., the keyword Nb, respectively, the target image is calculated.
  • the similarity between the keyword 1b and the keyword 1a, the similarity between the keyword 2b and the keyword 1a, ..., the similarity between the keyword Nb and the keyword 1a, and the similarity of the N keywords with the keyword 1a The degree is weighted and summed to obtain the degree of matching between the target image and the first sentence.
  • the key of the target image is calculated.
  • the similarity between the word 1b and the keyword 1a, and the similarity between the keyword 1b and the keyword 2a, and the similarity between the keyword 1b and the keyword 1a and the keyword 2a are weighted and summed to obtain the keyword 1b and the first
  • the matching degree of the statement similarly, the matching degree of the keyword 2b, ..., the keyword Nb and the first sentence can be obtained, and then the matching of the keyword 1b, the keyword 2b, ..., the keyword Nb with the first sentence is obtained.
  • the degree is weighted and summed to obtain the degree of matching between the target image and the first sentence.
  • the weight value can be determined in two ways: (1) setting the same weight value, that is, the keyword 1b and the keyword 1a.
  • the weight value of the similarity is the same as the weight value of the similarity of the keyword 1b and the keyword 2a, for example, both are 0.5; (2) setting a similarity threshold, and setting the weight value of the similarity smaller than the similarity threshold to 0, for example, if the similarity between the keyword 1b and the keyword 1a is less than the similarity threshold, the weight value of the similarity of the keyword 1b and the keyword 1a is set to 0; and the similarities of the similarity greater than or equal to the similarity threshold are similar.
  • the weight value of the degree is set to 1/W.
  • each keyword can be represented by a vector of one dimension according to the word vector model, and calculation is performed.
  • the similarity between the two keywords is actually the distance between the calculation vectors.
  • the distance can be calculated by using the Euclidean distance or the cosine distance, which is not limited.
  • the subject matter of the first statement may be determined in the application, and the first sentence belongs to the subject according to the topic and the topic and the weight value correspondence table.
  • a weight value corresponding to the theme and further obtaining the target image according to a similarity between the N keywords and a keyword included in the first sentence, and a weight value corresponding to a theme to which the first sentence belongs The degree of matching between the first statements.
  • the keywords of the statements in the statement library may be pre-supervised using unsupervised learning methods, such as the kmeans clustering algorithm, to obtain different categories, and one category is a topic, thereby obtaining each of the sentence libraries.
  • the topics to which each statement belongs may be inspirational, mood, food, sleep, life, work, reading, travel, and the like.
  • the theme to which each statement belongs may be stored in a plurality of manners.
  • the theme identifiers are theme 1, theme 2, theme 3, and the like. If the topic to which the statement a belongs is the theme 1, the label of the theme 1 can be set for the statement a. If the topic to which the statement b belongs is the theme 2, the label of the theme 1 can be set for the statement b, so that the label of each statement can be used according to the label. Determine the topic to which each statement belongs.
  • Another possible implementation manner is to store the topic to which each statement belongs in the form of a data table, as shown in Table 3, which is an example of the content of the topic part to which the statement and the statement belong.
  • the weight value corresponding to the topic to which the first sentence belongs may be obtained according to the topic and the weight value correspondence table, as shown in Table 4, which is the content of the topic and the weight value correspondence table. Indicate.
  • the average of the sum of the similarities between the N keywords of the target image and the keywords included in the first sentence may be multiplied by the weight of the user for the subject to which the first sentence belongs.
  • Value the degree of matching between the target image and the first sentence is obtained.
  • the first statement may be any one of the statements in the statement library. Therefore, according to the degree of matching between each statement in the statement library and the target image, the matching degree with the target image may be greater than or equal to the first
  • the threshold value is recommended to the user; the first threshold can be set by a person skilled in the art according to experience and actual conditions; in a specific implementation, the matching degree between each statement in the sentence library and the target image can be as high as possible.
  • the order is sorted in a low order, and a predetermined number of statements ranked first are recommended to the user.
  • the user may select one of the plurality of recommended sentences as the target sentence that matches the target image.
  • the target statement may be loaded into the target image in a pixel form, or may exist independently with the target image.
  • the specifics are not limited.
  • Figure 2c is a schematic diagram of the effect of the target sentence matching the target image.
  • weight values may be assigned to each topic in advance. Specifically, if there are n topics, the initial weight values assigned to each topic may be 1/n, as shown in Table 5.
  • the topic weight value can be updated subsequently based on the target statement selected by the user in the recommended statement. For example, if the target sentence selected by the user A in the recommended statement belongs to the theme 1, the weight value of the theme 1 may be adjusted for the user A, and the weight values of the other topics may be adjusted accordingly. small. Thereby, the update of the topic weight value correspondence table is realized.
  • the terminal may store the topic weight value correspondence table for the user who uses the terminal, and update the topic weight value correspondence table after the user selects the target statement, or the terminal also
  • the theme weight value correspondence table may be updated according to the target statement selected by the user according to the set period, so that the subsequent sentence recommended by the user according to the updated theme weight value correspondence table is more in line with the user's preference.
  • the server may store a topic weight value correspondence table for each user, or may store the user weight value of the plurality of users in a table, and after the user selects the target statement, The theme weight value correspondence table is updated, or the server may update the topic weight value correspondence table according to the target sentence selected by the user according to the set period.
  • the keyword association table may be updated in the application. Specifically, the target statement and the target image may be combined into a third sentence image pair, according to the third statement image pair. Obtaining a set of items of the third sentence image pair, according to the item set of the plurality of first sentence image pairs, the item set of the plurality of second sentence image pairs, and the item set of the third sentence image pair, Calculating the degree of association between each of the first keyword set and each of the second keyword set, and updating the keyword association table.
  • the update process may be performed after the user selects the target statement, or may be performed according to the target statement selected by the user according to the set period, which is not limited.
  • the subject weight value correspondence table and the keyword association table are updated according to the target sentence selected by the user, that is, updated according to the feedback information, so that the updated theme weight value correspondence table and the keyword association table can be subsequently made.
  • the statements recommended for the user are more in line with the user's preferences.
  • FIG. 2 is a schematic diagram of the overall process of the sentence recommendation method in the embodiment of the present application.
  • FIG. 2d illustrates, in a more visual manner, a process of recommending a sentence for a user in the embodiment of the present application and updating a keyword association table and a topic weight value correspondence table according to a target statement selected by the user from the recommended sentence.
  • the process which corresponds to the content described in the above embodiments, will not be specifically described herein.
  • the embodiment of the present invention further provides a sentence recommendation device, and the specific content of the device can be implemented by referring to the foregoing method.
  • FIG. 3 is a schematic structural diagram of a statement recommendation apparatus according to an embodiment of the present invention, where the apparatus is used to execute the foregoing method flow.
  • the sentence recommendation device 300 includes:
  • the obtaining module 301 is configured to acquire N keywords of the target image; the N keywords include M direct keywords obtained by parsing the target image and indirect keywords associated with the M direct keywords; , M is a positive integer, and N>M;
  • the processing module 302 is configured to calculate, for the first sentence in the statement library, a similarity between the N keywords and the keywords included in the first sentence, and according to the N keywords and the Determining the degree of similarity between the keywords included in the first sentence, obtaining a matching degree between the target image and the first sentence;
  • the first statement is any statement in the statement library;
  • the recommendation module 303 is configured to recommend, to the user, a statement that the matching degree between the statement library and the target image is greater than or equal to the first threshold.
  • the processing module 302 is specifically configured to: determine a topic to which the first statement belongs; determine, according to a topic to which the first statement belongs, and a topic and a weight value correspondence table, determine a topic to which the first sentence belongs a weight value; the target image and the first number are obtained according to a similarity between the N keywords and a keyword included in the first sentence, and a weight value corresponding to a theme to which the first sentence belongs The degree of matching between a statement.
  • the processing module 302 is further configured to: determine a target statement selected in the recommended statement and a topic to which the target statement belongs; and associate the subject and the weight value in the table with the target statement The weight value corresponding to the theme is adjusted.
  • the processing module 302 is specifically configured to: parse the target image to obtain the M direct keywords; and for each of the M direct keywords, according to keywords a correlation table, wherein a keyword whose degree of association with each of the direct keywords is greater than or equal to a second threshold is used as an indirect keyword associated with each of the direct keywords; and the keyword association table includes a plurality of keywords Degree of association; according to the indirect keywords associated with each of the direct keywords, Indirect keywords associated with the M direct keywords.
  • the processing module 302 is specifically configured to: parse the target image to obtain a feature of the target image; and if it is determined that the feature of the target image includes a facial feature, according to the facial feature Determining the expression of the face to obtain a face keyword; obtaining a scene keyword according to a feature other than the face feature in the feature of the target image; according to the face keyword and the scene keyword , get the M direct keywords.
  • processing module 302 is specifically configured to:
  • the training set includes a plurality of first sentence image pairs, and each of the plurality of first sentence image pairs includes a statement and an image corresponding to the statement;
  • the training set further includes a plurality of unpaired sentences and a plurality of unpaired images
  • the processing module 302 is specifically configured to:
  • processing module 302 is further configured to:
  • FIG. 4 is a schematic structural diagram of another sentence recommendation apparatus according to an embodiment of the present invention, where the apparatus is used to execute the foregoing method flow.
  • the sentence recommendation device 400 includes: a communication interface 401, a processor 402, a memory 403, and a bus system 404;
  • the memory 403 is used to store a program.
  • the program can include program code, the program code including computer operating instructions.
  • the memory 403 may be a random access memory (RAM) or a non-volatile memory, such as at least one disk storage. Only one memory is shown in the figure, of course, the memory can also be set to a plurality as needed. Memory 403 can also be a memory in processor 402.
  • the memory 403 stores the following elements, executable modules or data structures, or a subset thereof, or an extended set thereof:
  • Operation instructions include various operation instructions for implementing various operations.
  • Operating system Includes a variety of system programs for implementing various basic services and handling hardware-based tasks.
  • the processor 402 controls the operation of the sentence recommendation device 400, which may also be referred to as a CPU (Central Processing Unit).
  • a bus system 404 which may include, in addition to the data bus, a power bus, a control bus, a status signal bus, and the like.
  • bus system 404 may include, in addition to the data bus, a power bus, a control bus, a status signal bus, and the like.
  • bus system 404 may include, in addition to the data bus, a power bus, a control bus, a status signal bus, and the like.
  • bus system 404 may include, in addition to the data bus, a power bus, a control bus, a status signal bus, and the like.
  • various buses are labeled as bus system 404 in the figure. For ease of representation, only the schematic drawing is shown in FIG.
  • Processor 402 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the foregoing method may be completed by an integrated logic circuit of hardware in the processor 402 or an instruction in a form of software.
  • the processor 402 described above may be a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or discrete hardware. Component.
  • DSP digital signal processor
  • ASIC application specific integrated circuit
  • FPGA field programmable gate array
  • the methods, steps, and logical block diagrams disclosed in the embodiments of the present application can be implemented or executed.
  • the general purpose processor may be a microprocessor or the processor or any conventional processor or the like.
  • the steps of the method disclosed in the embodiments of the present application may be directly implemented by the hardware decoding processor, or may be performed by a combination of hardware and software modules in the decoding processor.
  • the software module can be located in a conventional storage medium such as random access memory, flash memory, read only memory, programmable read only memory or electrically erasable programmable memory, registers, and the like.
  • the storage medium is located in memory 403, and processor 402 reads the information in memory 403 and performs the above method steps in conjunction with its hardware.
  • N keywords of the target image are obtained, and the N keywords include M direct keywords obtained by parsing the target image and associated with the M direct keywords.
  • Indirect keywords N, M are positive integers, and N>M; for the first sentence in the statement library, respectively calculating the similarity between the N keywords and the keywords included in the first sentence, And obtaining a matching degree between the target image and the first sentence according to a similarity between the N keywords and a keyword included in the first sentence; the first statement is the statement Any statement in the library; recommending, to the user, a statement that the degree of matching between the statement library and the target image is greater than or equal to the first threshold.
  • the N keywords of the target image include both straight Key words, including indirect keywords associated with direct keywords, and indirect keywords associated with M direct keywords may be keywords that are based on M direct keywords and reflect user subjective ideas and image moods.
  • the recommended sentence can be more in line with the image mood and the user's subjective thinking statement, and improve user satisfaction.
  • embodiments of the present invention can be provided as a method, or a computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or a combination of software and hardware. Moreover, the invention can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) including computer usable program code.
  • a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) including computer usable program code.
  • the computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device.
  • the apparatus implements the functions specified in one or more blocks of a flow or a flow and/or block diagram of the flowchart.
  • These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce computer-implemented processing for execution on a computer or other programmable device.
  • the instructions provide steps for implementing the functions specified in one or more of the flow or in a block or blocks of a flow diagram.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

一种语句推荐方法及装置，获取目标图像的N个关键词，N个关键词中包括解析目标图像得到的M个直接关键词以及M个直接关键词关联的间接关键词；针对于语句库中的每一语句，得到目标图像与每一语句之间的匹配度；将语句库中与目标图像之间的匹配度大于等于第一阈值的语句推荐给用户。本申请中，由于N个关键词中既包括直接关键词，还包括直接关键词关联的间接关键词，使得后续可以根据直接关键词和间接关键词来推荐语句，从而能够使推荐的语句更符合图像意境和用户的主观思想的语句，提高用户满意度。

Description

一种语句推荐方法及装置 技术领域
本申请涉及图像处理和语句处理技术领域，尤其涉及一种语句推荐方法及装置。
背景技术
随着数码相机和智能手机等移动设备的普及，以及社交网络等传播媒介的发展，用户手中的照片数量越来越多。很多用户喜欢在社交网络比如微信、微博上选择自己终端设备中的照片来分享。在进行照片分享时，用户往往希望能为照片添加一些比较唯美有内涵，或者符合时代特征的语句，以便增加待分享内容的吸引力和自己的影响力。
为满足用户的这一需求，现有技术中用于照片处理的应用，比如美图秀秀，给用户提供了为图片添加文字，并转发添加了文字的图片到社交网络中的功能。对于写作水平高的用户来讲，很容易利用这些应用为自己选择分享的图片添加名言美句。然而，有些用户在进行图片分享时，无法直接写出吸引人的文字或者很快想到符合图片意境的名人名言，使得这种方式具有很大的局限性。
现有技术中另一种实现方式为，利用图像识别技术识别出图像内容，生成对应的描述。例如，针对于一个大女孩陪同一个小女孩玩耍的图片，生成的描述为“two young girls are playing with lego toy”；针对于一幅沙拉的图片，生成的描述为“the salad has many different types of vegetables in it”。这种方式生成的描述是对图片中物体之间的联系以及它们的属性和参与的活动进行客观性的陈述。然而，在进行图片分享时，这些客观性的陈述并不能给用户的图片分享带来积极影响。
综上，目前亟需一种语句推荐方法，用于为用户推荐更符合图像意境和用户的主观思想的语句，提高用户满意度。
发明内容
本申请提供一种语句推荐方法及装置，用于为用户推荐更符合图像意境和用户的主观思想的语句，提高用户满意度。
第一方面，本申请实施例提供一种语句推荐方法，包括：
获取目标图像的N个关键词；所述N个关键词中包括解析所述目标图像得到的M个直接关键词以及所述M个直接关键词关联的间接关键词；N，M为正整数，且N>M；
针对于语句库中的第一语句，分别计算所述N个关键词与所述第一语句包括的关键词之间的相似度，并根据所述N个关键词与所述第一语句包括的关键词之间的相似度，得到所述目标图像与所述第一语句之间的匹配度；所述第一语句为所述语句库中的任一语句；
将所述语句库中与所述目标图像之间的匹配度大于等于第一阈值的语句推荐给用户。
如此，由于目标图像的N个关键词中既包括直接关键词，还包括直接关键词关联的间接关键词，而M个直接关键词关联的间接关键词可以为根据M个直接关键词延伸出的能够反映用户主观思想和图像意境的关键词，因此，根据直接关键词和间接关键词来推荐语句，能够使推荐的语句更符合图像意境和用户的主观思想的语句，提高用户满意度。
可选地，得到所述目标图像与所述第一语句之间的匹配度之前，还包括：
确定所述第一语句所属的主题；根据所述第一语句所属的主题以及主题与权重值对应表，确定所述第一语句所属的主题对应的权重值；
根据所述N个关键词与所述第一语句包括的关键词之间的相似度，得到所述目标图像与所述第一语句之间的匹配度，包括：
根据所述N个关键词与所述第一语句包括的关键词之间的相似度、以及所述第一语句所属的主题对应的权重值，得到所述目标图像与所述第一语句之间的匹配度。
如此，确定目标图像与所述第一语句之间的匹配度时，考虑第一语句所 属的主题对应的权重值，从而增加了目标图像与所述第一语句之间的匹配度的计算依据，使得确定出确定目标图像与所述第一语句之间的匹配度更加准确合理，且更符合用户的喜好。
可选地，将所述语句库中与所述目标图像之间的匹配度大于等于第一阈值的语句推荐给用户之后，还包括：
确定在推荐的语句中被选择的目标语句以及所述目标语句所属的主题；将所述主题与权重值对应表中与所述目标语句所属的主题对应的权重值调大。
如此，根据被选择的目标语句来调整目标语句所属的主题对应的权重值，从而能够充分根据反馈信息来更新权重值，有利于为用户推荐出更符合用户需求的语句。
可选地，获取所述目标图像的N个关键词，包括：
对所述目标图像进行解析，得到所述M个直接关键词；
针对于所述M个直接关键词中的每个直接关键词，根据关键词关联表，将与所述每个直接关键词的关联度大于等于第二阈值的关键词作为所述每个直接关键词关联的间接关键词；所述关键词关联表中包括多个关键词之间的关联度；
根据所述每一直接关键词关联的间接关键词，得到所述M个直接关键词关联的间接关键词。
如此，根据关键词关联表得到直接关键词的间接关键词，从而扩展了目标图像的关键词的范围，由于间接关键词可以为根据直接关键词延伸出的能够反映用户主观思想和图像意境的关键词，从而能够使推荐的语句更符合图像意境和用户的主观思想的语句，提高用户满意度。
可选地，对所述目标图像进行解析，得到所述M个直接关键词，包括：
对所述目标图像进行解析，得到所述目标图像的特征；若确定所述目标图像的特征中包括人脸特征，则根据所述人脸特征确定所述人脸的表情，得到人脸关键词；根据所述目标图像的特征中除所述人脸特征以外的特征，得到场景关键词；根据所述人脸关键词和所述场景关键词，得到所述M个直接 关键词。
可选地，通过如下方式得到所述关键词关联表：
获取训练集合，所述训练集合中包括多个第一语句图像对，所述多个第一语句图像对的每个第一语句图像对中包括语句和所述语句对应的图像；
根据所述每个第一语句图像对中的图像的直接关键词和所述每个第一语句图像对中的语句的关键词，得到每个第一语句图像对的项集；
提取所述多个第一语句图像对中各个图像的直接关键词，得到第一关键词集合；提取所述多个第一语句图像对中各个语句的关键词，得到第二关键词集合；
根据所述多个第一语句图像对中每个第一语句图像对的项集，计算所述第一关键词集合中的各个关键词和所述第二关键词集合中的各个关键词之间的关联度，得到关键词关联表。
如此，根据获取到的训练集合来构建关键词关联表，使得关键词关联表具有理论依据，为后续为用户推荐语句奠定基础。
可选地，所述训练集合中还包括多个未配对语句和多个未配对图像；
根据所述多个第一语句图像对中每个第一语句图像对的项集，计算所述第一关键词集合中的各个关键词和所述第二关键词集合中的各个关键词之间的关联度，得到关键词关联表，包括：
根据所述多个第一语句图像对中每个第一语句图像对的项集，计算所述第一关键词集合中的各个关键词和所述第二关键词集合中的各个关键词之间的关联度，得到初始关键词关联表；
针对于所述多个未配对图像中的第一未配对图像，根据所述第一未配对图像的直接关键词，从所述初始关键词关联表中获取所述第一未配对图像的直接关键词关联的间接关键词；计算所述多个未配对语句中每个未配对语句中的关键词与所述第一未配对图像的直接关键词关联的间接关键词之间的相似度，将相似度大于等于第三阈值的未配对语句和所述第一未配对图像组成第二语句图像对；所述第一未配对图像为所述多个未配对图像中的任一未配 对图像；
根据多个第二语句图像对，得到每个第二语句图像对的项集；
根据所述多个第一语句图像对的的项集和多个第二语句图像对的项集，计算所述第一关键词集合中的各个关键词和所述第二关键词集合中的各个关键词之间的关联度，得到关键词关联表。
可选地，将所述语句库中与所述目标图像之间的匹配度大于等于第一阈值的语句推荐给用户之后，还包括：
确定在推荐的语句中被选择的目标语句；将所述目标语句和所述目标图像组成第三语句图像对；根据所述第三语句图像对，得到第三语句图像对的项集；
根据所述多个第一语句图像对的的项集、所述多个第二语句图像对的项集和所述第三语句图像对的项集，计算所述第一关键词集合中的各个关键词和所述第二关键词集合中的各个关键词之间的关联度，更新所述关键词关联表。
第二方面，本申请实施例提供一种语句推荐装置，用于实现上述第一方面中的任意一种方法，包括相应的功能模块，分别用于实现以上方法中的步骤。
第三方面，本申请实施例提供另一种语句推荐装置，用于实现上述第一方面中的任意一种方法，包括通信接口、处理器、存储器和总线系统，分别用于实现以上方法中的步骤。
本申请中，获取目标图像的N个关键词，所述N个关键词中包括解析所述目标图像得到的M个直接关键词以及所述M个直接关键词关联的间接关键词；N，M为正整数，且N>M；针对于语句库中的第一语句，分别计算所述N个关键词与所述第一语句包括的关键词之间的相似度，并根据所述N个关键词与所述第一语句包括的关键词之间的相似度，得到所述目标图像与所述第一语句之间的匹配度；所述第一语句为所述语句库中的任一语句；将所述语句库中与所述目标图像之间的匹配度大于等于第一阈值的语句推荐给用户。 本申请中，由于目标图像的N个关键词中既包括直接关键词，还包括直接关键词关联的间接关键词，而M个直接关键词关联的间接关键词可以为根据M个直接关键词延伸出的能够反映用户主观思想和图像意境的关键词，因此，根据直接关键词和间接关键词来推荐语句，能够使推荐的语句更符合图像意境和用户的主观思想的语句，提高用户满意度。
附图说明
为了更清楚地说明本申请中的技术方案，下面将对实施例描述中所需要使用的附图作简要介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例。
图1为本申请适用的一种系统架构示意图；
图2a为本申请提供的一种语句推荐方法所对应的流程示意图；
图2b为获取目标图像的直接关键词的流程示意图；
图2c为与目标图像匹配的目标语句效果示意图；
图2d为本申请实施例中语句推荐方法的整体流程示意图；
图3为本申请提供的一种语句推荐装置的结构示意图；
图4为本申请提供的另一种语句推荐装置的结构示意图。
具体实施方式
为了使本申请的目的、技术方案和优点更加清楚，下面将结合附图对本申请作进一步地详细描述。
本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别不同对象，而不是用于描述特定顺序。此外，术语“包括”和“具有”以及它们任何变形，意图在于覆盖不排他的包括。例如包括了一系列步骤或单元的过程、方法、系统、产品或设备没有限定于已列出的步骤或单元，而是可选地还包括没有列出的步骤或单元，或可选地还包括对于这些过程、方法、产品或设备固有的其它步骤或单元。
现有技术中为图像推荐语句的方式有多种，最常用的方式为根据图像内容生成图像标签，将图像标签与语句库中的语句包括的关键词进行匹配，计算图像与语句的匹配度，进而根据匹配度为图像推荐语句。其中，语句库中可以包括多种类型的语句，例如诗句、名言、优美的描述性语句等；或者，也可以根据需要筛选语句库中的语句，具体来说，若仅想要推荐诗句，则语句库中可以仅包括诗句。
举个例子，对图像进行解析后，识别出图像中包括大海，则生成的图像标签可以为“大海”或“海洋”。以语句库中包括诗句、名言、优美的描述性语句等多种类型的语句为例，将“大海”或“海洋”与语句库中的语句包括的关键词进行匹配，最终为用户推荐的语句可能为“大海是美丽的”以及其它包括“大海”的语句。
再举个例子，对图像进行解析后，识别出图像中包括大海，若想要为用户推荐诗句，则考虑到诗句中用于描述大海或海洋的语句包括的关键词多为“沧海”、“海上”、“江海”等，则可将大海对应的关键词“沧海”、“海上”、“江海”等作为图像标签，并将“沧海”、“海上”、“江海”与语句库中的语句包括的关键词进行匹配，最终为用户推荐的语句可能为“乘风破浪会有时，直挂云帆济沧海”、“海上升明月，天涯共此时”、“小舟从此逝，江海寄余生”等。
针对于上述内容可知，现有技术中的图像标签为直接根据图像内容生成的，这些图像标签往往为直接描述图像内容的词语，例如上述所提到的“大海”、“沧海”、“海上”等，因此，后续根据这些直接描述图像内容的词语为用户推荐的语句也往往为包括这些直接描述图像内容的词语的语句。然而，随着人们希望推荐的语句更加符合图像意境和人的主观思想这一需求的不断提高，采用上述方式推荐的语句已无法满足人们的这一需求。例如，若图像内容包括大海，由于大海能够表征的用户主观情感联想为“包容”或“神秘”，则用户可能还希望得到关于“包容”或“神秘”这一主观思想的语句，而上述直接包括图像标签的语句无法反映出用户的这一主观思想；再例如，若图 像内容包括朝阳，由于朝阳能够表征的用户主观情感联想为“奋斗”或“希望”，则用户可能还希望得到关于“奋斗”或“希望”这一主观思想的语句，同样地，若将朝阳作为图像标签，很难得到反映用户这一主观思想的语句。
基于此，本申请提供一种语句推荐方法，根据图像的M个直接关键词以及M个直接关键词关联的间接关键词为用户推荐语句，由于M个直接关键词关联的间接关键词可以为根据M个直接关键词延伸出的能够反映用户主观思想和图像意境的关键词，因此，根据直接关键词和间接关键词来推荐语句，能够使推荐的语句更符合图像意境和用户的主观思想的语句，提高用户满意度。
本申请中的语句推荐方法可适用于多种场景，第一种可能的场景为，用户在使用的终端上选中需要推荐语句的图片后，由终端执行本申请中的语句推荐方法为用户推荐语句，并呈现给用户；第二种可能的场景为，用户在使用的终端上选中需要推荐语句的图片后，由与终端连接的服务器执行本申请中的语句推荐方法为用户推荐语句，并通过终端呈现给用户。
针对于上述第一种可能的场景和第二种可能的场景，图1为本申请适用的一种系统架构示意图。如图1所示，该系统架构中包括服务器101、一个或多个终端，比如图1所示的第一终端1021、第二终端1022、第三终端1023。第一终端1021、第二终端1022、第三终端1023均可以通过网络(例如：无线网络)与服务器103进行通信。
在第一种可能的场景中，由第一终端1021、第二终端1022或第三终端1023执行语句推荐方法，为使用第一终端1021、第二终端1022或第三终端1023的用户推荐语句。此种情况下，服务器101中存储有关键词关联表和语句库，并将关键词关联表和语句库发送给终端，终端接收并存储关键词关联表和语句库，从而可基于关键词关联表以及语句库为用户推荐语句。且服务器101可按照设定周期对关键词关联表和语句库进行更新，并将更新后的关键词关联表和语句库发送给终端，以便于终端及时更新关键词关联表和语句库，从而为用户推荐更加合适的语句；或者，也可以由终端按照设定周期更 新关键词关联表和语句库。
在第二种可能的场景中，由服务器101执行语句推荐方法，为使用第一终端1021、第二终端1022或第三终端1023的用户推荐语句。此种情况下，服务器101中存储有关键词关联表以及语句库，而第一终端1021、第二终端1022或第三终端1023中无需存储有关键词关联表以及语句库。使用第一终端1021、第二终端1022或第三终端1023的用户在第一终端1021、第二终端1022或第三终端1023选中需要推荐语句的图片后，第一终端1021、第二终端1022或第三终端1023可将该图片发送给服务器101，服务器101根据接收到的图片执行语句推荐方法后，将推荐的语句发送给第一终端1021、第二终端1022或第三终端1023。同样地，服务器101可按照设定周期对关键词关联表和语句库进行更新，以便于为用户推荐更加合适的语句。
本申请中，考虑到图片的数据量较大，第一终端1021、第二终端1022或第三终端1023将图片发送给服务器会给网络带宽造成较大压力，因此，第一终端1021、第二终端1022或第三终端1023可对需要推荐语句的图片进行解析，得到图片的直接关键词并发送给服务器，由服务器根据图片的直接关键词为用户推荐语句，从而能够有效降低需要传输的数据量。
需要说明的是，更新关键词关联表和语句库的周期可以相同，也可以不相同，具体不做限定。关键词关联表也可以是在其它条件(例如，用户从推荐的语句中选择目标语句)的触发下进行更新的，后续具体介绍。
本申请中，终端可以为具有图片显示功能和语句呈现功能的设备，具体可以是具有无线连接功能的手持式设备、或连接到无线调制解调器的其他处理设备，经无线接入网与一个或多个核心网进行通信的移动终端。例如，终端可以为移动电话、计算机、平板电脑等。又如，终端也可以是便携式、袖珍式、手持式、计算机内置的或者车载的移动设备。再如，终端可以为用户设备(user equipment，简称UE)的一部分。服务器可以为具有处理能力的计算机设备等。
图2a为本申请提供的一种语句推荐方法所对应的流程示意图，如图2a所示，所述方法包括：
步骤201，获取目标图像的N个关键词；所述N个关键词中包括解析所述目标图像得到的M个直接关键词以及所述M个直接关键词关联的间接关键词；N，M为正整数，且N>M；
步骤202，针对于语句库中的第一语句，分别计算所述N个关键词与所述第一语句包括的关键词之间的相似度，并根据所述N个关键词与所述第一语句包括的关键词之间的相似度，得到所述目标图像与所一语句之间的匹配度；所述第一语句为所述语句库中的任一语句；
步骤203，将所述语句库中与所述目标图像之间的匹配度大于等于第一阈值的语句推荐给用户。
其中，目标图像可以为待推荐语句的图像，具体可以为终端所存储的相册中的图片，该图片可以为用户拍摄的图片，或者，也可以为用户绘画的图片，具体不做限定。本申请中，由于目标图像的N个关键词中既包括直接关键词，还包括直接关键词关联的间接关键词，而M个直接关键词关联的间接关键词可以为根据M个直接关键词延伸出的能够反映用户主观思想和图像意境的关键词，因此，根据直接关键词和间接关键词来推荐语句，能够使推荐的语句更符合图像意境和用户的主观思想的语句，提高用户满意度。
具体来说，步骤201中，对所述目标图像进行解析，得到所述目标图像的特征，若确定所述目标图像的特征中包括人脸特征，则根据所述人脸特征确定所述人脸的表情，得到人脸关键词；根据所述目标图像的特征中除所述人脸特征以外的特征，得到场景关键词；根据所述人脸关键词和所述场景关键词，得到所述M个直接关键词。
下面针对上述过程做进一步的解释说明。
具体实施中，对目标图像进行解析具体可以采用卷积神经网络做特征提取，通过解析目标图像得到目标图像的特征后，将目标图像的特征送入目标检测器中，得到目标图像中的物体，例如：咖啡杯、咖啡豆、人、石头等， 还可以确定出物体所属的类别，例如：水果、花卉、动物、植物、生活用品等。
针对图像中的人，进行人脸检测确定人脸区域，并将人脸输入到表情分类器得到该人脸是否为笑脸，其中，人脸检测可以采用现有基于harr特征的adaboost分类器的方法，检测到人脸后，可以用一个3层的卷积神经网络训练笑脸分类器。其中，若人脸为笑脸，则得到的人脸关键词可以为笑容、笑脸等。
针对于除人的特征以外的其它特征，将其送入场景分类器中，得到场景关键词。本申请中，场景分类器包括多个，例如事件场景分类器、地点场景分类器，以及其它场景分类器等，具体也可以采用基于卷积网络的多场景识别的方法来实现，相应地，场景关键词可以是指多种类型的关键词，例如，用于表征地点的关键词如球场、户外等，用于表征事件的关键词如吃饭、睡觉、踢足球等，以及其它关键词如冒险、旅程、挑战、城市、森林、草原、天空、溪流、海边、岩石、树林、摆饰、装饰、自然、云海、街道、建筑、湖水、稻田、动物、植物、花、烟火、村庄、海岸、生态、遗迹、废墟、夕阳、日出等。
例如，图2b为获取目标图像的直接关键词的流程示意图，如图2b所示，对目标图像进行解析后，得到特征1和特征2，将特征1和特征2输入目标检测分类器。根据特征1识别出人后，进行人脸检测，并输入到表情分类器，得到人脸关键词，例如，笑、开心；确定特征2为除人的特征以外的其它特征后，将其送入场景分类器，得到场景关键词，例如，球场、吃饭。进而根据人脸关键词和场景关键词，得到目标目标图像的直接关键词为笑、开心、球场、吃饭。
通过上述方式，得到M个直接关键词，分别为关键词1、关键词2、关键词3、……、关键词M，针对于所述M个直接关键词中的每个直接关键词，根据关键词关联表，将与所述每个直接关键词的关联度大于等于第二阈值的关键词作为所述每个直接关键词关联的间接关键词；根据所述每一直接关键 词关联的间接关键词，得到所述M个直接关键词关联的间接关键词。
本申请中，关键词关联表中包括多个关键词之间的关联度，如表1所示，
表1：关键词关联表中的部分内容示例
Figure PCTCN2016112163-appb-000001
根据表1，获取到与关键词1、关键词2、关键词3、……、关键词M的关联度大于等于第二阈值的关键词，从而得到M个直接关键词关联的间接关键词。其中，第二阈值可由本领域技术人员根据经验和实际情况来设置。
举个例子，关键词1为“日出”，查询表1后，得到“希望”与“日出”之间的关联度大于第二阈值，则“希望”即为“日出”的间接关键词。
本申请中，关键词关联表可以是通过离线训练得到的，下面针对于关键词关联表的生成过程进行介绍。
具体来说，首先获取训练集合，所述训练集合中包括多个第一语句图像对，所述多个第一语句图像对的每个第一语句图像对中包括语句和所述语句对应的图像；根据所述每个第一语句图像对中的图像的直接关键词和所述每个第一语句图像对中的语句的关键词，得到每个第一语句图像对的项集；提取所述多个第一语句图像对中各个图像的直接关键词，得到第一关键词集合；提取所述多个第一语句图像对中各个语句的关键词，得到第二关键词集合；根据所述多个第一语句图像对中每个第一语句图像对的项集，计算所述第一关键词集合中的各个关键词和所述第二关键词集合中的各个关键词之间的关联度，得到关键词关联表。
具体实施中，从互联网采集各种语句(包括名人名言、心情美句、关于 美食和生活的等主题的语句)和图像，将语句和所述语句对应的图像作为第一语句图像对，得到训练集合，训练集合中还包括多个未配对语句和多个未配对图像。其中，语句和所述语句对应的图像是指采集语句时，与所述语句已经匹配的图像，或者采集图像时，与所述图像已经匹配的语句，这样已经匹配的语句和图像构成第一语句图像对。优选地，本申请中从互联网采集各种语句后，可以对语句进行筛选，例如通过情感分类器过滤掉负面感情的语句；同样地，采集到图像后，也可以对图像进行筛选，以保证训练集合的合理性。
针对于多个第一语句图像对，利用图像识别技术得到图像的直接关键词，构成第一关键词集合；利用自然语言理解相关技术得到语句的关键词，构成第二关键词集合；如表2所示，为第一语句图像对中图像的直接关键词和语句的关键词部分内容示例。
表2：第一语句图像对部分内容示例
Figure PCTCN2016112163-appb-000002
根据表2可知，图像A和语句a构成第一语句图像对，解析图像A得到图像A的直接关键词为“日出”、“大海”，语句a为“后悔过去，不如奋斗未来”，因此，{日出，大海，奋斗，未来}可构成一个项集；图像B和语句b构成第一语句图像对，解析图像B得到图像B的直接关键词为“烟火”、“笑脸”、 “女孩”，语句b为“趁一切都来得及，做一个快乐的自己”，因此，{烟火，笑脸，女孩，快乐，自己}可构成一个项集；图像C和语句c构成第一语句图像对，解析图像C得到图像C的直接关键词为“蓝天”、“彩虹”、“大海”，语句c为“不经历风雨怎能见彩虹”，因此，{蓝天，大海，风雨，彩虹}可构成一个项集。
根据表1得到多个项集后，利用关联分析算法，比如频繁项集，得到第一关键词集合中的每个关键词和第二关键词集合中的每个关键词之间的关联程度confidence(Ai|Bj)，从而得到初始关键词关联表。
进一步地，针对于所述多个未配对图像中的第一未配对图像，根据所述第一未配对图像的直接关键词，从所述初始关键词关联表中获取所述第一未配对图像的直接关键词关联的间接关键词；计算所述多个未配对语句中每个未配对语句中的关键词与所述第一未配对图像的直接关键词关联的间接关键词之间的相似度，将相似度大于等于第三阈值的未配对语句和所述第一未配对图像组成第二语句图像对；所述第一未配对图像为所述多个未配对图像中的任一未配对图像。
具体实施中，解析第一未配对图像，得到第一未配对图像的直接关键词为“日出”，则可从初始关键词关联表中获取“日出”关联的间接关键词，例如为“奋斗”；计算多个未配对语句中每个未配对语句包括的关键词与“奋斗”之间的相似度，得到相似度大于等于第三阈值的未配对语句为“我们应当努力奋斗，有所作为”，从而可将该未配对语句和第一未配对图像组成第二语句图像对。其中第三阈值可由本领域技术人员根据经验和实际情况设置。
通过上述方式，得到多个第二语句图像对后，可得到多个第二语句图像对中每个语句图像对的项集，进而可根据多个第一语句图像对的的项集和多个第二语句图像对的项集，计算所述第一关键词集合中的各个关键词和所述第二关键词集合中的各个关键词之间的关联度，得到关键词关联表。
步骤202中，若第一语句包括的关键词为关键词1a，目标图像的N个关键词分别为关键词1b、关键词2b、……、关键词Nb，则分别计算目标图像 的关键词1b与关键词1a的相似度、关键词2b与关键词1a的相似度、……、关键词Nb与关键词1a的相似度，，并将N个关键词与关键词1a的相似度进行加权求和，得到目标图像与第一语句之间的匹配度。
又如，若第一语句包括的关键词为关键词1a和关键词2a，目标图像的N个关键词分别为关键词1b、关键词2b、……、关键词Nb，则计算目标图像的关键词1b与关键词1a的相似度，以及关键词1b与关键词2a的相似度，并将关键词1b与关键词1a、关键词2a的相似度进行加权求和，得到关键词1b与第一语句的匹配度；同理，可得到关键词2b、……、关键词Nb与第一语句的匹配度，进而将关键词1b、关键词2b、……、关键词Nb与第一语句的匹配度进行加权求和，得到目标图像与第一语句之间的匹配度。
下面对上述所提到的加权求和进行具体说明。以将关键词1b与关键词1a、关键词2a的相似度进行加权求和为例，权重值的确定可以有两种方式：(1)设置相同的权重值，即关键词1b与关键词1a的相似度的权重值和关键词1b与关键词2a的相似度的权重值相同，例如均为0.5；(2)设定一个相似度阈值，将小于相似度阈值的相似度的权重值设置为0，例如，关键词1b与关键词1a的相似度小于相似度阈值，则将关键词1b与关键词1a的相似度的权重值设置为0；将相似度大于等于相似度阈值的W个相似度的权重值均设置为1/W。
本申请中，以关键词1b和关键词1a为例，计算关键词1b和关键词1a之间的相似度时，可根据词向量模型，每个关键词都可以用一个维度的向量表示，计算两个关键词之间的相似度，实际上是计算向量之间的距离，其中，距离的计算可以采用欧式距离，或者也可以采用余弦距离，具体不做限定。
为了更准确地为用户推荐语句，本申请中还可确定所述第一语句所属的主题，并根据所述第一语句所属的主题以及主题与权重值对应表，确定所述第一语句所属的主题对应的权重值，进而根据所述N个关键词与所述第一语句包括的关键词之间的相似度、以及所述第一语句所属的主题对应的权重值，得到所述目标图像与所述第一语句之间的匹配度。
本申请中，可以预先将语句库中的语句的关键词采用非监督学习方法，比如kmeans聚类算法进行聚类后，得到不同的类别，一个类别即为一个主题，从而得到语句库中的各个语句所属的主题。具体来说，各个语句所属的主题可以为励志、心情、美食、睡觉、人生、工作、读书、旅游等。
进一步地，可采用多种方式存储各个语句所属的主题，例如，一种可能的实现方式为，为每个主题分配一个主题标识，例如，主题标识分别为主题1、主题2、主题3等，若语句a所属的主题为主题1，则可为语句a设置主题1的标签，若语句b所属的主题为主题2，则可为语句b设置主题1的标签，从而可根据各个语句的标签，确定出各个语句所属的主题。另一种可能的实现方式为，采用数据表的形式存储各个语句所属的主题，如表3所示，为语句与语句所属的主题部分内容示例。
表3：语句与语句所属的主题部分内容示例
Figure PCTCN2016112163-appb-000003
更进一步地，确定出第一语句所属的主题后，可根据主题与权重值对应表，得到第一语句所属的主题对应的权重值，如表4所示，为主题与权重值对应表部分内容示意。
表4：主题权重值对应表部分内容示意
Figure PCTCN2016112163-appb-000004
Figure PCTCN2016112163-appb-000005
由此，在步骤203中，可将目标图像的N个关键词与第一语句包括的关键词之间的相似度的和的平均值乘以所述用户针对于第一语句所属的主题的权重值，得到目标图像与所述第一语句之间的匹配度。其中，第一语句可以为语句库中的任一语句，因此，可根据语句库中的每一语句与目标图像之间的匹配度，将与所述目标图像之间的匹配度大于等于第一阈值的语句推荐给用户；第一阈值可由本领域技术人员根据经验和实际情况设置；具体实施中，也可以是将语句库中每一语句与所述目标图像之间的匹配度按照从高到低的顺序进行排序，选取排序靠前的预定数量的语句推荐给用户。
用户可从推荐的多个语句中选择其中的一个语句作为与目标图像匹配的目标语句，本申请中，目标语句可以像素形式加载到目标图像上，或者，也可以与目标图像一起独立存在。具体不做限定。图2c为与目标图像匹配的目标语句效果示意图。
下面针对于主题权重值对应表进行具体介绍。
本申请中，可预先针对每一主题分别分配权重值。具体来说，若有n个主题，则为每一主题分配的初始权重值可以都为1/n，如表5所示。
表5：初始主题权重值对应表
Figure PCTCN2016112163-appb-000006
后续可根据用户在推荐的语句中选择的目标语句，来对主题权重值进行更新。例如，用户A在推荐的语句中选择的目标语句所属的主题为主题1，则可针对于用户A将主题1的权重值调大，而将其它的主题的权重值相应调 小。从而实现对主题权重值对应表的更新。
需要说明的是，上述表4和表5中仅是示例性将多个用户的主题权重值存储在一个表格中，本申请中，也可以针对每个用户存储一个主题权重值对应表。
具体来说，若上述语句推荐方法由终端来执行，则终端可针对使用终端的用户存储主题权重值对应表，并在用户选择目标语句后，对主题权重值对应表进行更新，或者，终端也可以按照设定周期根据用户选择的目标语句对主题权重值对应表进行更新，从而能够使得后续根据更新后的主题权重值对应表为用户推荐的语句更符合用户的喜好。若上述语句推荐方法由服务器来执行，在服务器可针对每个用户存储一个主题权重值对应表，或者也可以将多个用户的主题权重值存储在一个表格中，并在用户选择目标语句后，对主题权重值对应表进行更新，或者，服务器也可以按照设定周期根据用户选择的目标语句对主题权重值对应表进行更新。
在用户选择目标语句后，本申请中还可以对关键词关联表进行更新，具体来说，可将所述目标语句和所述目标图像组成第三语句图像对，根据所述第三语句图像对，得到第三语句图像对的项集，根据所述多个第一语句图像对的的项集、所述多个第二语句图像对的项集和所述第三语句图像对的项集，计算所述第一关键词集合中的各个关键词和所述第二关键词集合中的各个关键词之间的关联度，更新所述关键词关联表。
具体实施中，可在用户选择目标语句后执行上述更新过程，或者，也可以按照设定周期根据用户选择的目标语句执行上述更新过程，具体不做限定。
由此可知，本申请中根据用户选择的目标语句更新主题权重值对应表和关键词关联表，即根据反馈信息进行更新，从而能够使得后续根据更新后的主题权重值对应表和关键词关联表为用户推荐的语句更符合用户的喜好。
图2d为本申请实施例中语句推荐方法的整体流程示意图。图2d以更形象的方式示意出了本申请实施例中为用户推荐语句的过程以及根据用户从推荐的语句中选择的目标语句来对关键词关联表和主题权重值对应表进行更新的 过程，其与上述实施例中所介绍的内容相对应，此处不再具体说明。
针对上述方法流程，本发明实施例还提供一种语句推荐装置，该装置的具体内容可以参照上述方法实施。
图3为本发明实施例提供的一种语句推荐装置的结构示意图，该装置用于执行上述方法流程。如图3所示，该语句推荐装置300包括：
获取模块301，用于获取目标图像的N个关键词；所述N个关键词中包括解析所述目标图像得到的M个直接关键词以及所述M个直接关键词关联的间接关键词；N，M为正整数，且N>M；
处理模块302，用于针对于语句库中的第一语句，分别计算所述N个关键词与所述第一语句包括的关键词之间的相似度，并根据所述N个关键词与所述第一语句包括的关键词之间的相似度，得到所述目标图像与所述第一语句之间的匹配度；所述第一语句为所述语句库中的任一语句；
推荐模块303，用于将所述语句库中与所述目标图像之间的匹配度大于等于第一阈值的语句推荐给用户。
可选地，所述处理模块302具体用于：确定所述第一语句所属的主题；根据所述第一语句所属的主题以及主题与权重值对应表，确定所述第一语句所属的主题对应的权重值；根据所述N个关键词与所述第一语句包括的关键词之间的相似度、以及所述第一语句所属的主题对应的权重值，得到所述目标图像与所述第一语句之间的匹配度。
可选地，所述处理模块302还用于：确定在推荐的语句中被选择的目标语句以及所述目标语句所属的主题；将所述主题与权重值对应表中与所述目标语句所属的主题对应的权重值调大。
可选地，所述处理模块302具体用于：对所述目标图像进行解析，得到所述M个直接关键词；针对于所述M个直接关键词中的每个直接关键词，根据关键词关联表，将与所述每个直接关键词的关联度大于等于第二阈值的关键词作为所述每个直接关键词关联的间接关键词；所述关键词关联表中包括多个关键词之间的关联度；根据所述每一直接关键词关联的间接关键词，得 到所述M个直接关键词关联的间接关键词。
可选地，所述处理模块302具体用于：对所述目标图像进行解析，得到所述目标图像的特征；若确定所述目标图像的特征中包括人脸特征，则根据所述人脸特征确定所述人脸的表情，得到人脸关键词；根据所述目标图像的特征中除所述人脸特征以外的特征，得到场景关键词；根据所述人脸关键词和所述场景关键词，得到所述M个直接关键词。
可选地，所述处理模块302具体用于：
获取训练集合，所述训练集合中包括多个第一语句图像对，所述多个第一语句图像对的每个第一语句图像对中包括语句和所述语句对应的图像；
根据所述每个第一语句图像对中的图像的直接关键词和所述每个第一语句图像对中的语句的关键词，得到每个第一语句图像对的项集；
提取所述多个第一语句图像对中各个图像的直接关键词，得到第一关键词集合；提取所述多个第一语句图像对中各个语句的关键词，得到第二关键词集合；
根据所述多个第一语句图像对中每个第一语句图像对的项集，计算所述第一关键词集合中的各个关键词和所述第二关键词集合中的各个关键词之间的关联度，得到关键词关联表。
可选地，所述训练集合中还包括多个未配对语句和多个未配对图像；
所述处理模块302具体用于：
根据所述多个第一语句图像对中每个第一语句图像对的项集，计算所述第一关键词集合中的各个关键词和所述第二关键词集合中的各个关键词之间的关联度，得到初始关键词关联表；
针对于所述多个未配对图像中的第一未配对图像，根据所述第一未配对图像的直接关键词，从所述初始关键词关联表中获取所述第一未配对图像的直接关键词关联的间接关键词；计算所述多个未配对语句中每个未配对语句中的关键词与所述第一未配对图像的直接关键词关联的间接关键词之间的相似度，将相似度大于等于第三阈值的未配对语句和所述第一未配对图像组成 第二语句图像对；所述第一未配对图像为所述多个未配对图像中的任一未配对图像；
根据多个第二语句图像对，得到每个第二语句图像对的项集；
根据所述多个第一语句图像对的的项集和多个第二语句图像对的项集，计算所述第一关键词集合中的各个关键词和所述第二关键词集合中的各个关键词之间的关联度，得到关键词关联表。
可选地，所述处理模块302还用于：
确定在推荐的语句中被选择的目标语句；
将所述目标语句和所述目标图像组成第三语句图像对；
根据所述第三语句图像对，得到第三语句图像对的项集；
根据所述多个第一语句图像对的的项集、所述多个第二语句图像对的项集和所述第三语句图像对的项集，计算所述第一关键词集合中的各个关键词和所述第二关键词集合中的各个关键词之间的关联度，更新所述关键词关联表。
图4为本发明实施例提供的另一种语句推荐装置的结构示意图，该装置用于执行上述方法流程。如图4所示，该语句推荐装置400包括：通信接口401、处理器402、存储器403和总线系统404；
其中，存储器403，用于存放程序。具体地，程序可以包括程序代码，程序代码包括计算机操作指令。存储器403可能为随机存取存储器(random access memory，简称RAM)，也可能为非易失性存储器(non-volatile memory)，例如至少一个磁盘存储器。图中仅示出了一个存储器，当然，存储器也可以根据需要，设置为多个。存储器403也可以是处理器402中的存储器。
存储器403存储了如下的元素，可执行模块或者数据结构，或者它们的子集，或者它们的扩展集：
操作指令：包括各种操作指令，用于实现各种操作。
操作系统：包括各种系统程序，用于实现各种基础业务以及处理基于硬件的任务。
处理器402控制语句推荐装置400的操作，处理器402还可以称为CPU(Central Processing Unit，中央处理单元)。具体的应用中，语句推荐装置400的各个组件通过总线系统404耦合在一起，其中总线系统404除包括数据总线之外，还可以包括电源总线、控制总线和状态信号总线等。但是为了清楚说明起见，在图中将各种总线都标为总线系统404。为便于表示，图4中仅是示意性画出。
上述本申请实施例揭示的方法可以应用于处理器402中，或者由处理器402实现。处理器402可能是一种集成电路芯片，具有信号的处理能力。在实现过程中，上述方法的各步骤可以通过处理器402中的硬件的集成逻辑电路或者软件形式的指令完成。上述的处理器402可以是通用处理器、数字信号处理器(DSP)、专用集成电路(ASIC)、现场可编程门阵列(FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。可以实现或者执行本申请实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本申请实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器403，处理器402读取存储器403中的信息，结合其硬件执行以上方法步骤。
从上述内容可以看出：本申请中，获取目标图像的N个关键词，所述N个关键词中包括解析所述目标图像得到的M个直接关键词以及所述M个直接关键词关联的间接关键词；N，M为正整数，且N>M；针对于语句库中的第一语句，分别计算所述N个关键词与所述第一语句包括的关键词之间的相似度，并根据所述N个关键词与所述第一语句包括的关键词之间的相似度，得到所述目标图像与所述第一语句之间的匹配度；所述第一语句为所述语句库中的任一语句；将所述语句库中与所述目标图像之间的匹配度大于等于第一阈值的语句推荐给用户。本申请中，由于目标图像的N个关键词中既包括直 接关键词，还包括直接关键词关联的间接关键词，而M个直接关键词关联的间接关键词可以为根据M个直接关键词延伸出的能够反映用户主观思想和图像意境的关键词，因此，根据直接关键词和间接关键词来推荐语句，能够使推荐的语句更符合图像意境和用户的主观思想的语句，提高用户满意度。
本领域内的技术人员应明白，本发明的实施例可提供为方法、或计算机程序产品。因此，本发明可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本发明可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。
本发明是参照根据本发明实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
尽管已描述了本发明的优选实施例，但本领域内的技术人员一旦得知了基本创造性概念，则可对这些实施例作出另外的变更和修改。所以，所附权 利要求意欲解释为包括优选实施例以及落入本发明范围的所有变更和修改。
显然，本领域的技术人员可以对本发明进行各种改动和变型而不脱离本发明的精神和范围。这样，倘若本发明的这些修改和变型属于本发明权利要求及其等同技术的范围之内，则本发明也意图包含这些改动和变型在内。

Claims (16)

  1. 一种语句推荐方法，其特征在于，所述方法包括：
    获取目标图像的N个关键词；所述N个关键词中包括解析所述目标图像得到的M个直接关键词以及所述M个直接关键词关联的间接关键词；N，M为正整数，且N>M；
    针对于语句库中的第一语句，分别计算所述N个关键词与所述第一语句包括的关键词之间的相似度，并根据所述N个关键词与所述第一语句包括的关键词之间的相似度，得到所述目标图像与所述第一语句之间的匹配度；所述第一语句为所述语句库中的任一语句；
    将所述语句库中与所述目标图像之间的匹配度大于等于第一阈值的语句推荐给用户。
  2. 根据权利要求1所述的方法，其特征在于，得到所述目标图像与所述第一语句之间的匹配度之前，还包括：
    确定所述第一语句所属的主题；
    根据所述第一语句所属的主题以及主题与权重值对应表，确定所述第一语句所属的主题对应的权重值；
    根据所述N个关键词与所述第一语句包括的关键词之间的相似度，得到所述目标图像与所述第一语句之间的匹配度，包括：
    根据所述N个关键词与所述第一语句包括的关键词之间的相似度、以及所述第一语句所属的主题对应的权重值，得到所述目标图像与所述第一语句之间的匹配度。
  3. 根据权利要求2所述的方法，其特征在于，将所述语句库中与所述目标图像之间的匹配度大于等于第一阈值的语句推荐给用户之后，还包括：
    确定在推荐的语句中被选择的目标语句以及所述目标语句所属的主题；
    将所述主题与权重值对应表中与所述目标语句所属的主题对应的权重值调大。
  4. 根据权利要求1-3中任一项所述的方法，其特征在于，获取所述目标图像的N个关键词，包括：
    对所述目标图像进行解析，得到所述M个直接关键词；
    针对于所述M个直接关键词中的每个直接关键词，根据关键词关联表，将与所述每个直接关键词的关联度大于等于第二阈值的关键词作为所述每个直接关键词关联的间接关键词；所述关键词关联表中包括多个关键词之间的关联度；
    根据所述每一直接关键词关联的间接关键词，得到所述M个直接关键词关联的间接关键词。
  5. 根据权利要求4所述的方法，其特征在于，对所述目标图像进行解析，得到所述M个直接关键词，包括：
    对所述目标图像进行解析，得到所述目标图像的特征；
    若确定所述目标图像的特征中包括人脸特征，则根据所述人脸特征确定所述人脸的表情，得到人脸关键词；
    根据所述目标图像的特征中除所述人脸特征以外的特征，得到场景关键词；
    根据所述人脸关键词和所述场景关键词，得到所述M个直接关键词。
  6. 根据权利要求4所述的方法，其特征在于，通过如下方式得到所述关键词关联表：
    获取训练集合，所述训练集合中包括多个第一语句图像对，所述多个第一语句图像对的每个第一语句图像对中包括语句和所述语句对应的图像；
    根据所述每个第一语句图像对中的图像的直接关键词和所述每个第一语句图像对中的语句的关键词，得到每个第一语句图像对的项集；
    提取所述多个第一语句图像对中各个图像的直接关键词，得到第一关键词集合；提取所述多个第一语句图像对中各个语句的关键词，得到第二关键词集合；
    根据所述多个第一语句图像对中每个第一语句图像对的项集，计算所述 第一关键词集合中的各个关键词和所述第二关键词集合中的各个关键词之间的关联度，得到关键词关联表。
  7. 根据权利要求6所述的方法，其特征在于，所述训练集合中还包括多个未配对语句和多个未配对图像；
    根据所述多个第一语句图像对中每个第一语句图像对的项集，计算所述第一关键词集合中的各个关键词和所述第二关键词集合中的各个关键词之间的关联度，得到关键词关联表，包括：
    根据所述多个第一语句图像对中每个第一语句图像对的项集，计算所述第一关键词集合中的各个关键词和所述第二关键词集合中的各个关键词之间的关联度，得到初始关键词关联表；
    针对于所述多个未配对图像中的第一未配对图像，根据所述第一未配对图像的直接关键词，从所述初始关键词关联表中获取所述第一未配对图像的直接关键词关联的间接关键词；计算所述多个未配对语句中每个未配对语句中的关键词与所述第一未配对图像的直接关键词关联的间接关键词之间的相似度，将相似度大于等于第三阈值的未配对语句和所述第一未配对图像组成第二语句图像对；所述第一未配对图像为所述多个未配对图像中的任一未配对图像；
    根据多个第二语句图像对，得到每个第二语句图像对的项集；
    根据所述多个第一语句图像对的的项集和多个第二语句图像对的项集，计算所述第一关键词集合中的各个关键词和所述第二关键词集合中的各个关键词之间的关联度，得到关键词关联表。
  8. 根据权利要求7所述的方法，其特征在于，将所述语句库中与所述目标图像之间的匹配度大于等于第一阈值的语句推荐给用户之后，还包括：
    确定在推荐的语句中被选择的目标语句；
    将所述目标语句和所述目标图像组成第三语句图像对；
    根据所述第三语句图像对，得到第三语句图像对的项集；
    根据所述多个第一语句图像对的的项集、所述多个第二语句图像对的项 集和所述第三语句图像对的项集，计算所述第一关键词集合中的各个关键词和所述第二关键词集合中的各个关键词之间的关联度，更新所述关键词关联表。
  9. 一种语句推荐装置，其特征在于，所述装置包括：
    处理器，用于获取目标图像的N个关键词；所述N个关键词中包括解析所述目标图像得到的M个直接关键词以及所述M个直接关键词关联的间接关键词；N，M为正整数，且N>M；针对于语句库中的第一语句，分别计算所述N个关键词与所述第一语句包括的关键词之间的相似度，并根据所述N个关键词与所述第一语句包括的关键词之间的相似度，得到所述目标图像与所述第一语句之间的匹配度；所述第一语句为所述语句库中的任一语句；
    通信接口，用于将所述语句库中与所述目标图像之间的匹配度大于等于第一阈值的语句推荐给用户。
  10. 根据权利要求9所述的装置，其特征在于，所述处理器具体用于：
    确定所述第一语句所属的主题；
    根据所述第一语句所属的主题以及主题与权重值对应表，确定所述第一语句所属的主题对应的权重值；
    根据所述N个关键词与所述第一语句包括的关键词之间的相似度、以及所述第一语句所属的主题对应的权重值，得到所述目标图像与所述第一语句之间的匹配度。
  11. 根据权利要求10所述的装置，其特征在于，所述处理器还用于：
    确定在推荐的语句中被选择的目标语句以及所述目标语句所属的主题；
    将所述主题与权重值对应表中与所述目标语句所属的主题对应的权重值调大。
  12. 根据权利要求9-11中任一项所述的装置，其特征在于，所述处理器具体用于：
    对所述目标图像进行解析，得到所述M个直接关键词；
    针对于所述M个直接关键词中的每个直接关键词，根据关键词关联表， 将与所述每个直接关键词的关联度大于等于第二阈值的关键词作为所述每个直接关键词关联的间接关键词；所述关键词关联表中包括多个关键词之间的关联度；
    根据所述每一直接关键词关联的间接关键词，得到所述M个直接关键词关联的间接关键词。
  13. 根据权利要求12所述的装置，其特征在于，所述处理器具体用于：
    对所述目标图像进行解析，得到所述目标图像的特征；
    若确定所述目标图像的特征中包括人脸特征，则根据所述人脸特征确定所述人脸的表情，得到人脸关键词；
    根据所述目标图像的特征中除所述人脸特征以外的特征，得到场景关键词；
    根据所述人脸关键词和所述场景关键词，得到所述M个直接关键词。
  14. 根据权利要求12所述的装置，其特征在于，所述处理器具体用于：
    获取训练集合，所述训练集合中包括多个第一语句图像对，所述多个第一语句图像对的每个第一语句图像对中包括语句和所述语句对应的图像；
    根据所述每个第一语句图像对中的图像的直接关键词和所述每个第一语句图像对中的语句的关键词，得到每个第一语句图像对的项集；
    提取所述多个第一语句图像对中各个图像的直接关键词，得到第一关键词集合；提取所述多个第一语句图像对中各个语句的关键词，得到第二关键词集合；
    根据所述多个第一语句图像对中每个第一语句图像对的项集，计算所述第一关键词集合中的各个关键词和所述第二关键词集合中的各个关键词之间的关联度，得到关键词关联表。
  15. 根据权利要求14所述的装置，其特征在于，所述训练集合中还包括多个未配对语句和多个未配对图像；
    所述处理器具体用于：
    根据所述多个第一语句图像对中每个第一语句图像对的项集，计算所述 第一关键词集合中的各个关键词和所述第二关键词集合中的各个关键词之间的关联度，得到初始关键词关联表；
    针对于所述多个未配对图像中的第一未配对图像，根据所述第一未配对图像的直接关键词，从所述初始关键词关联表中获取所述第一未配对图像的直接关键词关联的间接关键词；计算所述多个未配对语句中每个未配对语句中的关键词与所述第一未配对图像的直接关键词关联的间接关键词之间的相似度，将相似度大于等于第三阈值的未配对语句和所述第一未配对图像组成第二语句图像对；所述第一未配对图像为所述多个未配对图像中的任一未配对图像；
    根据多个第二语句图像对，得到每个第二语句图像对的项集；
    根据所述多个第一语句图像对的的项集和多个第二语句图像对的项集，计算所述第一关键词集合中的各个关键词和所述第二关键词集合中的各个关键词之间的关联度，得到关键词关联表。
  16. 根据权利要求15所述的装置，其特征在于，所述处理器还用于：
    确定在推荐的语句中被选择的目标语句；
    将所述目标语句和所述目标图像组成第三语句图像对；
    根据所述第三语句图像对，得到第三语句图像对的项集；
    根据所述多个第一语句图像对的的项集、所述多个第二语句图像对的项集和所述第三语句图像对的项集，计算所述第一关键词集合中的各个关键词和所述第二关键词集合中的各个关键词之间的关联度，更新所述关键词关联表。
PCT/CN2016/112163 2016-12-26 2016-12-26 一种语句推荐方法及装置 WO2018119593A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201680088593.9A CN109643332B (zh) 2016-12-26 2016-12-26 一种语句推荐方法及装置
PCT/CN2016/112163 WO2018119593A1 (zh) 2016-12-26 2016-12-26 一种语句推荐方法及装置

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2016/112163 WO2018119593A1 (zh) 2016-12-26 2016-12-26 一种语句推荐方法及装置

Publications (1)

Publication Number Publication Date
WO2018119593A1 true WO2018119593A1 (zh) 2018-07-05

Family

ID=62706590

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/112163 WO2018119593A1 (zh) 2016-12-26 2016-12-26 一种语句推荐方法及装置

Country Status (2)

Country Link
CN (1) CN109643332B (zh)
WO (1) WO2018119593A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108921918A (zh) * 2018-07-24 2018-11-30 Oppo广东移动通信有限公司 视频创建方法及相关装置
CN109508423A (zh) * 2018-12-14 2019-03-22 平安科技（深圳）有限公司 基于语义识别的房源推荐方法、装置、设备及存储介质
CN109783643A (zh) * 2019-01-09 2019-05-21 北京一览群智数据科技有限责任公司 一种近似语句推荐方法及装置
CN110298684A (zh) * 2019-05-22 2019-10-01 平安科技（深圳）有限公司 车型匹配方法、装置、计算机设备及存储介质

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110297934B (zh) * 2019-07-04 2024-03-15 腾讯科技（深圳）有限公司 一种图像数据处理方法、装置及存储介质
CN110414001B (zh) * 2019-07-18 2023-09-26 腾讯科技（深圳）有限公司 语句生成方法和装置、存储介质及电子装置
CN111797262A (zh) * 2020-06-24 2020-10-20 北京小米松果电子有限公司 诗词生成方法、装置、电子设备和存储介质

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020052901A1 (en) * 2000-09-07 2002-05-02 Guo Zhi Li Automatic correlation method for generating summaries for text documents
CN103226547A (zh) * 2013-04-28 2013-07-31 百度在线网络技术（北京）有限公司 为图片产生诗句的方法和装置
CN104008180A (zh) * 2014-06-09 2014-08-27 北京奇虎科技有限公司 结构化数据与图片的关联方法与关联装置

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9021052B2 (en) * 2012-09-28 2015-04-28 Interactive Memories, Inc. Method for caching data on client device to optimize server data persistence in building of an image-based project
CN104951554B (zh) * 2015-06-29 2018-03-06 浙江大学 为风景照配上符合其意境的诗句的方法

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020052901A1 (en) * 2000-09-07 2002-05-02 Guo Zhi Li Automatic correlation method for generating summaries for text documents
CN103226547A (zh) * 2013-04-28 2013-07-31 百度在线网络技术（北京）有限公司 为图片产生诗句的方法和装置
CN104008180A (zh) * 2014-06-09 2014-08-27 北京奇虎科技有限公司 结构化数据与图片的关联方法与关联装置

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108921918A (zh) * 2018-07-24 2018-11-30 Oppo广东移动通信有限公司 视频创建方法及相关装置
CN108921918B (zh) * 2018-07-24 2023-05-30 Oppo广东移动通信有限公司 视频创建方法及相关装置
CN109508423A (zh) * 2018-12-14 2019-03-22 平安科技（深圳）有限公司 基于语义识别的房源推荐方法、装置、设备及存储介质
CN109783643A (zh) * 2019-01-09 2019-05-21 北京一览群智数据科技有限责任公司 一种近似语句推荐方法及装置
CN110298684A (zh) * 2019-05-22 2019-10-01 平安科技（深圳）有限公司 车型匹配方法、装置、计算机设备及存储介质
CN110298684B (zh) * 2019-05-22 2023-06-06 平安科技（深圳）有限公司 车型匹配方法、装置、计算机设备及存储介质

Also Published As

Publication number Publication date
CN109643332A (zh) 2019-04-16
CN109643332B (zh) 2021-02-23

Similar Documents

Publication Publication Date Title
WO2018119593A1 (zh) 一种语句推荐方法及装置
Wu et al. Deep learning for video classification and captioning
CN106355170B (zh) 照片分类方法及装置
US11620001B2 (en) Pictorial symbol prediction
US9311530B1 (en) Summarizing a photo album in a social network system
Simo-Serra et al. Neuroaesthetics in fashion: Modeling the perception of fashionability
US11511436B2 (en) Robot control method and companion robot
CN113569088B (zh) 一种音乐推荐方法、装置以及可读存储介质
Park et al. Towards personalized image captioning via multimodal memory networks
CN105468596B (zh) 图片检索方法和装置
CN106959966A (zh) 一种信息推荐方法及系统
US11948558B2 (en) Messaging system with trend analysis of content
US20230013116A1 (en) Multimodal entity identification
US11750547B2 (en) Multimodal named entity recognition
Zhao et al. Jdnet: A joint-learning distilled network for mobile visual food recognition
CN112069326A (zh) 知识图谱的构建方法、装置、电子设备及存储介质
US20140233811A1 (en) Summarizing a photo album
Geng et al. Soft biometrics in online social networks: A case study on Twitter user gender recognition
Lonn et al. Smartphone picture organization: A hierarchical approach
CN112016548B (zh) 一种封面图展示方法及相关装置
Yamamoto et al. Image emotion recognition using visual and semantic features reflecting emotional and similar objects
US20210224310A1 (en) Electronic device and story generation method thereof
EP4384924A1 (en) Evolution of topics in a messaging system
WO2014186392A2 (en) Summarizing a photo album
Znaidia et al. CEA LIST's Participation to Visual Concept Detection Task of ImageCLEF 2011.

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16925700

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16925700

Country of ref document: EP

Kind code of ref document: A1