CN110909194A

CN110909194A - Input method and system thereof

Info

Publication number: CN110909194A
Application number: CN201910934777.8A
Authority: CN
Inventors: 施明
Original assignee: Shanghai Mengjia Network Technology Co Ltd
Current assignee: Shanghai Mengjia Network Technology Co Ltd
Priority date: 2019-09-29
Filing date: 2019-09-29
Publication date: 2020-03-24

Abstract

The invention relates to an input method and a system thereof, wherein the input method system comprises the following steps: a user input acquisition module configured to acquire a character string input by a user; a text-on-screen prediction module configured to predict text-on-screen based on a string of characters input by a user; a gallery module configured to obtain one or more candidate pictures/videos based on-screen text predicted by a user input string; and a teletext composition module configured to generate one or more teletext composite pictures/videos including the predicted onscreen text based on the one or more candidate pictures/videos. The invention inquires the picture/video matched with the character input by the user, and adds the on-screen characters predicted according to the character input by the user to the picture/video, so that the output content of the input method is not limited to the characters any more, and richer and more recreational output forms can be provided, thereby meeting the increasingly improved requirements of the user on the input method.

Description

Input method and system thereof

Technical Field

The invention relates to the technical field of application, in particular to an input method and a system thereof.

Background

The input method is an application program which is used at high frequency in daily life of people no matter at a PC end or a mobile end. The development of the existing input method has two obvious trends. One trend is the development of usability, with more convenient, more accurate, and more efficient input. Both the application of artificial intelligence to input method matching and speech recognition based input methods are representative of this direction. The other trend is the development of entertainment direction, and the input content is richer, more diversified and more intuitive. The continuous addition of input functions such as characters, expressions, emoticons and the like reflects the development of the input method in the direction. However, as the demand of people on expression is continuously increased, the existing input function can not meet the demand.

Disclosure of Invention

Aiming at the technical problems in the prior art, the invention provides an input method system, which comprises the following steps: a user input acquisition module configured to acquire a character string input by a user; a text-on-screen prediction module configured to predict text-on-screen based on a string of characters input by a user; a gallery module configured to obtain one or more candidate pictures/videos based on-screen text predicted by a user input string; and a teletext composition module configured to generate one or more teletext composite pictures/videos including the predicted onscreen text based on the one or more candidate pictures/videos.

Preferably, the upper screen text prediction module and the gallery module are located on a server.

Preferably, the word stock module at the client is configured to provide one or more candidate words based on the character string input by the user; the on-screen text prediction module updates predicted on-screen text in response to the candidate word selected by the user.

Preferably, the system further comprises a communication module at the client configured to send the character string input by the user to the server; and receiving one or more candidate pictures/videos from the gallery module from the server.

Preferably, wherein further comprising an interface module configured to provide an input interface, the input interface further comprising one or more of the following respective areas: an input area configured for inputting a character or a character string; a candidate word region configured to provide one or more candidate words; and a candidate picture area configured for displaying one or more candidate pictures/videos or corresponding thumbnails, or one or more teletext composite pictures/videos or corresponding thumbnails.

Preferably, the interface module is configured to output a corresponding teletext picture/video in response to a user selection of one of the teletext pictures/videos or corresponding thumbnail.

Preferably, the user input acquisition module is configured to store characters or character strings of historical user input, and when a user inputs a character, the character strings including the character which are input in the historical input are provided for the user according to the character input by the user.

According to another aspect of the present invention, there is provided an input method, including: acquiring a character string input by a user; transmitting a character string input by a user to a server; receiving one or more candidate pictures/videos obtained based on predicted on-screen text of a character string input by a user from a server; and generating one or more teletext composite pictures/videos containing the predicted onscreen text based on the predicted onscreen text and the one or more candidate pictures/videos.

Preferably, the method further comprises the following steps: and sending the new character input by the user to the server in response to receiving the new character input by the user.

Preferably, the method further comprises the following steps: querying a local thesaurus based on a character string input by a user to provide one or more candidate words; and sending the candidate word selected by the user to the server.

Preferably, the method further comprises the following steps: and responding to the candidate words selected by the user, and updating the predicted on-screen characters on the server by adopting the candidate words selected by the user.

Preferably, the method further comprises the following steps: and responding to the candidate words selected by the user, and updating the predicted on-screen characters in the image-text synthesis image/video by adopting the candidate words selected by the user.

Preferably, further comprising: inquiring a stored historical input character string according to characters input by a user; and providing a historical input string including the character.

Preferably, further comprising: providing one or more teletext composite pictures/videos including the predicted onscreen text to the user; and outputting the selected teletext composite picture/video in response to a selection by the user.

Preferably, further comprising: after generating one or more teletext composite pictures/videos containing the predicted onscreen text, corresponding teletext composite picture/video thumbnails are generated.

Preferably, further comprising: providing one or more teletext picture/video thumbnails including the predicted onscreen text to the user; and outputting the selected teletext composite picture/video in response to a selection by the user.

The invention queries the matched picture/video through the character string input by the user, and can greatly improve the query speed of the picture/video. In some embodiments, particularly when the user input speed is fast, the matched picture/video can be displayed quickly, and the problem therein can be updated quickly, thereby providing a better user experience.

Drawings

Preferred embodiments of the present invention will now be described in further detail with reference to the accompanying drawings, in which:

FIG. 1 is a schematic diagram of an input method system according to one embodiment of the invention;

FIG. 2 is a schematic diagram of an input interface according to one embodiment of the invention;

FIG. 3 is a schematic diagram of a gallery module according to one embodiment of the invention;

FIG. 4 is a schematic diagram of an input method system according to another embodiment of the invention.

FIG. 5 is a schematic diagram of an input interface according to another embodiment of the invention;

FIG. 6 is a schematic flow diagram of an input method according to one embodiment of the invention; and

fig. 7 is a flowchart illustrating a process of providing candidate pictures according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

In the following detailed description, reference is made to the accompanying drawings that form a part hereof and in which is shown by way of illustration specific embodiments of the application. In the drawings, like numerals describe substantially similar components throughout the different views. Various specific embodiments of the present application are described in sufficient detail below to enable those skilled in the art to practice the teachings of the present application. It is to be understood that other embodiments may be utilized and structural, logical or electrical changes may be made to the embodiments of the present application.

Some functions of the input method in the prior art, such as an emoticon function, enable the input method to input pictures. However, when using the emoticon, the user needs to download the emoticon in advance. The pictures which can be input in the input method are limited to the pictures provided in the facial expression package. In particular, the text in the emoticon picture cannot be modified. This greatly limits the use of the user.

Some embodiments of the invention provide an input method and system with more entertainment: based on the content input by the user, the content is combined with the picture/video to form the picture/video containing the content input by the user. The technical solution of the present invention is explained in detail by the examples of the drawings below. It will be appreciated by those skilled in the art that the inventive arrangements can also be applied to video in a similar manner, for example small videos with a time of less than 5 seconds, 10 seconds or 15 seconds.

FIG. 1 is a schematic diagram of an input method system according to one embodiment of the invention. As shown in fig. 1, the input method system 100 in this embodiment includes a user input acquisition module 102, a gallery module 104, an on-screen text prediction module 106, and a text composition module 108. Wherein the user input acquisition module 102 is configured to acquire a character string input by a user. In one embodiment, to facilitate the interaction of the system with the user, an interface module 101 is further included, which is configured to provide an input interface through which the user can input characters, and a user input acquisition module 102 acquires the character string input by the user through the user interface.

As shown in fig. 2, the user input interface provided for the interface module 101 in this embodiment is shown. On the user input interface, the user may enter a character or a string of characters. The user input interface includes: a character display area 202, a candidate picture area 204, and an input area 206. The character display area 202 is used for displaying characters or character strings input by a user, and the input mode of the user may be text input, voice input, and the like. The candidate picture area 204 shows the synthesized candidate teletext picture composition, hereinafter referred to as candidate composition picture. If the candidate composite map is larger, the thumbnail of the candidate composite map is displayed. In the candidate picture area 204, the user may select a candidate composite picture. For example, the user may directly click on one of the candidate composite maps in the candidate map area 204; alternatively, the user may click on a space and select the first candidate composite map of the candidate composite maps. In some embodiments, the candidate picture area 204 can be expanded to display more candidate composite pictures. For example, the candidate composite maps of the candidate map region 204 can be slid left and right to present other candidate composite maps. Alternatively, the candidate picture area 204 can be expanded into the input area 206 to present additional candidate composite pictures.

The input area 206 may provide a keyboard or voice input interface for use by a user. The keyboard includes but is not limited to: pinyin 9 key, pinyin 26 key, handwriting keyboard, chinese stroke keyboard, chinese five-stroke, etc.

In addition, the candidate picture area 204 of the input interface also has a separate area for displaying the candidate pictures provided by the library module 104, i.e., an area for displaying the candidate pictures without synthesized onscreen text, to provide an opportunity for selection of the candidate pictures by the user. For example, a "my" button and/or a cloud tag button (not shown in the figure) are provided in the candidate picture area 204. The user may click on the "my" button, so that the candidate picture area 204 shows that the user stores one or more candidate pictures, and usually, the picture corresponding to the "my" button is stored through user operation, and the storage location may be local or cloud. In some embodiments, the candidate pictures stored locally or in the cloud may be divided into a plurality of groups, for example: "favorite", "most frequently used", "pop-up", "animal", etc., whereby it is possible to more conveniently make the user find a picture desired by himself. And a cloud label button, corresponding to a cloud storage location. The user clicks on the "My" or cloud Mark button, and one or more candidate pictures in the storage location may be displayed for selection by the user.

The user inputs characters through the input area 206, and the input characters are displayed in the character display area 202. The user input acquisition module 102 acquires the character string input by the user from the character display area 202. And sends it to the library module 104 and the onscreen text prediction module 106. In another embodiment, the user input acquisition module 102 stores a string of historical inputs. When the user inputs a character through the input area 206, the user input acquisition module 102 provides the user with a character string including the character, which is input historically, according to the character, which is input by the user, so that the character string to be input by the user can be obtained quickly.

The gallery module 104 is configured to obtain one or more candidate pictures/videos based on a character string input by a user. In one embodiment, the gallery module 104 is located in the cloud, and all candidate pictures provided by the gallery module come from a cloud server. In another embodiment, the gallery module 104 provides both local storage and candidate pictures from the cloud server.

In some embodiments, the candidate picture includes a text region. The text area of the candidate picture is defined to be able to accommodate one or more texts. In some embodiments, the candidate pictures are adjusted to leave the position of the text area, so that the pictures with the text added are more attractive. Further, in order to ensure the aesthetic degree, one or more of the size, font, layout, and color of the text contained in the text region are predefined. Also, there is generally a limit to the number of words that a word region can accommodate. If the number of added characters exceeds the number of characters that can be accommodated by the character area, the character area may display only the maximum number of characters that can be accommodated, with the remaining characters being replaced with symbols such as ellipses. In some embodiments, the above-mentioned words include one or more of chinese characters, foreign words, numbers, punctuation marks, and the like. In some embodiments, the candidate pictures may be one or more of line drawings, grayscale drawings, color drawings, photographs, and the like. The background of the candidate picture may be white, gray, light blue, green, blue, black, etc. In some embodiments, the text in the text region has dynamic properties. For example, the text may be enlarged or reduced, rotated, discolored, edge-lit, and the like.

In some embodiments, the candidate picture may be a motion picture. For example, the candidate picture includes a motion picture of a plurality of sub-pictures. Each sub-picture comprises a respective text area. The text area of each sub-picture may be different. In some embodiments, the text added in the text area of each sub-picture is consistent. Thus, although the sub-picture is converted to form the motion picture, the characters presented to the user by the entire motion picture are consistent. In other embodiments, the text added in the text area of each sub-picture is not consistent. The text areas of the individual sub-pictures are combined to be added text. For example, the motion picture includes 3 sub-pictures, and the text to be added is "i love you"; then the text areas of the 3 sub-pictures are added with "i", "love" and "you", respectively. Thus, the candidate pictures dynamically present the added text "i love you" to the user. In some embodiments, the switching of adding text in each sub-picture of the candidate picture may have a special effect. These effects include, but are not limited to: fade-in and fade-out, small to large or large to small then disappear, left to right or right to left then disappear, top to bottom or bottom to top then disappear, etc. Those skilled in the art will appreciate that candidate videos may also be processed in a similar manner. In some examples, the candidate video is capable of playing on-screen text.

FIG. 3 is a schematic diagram of a gallery module according to one embodiment of the invention. As shown, the gallery module 300 includes a keyword extraction module 302, an attribute extraction module 303, a search module 304, an index gallery 305, a gallery 306, and a thumbnail gallery 308. The keyword extraction module 302 and the attribute extraction module 303 are optional modules, and in some embodiments, the search module 304 may directly search and match in the gallery 306 by using a character string input by a user. In other embodiments, in order to improve the speed and matching degree of the retrieval and make the provided candidate pictures more suitable for the needs of the user, the keyword extraction module 302 may be included to extract keywords in the character string, and the attribute extraction module 303 is used to extract attributes of the character string, such as recognition, derogation, neutrality, praise, irony, user history and preference, etc.

In the present embodiment, a plurality of candidate pictures are stored in the gallery 306. Each candidate thumbnail picture is included in the thumbnail library 308 as a corresponding candidate thumbnail picture. Those skilled in the art will appreciate that thumbnail library 308 is optional. The technical solution of the present invention can be implemented without including the thumbnail library 308.

The candidate pictures in the gallery 306 include picture descriptions. The picture description may be one or more words (e.g., keywords), a piece of text, or a combination of one or more words or text and mood. In some embodiments, the picture description describes lines or subtext that match the candidate picture, such as "you are really too beautiful", "i don't hold up the wall and get you" and so on. In some embodiments, the picture description illustrates scenes that the candidate picture fits in describing, such as "busy", "upside down", "halo", and the like. In some embodiments, the picture description illustrates the content, atmosphere, sound, smell, taste, etc. of the candidate picture, e.g., "yellow river," "true scent," "too sweet," etc. In some embodiments, the picture descriptions of the candidate pictures are one or more of the above types of picture descriptions. The above is merely a picture description exemplarily illustrating candidate pictures. Other types of picture descriptions may also be included for the candidate pictures to match the needs of the user.

In some embodiments, the candidate pictures include text. The candidate picture may include text that is considered to be part of the candidate picture and is not alterable. A candidate picture that includes text may contain a text region. Alternatively, a candidate picture including text may not contain a text region. When the candidate picture does not include the text area, if the user selects the candidate picture of the type, then in one case, the characters input by the user are the same as the texts included in the candidate picture, the picture containing the texts and desired by the user is obtained without the need of the text-text synthesis, and the step of the text-text synthesis can be omitted. In another case, the text on the screen is different from the text included in the candidate picture, and then the user selects the candidate picture different from the text on the screen to indicate that the user wants to change the content on the screen to obtain the desired picture including the text, which can be regarded as that the steps of changing the content on the screen and synthesizing the text are omitted. Therefore, even pictures that do not include text regions can be stored in the gallery 306 as candidates for the present invention. In some embodiments, the candidate picture and/or the supplemental candidate picture may include text but not include a text region. If the user selects such candidate pictures, the user can be considered to confirm that the characters included in the pictures are the characters on the screen.

In some embodiments, the candidate picture comprises a picture classification. The picture category describes the category to which the picture belongs. The picture classification helps to provide candidate pictures according to user preferences to better meet the needs of the user. For example, the user's preference is a lovely small animal. When candidate pictures are provided, candidate pictures that satisfy both animals and sprouts have increased weight when sorted. Thereby, the user can be more satisfied when providing the candidate picture. Likewise, in some embodiments, picture classification may also facilitate obtaining user preferences, alone or in combination with other user information, for a precise representation of a user.

Table 1 below is an example of candidate pictures in a gallery:

table 1: chart table

	Picture name	Characters in picture	Picture classification	Picture description
						1	Pick up hill 0028	Is free of	General purpose, children	Who? …
2	Octopus 0012	Is free of	Efficients, animals	Who is my? …
					3	Small red cap 0010	Asking who did i?	Sprout and children	Brave and wisdom …
4	…	…	…	…

In some embodiments, the index repository 305 is used to store indexes created from one or more of picture descriptions, text in pictures, and picture classifications, as will be understood by those skilled in the art, the methods for creating indexes known in the art can be applied to create indexes for the gallery 306. These indices are stored in an index repository 305.

The search module 304 uses the index stored in the index database 305, and uses the character string input by the user as a search condition to implement the search matching of the candidate picture and the character string.

In one embodiment, when a character string is long, its keywords may be extracted as search conditions. Wherein, the keyword is one or more words capable of indicating semantics. For example, the keyword extraction module 302 obtains the keywords of the character string in the following manner. First, a character string is divided into a plurality of words according to semantics. For example: "what is your arrangement today? "this string can be divided into the following 7 parts" today/you/arrangement/what/? ", where"? "is a punctuation mark. The character string includes 6 words. Then, the particle and pronouns are removed according to the nature of the word. For example, after removing the particle, the remaining words are "today, scheduled". Next, words are given different weights according to their attributes. For example, the weights of words representing subjects, predicates, and objects are greater than the weights of words representing determinants; the weights of words representing a fixed and complementary language are greater than the weights of words representing a shape. In the above example, the weight of "arrange" is greater than the weight of "today". Thus, as indicated above, in some embodiments, the keyword extraction module obtains keywords and weights for the keywords. In the above example, the keyword extraction module 302 obtained the keywords "arrange" and "today"; where the weight of "scheduled" is greater than the weight of "today". In some embodiments, the number of keywords obtained by the keyword extraction module is limited. The lower weighted keywords may be omitted. In some embodiments, the mood of the string is also extracted as a keyword.

As will be appreciated by those skilled in the art, the above method is merely illustrative of the technical solution of the present invention and does not limit the scope thereof. The prior art methods of automatic semantic analysis can be applied to extract keywords in a string. The retrieval process can be simplified by acquiring the keywords of the character string, and the speed and the accuracy of searching and matching are improved.

The attribute extraction module 303 is an optional module. The character string may be further analyzed by the attribute extraction module 303 after the character string is obtained, so as to obtain attributes of the character string, such as recognition, derogation, neutrality, praise, irony, user history and preference, etc. The attributes are advantageous for recommending candidate pictures to the user.

In some embodiments, the index library 305, gallery 306, and thumbnail gallery 308 may be located locally. In some embodiments, index library 305, gallery 306, and thumbnail gallery 308 may be located on a cloud server. As previously described, the local and cloud servers may each include an index library 305, a gallery 306, and a thumbnail gallery 308 to provide candidate pictures that match the character string.

In some embodiments, the search module 304 ranks the search match results and sends the top ranked result or results to the teletext composition module 108. In some embodiments, the search module 304 can obtain the number of pictures, e.g., 5, presented to the user in the input interface. The search module 304 obtains thumbnails of top-ranked candidate pictures from the thumbnail database, for example, thumbnails corresponding to top-ranked 5 candidate pictures are sent to the teletext synthesis module 108. In some embodiments, the search module provides more thumbnails of the candidate pictures than the number of pictures that the input interface can present to the user, so that the user can present more candidate pictures quickly when the user wishes to present more candidate pictures.

In some embodiments, the search module 304 ranks the matching results based on one or more of the following factors: (1) matching degree of characters or keywords thereof with picture description of the candidate picture and/or characters in the picture; (2) matching degree of characters or keywords thereof with candidate picture categories; (3) selecting a historical record of candidate pictures by a user; (4) matching degree of user preference and candidate picture category; (5) degree of match of user attributes to candidate picture categories (6) popularity of a candidate picture in its picture category; (7) the general degree of the candidate pictures; (8) the ratio of the candidate picture category in the retrieval result; and so on. As will be appreciated by those skilled in the art, the above is merely an exemplary illustration of some factors that may apply to candidate picture ordering and does not encompass all factors that may be possible. Other factors that are beneficial to provide the user's desired or better graphics effect may also be indicators of candidate picture ordering references.

In some embodiments, the above ranking factor of the candidate pictures is embodied by the ranking of the candidate pictures. For example, the higher the degree of matching, the higher the weight. In some embodiments, the weight of a string or its keywords that are completely consistent with the text in the picture is higher than the weight of a string or its keywords that are contained in the text in the picture. However, different factors have different top weights. For example, the highest weight of the matching degree of the character string or the keyword thereof with the characters in the candidate pictures is greater than the highest weight of the matching degree of the character string or the keyword thereof with the pictures in the candidate pictures. In other words, if the character is completely consistent with the text in the first candidate picture; likewise, also in full agreement with the picture description of the second candidate picture, the first candidate picture is ordered further forward than the second candidate picture. Other ranking factors can also be embodied in the ranking by adjustment of weights, as will be appreciated by those skilled in the art. In some embodiments, the search module 304 forms personalized search results by dynamically adjusting the weights of the candidate pictures to better match the needs of the user. Other methods related to weight adjustment in the prior art can also be applied to the method, so that the technical effect of the invention is better improved.

According to an embodiment of the present invention, due to the limited number of candidate pictures, the search module 304 has a much higher probability of "missing" in the process of retrieving a match, i.e., no matching candidate picture appears. Therefore, when there are no or insufficient candidate pictures to match, the search module 304 is required to provide additional candidate pictures. On the other hand, unlike character matching, the degree of freedom of picture matching is higher. In other words, characters with different styles may be matched with the same candidate picture to achieve a good effect. The picture descriptions cannot exhaust these collocations or effects. Thus, the rules for the search module to provide additional candidate pictures are more flexible.

In some embodiments, the search module provides the supplemental candidate pictures when there are no or insufficient number of matching candidate pictures. For example, the supplemental candidate pictures may be randomly obtained from a gallery. Since the picture collocation is more flexible, even if candidate pictures are randomly provided, the possibility that a user can select a suitable candidate picture from the randomly acquired candidate pictures is very high. Of course, it may be better to provide supplemental candidate pictures based on user history and preferences. Thus, in some embodiments, supplemental candidate pictures are provided according to the history of user selection of candidate pictures. In some embodiments, the supplemental candidate pictures are provided according to a user's preference. If user attribute information is available, supplemental candidate pictures may also be provided based on user attributes. In some embodiments, currently popular topics may also be good choices. For example, if a movie is currently being shown, then the candidate pictures that provide the theme of the movie may also meet the user's expectations.

In some embodiments, picture classification may also be useful when providing supplemental candidate pictures. For example, if candidate pictures are randomly provided from various picture categories, then more stylistic candidate pictures will be presented to the user. The likelihood that the user finds a satisfactory candidate picture is also higher.

As mentioned above, the candidate picture is provided based on the character string input by the user or the keyword thereof, and the supplementary candidate picture is provided for assistance, and the candidate picture provided by the invention can be better matched with the character input by the user or the keyword thereof, so that the requirement of the user is better met, and the better expression effect is achieved.

The onscreen text prediction module 106 is configured to predict onscreen text based on a character string input by a user. In some embodiments, the onscreen text prediction module 106 is located at a server. In some embodiments, the onscreen text prediction module 106 is similar to the thesaurus module 103. The network lexicon relied on by the on-screen text prediction module 106 is larger than the local lexicon and can be updated in real time, so the prediction is more accurate. Furthermore, other modes can be used for prediction, for example, historical input of a user is stored in the system, and the historical input containing the current character string is predicted as characters on a screen; or predicting the characters on the screen according to the user preference acquired in the system. The onscreen text prediction module 106 sends the predicted onscreen text to the gallery module 104 to search for candidate pictures/videos. In some embodiments, the client of the input method system includes a communication module to send the character string input by the user to the server. After obtaining the candidate pictures/videos via the onscreen text prediction module 106 and the gallery module 104 located on the server, the communication module receives one or more candidate pictures/videos from the gallery module from the server. In some embodiments, the communication module also receives predicted onscreen text from onscreen text prediction module 106 in the server.

The teletext synthesis module 108 receives the candidate pictures/videos from the gallery module 104 and the predicted onscreen text from the onscreen text prediction module 106, adds the predicted onscreen text to the candidate pictures/videos, and generates one or more teletext synthesis pictures/videos containing the predicted onscreen text.

Referring to fig. 2, the teletext composition module 108 displays the composited teletext composite picture/video in the candidate picture area 204 for selection by the user. In some embodiments, the number of candidate composite maps generated by the teletext composition module 108 is greater than the number of candidate picture regions 204 that can be presented. Thus, when the user wishes to view more candidate composite maps, the user can be presented with other candidate composite maps to which the candidate words on the screen are added more quickly.

Further, the teletext composition module 108 provides the one or more teletext composition pictures/videos to the user via the interface module 101. The user can click to select one of the teletext pictures/videos to which the predicted text is added or select the first teletext picture/video to which the predicted text is added by clicking a space. Optionally, the text area in the teletext picture/video has editable properties, i.e. allowing the user to adjust the position and/or size of the text area as well as the font size, font style, color, etc. of the text on the screen. And responding to the confirmation operation of the user, and outputting the image-text composite picture/video added with the upper screen characters. Alternatively, the user selects a teletext picture/video with the onscreen text added already enough to indicate a confirmation operation. To simplify the operation, the interface module 101 directly outputs the teletext picture/video with the text on the screen added thereto in response to a selection by the user.

In some embodiments, the teletext composition module 108 retrieves the predicted teletext and adds the predicted teletext to the thumbnails of the plurality of candidate pictures provided from the gallery module 104 to form a plurality of thumbnails of teletext pictures including the teletext. The thumbnail is smaller in size and more suitable for being displayed on the input interface. If the user selects a thumbnail of a candidate picture including the text on the screen, the interface module 101 outputs the picture-text composite picture including the text on the screen in the original size in response to the user's confirmation operation. The teletext composition module 108 may be used to create candidate composite images of the original size, including the text on the screen, either simultaneously with or after the creation of the thumbnail image.

In some embodiments, after the user input obtaining module 102 obtains a newly input character or character string, the newly input character or character string is sent to the on-screen text prediction module 104, the on-screen text prediction module 104 predicts again to generate a new predicted on-screen text, and sends the new predicted on-screen text to the image-text composition module 108, and the image-text composition module 108 updates the on-screen text in the original text composition map.

Further, the user input obtaining module 102 analyzes the character or the character string newly obtained by the user input, and when the newly added character does not affect the semantics, the attribute, and the like of the original character string, only the new character is sent to the screen text prediction module 104 for updating the screen text. If the newly added character causes the semantics, attributes and the like of the original character string to change, the newly added character is sent to the library module 104 to re-provide the candidate picture.

FIG. 4 is a schematic diagram of an input method system according to another embodiment of the invention. As shown in fig. 4, the input method system provided in this embodiment further includes a word library module 103 on the basis of the input method system shown in fig. 1. The lexicon module 103 can provide one or more candidate words from the characters entered by the user. The thesaurus module 103 may be local to the client of the input method system. Those skilled in the art will appreciate that the lexicon module 103 is not necessary.

When the user input acquisition module 102 acquires the characters input by the user, the characters are sent to the word stock module 103, and the word stock module 103 searches corresponding words from the word stock according to the characters and returns the words to the interface module 101 for display so as to be selected by the user. As will be appreciated by those skilled in the art, the prior art implementations for providing candidate words may be applied herein to provide candidate words that best match the characters entered by the user.

FIG. 5 is a schematic diagram of an input interface according to one embodiment of the invention. The input interface includes a candidate word area 208 where the user can select candidate words on the screen. The user may click directly on a candidate word in the candidate word area 208. When the user selects one candidate word, the communication module sends the candidate word selected by the user to the server. The text-on-screen prediction module 106 updates the original predicted text-on-screen according to the selected text-on-screen words. The gallery module 104 updates the candidate pictures/videos with the updated predicted onscreen text. When the new character or new candidate changes the semantics of the original predicted onscreen text, gallery module 104 performs a supplemental search based on the new character to provide a new candidate picture. The teletext composition module 108 regenerates the teletext composite image. Further, the updated predicted on-screen text is also sent to the teletext combiner module 108. The teletext synthesis module 108 updates the on-screen text in the original text synthesis map.

In an embodiment of the present invention, the text in the text area of the candidate picture has a dynamic attribute, for example, the text is displayed in a dynamic change manner such as highlighting, flashing, color changing, and the like. Optionally, the text area in the generated image-text composite picture has editability, that is, the text area is allowed to be adjusted by the user to the position and/or size of the text area, and the size, font, format, color and the like of the text on the screen.

FIG. 6 is a schematic flow diagram of an input method according to one embodiment of the invention. As shown in fig. 6, the input method includes the steps of:

step S600, a character string input by a user is acquired. In some embodiments, a user enters characters through a keyboard at an input interface, and a set of the entered characters is used as a character string input by the user. Alternatively, when the history input of the user is stored in the system, the stored history input character string may be queried according to the character input by the user, and the character string including the character input in the history may be used as the character string input by the user. As will be appreciated by those skilled in the art, any method of obtaining a character or string of characters input by a user in an input method known in the art may be applied to obtain the user input.

Step S601, sending the character string input by the user to a server; and predicting, at the server, on-screen text based on the character string input by the user. For example, on-screen text is predicted in terms of user history input, user preferences, and the like. This is similar to the process of predicting the user's on-screen text using a local thesaurus.

In step S602, one or more candidate pictures/videos are obtained at the server based on the character string input by the user. Specifically, as shown in fig. 7, a schematic diagram of a method for providing candidate pictures according to an embodiment of the present invention is shown. The method comprises the following steps:

step S700, generating a search condition, where the search condition may be a character string, and may be a keyword extracted from the character string, extracting a character string attribute, such as recognition, derogation, neutrality, praise, irony, etc., extracting a user history and preference, etc., so as to improve search efficiency and matching accuracy.

In step S701, a candidate picture matching the search condition is retrieved from the gallery. In some embodiments, the candidate pictures in the gallery include one or more of a picture description, a picture classification, and text in a picture. Searching for matching candidate pictures based on the character strings or keywords thereof by using indexes established according to one or more of candidate pictures including picture descriptions, picture classifications and words in the pictures. As will be appreciated by those skilled in the art, other ways of searching for matching candidate pictures may be applied herein to enable retrieval of candidate pictures.

In some embodiments, the method further comprises the step of ranking the matched candidate pictures. For example, the matching candidate pictures have different weights according to various factors such as the matching attribute and the matching degree. And sorting the candidate pictures according to the weights of the candidate pictures.

In step S702, in response to no matching candidate picture or insufficient number of candidate pictures, the search candidate picture is complemented. The number of candidate pictures is determined by the number of pictures that can be presented by the input interface and the number of additional candidate pictures that are provided to facilitate the user to quickly browse through more pictures. Therefore, the number of candidate pictures is generally greater than or equal to the number of pictures that can be presented by the input interface.

In some embodiments, the supplemental candidate picture is random. Alternatively, the supplemental candidate pictures are based on one or more of user history and/or user preferences, popularity of the candidate pictures, categories of the candidate pictures, and the like.

In step S703, a matching candidate picture and/or a supplementary candidate picture is provided.

Step S603, receiving one or more candidate pictures/videos obtained based on predicted on-screen text of a character string input by a user from a server; and generating one or more teletext composite pictures/videos containing the predicted onscreen text based on the predicted onscreen text and the one or more candidate pictures/videos. In one embodiment, the predicted onscreen text is added to a text region in the candidate picture, thereby generating a teletext picture. Optionally, a teletext picture thumbnail may also be generated.

Step S604, the generated image-text composite picture is provided to the user through the candidate picture area 206 in the user interface. The teletext picture composition may be displayed to the user via the candidate picture area 206, or a thumbnail of the teletext picture composition may be displayed.

Step 605, determining whether a newly input character is received, if the newly input character is received, in step S606, updating the predicted on-screen character according to the newly input character. If the character newly input by the user is not received, step S608 is performed.

Step S607, the predicted on-screen text in the image-text composite picture is updated.

Step S608, determining whether the image-text composite picture selected by the user is received, if not, returning to step S605, if so, responding to the selection of the user in step S609, and outputting the image-text composite picture.

In the embodiment of the invention, the character string is directly used for predicting the on-screen character and then searching the candidate picture/video at the server, the selection of a user does not need to be waited, and the picture/video searching speed can be greatly improved. In some embodiments, the user may not have to complete the entire input in some cases because of the higher accuracy of the prediction. As the user selects the candidate words, the characters in the candidate pictures/videos are changed, a feeling of inputting the characters in the pictures is provided for the user, and the user experience can be improved.

The above embodiments are provided only for illustrating the present invention and not for limiting the present invention, and those skilled in the art can make various changes and modifications without departing from the scope of the present invention, and therefore, all equivalent technical solutions should fall within the scope of the present invention.

Claims

1. An input method system, comprising:

a user input acquisition module configured to acquire a character string input by a user;

a text-on-screen prediction module configured to predict text-on-screen based on a string of characters input by a user;

a gallery module configured to obtain one or more candidate pictures/videos based on-screen text predicted by a user input string; and

a teletext composition module configured to generate one or more teletext composition pictures/videos including the predicted onscreen text based on the one or more candidate pictures/videos.

2. The input method system of claim 1, wherein the onscreen text prediction module and the gallery module are located on a server.

3. The input method system of claim 2, further comprising a thesaurus module at the client configured to provide one or more candidate words based on a character string input by a user; the on-screen text prediction module updates predicted on-screen text in response to the candidate word selected by the user.

4. The input method system of claim 2, further comprising a communication module at the client configured to send a user-entered string to the server; and receiving one or more candidate pictures/videos from the gallery module from the server.

5. The input method system of claim 1, further comprising an interface module configured to provide an input interface further comprising one or more of the following regions:

an input area configured for inputting a character or a character string;

a candidate word region configured to provide one or more candidate words; and

a candidate picture area configured for displaying one or more candidate pictures/videos or corresponding thumbnails, or one or more teletext composite pictures/videos or corresponding thumbnails.

6. The input method system of claim 3, wherein the interface module is configured to output a corresponding teletext picture/video in response to a user selection of one teletext picture/video or corresponding thumbnail.

7. The input method system of claim 1, wherein the user input acquisition module is configured to store a character or a character string of a user's historical input, and when a user inputs a character, the user is provided with the historically input character string including the character according to the character input by the user.

8. An input method, comprising:

acquiring a character string input by a user;

transmitting a character string input by a user to a server;

receiving one or more candidate pictures/videos obtained based on predicted on-screen text of a character string input by a user from a server; and

based on the predicted onscreen text and the one or more candidate pictures/videos, one or more teletext composite pictures/videos containing the predicted onscreen text are generated.

9. The input method of claim 8, further comprising: and sending the new character input by the user to the server in response to receiving the new character input by the user.

10. The input method of claim 8, further comprising: querying a local thesaurus based on a character string input by a user to provide one or more candidate words; and sending the candidate word selected by the user to the server.

11. The input method of claim 10, further comprising: and responding to the candidate words selected by the user, and updating the predicted on-screen characters on the server by adopting the candidate words selected by the user.

12. The input method of claim 10, further comprising: and responding to the candidate words selected by the user, and updating the predicted on-screen characters in the image-text synthesis image/video by adopting the candidate words selected by the user.

13. The input method of claim 8, further comprising: inquiring a stored historical input character string according to characters input by a user; and providing a historical input string including the character.

14. The input method of claim 8, further comprising: providing one or more teletext composite pictures/videos including the predicted onscreen text to the user; and outputting the selected teletext composite picture/video in response to a selection by the user.

15. The input method of claim 8, further comprising: after generating one or more teletext composite pictures/videos containing the predicted onscreen text, corresponding teletext composite picture/video thumbnails are generated.

16. The input method of claim 8, further comprising: providing one or more teletext picture/video thumbnails including the predicted onscreen text to the user; and outputting the selected teletext composite picture/video in response to a selection by the user.