WO2020056948A1

WO2020056948A1 - Method and device for data processing and device for use in data processing

Info

Publication number: WO2020056948A1
Application number: PCT/CN2018/121235
Authority: WO
Inventors: 刘文文; 李琳; 潘牧野
Original assignee: 北京搜狗科技发展有限公司
Priority date: 2018-09-20
Filing date: 2018-12-14
Publication date: 2020-03-26
Also published as: CN110929122A; CN110929122B

Abstract

Provided in the embodiments of the present application are a method and device for data processing and a device for use in data processing. The method specifically comprises: determining information to be replied from clipboard information; the information to be replied corresponding to the entirety or a portion of the clipboard information; determining a reply candidate corresponding to the information to be replied; and displaying the reply candidate. The embodiments of the present application increase reply efficiency and implement smart replying in a scenario of replying across communication windows.

Description

Data processing method, device and device for data processing

This application claims the priority of a Chinese patent application filed on September 20, 2018 with the Chinese Patent Office, application number 201811101152.5, and the invention name "a data processing method, device, and device for data processing". Incorporated by reference in this application.

Technical field

The present application relates to the field of communications technologies, and in particular, to a data processing method, device, and device for data processing.

Background technique

With the development of communication technology, communication applications such as SMS applications and instant messaging applications can provide users with information interaction functions to enable different users to exchange information. For example, different users can send text messages to each other through a text messaging application. As another example, different users can also send messages to each other through an instant messaging application.

In practical applications, when a user receives information sent by a communication peer, he or she usually needs to think about appropriate statements to reply to the information, and the response efficiency is low.

Summary of the Invention

The embodiments of the present application provide a data processing method and device, and a device for data processing, which can improve the response efficiency, and can realize intelligent reply in a reply scenario across communication windows.

In order to solve the above problem, an embodiment of the present application discloses a data processing method, including:

Determining to-be-reply information from the clipboard information; the to-be-reply information corresponds to all or part of the clipboard information;

Determining a reply candidate corresponding to the information to be replyed;

The reply candidate is displayed.

On the other hand, an embodiment of the present application discloses a data processing device, including:

A to-be-reply information determining module, configured to determine to-be-reply information from the clipboard information; the to-be-reply information corresponds to all or part of the clipboard information;

A reply candidate determination module, configured to determine a reply candidate corresponding to the information to be replyed; and

The reply candidate display module is configured to display the reply candidate.

In another aspect, an embodiment of the present application discloses a device for data processing, including a memory, and one or more programs. One or more programs are stored in the memory, and are configured to be read by one or one. The above processor executes the one or more programs including instructions for:

In another aspect, an embodiment of the present application discloses a machine-readable medium having instructions stored thereon that, when executed by one or more processors, cause a device to execute the data processing method according to one or more of the foregoing.

The embodiments of the present application include the following advantages:

The embodiment of the present application supports a user to carry information to be responded to through a clipboard, so that the information to be responded can be determined from the clipboard information, and a reply candidate corresponding to the information to be responded is provided, so the response efficiency can be improved. The embodiment of the present application can be applied to a response scenario across communication windows. Specifically, a user can copy information in a first communication window and jump to a second communication window. The embodiment of the present application can automatically The information to be answered is determined in the information, and a reply candidate corresponding to the information to be responded is provided. Therefore, an intelligent reply can be implemented in a reply scene across the communication window to improve the reply efficiency.

The embodiments of the present application may also be applied to a reply scenario in a communication window. The reply scene in the communication window is specifically: the information received by the user and the information sent are located in the same communication window. For operating systems with stricter security requirements, such as the IOS system, it cannot support the direct reading of the screen content. Therefore, the embodiment of the present application can determine the information to be responded by using the clipboard information.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to explain the technical solution of the embodiments of the present application more clearly, the drawings used in the description of the embodiments of the application will be briefly introduced below. Obviously, the drawings in the following description are just some embodiments of the application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without paying creative labor.

FIG. 1 is a schematic diagram of an application environment of a data processing method according to an embodiment of the present application; FIG.

2 is a flowchart of steps in a first embodiment of a data processing method according to the present application;

3 is a flowchart of steps in a second embodiment of a data processing method of the present application;

4 is a schematic diagram of an interface according to an embodiment of the present application;

5 is a structural block diagram of an embodiment of a data processing apparatus of the present application;

FIG. 6 is a block diagram of an apparatus 800 for data processing of the present application; and

FIG. 7 is a schematic structural diagram of a server in some embodiments of the present application.

detailed description

In the following, the technical solutions in the embodiments of the present application will be clearly and completely described with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

An embodiment of the present application provides a data processing scheme, which may include: determining information to be responded from the clipboard information; the information to be responded may correspond to all or part of the clipboard information; and determining the information to be responded to Reply candidates corresponding to the information; displaying the reply candidates.

In the embodiment of the present application, the clipboard is an area in the memory and is a plug-in in a preset program. With the clipboard, the user can use a simple cut, copy, paste and other actions to select the selected information in various ways. Pass and share between applications. The clipboard uses the terminal's internal resource memory, or virtual memory, to temporarily save cut and copied information.

The preset programs may specifically include: browser programs, instant messaging programs, social network (for example: Weibo, forum, news, etc.) programs, and other applications with clipboard functions.

The information in the clipboard is selected by the user. The type of clipboard information may include at least one of the following types: text, picture, audio, video, and page address.

In practical applications, the clipboard may include only one piece of information; or, the clipboard may include multiple pieces of information. For example, the clipboard may include three pieces of information, where the first piece of information may be a text type, the second piece of information may be a voice type, and the third piece of information may be a picture type. It can be understood that the embodiment of the present application does not limit the amount of the clipboard information and the specific type of the clipboard information.

The embodiments of the present application may be applicable to a reply scenario across communication windows. The cross-communication window may include: different communication windows of different applications, or different communication windows of the same application.

Different windows of different applications may include: a communication window of a short message application and a communication window of an instant messaging application, or a communication window of a first instant messaging application and a communication window of a second instant messaging application. For example, user A receives message A sent by user B through a short message application. Since sending of a short message consumes corresponding short message charges, the embodiment of the present application supports user A to send a reply corresponding to message A to user B through an instant messaging application. Correspondingly, user A can copy information A and jump to the communication window between user A and user B in the instant messaging application. In this embodiment of the present application, the clipboard information (that is, the content of information A) can be used as The information to be responded to, and a reply candidate is automatically provided for user A to choose, so that user A can achieve a quick reply in the case of cross-scenarios, which can improve the response efficiency.

Here are descriptions of different windows for the same application. For example, user C receives information B sent by user D through communication window A of the instant messaging application. Because communication window A is a group window, the reply corresponding to information B may involve privacy. Therefore, the embodiment of this application supports user C through the instant messaging application. Communication window B sends a reply corresponding to message B to user D. Correspondingly, the user C can copy the information B and jump to the communication window B between the user C and the user D in the instant messaging application. In this embodiment of the present application, the clipboard information (that is, the content of the information B) As the to-be-reply information, a reply candidate is automatically provided for user C to choose, so that user C can achieve a quick reply under the circumstance of cross-scenario, thereby improving the reply efficiency. For example, user D is the leader of user C, communication window A is the communication window of the work group, and communication window B is the communication window between user C and user D. Since the reply generated in the communication window of the work group may be excluded by user C See with users other than user D, so it is not convenient to reply to message B in the communication window of the work group.

It should be noted that the above-mentioned user jumps to the second communication window, which is only an optional embodiment. In fact, the solution in this embodiment of the present application may not depend on the jump of the communication window, even if the user does not jump to the communication window In the embodiment of the present application, the information to be responded can be automatically determined from the clipboard information, and a reply candidate corresponding to the information to be responded is provided.

The data processing method provided in the embodiment of the present application may be applied to an application environment such as a website and / or an APP (Application, Application) to improve the response efficiency. For example, the APP may be a communication application, and the website may be a webpage for providing a communication service, and the like.

The data processing method provided in the embodiment of the present application can be applied to the application environment shown in FIG. 1. As shown in FIG. 1, the client 100 and the server 200 are located in a wired or wireless network. Through the wired or wireless network, the client 100 Perform data interaction with the server 200.

Optionally, the client 100 may run on a terminal. The above terminals include, but are not limited to, a smartphone, a tablet, an e-book reader, and MP3 (Motion Picture Expert Compression Standard Audio Level 3, Moving Picture Experts Group Audio Layer III ) Player, MP4 (Moving Picture Expert Compression Standard Audio Level 4, Moving Picture Experts Group Audio Layer 4) player, laptop portable computer, car computer, desktop computer, set-top box, smart TV, wearable device and so on.

The client 100 may be an APP running on the device, such as an instant messaging APP, a short message APP, an input method APP, or an APP built into the operating system. The embodiment of the present application does not limit the specific APP corresponding to the client.

Method Example One

Referring to FIG. 2, a flowchart of steps in a first embodiment of a data processing method according to the present application is shown, which may specifically include the following steps:

Step 201: Determine to-be-reply information from the clipboard information; the to-be-reply information may correspond to all or part of the clipboard information;

Step 202: Determine a reply candidate corresponding to the information to be replyed.

Step 203: Display the reply candidates.

At least one step of the method embodiment shown in FIG. 2 may be executed by a server and / or a client. Of course, the embodiment of this application does not limit the specific execution subject of each step.

The method embodiment in FIG. 2 may correspond to a trigger condition.

According to an embodiment, the triggering condition may include: the input method keyboard is turned up. The keyboard of the input method is invoked, which can indicate that the user wants to reply by inputting. Therefore, the method in the embodiment of the present application can be triggered.

According to another embodiment, the trigger condition may include: the clipboard information is updated. The updated clipboard information may indicate that the user has generated new clipboard information, indicating that the user has a response requirement, and therefore, the method of the embodiment of the present application may be triggered.

According to yet another embodiment, the trigger condition may include: the clipboard information is updated and the input method keyboard is invoked. In this case, it indicates that the user has a requirement to reply through input, and therefore, the method in the embodiment of the present application can be triggered.

According to another embodiment, the trigger condition may include: after the clipboard information is updated, jumping to a communication window. Jumping to the communication window can refer to jumping from the interface before the clipboard operation to the communication window. In this case, it can be explained that the user wants to reply through the communication window after the jump, so the method of the embodiment of the present application can be triggered. . The interface before the clipboard operation can be a communication window or a non-communication window.

According to another embodiment, the trigger condition may include: after the clipboard information is updated, jumping to a communication window, and the input method keyboard is invoked.

It can be understood that the above trigger condition is only an optional embodiment. In fact, those skilled in the art can determine the above trigger condition according to actual application requirements. For example, the above trigger condition may also be a preset gesture of a user. The specific trigger conditions are not limited.

In step 201, the clipboard information can be obtained by accessing the clipboard. Clipboard information can include: one piece of content, or multiple pieces of content. For example, in response to the user's long-press operation, the selection interface corresponding to each piece of information can be displayed in the communication window, so that the user can select at least one piece of information to copy

Step 201 may determine to-be-reply information from the clipboard information, and the to-be-reply information may correspond to all or part of the clipboard information. For example, the clipboard information may include: the information and the sender identification of the information. The sender identification may be filtered from the clipboard information, and the information may be retained as the information to be responded to.

In the embodiment of the present application, optionally, the type of the clipboard information may include at least one of the following types: text, picture, audio, video, and page address. The embodiment of the present application may determine at least one type of information to be responded to, and determine a corresponding reply candidate for the at least one type of information to be responded to.

The embodiments of the present application may provide the following technical solutions for determining a response candidate corresponding to the information to be responded to:

In the technical solution 1, the step 202 of determining the reply candidate corresponding to the information to be specifically answered may include: determining a theme corresponding to the information to be responded to; and determining a reply candidate corresponding to the information to be responded according to the theme.

The subject can refer to the central idea of the message to be answered. In the embodiment of the present application, a topic may be characterized by a topic keyword, and the topic keyword may refer to a keyword that can reflect a topic to be responded to.

Optionally, the above determining the subject corresponding to the information to be responded to may specifically include:

Determining method A1, determining the theme corresponding to the page address according to the content of the page corresponding to the page address; and / or

A determination method A2: firstly identify a video stream and / or an audio stream corresponding to a video, and determine a theme corresponding to the video according to the obtained first recognition result; and / or

Determining method A3, performing second recognition on the picture, and determining a theme corresponding to the picture according to the obtained second recognition result; and / or

A determination method A4: Perform voice recognition on the audio, and determine a theme corresponding to the information to be responded according to the obtained voice recognition result.

Video usually consists of still pictures, which are called video frames. The video stream corresponding to the video can be used to represent consecutive video frames. The audio stream corresponding to the video can be used to represent a continuous audio signal, and the audio stream is synchronized with the continuous video frame to achieve the synchronous playback effect of the video picture and audio.

In practical applications, the audio stream corresponding to the video may correspond to video content such as the lines of the video, the soundtrack, and the soundtrack may include: theme songs, episodes, ending songs, and background music corresponding to the lines. It can be understood that the embodiment of the present application does not limit the specific video content corresponding to the audio stream.

In practical applications, the video stream and audio stream corresponding to the video can be located in the same file. In this case, audio can be extracted from the video file. Specifically, the video file can be converted into an audio file, for example, MP4 (Motion Picture Expert Compression Standard Audio Level 4, Moving Picture Experts Group Audio Layer 4) format video files are converted to MP3 (Motion Picture Expert Compression Standard Audio Level 3, Moving Picture Experts Group Audio Audio Layer III) format audio files. Alternatively, the video stream and audio stream corresponding to the video may be located in separate files, that is, the video file and the audio file may be independent. In this case, the audio file may be directly obtained. The audio file may include an audio stream corresponding to the video, so the audio stream corresponding to the video may be read from the audio file.

In practical applications, several video frames can be extracted from the video at preset time intervals, and the extracted video frames can be used as objects for image recognition. It can be understood that a person skilled in the art may determine the preset time interval according to actual application requirements. For example, the preset time interval may be a playback duration corresponding to N video frames, and N is a positive integer. It is understood that the embodiments of the present application There are no restrictions on the specific N and the preset time interval.

The embodiments of the present application can identify the video stream and / or audio stream corresponding to the video by using the following identification methods:

Recognition method 1. Perform image recognition on a video stream corresponding to a video to obtain corresponding image target information; and / or

Recognition method 2. Perform text recognition on a video stream corresponding to a video to obtain corresponding text information; and / or

Recognition method 3: Perform voice recognition on the audio stream corresponding to the video to obtain corresponding text information.

In the recognition method 1, image recognition refers to a technology that uses a machine to process, analyze, and understand an image to identify image objects in various modes. Specifically to the embodiment of the present application, a technology for processing, analyzing, and understanding a video frame by a machine to identify image targets in various modes can be used. Among them, the image target in the video frame may correspond to a certain image area in the video frame, and the image target in the video frame may include: objects, people, space, and so on. For example, a character may be a person in a video frame, an item may be an item worn by a person in a video frame, and a space may be an environmental space in which the character is located in the video frame, such as an outdoor environment, an indoor environment, etc. For example, an indoor environment may include an indoor It can be understood that the information such as the wall and the ground does not limit the specific image target in the video frame.

In an optional embodiment of the present application, the process of performing image recognition on a video frame corresponding to a video stream and / or an audio stream may include: detecting an image target in the video frame, and using a deep learning method on the acquired image The target is analyzed to obtain corresponding image target information. Therefore, the recognition result in the embodiment of the present application may include: image target information corresponding to a video frame. The above image target information may include: the image of the image target (that is, the image of the image target in the video frame, which usually corresponds to a certain closed area in the video frame), the recognition result of the image target (such as the recognized image Target name, category, etc.). For example, you can use face detection technology to detect faces in video frames, and use deep learning methods to analyze the faces to obtain information such as the gender and age of the character, and even the source of the character, such as which film and television source Drama, etc. You can even get a celebrity. Further, it is also possible to detect items worn by the character, such as clothing, shoes, watches, jewelry, etc. Alternatively, it is also possible to detect spatial information and the like where the character is located.

The text information in the video frame may include: text information included in the image, and / or text information in the subtitles.

For the identification method 2, text recognition technology may be used to perform text recognition on a video frame corresponding to a video stream and / or an audio stream. The above text recognition technology may include: OCR (Optical Character Recognition) technology, etc. The OCR technology may segment characters in an image after pre-processing such as noise reduction to obtain a single character image. And recognize the characters corresponding to a single character image. It can be understood that the embodiment of the present application does not limit the specific text recognition technology.

For the identification method 2, a subtitle file corresponding to the subtitle of the video frame can be obtained, and the text information in the subtitle can be obtained from the subtitle file. Get text information in subtitles. It can be understood that the embodiment of the present application does not limit the specific acquisition manner of the text information in the subtitles.

For the recognition method 3, a voice recognition technology can be used to convert the audio stream corresponding to the video into text information. If the audio stream corresponding to the video is recorded as S, a series of processing is performed on S to obtain a corresponding speech feature sequence O, which is denoted as O = {O1, O2, ..., Oi, ..., OT}, where Oi is The i-th speech feature, T is the total number of speech features. The sentence corresponding to the audio stream S can be regarded as a word string composed of many words, which is written as W = {w1, w2, ..., wn}, and n is a natural number. The process of speech recognition is to find the most likely word string W according to the known speech feature sequence O.

Specifically, speech recognition is a model matching process. In this process, you can first build a speech model based on the characteristics of people's speech, and analyze the input speech signals to extract the required features to establish the speech recognition requirements. The process of recognizing the voice input by the user is the process of comparing the features of the voice input by the user with the template, and finally determining the best template that matches the voice input by the user to obtain the result of voice recognition . Specific speech recognition algorithms can use statistics-based hidden Markov model training and recognition algorithms, neural network-based training and recognition algorithms, dynamic time-rounded matching-based recognition algorithms, and other algorithms. The application embodiment does not limit the specific speech recognition process.

In an application example of the present application, the user G receives the website A sent by the user H and copies the website A, then the embodiment of the present application can automatically analyze the page content of the website A to obtain the topic A corresponding to the website A The topic A can be related to entertainment gossip, national economy and people's livelihood, etc., and can automatically give a response candidate corresponding to the topic A, such as "I also look at this page", or "I also like the content corresponding to the topic A" and so on.

In another application example of this application, if user I receives video A sent by user J and copies video A, the embodiment of this application can automatically identify video A and obtain the subject B corresponding to video A. The Topic B can be related to the child's life in the kindergarten, etc., and can automatically give a response candidate corresponding to the topic B, such as "It seems that the child is happy in the kindergarten".

In another application example of the present application, user K receives picture A sent by user L and copies picture A, the embodiment of the present application can automatically identify picture A and obtain the subject C corresponding to picture A. The Theme C can be related to a piece of clothing, such as a coat, a down jacket, a skirt, etc., and can automatically give a reply candidate for Theme C, such as "Theme C is beautiful and worth buying", "Theme C is a bit old-fashioned" "Subject C is fat" and so on.

The embodiments of the present application may provide the following determination manners for determining a topic corresponding to the information to be responded to:

Determining method B1. A TF-IDF (term frequency-inverse document frequency algorithm) method is used to determine the topic keywords corresponding to the information to be responded to.

The main idea of TF-IDF is: If a word or phrase appears frequently in a document or a text and has a high TF, and rarely appears in other documents or texts, the word or phrase is considered to have a good category distinction. Capabilities, suitable for classification.

Determining method B2. The LDA (Latent Dirichlet Allocation) model is used to determine the topic keywords corresponding to the information to be answered.

The LDA model is a document generation model and an unsupervised machine learning technology. It thinks that a document or a text has multiple topics, and each topic corresponds to a different topic keyword. The process of constructing a document or a text, first select a certain topic with a certain probability, and then select a certain topic keyword with a certain probability under this topic, so the first one of this document is generated. Theme keywords. Repeating this process continuously produces a document or a text. The use of LDA is the inverse process of the above document generation process, that is, to find the theme of this document or this text, and the topic keywords corresponding to these topics according to a document or a text.

Determining method B3. A classification model is used to determine a category corresponding to the information to be responded to, and a topic keyword is obtained based on the information of the category. The classification model may include: a fasttext model. The fastText model can output the probability that the word sequence belongs to different categories for the input word sequence (a text or a sentence). The fastText model can combine words and phrases in a word sequence into a feature vector. The feature vector is mapped to the middle layer through a linear transformation, and the middle layer is then mapped to the corresponding preset category. Optionally, fastText may use a non-linear activation function in the process of mapping to the corresponding preset category. fastText has the advantages of fast speed and high accuracy. Of course, the embodiment of the present application does not limit the specific classification model.

It can be understood that a person skilled in the art may adopt any one or combination of the foregoing determination manner B1 to determination manner B3 according to actual application requirements, and the embodiment of the present application does not limit the specific process of determining the topic corresponding to the information to be responded to.

An embodiment of the present application may determine a reply candidate corresponding to the information to be responded according to the subject. Optionally, the information to be returned is notification-type information. In this case, the reply candidate may be "received", "good", and the like. The subject of the message to be answered "Come to the meeting room" can be "location notification", and the subject of the message to be answered "There is a place in your plan that needs to be modified" can be "work modification notice".

Optionally, the information to be answered may be inquiry-type information. In this case, the reply candidate may be a positive candidate, a negative candidate, one of a plurality of options, or an answer to a question. For example, if the information to be responded is “Go to place A for dinner tomorrow or place B for dinner”, the corresponding reply candidates may include: location A or location B. For another example, if the message to be answered is "Are you off work?", The corresponding reply candidate may include: Off or Off. As another example, if the message to be answered is "What are you doing", the corresponding reply candidates may include: eating, watching videos, and so on.

Technical solution 2,

In technical solution 2, step 202 determines a response candidate corresponding to the information to be responded to, specifically, may include: searching for a mapping relationship between the data to be responded to and the response data according to the information to be responded to obtain the response to be answered. Reply candidates corresponding to the reply information; wherein the mapping relationship may be obtained according to historical communication data corresponding to at least one user, and the historical communication data may include historical to-be-reply data and corresponding historical reply data.

In a specific implementation process, the above mapping relationship may be determined by: obtaining historical communication data of at least one user; historical communication data may include: historical to-be-reply data and its corresponding historical reply data; for each piece of historical to-be-reply data Extract the corresponding historical reply content; use the historical reply content corresponding to the historical to-reply data that meets the preset conditions as the reply data corresponding to the historical to-reply data, so that the historical to-reply data and its corresponding reply data can be based on, Determine the above mapping relationship.

The user may be a current terminal user or at least one sampling user in the entire network, and the historical communication data obtained is also different, for example:

① Acquire at least one set of historical communication data generated by a user of the current terminal. For example, the user's response data to the communication content can be obtained; at least one set of question and answer pairs can be extracted from the response data, each group of question and answer pairs can include: communication content, and response content to the corresponding communication content, the at least A set of question-answer pairs is the at least one set of historical communication data generated by the user.

For example, the terminal may often receive the text message "What time do you get off work?", The terminal sometimes responds with "9 o'clock", and sometimes it returns "8 o'clock", so "when do you get off work?" And "9 o'clock" constitute a set of question and answer pairs. "When do you get off work?" And "8 o'clock" form a set of question-and-answer pairs. You can form personalized cache data based on the question and answer pair: "when do you get off work? ① 9 o'clock 8 o'clock. When the terminal receives the same text message again, it will give There are two reply candidates: ① 9 o'clock and 8 o'clock. Users don't need to input, they can click to reply to the SMS.

② Determine at least one sampling user; acquire the at least one set of historical communication data generated by the at least one sampling user, for example, all users in the system, some users in the system, etc., based on the The solution does not need to analyze the historical communication data of each user separately, so it can improve the acquisition efficiency of historical communication data.

After obtaining at least one set of historical communication data, the historical to-be-reply data in each set of historical communication data may be obtained first, and then the same historical to-be-reply data is combined to obtain all historical to-be-reply data contained in the historical communication data, The historical to-reply data in one group of historical communication data refers to the above of the communication, and the historical to-reply data is the response content generated for the communication above; or, the historical to-be-reply data in the historical communication data refers to It is the network question data, and the historical reply data is the answer to the network question data and so on.

After obtaining all historical to-be-reply data contained in the historical communication data, for each piece of historical to-be-reply data, the corresponding historical reply data can be obtained, and the number of occurrences of each piece of historical reply data can be obtained.

Further, after obtaining each piece of historical to-be-reply data and its corresponding historical reply data, for each piece of historical to-reply data, historical reply data whose occurrences are greater than a preset number (for example: 20, 30, etc.) can be obtained as its Corresponding reply data; the historical reply data can also be sorted according to the number of occurrences, and then the historical reply data ranked in the first few positions (for example: 4, 5, etc.) is used as its reply data. This can obtain the mapping relationship between the data to be reply and the reply data.

Taking the number of occurrences greater than the preset number (for example: 20) as an example, the data to be returned "when do you get off work?" Contains two pieces of reply data: ① 8 o'clock 9 o'clock; the data to be reply "Did you eat?" Contains six pieces of response data, namely: ①have eaten ②not ③not yet ④eaten ⑤eaten ⑥have not yet; the data to be returned “rest early, good night” contains a piece of response data, specifically: ①good night. It can be understood that the embodiments of the present application are directed to specific mapping relationships.

Technical solution 3.

In technical solution 3, step 202 determines a reply candidate corresponding to the information to be responded, which may specifically include: extracting first feature information of the information to be responded to; based on a correspondence relationship between the pre-established feature information and a reply rule, A first reply rule corresponding to the first feature information is determined; at least one reply candidate is determined through the first reply rule.

For example, the characteristic information is, for example, a preset sentence, a preset sentence format, and the like, and the preset sentence is, for example, "have you eaten", "when did you sleep", "how are you", etc., the preset sentence format For example: "Did you go to XX for dinner or go to XX for dinner", "Eat XX or XX today" (where XX is the default word) and so on. In the specific implementation process, corresponding response rules can be constructed for each type of feature information. For example, the response rules for "have eaten" are: ① already eaten ② not yet eaten ③ ready to eat; To go to Wanzhou Pea Mi Noodles to eat or to eat fragrant meal "to establish the reply rules are: ① Wanzhou Pea Mi Noodles ② To make incense, etc. Of course, other reply rules can also be constructed, which are not limited in the examples of this application.

As an optional embodiment, determining the at least one reply candidate by using the first reply rule includes: extracting at least one specific keyword in the information to be responded to; and at least one specific keyword And combining with the first reply rule to obtain the at least one reply candidate.

For example, suppose the preset sentence format is "Do you plan to go to XX for dinner or go to XX for dinner"? The reply rule constructed for it is: I go to XX for dinner today, where XX represents the default item in the message to be answered, for example : The first data to be answered is "Do you plan to go home for dinner today, or go to Richang for dinner?", Then the default words (that is, specific keywords) include: go home, Richang, so that at least one response candidate constructed includes : ① I went to restaurant A for dinner today; ② I went to restaurant B for dinner today. In addition, since the user may not necessarily go to these two places for dinner, other reply candidates can be added, such as: ③ casually, etc. ; For example, for "eating dumplings or noodles today", construct a reply rule for it: eat XX today, then combine the default words "dumplings" and "noodles" with the reply rule, so that at least one reply candidate is constructed Including: ① eating dumplings today; ② eating noodles today and so on. Through the above solution, at least one reply candidate can be determined based on a specific keyword included in the information to be responded, so the determined reply candidate is more relevant and accurate.

Technical solution 4.

In the technical solution 4, step 202 determines the response candidate corresponding to the information to be responded to, which may specifically include: determining user status information through a running application of the terminal; and determining the information to be responded according to the user status information. The corresponding reply candidate.

Technical solution 4 uses the user status information in the process of generating the reply candidate, and the above user status information obtained through the running application can reflect the user's use of the application. For example, when the user receives a When the user's information is returned or the user responds to the information from the correspondent peer through the terminal, the use of the application by the user may specifically include a series of users such as settings, browsing, purchase, and viewing generated by the user through the running APP. The action is such that the reply candidate generated according to the user status information can carry deep information other than the judgment reply, so that the reply candidate can meet the user's precise reply intention, thereby improving the accuracy and richness of the quick reply candidate. In the case where the reply candidate matches the user's precise reply intention, the user can directly use the above reply candidate to reply to the information. Since the input cost when the user responds to the information can be further reduced, the reply efficiency can be improved.

For example, when user M receives a short message, he is watching a video through a video app. In this way, the corresponding user status information can be obtained through the video app, and when the content of the short message is, for example, a question "What are you doing?" According to the above user status information, a response candidate such as "watching a video" or "watching a TV series" is generated; or, when the content of the short message is, for example, "Did you eat?", "No, Candidates for replying to “watching video”, “not yet, watching TV series”, “watching TV series while eating”.

The process of determining a reply candidate corresponding to the information to be responded is described in detail through the technical solutions 1 to 4 above. It can be understood that those skilled in the art can use the technical solutions 1 to 4 according to the actual application requirements. Either or a combination, or other technical solutions may also be adopted. The embodiment of the present application does not limit the specific process of determining a response candidate corresponding to the information to be returned.

Step 203 may display the reply candidates for selection by the user. Optionally, in response to a user's trigger operation for any reply candidate, the target reply candidate corresponding to the trigger operation may be displayed on the screen, and the screen may be: outputting the target reply candidate to an input box of a communication window.

According to an embodiment, the displaying the response candidate may specifically include: displaying the response candidate above an input method keyboard. After the input method keyboard is called up, at least one reply candidate may be displayed above the input method keyboard without generating any input string or generating any input string by the user. According to another embodiment, the reply candidates may be displayed through a pop-up window or a mask. It can be understood that the embodiment of the present application does not limit the specific display manner of the above reply candidates.

In an optional embodiment of the present application, step 203 displays the reply candidates, which may specifically include: ranking multiple reply candidates according to the social relationship between the local user and the peer user; displaying the sorted Multiple reply candidates.

In practical applications, the social relationship between the local user and the peer user can be determined in the following ways: obtaining the communication content between the peer user and the local user; extracting all the content in the communication content A first predetermined keyword included; and based on the first predetermined keyword, determining a social relationship between a local user and a peer user.

For example, a database may be set in advance, and the database includes at least one social relationship and keywords corresponding to each social relationship. For example, the keywords for "couple relationship" can include: "Smith", "honey", "dear", and the keywords for "couple" relationship can include: "wife", "husband", "key of colleague relationship" Words can include: "project", "* 工", "Leader", "Madam", etc.

Furthermore, after obtaining the communication content between the peer user and the local user, it can be determined whether the communication content contains any of the above keywords, and if any of the above keywords are contained, it is extracted as the first predetermined keyword, and then The social relationship is determined by the first predetermined keyword.

Optionally, for each response candidate, the probability that it belongs to a social relationship may be determined, and then multiple response candidates are sorted according to the probability and output, for example, if the social relationship is a couple relationship, the response candidates include: ① Goodnight ② Anan ③ Good night, dear, the probability that these three response candidates belong to a couple relationship is: 0.1, 0.6, 0.9, then you can output the three response candidates in the following order: ① Good night, dear ② An An ③ Good night. In the above solution, multiple reply candidates can be sorted according to social relationships, so the rationality of the ranking results can be improved.

In summary, the data processing method in the embodiment of the present application supports a user to carry information to be responded to via a clipboard, so that the information to be responded can be determined from the information on the clipboard, and a reply candidate corresponding to the information to be responded is provided, so the response efficiency can be improved. . The embodiment of the present application can be applied to a response scenario across communication windows. Specifically, a user can copy information in a first communication window and jump to a second communication window. The embodiment of the present application can automatically The information to be answered is determined in the information, and a reply candidate corresponding to the information to be responded is provided. Therefore, an intelligent reply can be implemented in a reply scene across the communication window to improve the reply efficiency.

Method embodiment two

Referring to FIG. 3, a flowchart of steps in a second embodiment of a data processing method according to the present application is shown, which may specifically include the following steps:

Step 301: Determine to-be-reply information from the clipboard information; the to-be-reply information may correspond to all or part of the clipboard information;

Step 302: Determine a reply candidate corresponding to the information to be replyed.

Step 303: Display the reply candidates and the clipboard information.

The embodiment of the present application can display the reply candidate and the clipboard information at the same time, so that the comparison display effect between the clipboard information and the reply candidate can be achieved, so that the user knows that the reply candidate is responding to the clipboard information.

Referring to FIG. 4, a schematic diagram of an interface according to an embodiment of the present application is shown, which may specifically include: a communication window 401 and an input method interface 402;

The communication window 401 may include communication content and an input box. Taking the communication window between user A and user B as an example, the communication content may include: communication content 1 sent by user B and communication content 2 sent by user A, and the like.

The input method interface 402 may include an input method keyboard 421, a clipboard information area 422, and a reply candidate area 423. The clipboard information area 422 may be located above the input method keyboard 421, and the clipboard information area 422 may cover part or all of the input. Method tool; the reply candidate area 423 may be located in the clipboard information area 423.

The clipboard information area 422 may include: clipboard information and prompt information corresponding to the clipboard information, such as "from the clipboard" and the like.

The reply candidate area 423 may include: n (n is a natural number) reply candidates, and corresponding prompt information, such as “smart reply” and the like.

If a user's click operation on a reply candidate is received, the target reply candidate corresponding to the click operation can be output to the input box.

In an optional embodiment of the present application, it may be determined whether the above-mentioned clipboard information is included in the communication content of the communication window. If so, the above-mentioned clipboard information is not displayed, otherwise, the above-mentioned clipboard information is displayed. The above judgment can avoid duplication of clipboard information.

It should be noted that, for the sake of simple description, the method embodiments are all described as a series of action combinations. However, those skilled in the art should know that the embodiments of the present application are not limited by the described action sequence because According to the embodiment of the present application, some steps may be performed in other orders or simultaneously. Secondly, a person skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present application.

Device embodiment

Referring to FIG. 5, a structural block diagram of an embodiment of a data processing apparatus according to the present application is shown, which may specifically include:

The to-be-reply information determination module 501 is configured to determine the to-be-reply information from the clipboard information; the to-be-reply information may correspond to all or part of the clipboard information;

A reply candidate determination module 502 configured to determine a reply candidate corresponding to the information to be replyed; and

The reply candidate display module 503 is configured to display the reply candidates.

Optionally, the apparatus may further include:

The clipboard information display module is configured to display the clipboard information.

Optionally, the reply candidate display module 503 is specifically configured to display the reply candidate above the input method keyboard.

Optionally, the type of the clipboard information may include at least one of the following types: text, picture, audio, video, and page address.

Optionally, the reply candidate determination module 502 may include:

A topic determination module configured to determine a topic corresponding to the information to be responded to;

The candidate determination module is configured to determine a response candidate corresponding to the information to be returned according to the subject.

Optionally, the topic determination module may include:

A first theme determination module configured to determine a theme corresponding to the page address according to the content of the page corresponding to the page address; and / or

A second theme determination module configured to perform first recognition on a video stream and / or audio stream corresponding to a video, and determine a theme corresponding to the video according to the obtained first recognition result; and / or

A third theme determination module configured to perform second recognition on the picture, and determine a theme corresponding to the picture according to the obtained second recognition result; and / or

The fourth theme determining module is configured to perform voice recognition on the audio, and determine a theme corresponding to the information to be responded according to the obtained voice recognition result.

Optionally, the reply candidate determination module 502 may include:

A search module configured to perform a search in a mapping relationship between the data to be replied and the reply data according to the information to be replied to obtain a reply candidate corresponding to the information to be replied;

The mapping relationship is obtained based on historical communication data corresponding to at least one user. The historical communication data may include historical to-be-reply data and corresponding historical reply data.

Optionally, the reply candidate display module 503 may include:

A sorting module configured to sort multiple reply candidates based on the social relationship between the local user and the peer user;

The sorting display module is configured to display the sorted multiple reply candidates.

As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple. For the related parts, refer to the description of the method embodiment.

Each embodiment in this specification is described in a progressive manner. Each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments may refer to each other.

Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be described in detail here.

An embodiment of the present application provides a device for data processing, including a memory, and one or more programs. One or more programs are stored in the memory and configured to be executed by one or more processors. The one or more programs include instructions for: determining information to be responded from the clipboard information; the information to be responded to corresponds to all or part of the clipboard information; and determining that the information to be responded corresponds to ; Candidates for reply; showing the candidates for reply.

Fig. 6 is a block diagram of a device 800 for data processing according to an exemplary embodiment. For example, the device 800 may be a mobile phone, a computer, a digital broadcasting terminal, a messaging device, a game console, a tablet device, a medical device, a fitness equipment, a personal digital assistant, and the like.

6, the device 800 may include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, And communication component 816.

The processing component 802 generally controls the overall operations of the device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing element 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the method described above. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

The memory 804 is configured to store various types of data to support operation at the device 800. Examples of these data include instructions for any application or method operating on the device 800, contact data, phone book data, messages, pictures, videos, and the like. The memory 804 may be implemented by any type of volatile or non-volatile storage devices, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), Programming read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

The power component 806 provides power to various components of the device 800. The power component 806 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the device 800.

The multimedia component 808 includes a screen that provides an output interface between the device 800 and a user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, swipe, and gestures on the touch panel. The touch sensor may not only sense a boundary of a touch or slide action, but also detect duration and pressure related to the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operation mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have focal length and optical zoom capabilities.

The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC) that is configured to receive an external audio signal when the device 800 is in an operation mode, such as a call mode, a recording mode, and a voice data processing mode. The received audio signal may be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 further includes a speaker for outputting audio signals.

The I / O interface 812 provides an interface between the processing component 802 and a peripheral interface module. The peripheral interface module may be a keyboard, a click wheel, a button, or the like. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.

The sensor component 814 includes one or more sensors for providing status assessment of various aspects of the device 800. For example, the sensor component 814 can detect the on / off state of the device 800 and the relative positioning of the components, such as the display and keypad of the device 800. The sensor component 814 can also detect the change of the position of the device 800 or a component of the device 800 , The presence or absence of the user's contact with the device 800, the orientation or acceleration / deceleration of the device 800, and the temperature change of the device 800. The sensor component 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor component 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 814 may further include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 816 is configured to facilitate wired or wireless communication between the device 800 and other devices. The device 800 can access a wireless network based on a communication standard, such as WiFi, 2G, or 3G, or a combination thereof. In one exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency data processing (RFID) technology, infrared data association (IrDA) technology, ultra wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, the device 800 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable A gate array (FPGA), controller, microcontroller, microprocessor, or other electronic component is implemented to perform the above method.

In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions, such as a memory 804 including instructions, may be executed by the processor 820 of the device 800 to complete the foregoing method. For example, the non-transitory computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

FIG. 7 is a schematic structural diagram of a server in some embodiments of the present application. The server 1900 may have relatively large differences due to different configurations or performance, and may include one or more central processing units (CPUs) 1922 (for example, one or more processors) and memory 1932, one or one The above storage medium 1930 (eg, one or one storage device with an amount of Shanghai) storing application programs 1942 or data 1944. The memory 1932 and the storage medium 1930 may be temporary storage or persistent storage. The program stored in the storage medium 1930 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Furthermore, the central processing unit 1922 may be configured to communicate with the storage medium 1930, and execute a series of instruction operations in the storage medium 1930 on the server 1900.

The server 1900 may also include one or more power sources 1926, one or more wired or wireless network interfaces 1950, one or more input-output interfaces 1958, one or more keyboards 1956, and / or, one or more operating systems 1941. , Such as Windows ServerTM, Mac OSXTM, UnixTM, LinuxTM, FreeBSDTM and so on.

A non-transitory computer-readable storage medium, when instructions in the storage medium are executed by a processor of a device (server or terminal), enable the device to execute the data processing method shown in FIG. 2 or FIG. 3.

A non-transitory computer-readable storage medium, when instructions in the storage medium are executed by a processor of a device (server or terminal), enable the device to execute a data processing method, the method includes: from a clipboard Information to be answered is determined; the information to be responded to corresponds to all or part of the clipboard information; a response candidate corresponding to the information to be responded is determined; and the response candidate is displayed.

Those skilled in the art will readily contemplate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application. These variations, uses, or adaptations follow the general principles of this application and include common general knowledge or conventional technical means in the technical field not disclosed in this disclosure. . It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.

It should be understood that the present application is not limited to the precise structure that has been described above and shown in the drawings, and various modifications and changes can be made without departing from the scope thereof. The scope of the application is limited only by the accompanying claims.

The above is only a preferred embodiment of the present application and is not intended to limit the present application. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present application shall be included in the protection of the present application. Within range.

The data processing method, data processing device, and data processing device provided by the present application have been described in detail above. Specific examples are applied in this article to explain the principle and implementation of the present application. The descriptions of the above embodiments are only used to help understand the method and core ideas of the present application; meanwhile, for a person of ordinary skill in the art, according to the ideas of the present application, there will be changes in the specific implementation and application scope. In summary, the content of this specification should not be construed as a limitation on this application.

Claims

A data processing method, characterized in that the method includes:

Determining to-be-reply information from the clipboard information; the to-be-reply information corresponds to all or part of the clipboard information;

Determining a reply candidate corresponding to the information to be replyed;

The reply candidate is displayed.
The method according to claim 1, further comprising:

Displaying the clipboard information.
The method according to claim 1 or 2, wherein the displaying the reply candidate comprises:

Above the input method keyboard, the reply candidates are displayed.
The method according to claim 1 or 2, wherein the type of the clipboard information includes at least one of the following types: text, picture, audio, video, and page address.
The method according to claim 1 or 2, wherein the determining a reply candidate corresponding to the information to be returned comprises:

Determining a subject corresponding to the information to be responded to;

According to the subject, a reply candidate corresponding to the information to be reply is determined.
The method according to claim 5, wherein the determining a subject corresponding to the information to be returned comprises:

Determine the theme corresponding to the page address according to the content of the page corresponding to the page address; and / or

First identifying a video stream and / or an audio stream corresponding to a video, and determining a theme corresponding to the video according to the obtained first recognition result; and / or

Perform a second recognition on the picture, and determine the theme corresponding to the picture according to the obtained second recognition result; and / or

Perform voice recognition on the audio, and determine the theme corresponding to the information to be answered according to the obtained voice recognition result.
The method according to claim 1 or 2, wherein the determining a reply candidate corresponding to the information to be returned comprises:

Searching in the mapping relationship between the data to be replied and the reply data according to the information to be replied to obtain a reply candidate corresponding to the information to be replied;

The mapping relationship is obtained based on historical communication data corresponding to at least one user, and the historical communication data includes historical to-be-reply data and corresponding historical reply data.
The method according to claim 1 or 2, wherein the displaying the reply candidate comprises:

Sort multiple response candidates based on the social relationship between the local user and the peer user;

Multiple ranked candidates are displayed.
A data processing device, comprising:

A to-be-reply information determination module configured to determine to-be-reply information from the clipboard information; the to-be-reply information corresponds to all or part of the clipboard information;

A reply candidate determination module configured to determine a reply candidate corresponding to the information to be replyed; and

The reply candidate display module is configured to display the reply candidate.
The apparatus according to claim 9, further comprising:

The clipboard information display module is configured to display the clipboard information.
The device according to claim 9 or 10, wherein the reply candidate display module is specifically configured to display the reply candidate above an input method keyboard.
The device according to claim 9 or 10, wherein the type of the clipboard information includes at least one of the following types: text, picture, audio, video, and page address.
The apparatus according to claim 9 or 10, wherein the reply candidate determination module comprises:

A topic determination module configured to determine a topic corresponding to the information to be responded to;

The candidate determination module is configured to determine a response candidate corresponding to the information to be returned according to the subject.
The apparatus according to claim 13, wherein the theme determination module comprises:

A first theme determination module configured to determine a theme corresponding to the page address according to the content of the page corresponding to the page address; and / or

A second theme determination module configured to perform first recognition on a video stream and / or audio stream corresponding to a video, and determine a theme corresponding to the video according to the obtained first recognition result; and / or

A third theme determination module configured to perform second recognition on the picture, and determine a theme corresponding to the picture according to the obtained second recognition result; and / or

The fourth theme determining module is configured to perform voice recognition on the audio, and determine a theme corresponding to the information to be responded based on the obtained voice recognition result.
The apparatus according to claim 9 or 10, wherein the reply candidate determination module comprises:

A search module configured to perform a search in a mapping relationship between the data to be replied and the reply data according to the information to be replied to obtain a reply candidate corresponding to the information to be replied;

The mapping relationship is obtained based on historical communication data corresponding to at least one user, and the historical communication data includes historical to-be-reply data and corresponding historical reply data.
The apparatus according to claim 9 or 10, wherein the reply candidate display module comprises:

A sorting module configured to sort multiple reply candidates based on the social relationship between the local user and the peer user;

The sorting display module is configured to display the sorted multiple reply candidates.
An apparatus for data processing, comprising a memory and one or more programs, wherein one or more programs are stored in the memory and configured to be executed by one or more processors One or more programs contain instructions for:

Determining to-be-reply information from the clipboard information; the to-be-reply information corresponds to all or part of the clipboard information;

Determining a reply candidate corresponding to the information to be replyed;

The reply candidate is displayed.
The apparatus of claim 17, wherein the apparatus is further configured to be executed by one or more processors, the one or more programs containing instructions for performing the following operations:

Displaying the clipboard information.
The apparatus according to claim 17 or 18, wherein the displaying the reply candidate comprises:

Above the input method keyboard, the reply candidates are displayed.
The device according to claim 17 or 18, wherein the type of the clipboard information includes at least one of the following types: text, picture, audio, video, and page address.
The apparatus according to claim 17 or 18, wherein determining the response candidate corresponding to the information to be returned comprises:

Determining a subject corresponding to the information to be responded to;

According to the subject, a reply candidate corresponding to the information to be reply is determined.
The apparatus according to claim 21, wherein the determining a subject corresponding to the information to be returned comprises:

Determine the theme corresponding to the page address according to the content of the page corresponding to the page address; and / or

First identifying a video stream and / or an audio stream corresponding to a video, and determining a theme corresponding to the video according to the obtained first recognition result; and / or

Perform a second recognition on the picture, and determine the theme corresponding to the picture according to the obtained second recognition result; and / or

Perform voice recognition on the audio, and determine the theme corresponding to the information to be answered according to the obtained voice recognition result.
The apparatus according to claim 17 or 18, wherein the determining a reply candidate corresponding to the information to be returned comprises:

Searching in the mapping relationship between the data to be replied and the reply data according to the information to be replied to obtain a reply candidate corresponding to the information to be replied;

The mapping relationship is obtained based on historical communication data corresponding to at least one user, and the historical communication data includes historical to-be-reply data and corresponding historical reply data.
The apparatus according to claim 17 or 18, wherein the displaying the reply candidate comprises:

Sort multiple response candidates based on the social relationship between the local user and the peer user;

Multiple ranked candidates are displayed.
A machine-readable medium having stored thereon instructions which, when executed by one or more processors, cause a device to perform the data processing method according to one or more of claims 1 to 8.