CN113111269A

CN113111269A - Data processing method and device, computer readable storage medium and electronic equipment

Info

Publication number: CN113111269A
Application number: CN202110507901.XA
Authority: CN
Inventors: 常永炷; 浦嘉澍; 毛晓曦; 范长杰; 胡志鹏
Original assignee: Netease Hangzhou Network Co Ltd
Current assignee: Netease Hangzhou Network Co Ltd
Priority date: 2021-05-10
Filing date: 2021-05-10
Publication date: 2021-07-13
Anticipated expiration: 2041-05-10
Also published as: CN113111269B

Abstract

The embodiment of the application discloses a data processing method and device, a storage medium and electronic equipment. The method comprises the following steps: acquiring a target original message and target forwarding data on a message forwarding chain corresponding to the target original message; target forwarding comment information and target original information in the target forwarding data are used as target input data and input into a comment judging model to obtain a judging label; and determining a target object commented by the target forwarding comment message and comment attitude information of a publishing object of the target forwarding comment message to the target original message according to the discrimination tag, wherein the target object comprises the original object and/or the publishing object of the publishing target original message. According to the method and the device, deeper information in the message forwarding chain can be mined, so that the accuracy of target forwarding data analysis is improved, and meanwhile, the accuracy rate of the target forwarding data analysis is improved.

Description

Data processing method and device, computer readable storage medium and electronic equipment

Technical Field

The present application relates to the field of data processing technologies, and in particular, to a data processing method and apparatus, a computer-readable storage medium, and an electronic device.

Background

In the information age, social media have become an indispensable part of life, and it is very common to use data of social media for marketing analysis and user research. For example, forwarding and commenting of internet names for some hot events are very valuable for relevant departments to know current public opinion information, determine current public opinion forms and make decisions; aiming at the commodities, comments of merchants and the like, the method has certain help for the merchants to adjust market strategies and buyers to select the commodities; for microblogs, users can publish, forward and comment messages to mark life, share newsfeed events, express views and attitudes to certain events, and the like.

In the mass forwarding data of the social media, some users present a viewpoint and attitude to the original message, and some users issue corresponding attitudes again on the viewpoint and attitude of others to the original message. For example, the original post is: sleeping too late at night has a great influence on the body. The comments of user a on the original post are: this view is not necessarily true, user B forwards user a's view and replies to user a saying: i agree completely that user B also has the same attitude as user a with respect to the original post. However, when data analysis is performed, comment information of each user is usually analyzed separately, and therefore, the attitude of the user B is determined to support the original post, thus affecting the accuracy of the forwarded data analysis.

Disclosure of Invention

The embodiment of the application provides a data processing method and device, a computer readable storage medium and electronic equipment, which can improve the accuracy of forwarding data analysis.

The embodiment of the application provides a data processing method, which comprises the following steps:

acquiring a target original message and target forwarding data on a message forwarding chain corresponding to the target original message;

target forwarding comment information and the target original information in the target forwarding data are used as target input data and input into a comment judging model to obtain a judging label;

and determining a target object commented by the target forwarding comment message and comment attitude information of a publishing object of the target forwarding comment message to the target original message according to the discrimination tag, wherein the target object comprises the original object publishing the target original message and/or the publishing object.

An embodiment of the present application further provides a data processing apparatus, including:

the target acquisition module is used for acquiring a target original message and target forwarding data on a message forwarding chain corresponding to the target original message;

the tag determining module is used for inputting a target forwarding comment message and the target original message in the target forwarding data as target input data into a comment judging model to obtain a judging tag;

and the attitude determination module is used for determining a target object commented by the target forwarding comment message and comment attitude information of a publishing object of the target forwarding comment message to the target original message according to the discrimination tag, wherein the target object comprises the original object publishing the target original message and/or the publishing object.

An embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored, where the computer program is suitable for being loaded by a processor to perform the steps in the data processing method according to any of the above embodiments.

An embodiment of the present application further provides an electronic device, where the electronic device includes a memory and a processor, where the memory stores a computer program, and the processor executes the steps in the data processing method according to any of the above embodiments by calling the computer program stored in the memory.

According to the data processing method, the data processing device, the computer readable storage medium and the electronic equipment, the target forwarding comment message and the target original message in the message forwarding chain are input to the comment judging model together for processing, rather than only the target forwarding comment message of each forwarding object in the message forwarding chain is processed independently, so that deeper information in the message forwarding chain can be mined, and the accuracy of target forwarding data analysis is improved; in addition, a judgment tag can be obtained by using the comment judgment model, a target object to be commented by the target forwarding comment message and comment attitude information of a publishing object of the target forwarding comment message to the target original message are determined according to the judgment tag, and understandably, on one hand, the comment attitude information of the publishing object to the target original message can be determined by using the comment judgment model, so that the comment attitude of the publishing object of the target forwarding comment message to the target original message can be obtained, and the accuracy of target forwarding data analysis is improved; on the other hand, the target forwarding data is processed through the comment discrimination model, and the accuracy and the efficiency of analyzing the target forwarding data are further improved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a schematic flow chart of a data processing method according to an embodiment of the present application.

Fig. 2 is a schematic diagram of an original message and forwarding data of a message forwarding chain corresponding to the original message according to an embodiment of the present application.

Fig. 3 is another schematic flow chart of a data processing method according to an embodiment of the present application.

Fig. 4 is a schematic flowchart of a data processing method according to an embodiment of the present application.

Fig. 5 is a schematic diagram of a data structured format according to an embodiment of the present application.

Fig. 6 is a schematic diagram after the structured format of padding data is provided in the embodiment of the present application.

Fig. 7 is a schematic flowchart of a data processing method according to an embodiment of the present application.

Fig. 8 is a schematic flowchart of a data processing method according to an embodiment of the present application.

Fig. 9 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application.

Fig. 10 is another schematic structural diagram of a data processing apparatus according to an embodiment of the present application.

Fig. 11 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

The embodiment of the application provides a data processing method and device, a computer readable storage medium and electronic equipment. Specifically, the data processing method of the embodiment of the present application may be executed by an electronic device, where the electronic device may be a terminal or a server. The terminal may be a terminal device such as a smart phone, a tablet Computer, a notebook Computer, a touch screen, a game machine, a Personal Computer (PC), a Personal Digital Assistant (PDA), and the like. The server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a network service, cloud communication, middleware service, a domain name service, a security service, a CDN, a big data and artificial intelligence platform, and the like, but is not limited thereto.

A data processing method, an apparatus, a computer-readable storage medium, and an electronic device in the embodiments of the present application will be described in detail below. It should be noted that the following description of the embodiments is not intended to limit the preferred order of the embodiments.

Referring to fig. 1, fig. 1 is a schematic flow chart of a data processing method according to an embodiment of the present application, where the flow of the data processing method can be as follows.

101, acquiring a target original message and target forwarding data on a message forwarding chain corresponding to the target original message.

The original message refers to a message originally issued by a user, for example, an original post, an original microblog, an original news, and the like. When a user forwards a message, a forwarding path exists, and the message transmission is displayed in the path; meanwhile, in the forwarding process, the forwarding comment message of the user is also included. The process of delivering this message (information) is referred to as a message forwarding chain in this application.

The forwarding object may also be referred to as a forwarding user. When seeing a message, the user forwards the message and possibly reviews the message, so as to give out own insights, and in the application, the user who forwards the message (possibly reviews the message but possibly does not review the message) is called the forwarding user.

The published objects may also be referred to as publishing users. The user forwards a message and comments on the message, that is, posts comments, and correspondingly, the user is called a posting user. Alternatively, a user who issues a forwarding comment message among forwarding users may be referred to as a publishing user.

The commented object may also be referred to as a commented user or a response object. In the message forwarding chain, there are users who forward messages, but as described in the background, not every forwarding user is responding to the original message, and many users are other users on the reply forwarding chain. Other users of the forwarding user on the reply forwarding chain are called reply objects, namely the commented objects.

The message forwarding chain corresponding to each original message may include one or more messages. The forwarding data in the message forwarding chain comprises forwarding comment messages and/or forwarding object information. The forwarding comment message and the corresponding forwarding object in the forwarding data may be one or more. The forwarding object information includes information such as a user name of the forwarding object, a level of the forwarding object, and a head portrait, and may further include other various information. In the embodiment of the present application, the example that the forwarding object information includes the user name of the forwarding object is described.

Fig. 2 is a schematic diagram of an original message and forwarding data of a message forwarding chain corresponding to the original message provided in the embodiment of the present application. The original message is described by taking the original microblog as an example, the message forwarding chain is taken as an example, and the forwarding data is taken as an example.

The user name of an original user (original object) who publishes an original message is lie three, and the original message published by lie three is as follows: i tend to think that souls in journey are all escaped souls. The message forwarding chain corresponding to the original message comprises three.

The forwarding data in the first message forwarding chain of the original message comprises: semi-circle: but not with// @ litris: escape from the pubic but useless// @ moon: evasion pubic but useful. Wherein, each forwarding comment data in the message forwarding chain comprises: escape pubic but useful; escape from the pubis but not useful at all; but is not useful.

Forwarding data in a second message forwarding chain of the original message comprises: ABC: praise expression// @ litsan: escape is not pubic at all but is useless. Wherein, each forwarding comment data in the message forwarding chain comprises: like expressions; escape is not pubic at all but is useless.

Forwarding data in a third message forwarding chain of the original message comprises: blue and green: and forwarding the microblog. The message forwarding chain only displays that the forwarding object with the user name of 'Qingqing' forwards the original microblog, and does not comment the original microblog, namely, does not have any forwarding comment data.

It should be noted that the original message and the message forwarding chain of the original message shown in fig. 2 are only a simple example illustration. When a forwarding user forwards each original message, a message forwarding chain exists, at least one corresponding message forwarding chain exists, and forwarding data corresponds to each message forwarding chain.

The original message, message forwarding chain, forwarding data, etc. are described above. It should be noted that the target original message in this step refers to the original message to be processed, which is essentially the original message; the target forwarding data refers to forwarding data on a message forwarding chain corresponding to the target original message, or may be simply understood as forwarding data to be processed, which is essentially the forwarding data.

The target forwarding data comprises one or more target forwarding comment messages and/or target forwarding user messages.

And 102, taking the target forwarding comment message and the target original message in the target forwarding data as target input data, and inputting the target forwarding comment message and the target original message into a comment judgment model to obtain a judgment label.

Specifically, each target forwarding comment message and each target original message in the target forwarding data are used as corresponding target input data of each group, and the target input data are input into the comment judgment model, so that each judgment label corresponding to each target forwarding comment message is obtained. Each discrimination label is a predicted label obtained by prediction using the comment discrimination model. The method comprises the steps that a target forwarding comment message and a target original message corresponding to the target forwarding comment message serve as a group of target input data, and the group of target input data are input into a comment judging model to obtain a judging label corresponding to the target forwarding comment message. And taking the plurality of target forwarding comment messages and the corresponding target original messages as a plurality of groups of target input data, and inputting the plurality of groups of target input data into the comment judging model to obtain a plurality of judging labels corresponding to the plurality of target forwarding comment messages.

The comment discrimination model can be obtained through a pre-training process, and the step of obtaining the comment discrimination model through the training process will be described in detail later, specifically please refer to the corresponding description later.

The comment judgment model can realize feature processing on each group of target input data to obtain corresponding target feature vectors. In addition, the comment judging model also comprises a plurality of different discriminators, each different discriminator respectively processes the same target characteristic vector to obtain labels corresponding to different discriminators, and the labels corresponding to the different discriminators are used as the judging labels corresponding to the target characteristic vector.

And if the number of the forwarded comment messages of the target original message is i, corresponding to i groups of target input data. And inputting each group of target input data into the comment judgment model to obtain a processing result, wherein the processing result is the corresponding judgment label. As shown in table 1, i sets of target input data and corresponding processing results are included.

TABLE 1 schematic table of target input data and processing results for each group

The judging tags at least include a target object tag of a target object commented by the target forwarding comment message (or other tags capable of representing the target object commented by the target forwarding comment message), and a comment attitude tag of a publishing object of the target forwarding comment message to the target original message. The target object includes an original object and/or a published object from which the target original message was published. The target object tag (or other tags), comment attitude tag, and the like may be represented by information similar to 0,1, and the like, may be represented by yes, no, and the like, or may be represented by information in other formats.

103, determining a target object commented by the target forwarding comment message and comment attitude information of a publishing object of the target forwarding comment message to the target original message according to the discrimination tag, wherein the target object comprises the original object and/or the publishing object of the publishing target original message.

Specifically, a target object commented by each target forwarding comment message and comment attitude information of a publishing object of each target forwarding comment message to each target original message are determined according to each discrimination label.

The judging tags can include target object tags of target objects commented by the target forwarding comment messages and information of target attitude tags of the publishing objects of the target forwarding comment messages to the target original messages. Therefore, whether each forwarded comment message responds to the original message or not can be obtained, if not, which publishing user responds to the original message, and the reply attitude information (comment attitude information) and the like can be obtained.

The target object tag and the comment attitude tag in each discrimination tag are represented by similar information such as 0,1, and the like, and the similar information such as 0,1 is converted into specific target object and comment attitude information, and the like.

In one embodiment, for the comment attitude tag, 0 represents support attitude, 1 represents object attitude, and the corresponding discrimination tag is data such as 0 and 1, so that 0 and 1 are converted into comment attitude information such as support attitude and object attitude.

In one embodiment, for the target object tag, 1 is used to represent the target object that is commented on by the target forward comment message, and 0 is used to represent the non-target object. The sum of the numbers of 0 and 1 in the target object tag is the same as the total number of the target forwarding comment messages in the longest message forwarding chain in the message forwarding chains corresponding to the target original message plus 1 (the 1 refers to the target original message). If the number of the target forwarding comment messages of the longest message forwarding chain is 7, the target object label can be 0-0-0-0-1-0-0-0, and it indicates that the forwarding comment message comment is the 4 th forwarding user in the message forwarding chain, where the first 0 indicates that the target object commented by the target forwarding comment message is not the original object. In other embodiments, the target object that is reviewed by the target forward review message may also be determined in other ways.

In the embodiment, the target forwarding comment message and the target original message in the message forwarding chain are input to the comment judging model together for processing, rather than only the target forwarding comment message of each forwarding object in the message forwarding chain is processed separately, so that deeper information in the message forwarding chain can be mined to improve the accuracy of target forwarding data analysis; in addition, a judgment tag can be obtained by using the comment judgment model, a target object to be commented by the target forwarding comment message and comment attitude information of a publishing object of the target forwarding comment message to the target original message are determined according to the judgment tag, and understandably, on one hand, the comment attitude information of the publishing object to the target original message can be determined by using the comment judgment model, so that the comment attitude of the publishing object of the target forwarding comment message to the target original message can be obtained, and the accuracy of target forwarding data analysis is improved; on the other hand, the target forwarding data is processed through the comment discrimination model, and the accuracy and the efficiency of analyzing the target forwarding data are further improved.

Fig. 3 is a schematic flowchart of another data processing method according to an embodiment of the present application. The data processing method includes the following steps.

201, obtaining a target original message and target forwarding data on a message forwarding chain corresponding to the target original message.

And 202, taking each target forwarding comment message and each target original message in the target forwarding data as corresponding target input data of each group, and performing feature processing on each group of target input data by using a comment judgment model to obtain corresponding target feature vectors.

Each set of target input data comprises a target forwarding comment message and a target original message. And performing feature processing on each group of input data by using the comment discrimination model to obtain a target feature vector corresponding to the group of input data. Each set of target input data corresponds to a target feature vector.

And 203, analyzing and processing each target feature vector by using a comment original message discriminator, a commented object position discriminator and a comment attitude discriminator of the comment discrimination model to respectively obtain a target comment original message tag, a target commented object position tag and a target comment attitude tag corresponding to each target feature vector.

The comment judging model in the embodiment at least comprises a comment original message discriminator, a commented object position discriminator and a comment attitude discriminator. The comment attitude information corresponding to the target forwarding comment message can be determined through the comment original message discriminator.

And simultaneously inputting each target feature vector into a comment original message discriminator, a commented object position discriminator and a comment attitude discriminator of the comment discriminating model.

If each target feature vector is input into the comment original message discriminator to be analyzed and processed, so as to obtain the target comment original message probability of each target feature vector; determining a target comment original message label of each target feature vector according to the probability of the target comment original message; inputting each target feature vector into a commented object position discriminator for analysis processing to obtain target commented object position probability of each target feature vector; determining the position label of the target commented object of each target characteristic vector according to the position probability of the target commented object; inputting each target feature vector into a comment attitude discriminator for analysis processing to obtain a target comment attitude probability of each target feature vector; and determining the target comment attitude tag of each target feature vector according to the target comment attitude probability.

The target comment original message label is used for indicating whether a target forwarding comment message is commenting on the target original message (whether the target original message is responded) in the forwarding process; if it can be represented by 1, if it can be represented by 0. The target commented object position tag is used for indicating that if the target forwarding comment message is not in the comment target original message, the target forwarding user of the publishing user corresponding to the target forwarding comment message on the reply message forwarding chain is determined, namely the position information of the commented object in the message forwarding chain. Wherein the location information may also be represented by 0/1; if there are three nodes (three forwarding comment messages and related information) a-B-C in the message forwarding chain, a target forwarding comment message may be represented by 0-1-0 in response to a second target forwarding comment message, and a target forwarding comment message may be represented by 1-0-0 in response to a first target forwarding comment message. The target comment attitude tag is used for determining comment attitude information of a publishing user corresponding to the target forwarding comment message on the target original message. The comment attitude information can also be represented by 0/1, 0 represents the anti-attitude, and 1 represents the support attitude. It should be noted that the individual tags may also be represented in other ways.

The target comment original message label, the target comment object position label and the target comment attitude label are predicted labels obtained after prediction is carried out by using the comment judgment model. The judging label comprises a target comment original message label, a target comment object position label and a target comment attitude label. In this embodiment, functions corresponding to the target comment original message tag and the target comment object location tag are the same as those corresponding to the target object tag in the embodiment shown in fig. 1.

The sum of the number of 0 s and the number of 1 s in the target commented object position label is the same as the number of target forwarded comment messages in the longest message forwarding chain in the message forwarding chains corresponding to the target original message. If the number of target forwarding comment messages of the longest message forwarding chain is 7, the target object position label to be commented can be 0-0-0-0-1-0-0, and it indicates that the forwarding comment message comment is the 5 th forwarding user in the message forwarding chain.

And 204, determining a target object commented by each target forwarding comment message and comment attitude information of a target forwarding comment message to the target original message according to the target comment original message tag, the target commented object position tag and the target comment attitude tag, wherein the target object comprises an original object and/or a published object publishing the target original message.

Since the discrimination labels are all information similar to 0 and 1 (note that the information in the discrimination labels has a relationship with the setting, if the discrimination labels are set to be represented by 0 and 1, the predicted corresponding discrimination labels are also represented by 0 and 1, and if the discrimination labels are set to be represented by yes and no, the predicted corresponding discrimination labels are also represented by yes and no; in the embodiment of the present application, 0 and 1 are taken as an example for explanation, wherein, the comment attitude label of the target original message can also be set to be 0,1, 2, etc., where 0 can be expressed as support attitude, 1 is expressed as objection attitude, 2 is expressed as neutral attitude, etc.), the information similar to 0 and 1 needs to be regularized to determine the target object to be commented by each target forwarding comment message, and comment attitude information of the target original message of the publishing object of the target forwarding comment message. Here, the regularization process may be understood as a process of formatting three tags.

For example, a message forwarding chain has three nodes (meaning that there are three target forwarding comment messages corresponding to the message forwarding chain) a-B-C, and the predicted result is a (1,0-0-0,1) -B (0,1-0-0,0) -C (0,0-1-0, 1). The three labels corresponding to the label A are (1,0-0-0,1), and after regularization processing is performed, the obtained information can be (comment original message, the object to be commented is original user, and comment attitude of the target original message is support attitude); the three labels corresponding to the B are (0,1-0-0,0), and after regularization processing, the obtained information can be (the information is not the comment original message, the comment object is a forwarding user corresponding to a first node in a message forwarding chain, and the comment attitude of the target original message is an inverse attitude); the three labels corresponding to C are (0,0-1-0,1), and after regularization processing, the obtained information may be (instead of reviewing the original message, the reviewed object is a forwarding user corresponding to the second node in the message forwarding chain, and the review attitude of the target original message is a support attitude).

Thus, the target objects commented by the target forwarding comment messages and the comment attitude information of the publishing objects of the target forwarding comment messages to the target original messages are determined.

According to the embodiment, the comment judgment model is used for predicting the target forwarding data on the target original message and the message forwarding chain corresponding to the target original message, so that the target object commented by each target forwarding comment message in the target forwarding data and the comment attitude information of the target forwarding comment message publishing object to the target original message are obtained, and the accuracy of analyzing the forwarding data is improved.

In an embodiment, if forwarding comment messages corresponding to target forwarding data in a message forwarding chain are directly input into a comment judgment model for processing, for example, forwarding comment messages in the message forwarding chain shown in fig. 2 are directly input into the comment judgment model for processing, the implementation is troublesome; on the other hand, the message forwarding chain corresponding to the microblog is different from the message forwarding chain corresponding to the news message, and the comment judging model is required to be applied to different message forwarding chains, so that the processing efficiency and the applicability of the comment judging model are reduced. Therefore, the target forwarding data in the message forwarding chain needs to be structured and processed as the structural data. And extracting each forwarding comment message after the structuralization processing, and inputting the forwarding comment message into a comment judgment model for processing, so that the formats of input data of the comment judgment model in all scenes are kept consistent.

In an embodiment, before/after the target forwarding data in the message forwarding chain is subjected to the structural processing, the target original message and the target forwarding data corresponding to the message forwarding chain of the target original message need to be cleaned, so as to avoid influence of dirty data in the target original message on training and influence on accuracy of the comment judgment model.

In an embodiment, as shown in fig. 4, a flow diagram of another data processing method provided in the embodiment of the present application is shown, and the flow of the data processing method may be as follows.

301, obtaining a target original message and target forwarding data on a message forwarding chain corresponding to the target original message.

And 302, cleaning the target original message and the target forwarding data to obtain the cleaned target original message and the cleaned target forwarding data.

And if the target original message does not have any message forwarding chain, deleting the target original message and prompting. If the target original message has a message forwarding chain, cleaning the target original message and the target forwarding data to obtain the cleaned target original message and the cleaned target forwarding data, comprising the following steps: deleting the message forwarding chains without the target comment forwarding message in the message forwarding chains corresponding to the target original message to obtain the remaining message forwarding chains; and deleting the target forwarding comment message of which the content meets the preset condition and the target forwarding data behind the target forwarding comment message in the remaining message forwarding chain to obtain the cleaned target original message and the corresponding target forwarding data.

The message forwarding chain without the target forwarding comment message means that only the target forwarding user information exists on the message forwarding chain, namely the target forwarding user corresponds to the message forwarding chain, but the target forwarding user does not make any comment. For example, in the third message forwarding chain in fig. 2, if the target forwarding user who only displays "cyan" forwards the target original message but does not make any comment, the message forwarding chain is deleted.

The step of deleting the forwarding data after the target forwarding comment message whose target forwarding comment message content meets the preset condition includes: deleting the target forwarding comment message and the target forwarding data after the target forwarding comment message, wherein the word number of the content of the target forwarding comment message is smaller than the preset word number; and/or deleting the target forwarding comment message and the target forwarding data after the target forwarding comment message, wherein the content of the target forwarding comment message meets the preset content; and/or deleting the target forwarding comment message and the target forwarding data after the target forwarding comment message, wherein the content of the target forwarding comment message is neither the comment original message nor the target forwarding object on the comment message forwarding chain.

The preset word number can be set according to requirements, such as a natural number of 3 or 5. And deleting the target forwarding comment message and the target forwarding data after the target forwarding comment message in the message forwarding chain, wherein the word number of the target forwarding comment message content is less than the preset word number. Assuming that the preset word number is 4, as in the first message forwarding chain in fig. 2, the target forwarding comment data corresponding to the target forwarding user with the user name of "semicircle" is: but not useful, the word number of the target forwarding comment message content is less than the preset word number, so the target forwarding comment message and the corresponding target forwarding user information are deleted. Meanwhile, in the message forwarding chain, after the target forwarding user is a semicircle, the target forwarding user also forwards comments, and the corresponding target forwarding comment message is deleted no matter how long the target forwarding comment message is. That is, not only the target forwarding comment message with the number of words in the target forwarding comment message content smaller than the preset number of words is deleted, but also the target forwarding data after the target forwarding comment message with the number of words in the target forwarding comment message content smaller than the preset number of words is deleted, so as to avoid the influence of the shorter target forwarding comment message on the determination result, that is, the accuracy of the determination result is influenced.

The preset contents may be meaningless preset characters, preset expressions, and the like. And deleting the target forwarding comment message of which the content meets the preset content and the target forwarding data behind the target forwarding comment message so as to avoid the influence of the meaningless preset content on the accuracy of the judgment result.

According to specific requirements, the target original message and the target forwarding data in the message forwarding chain corresponding to the target original message may be subjected to other aspects of cleaning (except for the above cleaning) to improve the accuracy of the determination result.

303, performing structural processing on the cleaned target original message and the target forwarding data to convert the target original message and the target forwarding data into corresponding target structure data, and extracting each target forwarding comment message and each target original message in the target structure data.

The cleaned target original message and the corresponding target forwarding data are subjected to structural processing, which can be understood as that the cleaned target original message and the corresponding target forwarding data are converted into a certain format.

Carrying out structural processing on the cleaned target original message and corresponding target forwarding data, on one hand, in order to enable the comment judging model to be applicable to various message forwarding chains, such as microblog forwarding chains, news forwarding chains and other message forwarding chains with different forms/formats; in addition, after the cleaned target original message and the corresponding target forwarding data are converted into a certain format, the format is relatively neat and convenient to store; and the target original message and the corresponding target forwarding data which are converted into a certain format are input into the comment judging model, so that the understanding of the comment judging model on the target input data is improved, and the efficiency and the accuracy of the comment judging model for obtaining the corresponding judging result are improved.

The method for processing the target original message after cleaning and the corresponding target forwarding data in a structured manner to obtain the target structure data corresponding to the target original message comprises the following steps: acquiring a preset data structured format and a plurality of fields in the data structured format; according to the fields, a preset analysis interface is called to analyze the target original message and target forwarding data corresponding to the target original message so as to obtain target result data; and filling the corresponding field by using the target result data to obtain target structure data corresponding to the target original message.

The data structured format and the field are designed on the basis of convenient storage, and can be kept consistent with the corresponding data structured format and the corresponding field in the comment judgment model obtained through training. The data structured format and fields are designed to hold the corresponding data. The data structured format may be in a tree-like format.

Fig. 5 is a schematic diagram of a data structured format provided in an embodiment of the present application. Wherein, root represents the following node, namely the original message; 1item indicates that the list length is 1; 7keys represents that 7 fields exist, namely 7 fields exist below the original message, wherein id, name, parent, text, up _ text and the like are all corresponding fields; id, which refers to the number id corresponding to the original message; parent refers to the number id of a father node, namely the forwarded comment message is forwarded from, if the number is 0 as the default of the original microblog; text represents a text message corresponding to the original message; up _ text represents a text message corresponding to a node (previous message) preceding the node (message), and the original message defaults to None.

children indicates that the original message has several forwarding, and 2items indicates that two forwarding users forward; 0 represents a first forwarding node; 10keys indicates that there are 11 fields; child _ count indicates several hops, and a hop of 1. In fig. 5, there are four more fields, namely, table 1, table 2, table 3 and table _ count, in children, and the three fields, namely, table 1, table 2 and table 3, are added to the structure data after the corresponding target comment original message tag, target comment object position tag and target comment attitude tag are obtained by the subsequent comment judgment model, and the other fields in children are consistent with those in root. Wherein label1 indicates whether the original message is being commented on; label2 shows the position information of the corresponding target object to be commented on the message forwarding chain if the target object is not the original message of the comment; label3 represents comment attitude information of the target original message for the published object of the target forward comment message.

The method for analyzing the target original message and the target forwarding data corresponding to the target original message by calling the preset analysis interface according to the field to obtain the target result data comprises the following steps: aggregating target forwarding data in a message forwarding chain corresponding to the target original message according to the target forwarding object information of the target forwarding comment message; and calling a preset analysis interface to analyze the aggregated target original message and the target forwarding data corresponding to the target original message according to the field so as to obtain target result data.

The aggregation is to aggregate all the target forwarding comment messages of the target original message together, and aggregate all the forwarding comment messages (child nodes) of one target forwarding comment message (node). As shown in fig. 2, in the first message forwarding chain and the second message forwarding chain, for @ lie three: to avoid the forwarding comment message (node) that is not pubic but is not useful at all, there are two child nodes, which are: semi-circle: but not useful; ABC: like an expression. The two child nodes are clustered together.

The preset parsing interface is an interface for parsing the aggregated target original message and the target forwarding data corresponding to the target original message, and may be a Python interface developed using Python language.

And according to the fields, calling a preset analysis interface to analyze the aggregated target original message and the target forwarding data corresponding to the target original message, such as analyzing a number corresponding to the target original message, a number corresponding to each target forwarding comment message, a text message corresponding to the target original message, a text message corresponding to each target forwarding comment message, a previous target forwarding comment message corresponding to each target forwarding comment message, a parent node number corresponding to each target forwarding comment message, counting the number of child nodes corresponding to each target forwarding comment message, and the like.

And after the target result data is obtained through analysis, filling corresponding fields with the target result data to obtain target structure data corresponding to the target original message.

The populated target structure data is shown in fig. 6. Wherein, in the root node, id: 9527 (a); name: "Lisansan"; parent: 0; text: "I tend to think the souls in the journey are all evasive souls" "" I tend to feel the lead that is on the soul of escapaping "; up _ text: "None", and the like. In the child node, id: 9832; name: "moon" and "moon"; parent: 9527 (a); text: "Escape pubic but useful" "" Escape is shameful but usefull "; child _ count: 1; up _ text: "I tend to think that souls in the journey are all evasive souls" "I tend to feel the fat on the journey is the fat of escapaping" "and so on. Please refer to the corresponding content described above, and will not be described again.

And structuring the data to obtain target structure data, wherein one target original message corresponds to one target structure data.

And 304, taking each target forwarding comment message and each target original message as corresponding target input data of each group, and performing feature processing on the target input data of each group by using a comment discrimination model to obtain corresponding target feature vectors.

305, analyzing and processing each target feature vector by using a comment original message discriminator, a commented object position discriminator and a comment attitude discriminator of the comment discriminating model to respectively obtain a target comment original message tag, a target commented object position tag and a target comment attitude tag corresponding to each target feature vector.

And 306, determining a target object commented by each target forwarding comment message and comment attitude information of a target forwarding comment message to the target original message according to the target comment original message tag, the target commented object position tag and the target comment attitude tag, wherein the target object comprises an original object and/or a published object publishing the target original message.

Wherein, the steps 301 to 306 are the same as those in the above embodiments, please refer to the corresponding description above, and are not repeated herein.

In an embodiment, steps 307 and 308 are also included.

307, according to the comment attitude information corresponding to each target forwarding comment message in the target forwarding data, determining a first number of forwarding objects of which the comment attitude information is support attitude, and determining a second number of forwarding objects of which the comment attitude information is anti-attitude.

Wherein the comment attitude comprises an objection attitude and a support attitude. And counting a first number of forwarding objects with support attitudes in each target forwarding comment message in the target forwarding data corresponding to the target original message, and counting a second number of forwarding objects with comment attitude information of anti-attitude.

And 308, determining whether the target original message is a false message according to the first quantity and the second quantity.

In one case, if the first number is greater than the second number, determining that the target original message is not a rumor false message; and if the first number is smaller than the second number, determining that the target original message is a false message. For example, the false message may be a rumor, false news, fraud information, and the like.

In one case, when the ratio of the second quantity to the first quantity exceeds a preset ratio, determining that the target original message is a false message; and when the ratio of the second quantity to the first quantity does not exceed the preset ratio, determining that the target original message is not a false message. The preset ratio may be any value greater than 1, such as 1.5.

In one case, when the ratio of the second number to the total number (the sum of the first number and the second number) is a preset ratio, the probability that the target original message has the preset ratio is considered as a false message. If the ratio of the second number to the total number is 0.8, the target original message is considered as a false message with a probability of 80%.

In the embodiment, the target comment attitude tag predicted by the comment discrimination model is further utilized to detect the false messages, so that the accuracy of the false message detection is greatly improved, and the propagation of the false messages can be fundamentally avoided.

In the embodiment, the comment judging model is used for processing the target original message and the target forwarding data on the message forwarding chain corresponding to the target original message so as to determine the target object commented by the target forwarding comment message and comment attitude information of the target original message about the publishing object of the target forwarding comment message.

The target forwarding comment message and the target original message in the message forwarding chain are input to the comment judging model together for processing, rather than only the target forwarding comment message of each forwarding object on the message forwarding chain is processed independently, so that deeper information in the message forwarding chain can be mined, and the accuracy of target forwarding data analysis is improved.

In addition, a judgment tag can be obtained by using the comment judgment model, a target object to be commented by the target forwarding comment message and comment attitude information of a publishing object of the target forwarding comment message to the target original message are determined according to the judgment tag, and understandably, on one hand, the comment attitude information of the publishing object to the target original message can be determined by using the comment judgment model, so that the comment attitude of the publishing object of the target forwarding comment message to the target original message can be obtained, and the accuracy of target forwarding data analysis is improved; on the other hand, the target forwarding data is processed through the comment discrimination model, and the accuracy and the efficiency of analyzing the target forwarding data are further improved.

The comment discrimination model in the above embodiment may be obtained by a training process in advance. The process of training to obtain the comment discriminant model will be described in detail below.

Specifically, as shown in fig. 7, a schematic flow chart of a data processing method provided in the embodiment of the present application is provided, where the data processing method can implement obtaining a corresponding comment discrimination model through training processing, and a flow chart of the data processing method may be as follows.

401, forwarding data on a message forwarding chain corresponding to each original message in the original message set and the original message set is obtained.

The original message set is used to train a comment discriminative model. The original message set comprises a plurality of original messages, each original message in the original message set corresponds to a message forwarding chain, and forwarding data correspond to the message forwarding chains. The message forwarding chain corresponding to each original message may include one or more message forwarding chains. The forwarding data in the message forwarding chain comprises forwarding comment messages and/or forwarding object information. The forwarding comment message and the corresponding forwarding object in the forwarding data may be one or more.

402, obtaining the forwarding comment message in the forwarding data, and a comment original message tag, a comment object position tag and a comment attitude tag corresponding to the forwarding comment message.

Specifically, forwarding comment messages in forwarding data, and comment original message tags, comment object position tags and comment attitude tags corresponding to the forwarding comment messages are obtained.

And labeling each original message in the original message set and each forwarded comment message on the message forwarding chain corresponding to each original message to respectively obtain a comment original message label, a comment object position label and a comment attitude label.

The comment original message label, the commented object position label and the comment attitude label correspond to the target comment original message label, the target commented object position label and the target comment attitude label in the above manner one by one. If the comment original message label is used for indicating whether the comment original message is being commented (whether the original message is being replied) or not in the forwarding process of the forwarded comment message; if it can be represented by 1, if it can be represented by 0. The comment object location tag is used for indicating which forwarding user on the reply message forwarding chain of the publishing user corresponding to the forwarding comment message is determined if the forwarding comment message is not the comment original message, namely the comment object location tag is the location information of the comment object in the message forwarding chain. Wherein the location information may also be represented by 0/1; if there are three nodes (three forwarding comment messages and related information) a-B-C in the message forwarding chain, the forwarding comment message can be represented by 0-1-0 in response to the second forwarding comment message, and the forwarding comment message can be represented by 1-0-0 in response to the first forwarding comment message. And the comment attitude tag is used for determining comment attitude information of the publishing user corresponding to the forwarded comment message to the original message. The comment attitude information can also be represented by 0/1, 0 represents the anti-attitude, and 1 represents the support attitude. It should be noted that the individual tags may also be represented in other ways.

The method for labeling each original message in the original message set and each forwarded comment message on the message forwarding chain corresponding to each original message includes manual labeling, and automatic labeling can also be realized in other modes. Because the pre-training language model (described in detail below) has been pre-trained, the pre-training language model can be fine-tuned using fewer original message sets, and because the original message sets have fewer data, the fine-tuning can be performed using an artificial labeling method, and the accuracy of the labeling can also be improved. Or automatic labeling is carried out by using an automatic labeling mode, and then the automatic labeling result is adjusted.

And after marking, acquiring a comment original message label, a comment object position label and a comment attitude label corresponding to each forwarded comment message.

And 403, taking the forwarded comment message and the corresponding original message, and the comment original message label, the commented object position label and the comment attitude label corresponding to the forwarded comment message as input data, and performing feature processing on the input data by using a pre-training language model to obtain a corresponding feature vector.

The pre-trained language model in the embodiment of the present application will be described first.

The pre-trained language model may be a BERT (bidirectional Encoder replication from transformations) language model, or may be other pre-trained language models. The pre-training language model can be trained in advance, and the pre-training language model which is trained in advance can be directly obtained.

The method for training the pre-training language model comprises the following steps: obtaining a corpus data set; preprocessing a corpus data set; and training a pre-training network structure (BERT network structure) by using the preprocessed corpus data set to obtain a pre-training language model (BERT language model).

The corpus data set comprises social media, known comment data of all E-commerce media, Wikipedia, Baidu encyclopedia, daily reports of people and other corpus data of a plurality of different fields, so that more corpuses can be received as far as possible, and the migration capability of the pre-training language model is enhanced.

The step of preprocessing the corpus data set comprises the following steps: and cleaning the corpus data set, and analyzing the cleaned corpus data set into a format which can be processed by a pre-training network structure. Wherein, wash the material data set, include: and deleting the webpage links, the label information, the invalid characters and the like in the corpus data set. The preset cleaning interface can be called to clean the material data set, and the preset cleaning interface can be a Python script written by a Python language.

Analyzing the cleaned corpus data set into a format which can be processed by a pre-training network structure, wherein the format comprises the following steps: for a paragraph, segmenting a plurality of sentences in the paragraph into a plurality of combined sentences in a form of window size 2, namely, taking every two adjacent sentences in the paragraph as one combined sentence to split the paragraph so as to form a plurality of combined sentences, wherein each combined sentence has only two sentences; the random shielding of words in the sentence, for example, the random shielding of words in the sentence according to a certain proportion, can be set to any proportion of 15% -50%. The method comprises the steps of segmenting a plurality of sentences in a paragraph and randomly masking words in the sentences respectively so as to be suitable for two different pre-training tasks.

The two pre-training tasks used in training the pre-training network structure are mlm (masked Language model) and nsp (next sequence prediction), respectively. MLM refers to the task of masking out some words from the input corpus at the time of training and then predicting the masked out words through context, which can also be simply understood as completing the blank filling. The NSP task is to determine whether sentence B is a context of sentence a, and if so, output "IsNext", otherwise, output "NotNext".

If the pre-training network structure is based on the BERT network structure, a 12-layer Tranfromers structure can be adopted, the dimensionality of the initialized word vector has 768 dimensions, and 12 attention layers are provided, wherein the size of a dictionary is 21128. Other BERT network architectures may also be employed.

And respectively calculating the losses corresponding to the two pre-training tasks in the process of training the pre-training network structure, and stopping training when the sum of the loss values corresponding to the two pre-training tasks is detected not to be reduced any more. Thus, a pre-training network model is obtained.

Wherein, the pre-training language model is used, and retraining is not needed when the domain migration is carried out every time. Because the corpus is used as much as possible in the pre-training process, the pre-training network model already contains knowledge of each field after the pre-training network structure is trained by using the corpus data set comprising the corpus to obtain the pre-training network model, the re-training is not needed during the field migration, and the training cost and the training time are saved.

After the pre-training language model is obtained, all the forwarded comment messages and the corresponding original messages, and comment original message labels, commented object position labels and comment attitude labels corresponding to all the forwarded comment messages are used as all the groups of input data and input into the pre-training language model, and the pre-training language model is used for carrying out feature processing on all the groups of input data to obtain corresponding feature vectors.

It can be understood that each set of input data includes a forwarded comment message, an original message corresponding to the forwarded comment message, and three tags corresponding to the forwarded comment message: comment original message label, commented object position label and comment attitude label. As shown in table 2, a schematic table of each set of input data.

Table 2 schematic table of each set of input data

And inputting each group of input data into a pre-training language model for feature processing to obtain corresponding feature vectors. Wherein each set of input data corresponds to a feature vector.

404, training the established comment original message discriminator, the comment object position discriminator and the comment attitude discriminator according to the feature vector and the corresponding comment original message label, the comment object position label and the comment attitude label to obtain a comment discrimination model.

Because modeling needs to be performed on three labels of the labeled original message set, three types of discriminators are established in the application, namely a comment original message discriminator, a commented object position discriminator and a comment attitude discriminator. Each discriminator corresponds to one target task, and the three tasks are trained and learned by adopting cross entropy respectively. The three types of discriminators cannot be used for discrimination until untrained.

Specifically, three types of discriminators are trained according to each feature vector, and a comment original message label, a comment object position label and a comment attitude label corresponding to each feature vector.

Specifically, training a comment original message discriminator according to each feature vector and a comment original message label corresponding to each feature vector to obtain a first loss value corresponding to a target task one; training a commented object position discriminator according to the feature vectors and commented object position labels corresponding to the feature vectors to obtain a second loss value corresponding to the target task II; training a comment attitude discriminator according to each feature vector and the comment attitude label corresponding to each feature vector to obtain a third loss value corresponding to the target task three; and adding the first loss value, the second loss value and the third loss value to obtain a loss value, performing back propagation to update parameters of the neural network through the loss value, calculating a new loss value in a forward direction by using the updated parameters until the new loss value is not reduced any more, and obtaining three types of trained discriminators to obtain a comment discrimination model. The comment judging model at least comprises a comment original message discriminator, a commented object position discriminator and a comment attitude discriminator.

It should be noted that, after the pre-training language model is obtained by training using the corpus data set, the above steps are processes of fine tuning the pre-training language model, so that the pre-training language model is applicable to the application scenario of the message forwarding chain in the embodiment of the present application. Understandably, the pre-training language model is used for retraining in the new three target tasks in the embodiment of the application, so that the parameters in the pre-training language model are ensured to be fixed, only the newly added neural network is trained, and finally the comment discrimination model is obtained. The comment judging model comprises a comment original message discriminator, a commented object position discriminator, a comment attitude discriminator and a pre-training language model.

After the comment judging model is obtained, the obtained comment judging model can be used for processing the target forwarding data on the target original message and the message forwarding chain corresponding to the target original message, so that comment original message labels, commented object position labels and comment attitude labels corresponding to all target forwarding comment messages in the target forwarding data are obtained. Therefore, whether each forwarded comment message responds to the original message or not can be obtained, if not, which publishing user responds to the original message, and the reply attitude information (comment attitude information) and the like can be obtained.

According to the embodiment of the application, the message forwarding chain is processed together with the corresponding original message as a whole instead of independently processing the forwarding comment message of each forwarding object on the message forwarding chain, so that deeper information in the message forwarding chain can be mined, and the accuracy of forwarding data analysis is improved.

In addition, after obtaining the comment original message label, the commented object position label and the comment attitude label, the pre-training language model is used for processing and training each forwarded comment message, the original message, the corresponding comment original message label, the commented object position label and the comment attitude label in the message forwarding chain to obtain a comment judgment model so as to respectively obtain three different discriminators, the three discriminators of the comment judgment model can be used for processing the target forwarding data of the message forwarding chain corresponding to the target original message and the target original message, and can determine whether the forwarded comment message is in the commented original message, the commented object position information, the comment attitude information and the like, namely the comment judgment model can determine whether the forwarded comment message is in the commented original message and forward the position information of the commented object on the message forwarding chain, and comment attitude information and the like corresponding to the forwarded comment messages, and the target forwarding data corresponding to the target original message and the target original message are processed through the comment judging model, so that the accuracy and the efficiency of analyzing the forwarding data are improved.

In an embodiment, if forwarding comment messages corresponding to forwarding data in a message forwarding chain are directly input to a pre-training language model for processing, for example, forwarding comment messages in the message forwarding chain shown in fig. 2 are directly input to the pre-training language model for processing, the implementation is troublesome; on the other hand, the message forwarding chain corresponding to the microblog is different from the message forwarding chain corresponding to the news message, and the pre-training language model is required to be respectively suitable for the different message forwarding chains, so that the processing efficiency and the applicability of the pre-training language model are reduced. Therefore, the forwarding data in the message forwarding chain needs to be structured and processed into structural data. And extracting each forwarded comment message after the structuralization processing, and inputting the message into a pre-training language model for processing, so that the formats of input data of the pre-training language model under all scenes are kept consistent.

In an embodiment, before/after the forwarding data in the message forwarding chain is subjected to the structuring processing, the original message and the forwarding data corresponding to the message forwarding chain of the original message need to be cleaned, so as to avoid that dirty data in the original message affects training and affects the accuracy of the comment judgment model.

Specifically, as shown in fig. 8, a schematic flow chart of the data processing method provided in the embodiment of the present application is shown. The data processing method can achieve the purpose that the corresponding comment judgment model is obtained through training processing, and comprises the following steps.

501, obtaining forwarding data on a message forwarding chain corresponding to each original message in an original message set and an original message set.

Wherein, the forwarding data comprises each forwarding comment message and/or forwarding object information/forwarding user information.

502, the original messages and the forwarding data in the message forwarding chain corresponding to the original messages are cleaned to obtain the cleaned original messages and the corresponding forwarding data.

And cleaning forwarding data in each original message in the original message set and a message forwarding chain corresponding to each original message. Wherein, include: deleting the original message without any message forwarding chain in the original message set; deleting the message forwarding chains without comment messages in the message forwarding chains corresponding to the original messages in the remaining original messages to obtain remaining message forwarding chains; and deleting the forwarding comment messages of which the contents meet the preset conditions and the forwarding data behind the forwarding comment messages in the remaining message forwarding chains to obtain the cleaned original messages and the corresponding forwarding data. The method comprises the steps of cleaning original messages in an original message set, and cleaning forwarding data in a message forwarding chain corresponding to the original messages.

The message forwarding chain without comment message includes that only forwarding user information exists on the message forwarding chain, namely, forwarding users correspond to the message forwarding chain, but the forwarding users do not comment. As in the third message forwarding chain in fig. 2, if only the forwarding user who displays "cyan" forwards the original message but does not comment, the message forwarding chain is deleted.

The step of deleting the forwarding data after the comment forwarding message whose contents meet the preset conditions includes: deleting the forwarded comment message and the forwarded data after the forwarded comment message, wherein the number of words of the content of the forwarded comment message is less than the preset number of words; and/or deleting the forwarding comment message and the forwarding data after the forwarding comment message, wherein the content of the forwarding comment message meets the preset content; and/or deleting the forwarded comment message and the forwarded data after the forwarded comment message, wherein the content of the forwarded comment message is neither the comment original message nor the forwarded object on the comment message forwarding chain. Please refer to the description of the corresponding steps above.

According to specific requirements, other aspects of cleaning (except for the cleaning) can be performed on the original messages and the forwarding data in the message forwarding chain corresponding to the original messages, so as to improve the accuracy of the subsequent comment judgment model.

And 503, performing structuring processing on each cleaned original message and corresponding forwarding data to obtain structural data corresponding to each original message.

And performing structuring processing on each cleaned original message and corresponding forwarding data, which can be understood as converting each cleaned original message and corresponding forwarding data into a certain format.

Carrying out structural processing on each cleaned original message and corresponding forwarding data, on one hand, in order to be suitable for various message forwarding chains, such as message forwarding chains with different forms/formats, such as microblog forwarding chains, news forwarding chains and the like; in addition, after each cleaned original message and corresponding forwarding data are converted into a certain format, the format is relatively neat and convenient to store; and the original messages and the corresponding forwarding data which are converted into a certain format and the three labels corresponding to the forwarding comment messages in the forwarding data are used as the input of the pre-training language model, so that the pre-training language model can understand the input data conveniently, and the efficiency and the accuracy of the training comment judgment model are improved.

The step of performing structured processing on each cleaned original message and corresponding forwarding data to obtain structural data corresponding to each original message includes: acquiring a preset data structured format and a plurality of fields in the data structured format; according to the fields, calling a preset analysis interface to analyze each original message and forwarding data corresponding to each original message to obtain result data; and filling the corresponding field by using the result data to obtain the structure data corresponding to each original message.

The method comprises the following steps of calling a preset analysis interface to analyze each original message and forwarding data corresponding to each original message according to fields to obtain result data, wherein the steps comprise: according to the forwarding object information, aggregating the forwarding data in the message forwarding chain corresponding to each original message; and calling a preset analysis interface to analyze the aggregated original messages and the forwarding data corresponding to the original messages according to the fields so as to obtain result data.

And according to the fields, calling a preset analysis interface to analyze the aggregated original messages and the forwarding data corresponding to the original messages, such as analyzing the numbers corresponding to the original messages, the numbers corresponding to the forwarding comment messages, the text messages corresponding to the original messages, the text messages corresponding to the forwarding comment messages, the previous forwarding comment messages corresponding to the forwarding comment messages, the numbers of the father nodes corresponding to the forwarding comment messages, counting the number of the child nodes corresponding to the forwarding comment messages, and the like.

And after analyzing the obtained result data, filling corresponding fields with the result data to obtain the structural data corresponding to each original message. The populated structure data can be seen in fig. 6.

And obtaining structural data after structuring the data, wherein each original message corresponds to one structural data. Thus, a plurality of configuration data can be obtained.

And 504, acquiring a comment original message tag, a commented object position tag and a comment attitude tag corresponding to each forwarded comment message in the forwarded data, acquiring a pre-trained language model, and adding the comment original message tag, the commented object position tag and the comment attitude tag corresponding to each forwarded comment message into the structural data.

The steps of obtaining the comment original message tag, the comment object position tag and the comment attitude tag corresponding to each forwarded comment message and obtaining the pre-training language model please refer to the above description.

After obtaining the comment original message label, the commented object position label and the comment attitude label corresponding to each forwarded comment message, adding the comment original message label, the commented object position label and the comment attitude label corresponding to each forwarded comment message into the structural data, so that three fields including label1, label2 and label3 and label values corresponding to the three fields are added in the structural data. Or there are three corresponding fields, namely, label1, label2 and label3, in the structure data, before the comment original message label, the comment object position label and the comment attitude label corresponding to each forwarded comment message are not obtained, the values of the three fields are null, and after the corresponding label values are obtained, the three fields, namely, label1, label2 and label3 in the structure data are filled with the corresponding label values.

505, extracting each forwarding comment message and original message in the structure data, comment original message label corresponding to each forwarding comment message, position label of object to be commented and comment attitude label.

And extracting each forwarded comment message and original message, comment original message label corresponding to each forwarded comment message, position label of the object to be commented and comment attitude label from the structural data, and using the extracted information as the input of the pre-trained language model, so that the pre-trained language model can understand the input data conveniently, and the efficiency and accuracy of training the comment judgment model are improved.

And 506, taking each forwarded comment message and the corresponding original message, and the comment original message label, the commented object position label and the comment attitude label corresponding to each forwarded comment message as each group of input data, and performing feature processing on each group of input data by using a pre-training language model to obtain each corresponding feature vector.

507, training the established comment original message discriminator, the established comment object position discriminator and the established comment attitude discriminator according to each feature vector and the corresponding comment original message label, the comment object position label and the comment attitude label to obtain a comment discrimination model.

Please refer to the corresponding description in the above embodiments for the steps corresponding to the steps 401 to 404 in the steps 501 to 507, which are not repeated herein.

The embodiment cleans the original messages and the forwarding data in the message forwarding chain corresponding to the original messages so as to improve the quality of the input data and avoid the influence of dirty data on the accuracy of the comment judgment model. Carrying out structuralization processing on each cleaned original message and corresponding forwarding data so as to be suitable for various message forwarding chains; meanwhile, the structure data is stored to facilitate storage; in addition, the input data of the pre-training language model is extracted from the structural data, so that the pre-training language model can understand the input data conveniently, and the efficiency and the accuracy of training the comment discrimination model are improved.

It should be noted that, the above-mentioned data processing method for implementing the corresponding comment judgment model and the data processing method for processing the target original message and the target forwarding data on the message forwarding chain corresponding to the target original message by using the comment judgment model can be applied to the same electronic device, and can also be applied to different electronic devices. When the comment judging model is applied to the same electronic equipment, the corresponding comment judging model can be realized, and the steps of processing the target original message and the target forwarding data on the message forwarding chain corresponding to the target original message by using the comment judging model are executed in the same electronic equipment. When the comment identification model is applied to different electronic equipment, the steps corresponding to the corresponding comment identification model can be realized and executed in one electronic equipment, and the steps of processing the target original message and the target forwarding data on the message forwarding chain corresponding to the target original message by using the comment identification model are executed in the other electronic equipment.

All the above technical solutions can be combined arbitrarily to form the optional embodiments of the present application, and are not described herein again.

In order to better implement the data processing method of the embodiment of the present application, an embodiment of the present application further provides a data processing apparatus. Please refer to fig. 9, fig. 9 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application. The data processing apparatus may include a target acquisition module 601, a tag determination module 602, and an attitude determination module 603.

The target obtaining module 601 is configured to obtain a target original message and target forwarding data on a message forwarding chain corresponding to the target original message.

The tag determining module 602 is configured to input a target forwarding comment message in the target forwarding data and the target original message as target input data to the comment determination model to obtain a determination tag.

In an embodiment, when the tag determining module 602 performs the step of inputting the target forwarding comment message and the target original message in the target forwarding data as target input data to the comment judgment model to obtain the judgment tag, specifically performs: taking a target forwarding comment message and a target original message of the target forwarding data as corresponding target input data, and performing feature processing on the target input data by using a comment discrimination model to obtain corresponding target feature vectors; and analyzing and processing the target characteristic vector by using a comment original message discriminator, a commented object position discriminator and a comment attitude discriminator of the comment discrimination model to respectively obtain a target comment original message label, a target commented object position label and a target comment attitude label which correspond to the target characteristic vector. The judging tags comprise target comment original message tags, target comment object position tags and target comment attitude tags.

The attitude determination module 603 is configured to determine, according to the discrimination tag, a target object commented on each target forwarding comment message and comment attitude information of a target original message about a target object of the target forwarding comment message, where the target object includes an original object of the target original message and a published object.

In one embodiment, the data processing apparatus further includes: an object cleaning module 604. The target cleaning module 604 is configured to, after target forwarding data on a message forwarding chain corresponding to the target original message and the target original message are obtained, clean the target original message and the target forwarding data to obtain a cleaned target original message and the cleaned target forwarding data.

In one embodiment, the data processing apparatus further includes: a target structuring module 605 and a target extraction module 606. The target structuring module 605 is configured to perform a structuring process on the target original message and the target forwarding data to convert the target original message and the target forwarding data into corresponding target structure data. And the target extraction module 606 is configured to extract each target forwarding comment message and each target original message in the target structure data as each set of input data.

In an embodiment, the target structuring module 605 is specifically configured to obtain a preset data structuring format and a plurality of fields in the data structuring format; according to the field, calling a preset analysis interface to analyze the target original message and the target forwarding data corresponding to the target original message to obtain target result data; and filling the corresponding field by using the target result data to obtain target structure data corresponding to the target original message.

In an embodiment, when executing the step of invoking a preset parsing interface to parse the target original message and the target forwarding data corresponding to the target original message according to the field to obtain the target result data, the target structuring module 605 specifically executes: aggregating the target forwarding data in the message forwarding chain corresponding to the target original message according to the target forwarding object information of the target forwarding comment message; and calling a preset analysis interface to analyze the aggregated target original message and the target forwarding data corresponding to the target original message according to the field so as to obtain target result data.

In one embodiment, the data processing apparatus further includes: a quantity determination module 607 and a dummy message determination module 608. The quantity determining module 607 is configured to determine, according to the comment attitude information corresponding to each target forwarding comment message in the target forwarding data, a first quantity of forwarding objects whose comment attitude information is a support attitude, and a second quantity of forwarding objects whose comment attitude information is an objection attitude. A false message determination module 608, configured to determine whether the target original message is a false message according to the first number and the second number.

In an embodiment, as shown in fig. 10, a schematic structural diagram of a data processing apparatus provided in the embodiment of the present application is shown. The data processing apparatus may include a first acquisition module 701, a second acquisition module 702, a training module 703, and a processing module 704.

A first obtaining module 701, configured to obtain forwarding data on a message forwarding chain corresponding to an original message set and each original message in the original message set, where the forwarding data includes each forwarding comment message.

The second obtaining module 702 is configured to obtain a comment original message tag, a comment object position tag, and a comment attitude tag corresponding to each forwarded comment message, and obtain a pre-training language model.

The training module 703 is configured to use each forwarded comment message and the corresponding original message, and a comment original message tag, a commented object position tag, and a comment attitude tag corresponding to each forwarded comment message as each set of input data, and perform feature processing on each set of input data by using a pre-training language model to obtain corresponding feature vectors; and training the established comment original message discriminator, the comment object position discriminator and the comment attitude discriminator according to each feature vector and the corresponding comment original message label, the comment object position label and the comment attitude label to obtain a comment discrimination model.

In an embodiment, the data processing apparatus may further include a cleaning module 704. The cleaning module 704 is configured to clean each original message and forwarding data corresponding to each original message to obtain each cleaned original message and corresponding forwarding data.

In an embodiment, the data processing apparatus may further include a structuring processing module 705, an adding module 706, and an extracting module 707. The structural processing module 705 is configured to perform structural processing on each original message and forwarding data corresponding to each original message to obtain structural data corresponding to each original message. The adding module 706 is configured to add the comment original message tag, the commented object position tag, and the comment attitude tag corresponding to each forwarded comment message to the structural data after the second obtaining module obtains the comment original message tag, the commented object position tag, and the comment attitude tag corresponding to each forwarded comment message. And the extracting module 707 is configured to extract each forwarded comment message and original message in the structure data, a comment original message tag corresponding to each forwarded comment message, a comment object position tag, and a comment attitude tag, to serve as each group of input data.

In other embodiments, the data processing apparatus includes a plurality of or all of the module units corresponding to fig. 9 and 10. And will not be described in detail herein.

All the above technical solutions can be combined arbitrarily to form the optional embodiments of the present application, and are not described herein again. For the detailed implementation technical means and the achieved beneficial effects of the data processing apparatus, please refer to the corresponding parts of the data processing method, which are not described herein again.

Correspondingly, an embodiment of the present application further provides an electronic device, where the electronic device may be a terminal or a server, and the terminal may be the above-mentioned device such as the terminal or the server. As shown in fig. 11, fig. 11 is a schematic structural diagram of an electronic device according to an embodiment of the present application. The electronic device 800 includes a processor 801 with one or more processing cores, a memory 802 with one or more computer-readable storage media, and a computer program stored on the memory 802 and executable on the processor. The processor 801 is electrically connected to the memory 802.

The processor 801 is a control center of the electronic device 800, connects various parts of the entire electronic device 800 using various interfaces and lines, and performs various functions of the electronic device 800 and processes data by running or loading software programs and/or modules stored in the memory 802 and calling data stored in the memory 802, thereby performing overall monitoring of the electronic device 800.

In the embodiment of the present application, the processor 801 in the electronic device 800 loads instructions (computer programs) corresponding to processes of one or more application programs into the memory 802, and the processor 801 executes the application programs stored in the memory 802 according to the following steps, so as to implement various functions:

The above operations can be implemented in the foregoing embodiments, and are not described in detail herein.

Optionally, as shown in fig. 11, the electronic device 800 further includes: a touch display 803, a radio frequency circuit 804, an audio circuit 805, an input unit 806, and a power supply 807. The processor 801 is electrically connected to the touch display screen 803, the radio frequency circuit 804, the audio circuit 805, the input unit 806, and the power supply 807, respectively. Those skilled in the art will appreciate that the electronic device configurations shown in the figures do not constitute limitations of the electronic device, and may include more or fewer components than shown, or some components in combination, or a different arrangement of components.

The touch display screen 803 can be used for displaying a graphical user interface and receiving operation instructions generated by a user acting on the graphical user interface. The touch display 803 may include a display panel and a touch panel. The display panel may be used, among other things, to display information entered by or provided to a user and various graphical user interfaces of the electronic device, which may be made up of graphics, text, icons, video, and any combination thereof. Alternatively, the Display panel may be configured in the form of a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), or the like. The touch panel may be used to collect touch operations of a user on or near the touch panel (for example, operations of the user on or near the touch panel using any suitable object or accessory such as a finger, a stylus pen, and the like), and generate corresponding operation instructions, and the operation instructions execute corresponding programs. Alternatively, the touch panel may include two parts, a touch detection device and a touch controller. The touch detection device detects the touch direction of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch sensing device, converts the touch information into touch point coordinates, sends the touch point coordinates to the processor 801, and can receive and execute commands (computer programs) sent by the processor 801. The touch panel may overlay the display panel, and when the touch panel detects a touch operation thereon or nearby, the touch panel transmits the touch operation to the processor 801 to determine the type of the touch event, and then the processor 801 provides a corresponding visual output on the display panel according to the type of the touch event. In the embodiment of the present application, a touch panel and a display panel may be integrated into the touch display screen 803 to realize input and output functions. However, in some embodiments, the touch panel and the touch panel can be implemented as two separate components to perform the input and output functions. That is, the touch display 803 may also be used as a part of the input unit 806 to implement an input function.

The radio frequency circuit 804 may be configured to transmit and receive radio frequency signals to establish wireless communication with a network device or other electronic devices through wireless communication, and transmit and receive signals with the network device or other electronic devices.

The audio circuit 805 may be used to provide an audio interface between a user and an electronic device through a speaker, microphone, or the like. The audio circuit 805 may transmit the electrical signal converted from the received audio data to a speaker, and convert the electrical signal into an audio signal for output; on the other hand, the microphone converts the collected sound signal into an electrical signal, which is received by the audio circuit 805 and converted into audio data, and the audio data is processed by the audio data output processor 801 and then sent to another electronic device via the rf circuit 804, or the audio data is output to the memory 802 for further processing. The audio circuit 805 may also include an earbud jack to provide communication of a peripheral headset with the electronic device.

The input unit 806 may be used to receive input numbers, character information, or user characteristic information (e.g., fingerprint, iris, facial information, etc.), and generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control.

The power supply 808 is used to power the various components of the electronic device 800. Optionally, the power supply 808 may be logically connected to the processor 801 through a power management system, so as to implement functions of managing charging, discharging, power consumption management, and the like through the power management system. The power supply 807 may also include any component of one or more dc or ac power sources, recharging systems, power failure detection circuitry, power converters or inverters, power status indicators, and the like.

Although not shown in fig. 11, the electronic device 800 may further include a camera, a sensor, a wireless fidelity module, a bluetooth module, etc., which are not described in detail herein.

In the foregoing embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

It will be understood by those skilled in the art that all or part of the steps of the methods of the above embodiments may be performed by instructions (computer programs) or by instructions controlling associated hardware, and the instructions may be stored in a computer readable storage medium and loaded and executed by a processor.

To this end, the present application provides a computer-readable storage medium, in which a plurality of computer programs are stored, and the computer programs can be loaded by a processor to execute the steps in any data processing method provided by the present application. For example, the computer program may perform the steps of:

Wherein the storage medium may include: read Only Memory (ROM), Random Access Memory (RAM), magnetic or optical disks, and the like.

Since the computer program stored in the storage medium can execute the steps in any data processing method provided in the embodiments of the present application, beneficial effects that can be achieved by any data processing method provided in the embodiments of the present application can be achieved, and detailed descriptions are omitted here for the foregoing embodiments.

The foregoing detailed description is directed to a data processing method, an apparatus, a storage medium, and an electronic device provided in the embodiments of the present application, and specific examples are applied in the present application to explain the principles and implementations of the present application, and the descriptions of the foregoing embodiments are only used to help understand the method and the core ideas of the present application; meanwhile, for those skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.

Claims

1. A data processing method, comprising:

2. The data processing method according to claim 1, wherein the comment discriminating model is obtained by a training process in advance, and the training process includes:

acquiring an original message set and forwarding data on a message forwarding chain corresponding to each original message in the original message set;

acquiring a forwarding comment message in the forwarding data, and a comment original message tag, a comment object position tag and a comment attitude tag corresponding to the forwarding comment message;

taking the forwarded comment message and the corresponding original message as well as the comment original message label, the commented object position label and the comment attitude label corresponding to the forwarded comment message as input data, and performing feature processing on the input data by using a pre-training language model to obtain a corresponding feature vector;

and training the established comment original message discriminator, the comment object position discriminator and the comment attitude discriminator according to the feature vector and the corresponding comment original message label, the comment object position label and the comment attitude label to obtain a comment discrimination model.

3. The data processing method according to claim 2, further comprising, after the step of obtaining forwarding data on a message forwarding chain corresponding to each original message in the original message set and the original message set, the step of:

carrying out structuralization processing on each original message and the forwarding data corresponding to each original message to obtain structure data corresponding to each original message;

after the step of obtaining the comment original message tag, the comment object position tag and the comment attitude tag corresponding to each forwarded comment message, the method further includes: adding a comment original message tag, a comment object position tag and a comment attitude tag corresponding to each forwarded comment message into the structural data;

extracting each forwarding comment message and the original message in the structural data, and the comment original message label, the commented object position label and the comment attitude label corresponding to each forwarding comment message.

4. The data processing method according to claim 1, wherein the step of inputting the target forwarding comment message and the target original message in the target forwarding data as target input data to a comment discrimination model to obtain a discrimination tag includes:

taking a target forwarding comment message and the target original message in the target forwarding data as target input data, and performing feature processing on the target input data by using a comment discrimination model to obtain a corresponding target feature vector;

analyzing and processing the target feature vector by using a comment original message discriminator, a commented object position discriminator and a comment attitude discriminator of the comment discrimination model to respectively obtain a target comment original message label, a target commented object label and a target comment attitude label which correspond to the target feature vector, wherein the discrimination labels comprise the target comment original message label, the target commented object label and the target comment attitude label.

5. The data processing method according to claim 1, further comprising, after the step of obtaining the target original message and the target forwarding data on the message forwarding chain corresponding to the target original message:

performing structural processing on the target original message and the target forwarding data corresponding to the target original message to obtain target structural data corresponding to the target original message;

and extracting the target forwarding comment message and the target original message in the target structure data.

6. The data processing method according to claim 5, wherein the step of performing a structuring process on the target original message and the target forwarding data corresponding to the target original message to obtain target structure data corresponding to the target original message comprises:

acquiring a preset data structured format and a plurality of fields in the data structured format;

according to the field, calling a preset analysis interface to analyze the target original message and the target forwarding data corresponding to the target original message to obtain target result data;

and filling the corresponding field by using the target result data to obtain target structure data corresponding to the target original message.

7. The data processing method according to claim 6, wherein the step of invoking a preset parsing interface to parse the target original message and the target forwarding data corresponding to the target original message according to the field to obtain target result data comprises:

aggregating the target forwarding data in the message forwarding chain corresponding to the target original message according to the target forwarding object information of the target forwarding comment message;

and calling a preset analysis interface to analyze the aggregated target original message and the target forwarding data corresponding to the target original message according to the field so as to obtain target result data.

8. The data processing method according to claim 5, further comprising, before the step of performing the structured processing on the target original message and the target forwarding data corresponding to the target original message:

cleaning the target original message and the target forwarding data corresponding to the target original message to obtain the cleaned target original message and the corresponding target forwarding data;

the step of performing structured processing on the target original message and the target forwarding data corresponding to the target original message includes: and carrying out structural processing on the cleaned target original message and the corresponding target forwarding data to obtain target structural data corresponding to the target original message.

9. The data processing method according to claim 8, wherein the step of cleaning the target original message and the target forwarding data corresponding to the target original message to obtain the cleaned target original message and the corresponding target forwarding data comprises:

deleting the message forwarding chains without the target forwarding comment message in the message forwarding chains corresponding to the target original message to obtain residual message forwarding chains;

and deleting the target forwarding comment message of which the content meets the preset condition and the subsequent target forwarding data in the residual message forwarding chain to obtain the cleaned target original message and the corresponding target forwarding data.

10. The data processing method of claim 1, wherein the review attitude information includes a support attitude and an objection attitude, the data processing method further comprising:

according to comment attitude information corresponding to the target forwarding comment message in the target forwarding data, determining a first number of published objects of which the comment attitude information is support attitude and a second number of published objects of which the comment attitude information is against attitude;

and determining whether the target original message is a false message according to the first quantity and the second quantity.

11. The data processing method of claim 10, wherein the step of determining whether the target original message is a dummy message according to the first number and the second number comprises:

when the ratio of the second quantity to the first quantity exceeds a preset ratio, determining that the target original message is a false message;

when the ratio of the second quantity to the first quantity does not exceed a preset ratio, determining that the target original message is not a false message.

12. A data processing apparatus, comprising:

13. A computer-readable storage medium, characterized in that it stores a computer program adapted to be loaded by a processor for performing the steps of the data processing method according to any one of claims 1-11.

14. An electronic device, characterized in that the electronic device comprises a memory in which a computer program is stored and a processor, which performs the steps in the data processing method according to any one of claims 1-11 by calling the computer program stored in the memory.