CN112637621A

CN112637621A - Live broadcast auditing method and device, electronic equipment and storage medium

Info

Publication number: CN112637621A
Application number: CN202011451949.5A
Authority: CN
Inventors: 周杰; 王鸣辉; 孙振邦; 王长虎
Original assignee: Beijing ByteDance Network Technology Co Ltd
Current assignee: Douyin Vision Co Ltd; Douyin Vision Beijing Co Ltd
Priority date: 2020-12-09
Filing date: 2020-12-09
Publication date: 2021-04-09
Anticipated expiration: 2040-12-09
Also published as: CN112637621B

Abstract

The present disclosure provides a live broadcast auditing method, apparatus, electronic device and storage medium, wherein the method comprises: extracting live broadcast characteristic information from live broadcast media content generated in each historical time period in a plurality of historical time periods in a live broadcast room of the current time period, and extracting audit-sending characteristic information from historical audit-sending records corresponding to each historical time period; processing the submission characteristic information and the live broadcast characteristic information at least once to obtain processed characteristic information after each processing; and determining the submission result of the live broadcast room in the future time period based on the processed characteristic information after each processing. Therefore, the historical submission characteristic information is utilized to process the live broadcast characteristic information in different modes, so that the relation between the historical submission characteristic information and the live broadcast characteristic information is excavated as much as possible, the submission result of the determined future time period is accurate, the process is automatically processed, manual participation is not needed, and the efficiency and the accuracy of review are high.

Description

Live broadcast auditing method and device, electronic equipment and storage medium

Technical Field

The present disclosure relates to the field of information technologies, and in particular, to a live broadcast auditing method and apparatus, an electronic device, and a storage medium.

Background

With the continuous development of information technology, multimedia live broadcast is widely concerned by people, for example, one live broadcast can bring millions of traffic, so that the information propagation speed is high and is difficult to measure. This is not only to enrich the diversified viewing needs of the audience, but also to cause the broadcast content produced by the anchor in the broadcast industry to be varied, for example, some anchors broadcast illegal content on the broadcast platform in order to improve popularity, which may have a severe impact on the society. In order to reduce the spreading of illegal contents, illegal auditing of live contents on a live platform needs to be performed in time.

Currently, auditing of live broadcast contents is mainly manual auditing, auditors judge whether a current live broadcast room violates rules or not by watching a live broadcast frame drawing picture, and once the violating contents are found, warnings can be given to even punishment such as closing the live broadcast room. However, the above manual review mode has low review efficiency, and with the rapid growth of live content, the requirements of the live platform in the aspect of live content review cannot be met.

Disclosure of Invention

The embodiment of the disclosure provides at least one live broadcast auditing scheme, and the auditing prediction is automatically carried out by combining historical auditing records and live broadcast media contents, so that manual participation is not required, and the auditing efficiency is higher.

Mainly comprises the following aspects:

in a first aspect, an embodiment of the present disclosure provides a live broadcast auditing method, where the method includes:

acquiring live broadcast media content generated in each historical time period in a live broadcast room of a current time period in a plurality of historical time periods, and historical review records corresponding to each historical time period in the plurality of historical time periods;

extracting live broadcast characteristic information from live broadcast media content generated in each historical time period, and extracting review characteristic information from historical review records corresponding to each historical time period;

processing the review feature information and the live broadcast feature information at least once to obtain processed feature information after each processing;

and determining the submission result of the live broadcast room in the future time period based on the processing characteristic information after each processing.

In one embodiment, the processing the review feature information and the live feature information at least once to obtain processed feature information after each processing includes:

processing the review feature information and the live broadcast feature information according to a first operation set to obtain first processing feature information, and,

processing the submission feature information and the live broadcast feature information according to a second operation set to obtain second processing feature information; the first set of arithmetic operations and the second set of arithmetic operations have different arithmetic operations.

In one embodiment, the first set of arithmetic operations includes a first stitching operation, a fusion operation, and a second stitching operation; the processing the review feature information and the live broadcast feature information according to a first operation set to obtain first processing feature information includes:

performing a first splicing operation on the review feature information and the live broadcast feature information of each historical time period to obtain first splicing feature information corresponding to each historical time period;

performing feature fusion on the first splicing feature information of each historical time period by using the trained neural network to obtain fusion feature information corresponding to a historical time period closest to the current time period;

performing second splicing operation on the fusion characteristic information corresponding to the historical time period closest to the current time period and the review sending characteristic information of each historical time period to obtain second splicing characteristic information;

determining the first processing characteristic information based on the second splicing characteristic information.

In one embodiment, the first set of arithmetic operations further comprises a first fully-join operation and a second fully-join operation; the method for performing feature fusion on the first splicing feature information of each historical time period by using the trained neural network to obtain fusion feature information corresponding to a historical time period closest to the current time period includes:

aiming at each historical time period, performing full connection operation on the first splicing characteristic information of the historical time period by using a trained first full connection layer to obtain first splicing characteristic information with a first preset full connection dimension corresponding to the historical time period;

fusing the first splicing characteristic information with a first preset full-connection dimension corresponding to each historical time period by using the trained neural network, and determining the fusion characteristic information of one historical time period closest to the current time period;

the determining the first processing characteristic information based on the second splicing characteristic information includes:

performing full connection operation on the second splicing characteristic information by using a trained second full connection layer to obtain second splicing characteristic information with a second preset full connection dimension;

and determining the second splicing characteristic information with the second preset full-connection dimension as the first processing characteristic information.

In one embodiment, the neural network comprises a plurality of network layers; each network layer includes a plurality of neurons; the method for performing feature fusion on the first splicing feature information of each historical time period by using the trained neural network to obtain fusion feature information corresponding to a historical time period closest to the current time period includes:

determining whether each neuron of each network layer in the trained neural network transmits the first splicing characteristic information of each historical time period to other neurons of the next network layer or not;

and if so, performing feature fusion on the first splicing feature information of each historical time period by using the neuron in the trained neural network to obtain fusion feature information corresponding to one historical time period closest to the current time period.

In one embodiment, the second set of arithmetic operations comprises a multiplication operation and a summation operation; the processing the review feature information and the live broadcast feature information according to a second operation set to obtain second processing feature information includes:

multiplying the review feature information and the live broadcast feature information of each historical time period to obtain multiplied feature information corresponding to each historical time period;

summing the multiplied characteristic information of each historical time period to obtain summed characteristic information;

determining the second processing characteristic information based on the summed characteristic information.

In one embodiment, the second operation set further includes a third full join operation, and the summing the multiplied feature information of each historical time period to obtain summed feature information includes:

aiming at each historical time period, performing full connection operation on the multiplied feature information of the historical time period by using a trained third full connection layer to obtain the multiplied feature information with a third preset full connection dimension corresponding to the historical time period;

and summing the multiplied feature information with the third preset full-connection dimension corresponding to each historical time period to obtain summed feature information.

In one embodiment, the second set of arithmetic operations further includes a normalization operation, the determining the second processing characteristic information based on the summed characteristic information includes:

determining whether a feature value to which the summed feature information points is greater than 0;

if so, determining the feature value pointed by the summation feature information as the second processing feature information; and if not, determining the value 0 as the second processing characteristic information.

In one embodiment, after extracting review feature information from the historical review record corresponding to each historical time period, the method further includes:

for each historical time period, coding the review feature information of the historical time period by using a trained coding layer to obtain review feature information with a preset coding dimension;

processing the submission feature information and the live broadcast feature information according to a first operation set to obtain first processing feature information, and processing the submission feature information and the live broadcast feature information according to a second operation set to obtain second processing feature information, wherein the processing method comprises the following steps:

processing the submission characteristic information with the preset coding dimension and the live broadcast characteristic information according to a first operation set to obtain first processing characteristic information, and processing the submission characteristic information with the preset coding dimension and the live broadcast characteristic information according to a second operation set to obtain second processing characteristic information.

In one embodiment, the determining, based on the processed feature information after each processing, a result of the review of the live broadcast room in a future time period includes:

performing difference operation on the processed characteristic information after each time of processing to obtain a difference result;

normalizing the difference result by using a preset activation function to obtain a normalized difference result;

and determining the normalized difference result as a trial sending result of the live broadcast room in a future time period.

In one embodiment, the method further comprises:

judging whether the submission result meets a preset submission condition or not;

and if so, sending the live broadcast information corresponding to the live broadcast room to an auditing user side.

In a second aspect, an embodiment of the present disclosure further provides a live broadcast auditing apparatus, where the apparatus includes:

the acquisition module is used for acquiring live broadcast media content generated in each historical time period in a live broadcast room of a current time period in a plurality of historical time periods and historical review records corresponding to each historical time period in the plurality of historical time periods;

the extraction module is used for extracting live broadcast characteristic information from live broadcast media content generated in each historical time period and extracting review characteristic information from historical review records corresponding to each historical time period;

the processing module is used for processing the review feature information and the live broadcast feature information at least once to obtain processed feature information after each processing;

and the auditing module is used for determining the auditing result of the live broadcast room in the future time period based on the processed characteristic information after each time of processing.

In a third aspect, an embodiment of the present disclosure further provides an electronic device, including: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory communicating via the bus when the electronic device is running, the machine-readable instructions when executed by the processor performing the steps of the live auditing method according to the first aspect and any of its various embodiments.

In a fourth aspect, this disclosed embodiment also provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the live broadcast auditing method according to the first aspect and any of its various implementation manners are executed.

By adopting the live broadcast auditing scheme, firstly, live broadcast characteristic information can be extracted from the obtained live broadcast media content of the live broadcast room generated in each historical time period in a plurality of historical time periods, and audit-sending characteristic information can be extracted from the historical audit-sending records corresponding to each historical time period, and then the audit-sending characteristic information and the live broadcast characteristic information can be subjected to characteristic processing based on different operation operations, so that the audit-sending result of the live broadcast room in the future time period can be determined according to the processed characteristic information. That is, the live broadcast auditing scheme utilizes the historical submission characteristic information to process the live broadcast characteristic information in different modes, so as to excavate the relation between the historical submission characteristic and the live broadcast characteristic information as much as possible, so that the submission result of the determined future time period is more accurate, and meanwhile, the process is automatically processed without manual participation, so that the auditing efficiency and accuracy are higher.

In order to make the aforementioned objects, features and advantages of the present disclosure more comprehensible, preferred embodiments accompanied with figures are described in detail below.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly described below, and the drawings herein incorporated in and forming a part of the specification illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the technical solutions of the present disclosure. It is appreciated that the following drawings depict only certain embodiments of the disclosure and are therefore not to be considered limiting of its scope, for those skilled in the art will be able to derive additional related drawings therefrom without the benefit of the inventive faculty.

Fig. 1 shows a flowchart of a live review method provided in an embodiment of the present disclosure;

fig. 2 is a flowchart illustrating a specific method for determining first processing characteristic information in a live broadcast auditing method according to a first embodiment of the present disclosure;

fig. 3 is a flowchart illustrating a specific method for determining second processing characteristic information in a live broadcast auditing method according to a first embodiment of the present disclosure;

fig. 4 is a schematic application diagram of a live broadcast auditing method according to a first embodiment of the present disclosure;

fig. 5 is a schematic diagram illustrating a live broadcast auditing apparatus provided in a second embodiment of the present disclosure;

fig. 6 shows a schematic diagram of an electronic device provided in a third embodiment of the present disclosure.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be described clearly and completely with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are only a part of the embodiments of the present disclosure, not all of the embodiments. The components of the embodiments of the present disclosure, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present disclosure, presented in the figures, is not intended to limit the scope of the claimed disclosure, but is merely representative of selected embodiments of the disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the disclosure without making creative efforts, shall fall within the protection scope of the disclosure.

Research shows that the auditing of the live broadcast content in the related technology is mainly based on manual auditing, and the auditing efficiency of the mode is low, so that the requirement of a live broadcast platform on the auditing of the live broadcast content cannot be met along with the rapid growth of the live broadcast content.

Based on the research, the embodiment of the disclosure provides at least one live broadcast auditing scheme, and combines the historical auditing record and the live broadcast media content to automatically perform auditing prediction, so that manual participation is not required, and the auditing efficiency is high.

The above-mentioned drawbacks are the results of the inventor after practical and careful study, and therefore, the discovery process of the above-mentioned problems and the solutions proposed by the present disclosure to the above-mentioned problems should be the contribution of the inventor in the process of the present disclosure.

The technical solutions in the present disclosure will be described clearly and completely with reference to the accompanying drawings in the present disclosure, and it is to be understood that the described embodiments are only a part of the embodiments of the present disclosure, and not all of the embodiments. The components of the present disclosure, as generally described and illustrated in the figures herein, may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present disclosure, presented in the figures, is not intended to limit the scope of the claimed disclosure, but is merely representative of selected embodiments of the disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the disclosure without making creative efforts, shall fall within the protection scope of the disclosure.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.

In order to facilitate understanding of the present embodiment, a live broadcast auditing method disclosed in the embodiments of the present disclosure is first described in detail, where an execution subject of the live broadcast auditing method provided in the embodiments of the present disclosure is generally an electronic device with a certain computing capability, and the electronic device includes: a server or other processing device. In some possible implementations, the live audit method may be implemented by a processor invoking computer readable instructions stored in a memory.

The live broadcast auditing method provided by the embodiment of the disclosure is described below by taking an execution subject as a terminal device as an example.

Example one

Referring to fig. 1, which is a flowchart of a live broadcast auditing method provided in an embodiment of the present disclosure, the method includes steps S101 to S104, where:

s101, acquiring live broadcast media content generated in each historical time period in a live broadcast room of a current time period in a plurality of historical time periods, and obtaining historical review records corresponding to each historical time period in the plurality of historical time periods.

Here, in order to facilitate understanding of the live broadcast auditing method provided by the embodiment of the present disclosure, first, an application scenario of the live broadcast auditing method is briefly described. The live broadcast auditing method is mainly applied to the monitoring process of the current live broadcast platform, and mainly considers that some illegal operations can exist in the live broadcast platform along with the rapid development of the live broadcast platform, and the live broadcast behavior can be effectively supervised through the monitoring of the live broadcast platform, so that a better live broadcast environment is created. The monitoring result in the embodiment of the present disclosure may be an audit result determined in time-sharing manner along with the live broadcast, where the time period of division is not too long nor too short, the too long time period division may cause that an illegal action in the live broadcast cannot be found in time due to the sufficient division granularity, and the too short time period division may cause a large amount of computation, so that the embodiment of the present disclosure may determine the audit result every 5 minutes.

To determine the review results of the live broadcast room in the future time period, the relevant data required by the embodiments of the present disclosure may be the live broadcast media content and the corresponding historical review record generated by the live broadcast room in each of the plurality of historical time periods.

The live media content may be all media contents related to live broadcast collected in a live broadcast process of a live broadcast room, where the media content may be recorded anchor video (including audio), text chat content between an anchor and a user, a picture including an anchor captured from the live broadcast room, or other media contents related to live broadcast, and this is not limited in this disclosure.

In addition, the historical review record is used for indicating the historical review condition of the live broadcast room in the live broadcast process, that is, whether the live broadcast room is reviewed in a historical time period or not can be judged through the historical review record. In a specific application, 0 and 1 can be used to respectively indicate that the historical event end has not been checked and has been checked.

S102, extracting live broadcast characteristic information from live broadcast media content generated in each historical time period, and extracting review characteristic information from historical review records corresponding to each historical time period.

Here, after acquiring the live media content and the historical review record, the disclosed embodiment may extract live characteristic information and review characteristic information from the live media content and the historical review record, respectively.

The live broadcast characteristic information can be bullet screen risk characteristics, speech risk characteristics, vulgar risk characteristics, the number of fans in the live broadcast room, the number of times of being reported in the live broadcast room and the like. The barrage risk characteristics can represent the risk degree of the live barrage content, for example, the risk degree can be digitalized, and the higher the corresponding numerical value is, the higher the possibility of illegal operation in the live broadcast room is; the language risk characteristics can represent the possibility that language contents sent by a main broadcast in a live broadcast room are dangerous, and can also be digitalized, and similarly, the higher the corresponding numerical value is, the higher the possibility that illegal operation exists in the live broadcast room is; the vulgar risk characteristics can represent the possibility that the body action sent by the direct broadcasting room anchor has violation, and can also be digitalized, and are not described again; the reported times of the live broadcast room can be used for representing the related information of the live broadcast room, and the possibility of illegal operation in the future of the live broadcast room can be indicated to a certain extent. In addition, other information related to the live broadcast may also be used in the embodiments of the present disclosure, and is not limited specifically herein.

In a specific application, the live broadcast feature information of the live broadcast room in each historical time period may be determined, for example, when the granularity is divided by taking 5 minutes as time and 10 historical time periods are taken, if the current time period falls into (14:50, 14:55), the 1 st to 10 th historical time periods may be sequentially determined to be (14:00, 14:05), (14:05, 14:10), (14:10, 14:15) (14:15, 14:20), (14:20, 14:25), (14:25, 14:30), (14:30, 14:35), (14:35, 14:40), (14:40, 14:45), and (14:45, 14:50), so that there may be 132 feature dimensions of live broadcast feature information for each historical time period.

S103, processing the review feature information and the live broadcast feature information at least once to obtain processed feature information after each processing.

Here, in order to further dig out the association between the review feature information and the live broadcast feature information, the live broadcast review method provided by the embodiment of the disclosure may process the review feature information and the live broadcast feature information through two different operation operations to obtain the first processing feature information and the second processing feature information.

The first processing characteristic information may be obtained based on correlation processing of the first set of arithmetic operations, and the second processing characteristic information may be obtained based on correlation processing of the second set of arithmetic operations. The first operation set may include operations such as splicing, fusion, full connection, and the like, where the splicing operation may be a combination of related features on the number of channels, the fusion operation may be a process of fusing features of multiple historical time periods, and the full connection operation may be a process of performing dimension transformation according to a preset full connection dimension; the second operation set may include operations such as multiplication, summation, full concatenation, and normalization, and the multiplication operation may further extract features that have been further reviewed, and other related operations are not described herein again.

It should be noted that, regardless of whether the first processing is performed according to the first set of operation operations or the second processing is performed according to the second set of operation operations, the dimensions of the obtained first processing feature information and the second feature information may be kept consistent.

In addition to processing the review feature information and the live feature information through the two different operation operations, the embodiments of the present disclosure may also perform related processing through other operation operations, where the processing may be determined by combining actual application requirements, and is not limited herein.

And S104, determining a review result of the live broadcast room in a future time period based on the processed characteristic information after each processing.

Here, taking the example of processing the review feature information and the live broadcast feature information through the two different operation operations, the live broadcast review method provided by the embodiment of the disclosure can determine the review result of the live broadcast in the future time period based on the first processing feature information and the second processing feature information. In a particular application, the review result may be determined based on a difference between the first processing characteristic information and the second processing characteristic information. Therefore, the second processing characteristic information contains the relevant characteristics related to the review, so that the probability of the review in the future time period is reduced if the characteristics of the historical review are more obvious, the possibility of the review for multiple times in one live broadcast room is avoided, the misjudgment operation of the multiple review is avoided on the premise of ensuring the review efficiency, and the review cost is further reduced.

Considering the key role of the first processing characteristic information and the second processing characteristic information in determining the review result, the following two aspects may specifically describe the determination process of the first processing characteristic information and the second processing characteristic information respectively:

in a first aspect: as shown in fig. 2, the first processing characteristic information may be determined according to the following steps:

s201, performing first splicing operation on the submission characteristic information and the live broadcast characteristic information of each historical time period to obtain first splicing characteristic information corresponding to each historical time period;

s202, performing feature fusion on the first splicing feature information of each historical time period by using the trained neural network to obtain fusion feature information corresponding to a historical time period closest to the current time period;

s203, performing second splicing operation on the fusion characteristic information corresponding to one historical time period closest to the current time period and the review sending characteristic information of each historical time period to obtain second splicing characteristic information;

and S204, determining first processing characteristic information based on the second splicing characteristic information.

Here, the review feature information and the live broadcast feature information of each historical time period may be first spliced to obtain first splicing feature information corresponding to each historical time period, that is, each time period may correspond to one first splicing feature information. The splicing operation of the embodiment of the present disclosure may include merging of the number of channels, and in a specific application, the review feature information may be spliced after the live feature information, so that the description feature is increased. Here, taking live broadcast feature information of 132 feature dimensions and review feature information of 1 feature dimension as an example, the dimension of the first splicing feature information obtained by splicing may be 133 dimensions.

After the first splicing feature information of each historical time period is obtained, feature fusion can be performed on the basis of the trained neural network, so that fusion feature information corresponding to one historical time period closest to the current time period is obtained. The Neural Network used in the embodiment of the present disclosure may be a Long Short-Term Memory Network (LSTM), and the LSTM is a special Recurrent Neural Network (RNN), and can gate the state to control the transmission state, and remember that unimportant information is forgotten and needs to be memorized for a Long time. That is, after the LSTM passes, the corresponding output of the first splicing feature information corresponding to each historical time segment is not only related to the output of the current historical time segment, but also related to the outputs of other historical time segments before the current historical time segment (specifically, related to the historical time segment, the gating state depends on the gating state), and the fused feature information is obtained. In the embodiment of the present disclosure, the fusion feature information corresponding to a historical time period closest to the current time period may be selected, and the fusion feature information corresponding to each historical time period may also be selected, where the fusion feature information corresponding to a historical time period closest to the current time period may be directly selected in consideration of the influence of the features of the recent historical time period on the predicted review result of the future time period.

In this way, the second splicing operation is performed on the fused feature information and the review feature information of each historical time period, and the first processing feature information can be determined based on the second splicing feature information obtained by splicing.

In the live broadcast auditing method provided by the embodiment of the disclosure, when the first processing feature information is determined, feature processing can be performed by combining full connection operation. The fully connected operation is mainly to linearly transform one feature space to another, and any dimension of the target space can be influenced by each dimension of the source space, that is, deeper features can be mined.

Here, the full-join operation may be performed on the first concatenation feature information of the historical time periods before feature fusion is performed by using the trained neural network, and the first concatenation feature information having a first preset full-join dimension is obtained, for example, for each historical time period, the first concatenation feature information obtained by concatenation is 133 dimensions, and may be converted into a 266-dimensional feature space. Therefore, fusion operation can be carried out based on the first splicing characteristic information after characteristic transformation.

In addition, before the first processing characteristic information is determined based on the second splicing characteristic information, a second full-connection layer can be used for performing full-connection operation on the second splicing characteristic information, so that the first processing characteristic information is determined according to the second splicing characteristic information with a second preset full-connection dimension after the characteristic transformation.

It should be noted that, in the embodiment of the present disclosure, not only the full-connection operation may be adopted in the two processing links, but also the full-connection operation may be adopted in other processing links, and no specific limitation is made herein.

In order to avoid the problem of overfitting in the neural network training process, the live broadcast auditing method provided by the embodiment of the disclosure can improve the network generalization capability based on dropout (a random inactivation method). By using the dropout method, in each round of training of the neural network, part of neurons (which may be neurons in a hidden layer included in the neural network) can be discarded with a probability P, other neurons are retained with a probability q of 1-P, and the output of the discarded neurons is set to zero, that is, the first stitching feature information of each historical time period is not transmitted to other neurons of the next network layer, and the feature fusion effect is lost in this round. In each round of training process, the discarded neurons can be randomly selected, so that the neural network obtained by training is less dependent on some local features and has generalization capability, and the over-fitting problem is avoided.

In a second aspect: as shown in fig. 3, the second processing characteristic information may be determined according to the following steps:

s301, multiplying the review feature information and the live broadcast feature information of each historical time period to obtain multiplied feature information corresponding to each historical time period;

s302, summing the multiplied characteristic information of each historical time period to obtain summed characteristic information;

and S303, determining second processing characteristic information based on the summation characteristic information.

Here, the review feature information and the live broadcast feature information for each of the historical time periods may be first multiplied, so that, when it is determined that the review feature information that has been reviewed is 1 and the review feature information that has not been reviewed is 0 for each of the historical time periods, the multiplication feature information that has not been reviewed will become 0 based on the multiplication operation, and the reviewed multiplication feature information will not change. At this time, the multiplied feature information of the respective history periods may be merged together to determine the second processing feature information based on the resultant summed feature information.

Before determining the summation characteristic information, performing full-connection operation on the multiplied characteristic information of each historical time period according to third full-connection operation, so as to determine the summation characteristic information based on the summation operation of the multiplied characteristic information with third preset full-connection dimension obtained by the full-connection operation.

The second operation set in the embodiment of the present disclosure further includes a normalization operation, where it may be determined whether the feature value pointed by the summed feature information is greater than 0, and if so, the feature value pointed by the summed feature information may be determined as the second processing feature information; if not, a value of 0 may be determined as the second processing characteristic information.

In a specific application, the normalization process can be implemented by using Relu mapping.

In order to facilitate feature processing, the live broadcast auditing method provided by the embodiment of the disclosure may further encode review feature information of each historical time period by using a trained encoding layer, so as to obtain review feature information with a preset encoding dimension. For example, for review feature information whose value is 1 and points to the 1 st historical time period that has been reviewed historically, it may be expanded to 16-dimensional review feature information, and the feature value corresponding to the 1 st dimension is 1, and the feature values corresponding to the other dimensions are 0, and similarly, for review feature information whose value is 1 and points to the 2 nd historical time period that has been reviewed historically, it may also be expanded to 16-dimensional review feature information, but at this time, the feature value corresponding to the 2 nd dimension is 1, and the feature values corresponding to the other dimensions are 0, and so on, and no further description is given.

In this way, the censored feature information and the live broadcast feature information with the preset coding dimension obtained by the coding can be processed according to the first operation set and the second operation set respectively to obtain the processed first processing feature information and the processed second processing feature information.

In the embodiment of the present disclosure, the influence of the historical review record and the live media content on the future review result prediction is considered comprehensively, and the embodiment of the present disclosure may be a review result determined based on a difference result between the first processing characteristic information and the second processing characteristic information.

Here, the difference result may be normalized based on a preset activation function. And if the processing flow of the first operation set is taken as a main network, the correspondingly obtained first processing characteristic information is a, the processing flow of the second operation set is taken as a secondary network, and the correspondingly obtained second processing characteristic information is b, so that the obtained difference result is a-b. The difference result is less than a, and then the sigmoid is activated, and due to monotonicity of the sigmoid, sigmoid (c) < ═ sigmoid (a) can be determined, that is, compared with the case of only using the main network for auditing, the response value of the embodiment of the disclosure is reduced, so that the purpose of inhibiting the auditing is achieved.

That is, the more important the characteristic of the historical review is, the larger the value of the second processing characteristic information is, and at this time, the smaller the possibility of the determined future review is, so that for the current live broadcast room, the review can be determined under the condition that the live broadcast is abnormal, and simultaneously, the probability of the future review can be reduced after the live broadcast room is determined to be subjected to historical review, so that the possible misjudgment condition caused by similar characteristics of the live broadcast room can be avoided, and the accuracy is higher.

In a specific application, the live broadcast auditing method provided by the embodiment of the disclosure can send live broadcast information corresponding to the live broadcast room to the auditing user terminal when judging that the submission result does not meet the preset submission condition. For example, when it is determined that the audit-sending probability is greater than the set threshold (e.g., 0.75), the live broadcast number, the live broadcast time, the participating anchor information, and the like corresponding to the live broadcast room may be sent to the auditing user side, and a reminder about "there is an illegal operation in the live broadcast room" may also be sent, so that the auditor can follow up the live broadcast activity in time, and a good environment of the live broadcast platform is ensured.

In order to facilitate understanding of the live broadcast auditing method provided by the embodiments of the present disclosure, a detailed description may be given below with reference to a flowchart shown in fig. 4. The correlation flow shown here mainly combines feature dimensions and a specific feature processing procedure.

As shown in fig. 4, for live feature information of one live broadcast room in 10 historical time periods, when it is determined that the live feature information is 132 dimensions, the corresponding input data may be 10 × 132. After the full join operation is performed, processed feature information of 10 × 256 dimensions may be obtained, and based on the processed feature information and review feature information (corresponding to dimension 10 × 16) corresponding to the encoded 10 history time periods, first processing may be performed according to a first operation set, and second processing may be performed according to a second operation set. The following two aspects are explained in detail.

For the first processing, the processed features and the review feature information may be first subjected to a stitching operation, so as to obtain first stitching feature information corresponding to 10 historical time periods, where a dimension of the first stitching feature information may be a sum of 10 × 256 and 10 × 16, that is, 10 × 272. A full join operation may then be performed to convert the first stitching feature information of dimension 10 x 272 to dimension 10 x 256. Before feature fusion is performed based on the trained neural network, information splitting can be performed to obtain 10 pieces of split information with 1 × 256 dimensions, so that feature fusion can be performed on the first spliced feature information of 10 historical time periods by using the trained neural network, and here, fused feature information (the corresponding dimension is 1 × 256) corresponding to the 10 th historical time period can be obtained. In this way, the second splicing operation may be performed on the fusion feature information corresponding to the 10 th historical time period and the review feature information of each historical time period to obtain second splicing feature information (corresponding dimension is 416), and then the full connection operation may be performed to convert the 416-dimensional second splicing feature information into 1-dimensional information, that is, the first processing feature information is obtained.

For the second processing, firstly, the processed features and the review feature information may be multiplied to obtain multiplied feature information corresponding to 10 historical time periods, and the dimension of the multiplied feature information may be 10 × 256. Then, a full connection operation is performed to convert the multiplied feature information of 10 × 256 dimensions into 10 × 1 dimensions. By using the summation operation between 10 pieces of multiplied feature information, the summed feature information (corresponding to dimension 1) can be obtained, that is, the second processing feature information is obtained.

After the first processing and the second processing are executed, the review result of the future time period can be determined through the difference information between the first processing characteristic information and the second processing characteristic information, so that automatic review prediction is realized, manual participation is not needed, and the review effect is high.

It will be understood by those skilled in the art that in the method of the present invention, the order of writing the steps does not imply a strict order of execution and any limitations on the implementation, and the specific order of execution of the steps should be determined by their function and possible inherent logic.

Based on the same inventive concept, a live broadcast auditing device corresponding to the live broadcast auditing method is also provided in the embodiment of the present disclosure, and as the principle of problem solving of the device in the embodiment of the present disclosure is similar to that of the live broadcast auditing method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and repeated parts are not described again.

Example two

Referring to fig. 5, which is a schematic diagram of a live broadcast auditing apparatus provided in a second embodiment of the present disclosure, the apparatus includes: the system comprises an acquisition module 501, an extraction module 502, a processing module 503 and an auditing module 504; wherein the content of the first and second substances,

an obtaining module 501, configured to obtain live broadcast media content generated in each historical time period in a live broadcast room of a current time period in a plurality of historical time periods, and a historical review record corresponding to each historical time period in the plurality of historical time periods;

an extracting module 502, configured to extract live broadcast feature information from live broadcast media content generated in each historical time period, and extract review feature information from a historical review record corresponding to each historical time period;

the processing module 503 is configured to perform at least one processing on the review feature information and the live broadcast feature information to obtain processed feature information after each processing;

and the auditing module 504 is configured to determine an audit result of the live broadcast room in a future time period based on the processed characteristic information after each time of processing.

In an embodiment, the processing module 503 is configured to perform at least one processing on the review feature information and the live broadcast feature information according to the following steps to obtain processed feature information after each processing:

processing the submission feature information and the live broadcast feature information according to a first operation set to obtain first processing feature information, and,

processing the submission characteristic information and the live broadcast characteristic information according to a second operation set to obtain second processing characteristic information; the first set of arithmetic operations and the second set of arithmetic operations have different arithmetic operations.

In one embodiment, the first set of arithmetic operations includes a first stitching operation, a fusion operation, and a second stitching operation; the processing module 503 is configured to process the review feature information and the live broadcast feature information according to a first operation set to obtain first processing feature information according to the following steps:

performing first splicing operation on the submission characteristic information and the live broadcast characteristic information of each historical time period to obtain first splicing characteristic information corresponding to each historical time period;

performing second splicing operation on the fusion characteristic information corresponding to one historical time period closest to the current time period and the review characteristic information of each historical time period to obtain second splicing characteristic information;

and determining first processing characteristic information based on the second splicing characteristic information.

In one embodiment, the first set of arithmetic operations further comprises a first fully-join operation and a second fully-join operation; a processing module 503, configured to determine the first processing characteristic information according to the following steps:

fusing first splicing characteristic information with a first preset full-connection dimension corresponding to each historical time period by using the trained neural network, and determining the fusion characteristic information of one historical time period closest to the current time period;

performing full connection operation on the second splicing characteristic information by using the trained second full connection layer to obtain second splicing characteristic information with a second preset full connection dimension;

and determining second splicing characteristic information with a second preset full-connection dimension as first processing characteristic information.

In one embodiment, a neural network includes a plurality of network layers; each network layer includes a plurality of neurons; the processing module 503 is configured to perform feature fusion on the first splicing feature information of each historical time period by using the trained neural network according to the following steps, so as to obtain fusion feature information corresponding to a historical time period closest to the current time period:

determining whether each neuron of each network layer in the trained neural network transmits first splicing characteristic information of each historical time period to other neurons of the next network layer or not;

In one embodiment, the second set of arithmetic operations comprises a multiply operation and a sum operation; the processing module 503 is configured to process the review feature information and the live broadcast feature information according to a second operation set to obtain second processing feature information according to the following steps:

based on the summed feature information, second processing feature information is determined.

In an embodiment, the second operation set further includes a third full join operation, and the processing module 503 is configured to perform a summation operation on the multiplied feature information of each historical time period according to the following steps to obtain summed feature information:

In one embodiment, the second set of arithmetic operations further includes a normalization operation, and the processing module 503 is configured to determine the second processing characteristic information according to the following steps:

determining whether a feature value pointed to by the summed feature information is greater than 0;

if so, determining the characteristic value pointed by the summed characteristic information as second processing characteristic information; if not, determining the value 0 as the second processing characteristic information.

In one embodiment, the processing module 503 is configured to obtain the first processing characteristic information and the second processing characteristic information according to the following steps:

after extracting the review feature information from the historical review record corresponding to each historical time period, coding the review feature information of the historical time period by using a trained coding layer aiming at each historical time period to obtain the review feature information with a preset coding dimension;

processing the submission characteristic information and the live broadcast characteristic information with the preset coding dimension according to a first operation set to obtain first processing characteristic information, and processing the submission characteristic information and the live broadcast characteristic information with the preset coding dimension according to a second operation set to obtain second processing characteristic information.

In one embodiment, the auditing module 504 is configured to determine review results for the live broadcast room in a future time period according to the following steps:

and determining the normalized difference result as a review sending result of the live broadcast room in a future time period.

In one embodiment, the above apparatus further comprises:

a judging module 505, configured to judge whether the submission result meets a preset submission condition; and if so, sending the live broadcast information corresponding to the live broadcast room to the auditing user side.

EXAMPLE III

An embodiment of the present disclosure provides an electronic device, as shown in fig. 6, the electronic device includes: processor 601, memory 602, and bus 603, where the memory 602 stores machine-readable instructions executable by the processor 601 (such as execution instructions corresponding to the acquisition module 501, the extraction module 502, the processing module 503, and the auditing module 504 in the live auditing apparatus in fig. 5, and the like), and when the electronic device is operated, the processor 601 and the memory 602 communicate via the bus 603, and the machine-readable instructions, when executed by the processor 601, perform the following processes:

processing the submission characteristic information and the live broadcast characteristic information at least once to obtain processed characteristic information after each processing;

and determining the submission result of the live broadcast room in the future time period based on the processed characteristic information after each processing.

In one embodiment, the instructions executed by the processor 601 to perform at least one processing on the review feature information and the live broadcast feature information to obtain processed feature information after each processing includes:

In one embodiment, the first set of arithmetic operations includes a first stitching operation, a fusion operation, and a second stitching operation; in the instruction executed by the processor 601, processing the review feature information and the live broadcast feature information according to a first operation set to obtain first processing feature information, where the processing feature information includes:

In one embodiment, the first set of arithmetic operations further comprises a first fully-join operation and a second fully-join operation; in the instruction executed by the processor 601, the trained neural network is used to perform feature fusion on the first splicing feature information of each historical time period, so as to obtain fusion feature information corresponding to a historical time period closest to the current time period, where the fusion feature information includes:

in the instructions executed by the processor 601, determining the first processing characteristic information based on the second splicing characteristic information includes:

In one embodiment, a neural network includes a plurality of network layers; each network layer includes a plurality of neurons; in the instruction executed by the processor 601, the trained neural network is used to perform feature fusion on the first splicing feature information of each historical time period, so as to obtain fusion feature information corresponding to a historical time period closest to the current time period, where the fusion feature information includes:

In one embodiment, the second set of arithmetic operations comprises a multiply operation and a sum operation; in the instruction executed by the processor 601, processing the review feature information and the live broadcast feature information according to a second operation set to obtain second processing feature information, where the second processing feature information includes:

In an embodiment, the second operation set further includes a third full join operation, and the instructions executed by the processor 601 perform a summation operation on the multiplied feature information of each historical time period to obtain summed feature information, where the summation operation includes:

In one embodiment, in the instructions executed by the processor 601, the second operation set further includes a normalization operation, and determining the second processing characteristic information based on the summed characteristic information includes:

In one embodiment, after extracting review feature information from the historical review record corresponding to each historical time period, the instructions executed by the processor 601 further include:

in the instruction executed by the processor 601, processing the submission feature information and the live broadcast feature information according to a first operation set to obtain first processing feature information, and processing the submission feature information and the live broadcast feature information according to a second operation set to obtain second processing feature information, which includes:

In one embodiment, the instructions executed by the processor 601, which are used for determining the result of the review of the live broadcast room in the future time period based on the processed characteristic information after each processing, include:

In one embodiment, the instructions executed by the processor 601 further include:

and if so, sending the live broadcast information corresponding to the live broadcast room to the auditing user side.

The embodiment of the present disclosure further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by the processor 601, the steps of the live broadcast auditing method in the above method embodiments are executed. The storage medium may be a volatile or non-volatile computer-readable storage medium.

The computer program product of the live broadcast auditing method provided by the embodiments of the present disclosure includes a computer-readable storage medium storing program codes, where instructions included in the program codes may be used to execute steps of the live broadcast auditing method described in the above method embodiments, which may be referred to in the above method embodiments specifically, and are not described herein again.

The embodiments of the present disclosure also provide a computer program, which when executed by a processor implements any one of the methods of the foregoing embodiments. The computer program product may be embodied in hardware, software or a combination thereof. In an alternative embodiment, the computer program product is embodied in a computer storage medium, and in another alternative embodiment, the computer program product is embodied in a Software product, such as a Software Development Kit (SDK), or the like.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the system and the apparatus described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, apparatus, and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present disclosure may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solution of the present disclosure may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing an electronic device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present disclosure. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

Finally, it should be noted that: the above-mentioned embodiments are merely specific embodiments of the present disclosure, which are used for illustrating the technical solutions of the present disclosure and not for limiting the same, and the scope of the present disclosure is not limited thereto, and although the present disclosure is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive of the technical solutions described in the foregoing embodiments or equivalent technical features thereof within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present disclosure, and should be construed as being included therein. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.

Claims

1. A live broadcast auditing method is characterized by comprising the following steps:

2. The method according to claim 1, wherein the processing the review feature information and the live feature information at least once to obtain processed feature information after each processing comprises:

3. The method of claim 2, wherein the first set of operation operations comprises a first stitching operation, a fusion operation, and a second stitching operation; the processing the review feature information and the live broadcast feature information according to a first operation set to obtain first processing feature information includes:

4. The method of claim 3, wherein the first set of arithmetic operations further comprises a first fully-join operation and a second fully-join operation; the method for performing feature fusion on the first splicing feature information of each historical time period by using the trained neural network to obtain fusion feature information corresponding to a historical time period closest to the current time period includes:

5. The method of claim 3, wherein the neural network comprises a plurality of network layers; each network layer includes a plurality of neurons; the method for performing feature fusion on the first splicing feature information of each historical time period by using the trained neural network to obtain fusion feature information corresponding to a historical time period closest to the current time period includes:

6. The method of any of claims 2 to 5, wherein the second set of arithmetic operations comprises a multiplication operation and a summation operation; the processing the review feature information and the live broadcast feature information according to a second operation set to obtain second processing feature information includes:

7. The method of claim 6, wherein the second set of arithmetic operations further comprises a third fully-concatenated operation, and wherein summing the multiplied feature information for each historical time period to obtain summed feature information comprises:

8. The method of claim 6, wherein the second set of arithmetic operations further comprises a normalization operation, and wherein determining the second processing characteristic information based on the summed characteristic information comprises:

9. The method according to claim 1, wherein after extracting review feature information from the historical review record corresponding to each historical time period, the method further comprises:

10. The method of claim 1, wherein determining the review results of the live broadcast room in the future time period based on the processed feature information after each processing comprises:

11. The method of claim 1, further comprising:

12. A live broadcast auditing apparatus, the apparatus comprising:

13. An electronic device, comprising: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory communicating over the bus when the electronic device is operating, the machine-readable instructions when executed by the processor performing the steps of the live auditing method of any of claims 1-11.

14. A computer-readable storage medium, having stored thereon a computer program for performing, when executed by a processor, the steps of a live auditing method according to any one of claims 1-11.