CN113613035A - Sensitive information processing method and device, electronic equipment and storage medium - Google Patents
Sensitive information processing method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113613035A CN113613035A CN202110872578.6A CN202110872578A CN113613035A CN 113613035 A CN113613035 A CN 113613035A CN 202110872578 A CN202110872578 A CN 202110872578A CN 113613035 A CN113613035 A CN 113613035A
- Authority
- CN
- China
- Prior art keywords
- audio
- video data
- sensitive information
- processing
- sensitive
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
- H04N21/23418—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
- G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
- G10L25/57—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination for processing of video signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/233—Processing of audio elementary streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/439—Processing of audio elementary streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/439—Processing of audio elementary streams
- H04N21/4394—Processing of audio elementary streams involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
Abstract
The disclosure provides a method and a device for processing sensitive information, electronic equipment and a storage medium, and belongs to the technical field of internet. The method comprises the following steps: acquiring audio and video data of a main broadcast user in a live broadcast process; sensitive information detection is carried out on each frame of audio and video data; when any frame of audio/video data is detected to contain sensitive information, processing the sensitive information in the audio/video data to obtain processed audio/video data, wherein the processed audio/video data does not contain the sensitive information; and transmitting the processed audio and video data to audience users. The method and the device have the advantages that the audio and video data of the anchor user in the live broadcasting process are detected in real time, the sensitive information is contained in the detected audio and video data, and the sensitive information in the audio and video data is processed in time, so that adverse effects caused by the fact that the audio and video data containing the sensitive information are spread on the network are avoided, and the network environment is effectively maintained.
Description
Technical Field
The present disclosure relates to the field of internet technologies, and in particular, to a method and an apparatus for processing sensitive information, an electronic device, and a storage medium.
Background
With the development of internet technology and the popularization of network live broadcast, many lawbreakers use live broadcast platforms to transmit sensitive information related to yellow, storm, political affairs and the like. Since the propagation of the sensitive information damages the network environment and affects the stability of the society, the sensitive information in the live broadcast scene needs to be processed.
In the related technology, a server receives audio and video data sent by a main broadcasting user, detects whether the audio and video data contains sensitive information, if the audio and video data contains the sensitive information, reports the audio and video data and a detection result to a technical staff for rechecking, and simultaneously sends the audio and video data to audience users. And when the rechecking result shows that the audio and video data contains sensitive information, punishment is carried out on the anchor user.
Although the method can punish the anchor user who issues the sensitive information, the audio and video data containing the sensitive information is widely spread on the network and has adverse effects due to the fact that the sensitive information is not processed timely.
Disclosure of Invention
The embodiment of the disclosure provides a method and a device for processing sensitive information, an electronic device and a storage medium, which are used for timely processing the sensitive information in audio and video data, so that the audio and video data containing the sensitive information are prevented from being transmitted on a network, and the network environment is effectively maintained. The technical scheme is as follows:
in a first aspect, a method for processing sensitive information is provided, where the method includes:
acquiring audio and video data of a main broadcast user in a live broadcast process;
sensitive information detection is carried out on each frame of audio and video data;
when any frame of audio and video data is detected to contain sensitive information, processing the sensitive information in the audio and video data to obtain processed audio and video data, wherein the processed audio and video data does not contain sensitive information;
and sending the processed audio and video data to audience users.
In another embodiment of the present disclosure, the performing sensitive information detection on each frame of audio/video data includes:
inputting each frame of audio and video data into a sensitive information detection model, and outputting a sensitive information detection result, wherein the sensitive information detection model is used for detecting all types of sensitive information or specified types of sensitive information in the audio and video data.
In another embodiment of the present disclosure, the processing sensitive information in the audio/video data to obtain processed audio/video data includes:
acquiring the sensitivity level and the sensitivity score of the sensitive information;
determining the risk degree of the sensitive information according to the sensitivity grade and the sensitivity score;
and processing the sensitive information according to a processing mode corresponding to the risk degree to obtain the processed audio and video data.
In another embodiment of the present disclosure, the processing the sensitive information according to a processing manner corresponding to the risk degree to obtain the processed audio/video data includes:
when the sensitive information is a sensitive image, performing fuzzification or noise addition processing on the sensitive image to different degrees to obtain a processed image; and/or the presence of a gas in the gas,
and when the sensitive information is sensitive audio, performing noise addition or shearing processing of different degrees on the sensitive audio to obtain processed audio.
In another embodiment of the present disclosure, the processing sensitive information in the audio/video data to obtain processed audio/video data includes:
acquiring target audio and video data with the same time length as the audio and video data from historical audio and video data of the anchor user;
and replacing the audio and video data with the target audio and video data to obtain processed audio and video data.
In another embodiment of the present disclosure, the method further comprises:
and reporting a sensitive information detection result aiming at the audio and video data, wherein the sensitive information detection result is used for manual rechecking.
In another embodiment of the present disclosure, after reporting the sensitive information detection result for the audio/video data, the method further includes:
and when the rechecking result shows that the audio and video data does not contain sensitive information, sending a prompt message to the audience user, wherein the prompt message is used for prompting the audience user to play the restored data of the processed audio and video data.
In another embodiment of the present disclosure, after reporting the sensitive information detection result for the audio/video data, the method further includes:
and when the rechecking result indicates that the audio and video data does not contain sensitive information, optimizing the sensitive information detection model according to the audio and video data and the rechecking result to obtain the optimized sensitive information detection model.
In a second aspect, an apparatus for processing sensitive information is provided, the apparatus comprising:
the acquisition module is used for acquiring audio and video data of a main broadcast user in a live broadcast process;
the detection module is used for detecting sensitive information aiming at each frame of audio and video data;
the processing module is used for processing the sensitive information in the audio and video data to obtain processed audio and video data when detecting that any frame of audio and video data contains sensitive information, wherein the processed audio and video data does not contain sensitive information;
and the sending module is used for sending the processed audio and video data to audience users.
In another embodiment of the disclosure, the detection module is configured to input each frame of audio/video data into a sensitive information detection model, and output a sensitive information detection result, where the sensitive information detection model is configured to detect all types of sensitive information or specified types of sensitive information in the audio/video data.
In another embodiment of the present disclosure, the processing module is configured to obtain a sensitivity level and a sensitivity score of the sensitive information; determining the risk degree of the sensitive information according to the sensitivity grade and the sensitivity score; and processing the sensitive information according to a processing mode corresponding to the risk degree to obtain the processed audio and video data.
In another embodiment of the present disclosure, the processing module is configured to, when the sensitive information is a sensitive image, perform blurring or denoising processing on the sensitive image to different degrees to obtain a processed image; and/or the presence of a gas in the gas,
and the processing module is used for carrying out noise adding or shearing processing of different degrees on the sensitive audio to obtain the processed audio when the sensitive information is the sensitive audio.
In another embodiment of the present disclosure, the processing module is configured to obtain target audio/video data with a duration the same as that of the audio/video data from historical audio/video data of the anchor user; and replacing the audio and video data with the target audio and video data to obtain processed audio and video data.
In another embodiment of the present disclosure, the apparatus further comprises:
and the reporting module is used for reporting the sensitive information detection result aiming at the audio and video data, and the sensitive information detection result is used for manually rechecking.
In another embodiment of the disclosure, the sending module is further configured to send a prompt message to the audience user when the review result indicates that the audio/video data does not contain sensitive information, where the prompt message is used to prompt the audience user to play the restored data of the processed audio/video data.
In another embodiment of the present disclosure, the apparatus further comprises:
and the optimization module is used for optimizing the sensitive information detection model according to the audio and video data and the rechecking result to obtain the optimized sensitive information detection model when the rechecking result indicates that the audio and video data does not contain sensitive information.
In a third aspect, an electronic device is provided, which includes a processor and a memory, where at least one program code is stored in the memory, and the at least one program code is loaded and executed by the processor to implement the method for processing sensitive information according to the first aspect.
In a fourth aspect, a computer-readable storage medium is provided, in which at least one program code is stored, and the at least one program code is loaded and executed by a processor to implement the method for processing sensitive information according to the first aspect.
The technical scheme provided by the embodiment of the disclosure has the following beneficial effects:
the method and the device have the advantages that the audio and video data of the anchor user in the live broadcast process are detected in real time, the sensitive information is contained in the detected audio and video data, and the sensitive information in the audio and video data is processed in time, so that the adverse effect caused by the propagation of the audio and video data containing the sensitive information on the network is avoided, and the network environment is effectively maintained.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present disclosure, and it is obvious for those skilled in the art to obtain other drawings based on the drawings without creative efforts.
Fig. 1 is a schematic diagram of an implementation environment related to a method for processing sensitive information according to an embodiment of the present disclosure;
FIG. 2 is a flowchart of a method for processing sensitive information according to an embodiment of the present disclosure;
FIG. 3 is a flow chart of another method for processing sensitive information provided by the embodiments of the present disclosure;
FIG. 4 is a schematic structural diagram of a device for processing sensitive information according to an embodiment of the present disclosure;
fig. 5 shows a block diagram of an electronic device according to an exemplary embodiment of the present disclosure.
Detailed Description
To make the objects, technical solutions and advantages of the present disclosure more apparent, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.
It is to be understood that the terms "each," "a plurality," and "any" and the like, as used in the embodiments of the present disclosure, are intended to encompass two or more, each referring to each of the corresponding plurality, and any referring to any one of the corresponding plurality. For example, the plurality of words includes 10 words, and each word refers to each of the 10 words, and any word refers to any one of the 10 words.
Fig. 1 shows an implementation environment related to a processing method of sensitive information provided by an embodiment of the present disclosure, and referring to fig. 1, the implementation environment includes: a first terminal 101, a server 102, a second terminal 103, and the like.
The first terminal 101 is a terminal used by the anchor user. The first terminal is provided with a live broadcast application, and based on the installed live broadcast application, the first terminal 101 collects audio and video data of a main broadcast user in a live broadcast process, sends the collected audio and video data to the server 102, and sends the collected audio and video data to the second terminal 103 through the server 102. The first terminal 101 may be installed with a sensitive information detection model, and based on the sensitive information detection model, the first terminal 101 may perform sensitive information detection on the acquired audio/video data of the anchor user in the live broadcast process of the anchor user, and timely process the sensitive information when detecting that any frame of audio/video data contains sensitive information, so as to prevent the audio/video data containing the sensitive information from being transmitted on the network. By detecting sensitive information of audio and video data at the anchor user end, the processing pressure of the server 102 is greatly reduced, the performance of the server is improved, and especially in a live broadcast scene, the live broadcast experience of audience users can be greatly improved by improving the performance of the server. Of course, considering that the computing power of the first terminal 101 is limited, the sensitive information may be detected not on the first terminal side but on the server side.
The second terminal 103 is a terminal used by audience users in the live room. The second terminal 103 has a live application installed therein, and based on the installed live application, the second terminal 103 can receive audio and video data of the anchor user sent by the server 102.
The first terminal 101 and the second terminal 103 may be a smart phone, a tablet computer, a notebook computer, a smart speaker, a smart watch, and the like, and the embodiment of the disclosure does not specifically limit the product types of the first terminal 101 and the second terminal 103.
The server 102 is a background server of the live broadcast application, and the server 102 may be an independent physical server, or a server cluster or a distributed system formed by a plurality of physical servers. The server 102 may be installed with a sensitive information detection model, and based on the installed sensitive information detection model, upon receiving the audio/video data of the anchor user sent by the first terminal 101, the server 102 can perform sensitive information detection on the audio/video data of the anchor user, and process the sensitive information when detecting that any frame of audio/video data contains sensitive information, thereby preventing the audio/video data containing sensitive information from being transmitted on the network, and effectively purifying the network environment.
The first terminal 101, the second terminal 103 and the server 102 may be directly or indirectly connected through wired or wireless communication, and the embodiment of the present disclosure is not particularly limited.
The embodiment of the present disclosure provides a method for processing sensitive information, which takes an electronic device as an example, where the electronic device is the first terminal 101 or the server 102 in fig. 1. Referring to fig. 2, a method flow provided by the embodiment of the present disclosure includes:
201. and audio and video data of the anchor user in the live broadcast process are obtained.
The anchor user can carry out voice live broadcast and video live broadcast in the live broadcast room, and when the anchor user carries out voice live broadcast, the audio data can be obtained in the live broadcast process of the anchor user; when the anchor user performs video live broadcasting, video data including audio data and image data can be acquired in the live broadcasting process of the anchor user.
202. And sensitive information detection is carried out aiming at each frame of audio and video data.
The sensitive information comprises information of contents related to yellow, political and storm. The sensitive information is in the form of at least one of a sensitive image and sensitive audio. For any frame of audio/video data, if at least one of the image data and the audio data in the audio/video data contains sensitive information, the frame of audio/video data can be determined to contain the sensitive information. For example, when it is detected that image data of the audio/video data contains a sensitive image and audio data does not contain a sensitive audio, it is determined that the audio/video data contains sensitive information; for another example, when it is detected that the image data of the audio/video data does not contain a sensitive image and the audio data contains a sensitive audio, it may be determined that the audio/video data contains sensitive information; for another example, when it is detected that the image data of the audio/video data includes a sensitive image and the audio data includes a sensitive audio, it may be determined that the audio/video data includes sensitive information.
203. And when any frame of audio and video data is detected to contain sensitive information, processing the sensitive information in the audio and video data to obtain processed audio and video data.
In order to prevent audio and video data containing sensitive information from being transmitted on a network, when any frame of audio and video data containing sensitive information is detected, the electronic equipment can perform processing such as fuzzification, noise addition and the like on the sensitive information in the audio and video data, so that the processed audio and video data does not contain the sensitive information. Even if the processed audio and video data are transmitted on the network, no adverse effect is generated.
204. And transmitting the processed audio and video data to audience users.
The method provided by the embodiment of the disclosure detects the audio and video data of the anchor user in real time in the live broadcasting process, and timely processes the sensitive information in the audio and video data when the audio and video data is detected to contain the sensitive information, thereby avoiding adverse effects caused by the transmission of the audio and video data containing the sensitive information on the network and effectively maintaining the network environment.
The embodiment of the present disclosure provides a method for processing sensitive information, which takes an electronic device as an example, where the electronic device is the first terminal 101 or the server 102 in fig. 1. Referring to fig. 3, an embodiment of the present disclosure is an example, and a method flow provided by the embodiment of the present disclosure includes:
301. the electronic equipment acquires audio and video data of a anchor user in a live broadcast process.
When the electronic equipment is a first terminal, namely the first terminal executes sensitive information detection operation, the first terminal collects audio and video data of a main broadcast user in real time in the live broadcast process of the main broadcast user and carries out sensitive information detection on the collected audio and video data; when the electronic equipment is a server, namely the server executes the detection operation of the sensitive information, the first terminal collects the audio and video data of the anchor user in real time and sends the collected audio and video data to the server in the live broadcast process of the anchor user.
The method comprises the steps that audio and video data obtained by electronic equipment are different according to different live broadcast types of anchor users, and if the anchor users carry out audio live broadcast, the electronic equipment obtains the audio data; if the anchor user carries out video live broadcast, the electronic equipment acquires video data.
302. Electronic equipment carries out sensitive information detection aiming at each frame of audio and video data
In order to better detect the audio and video data of the anchor user and avoid adverse effects caused by the transmission of any frame of audio and video data containing sensitive information on the network, the electronic equipment detects the sensitive information of each frame of audio and video data of the anchor user by taking a frame as a unit.
In one possible implementation manner, the electronic device may store a sensitive information database in advance, where the sensitive information database includes a sensitive image library and a sensitive audio library, a sensitive image related to the sensitive information is stored in the sensitive image library, and a sensitive audio related to the sensitive information is stored in the sensitive audio library. When the sensitive information of the audio and video data is detected, aiming at the image data in the audio and video data, the electronic equipment can match the image data with the sensitive image in the sensitive image library, and when the image data is matched with any sensitive image in the sensitive image library, the image data is determined to contain the sensitive information; the electronic equipment can match the audio data with the sensitive audio in the sensitive audio library aiming at the audio data in the audio and video data, and when the audio data is matched with the sensitive audio in the sensitive audio library, the audio data is determined to contain the sensitive information.
In another possible implementation manner, a sensitive information detection model is installed in the electronic device, the sensitive information detection model can detect all types of sensitive information or specified types of sensitive information in the audio and video data, and the specified types are determined according to actual application scenarios and are the types of information focused in the actual application scenarios. If the sensitive information detection model can detect all types of sensitive information in the audio and video data, training the initial sensitive information detection model based on the audio and video data containing all types of sensitive information is needed, so that the sensitive information detection model capable of detecting all types of sensitive information is obtained; if the sensitive information detection model can detect the sensitive information of the specified type in the audio and video data, the initial sensitive information detection model needs to be trained based on the audio and video data containing the sensitive information of the specified type, so that the sensitive information detection model capable of detecting the sensitive information of the specified type is obtained. Based on the installed sensitive information detection model, the electronic equipment inputs each frame of audio and video data into the sensitive information detection model and outputs a sensitive information detection result. The sensitive information detection result comprises whether sensitive information is contained or not, and the sensitivity level, the sensitivity score and the like of the sensitive information, wherein the sensitivity level and the sensitivity score can be determined according to the content of the sensitive information. If the sensitive information is a sensitive image, the sensitive information detection result may further include a sensitive image area, and if the sensitive information is a sensitive audio, the sensitive information detection result may further include a start time, an end time, and the like of the sensitive audio.
According to the method provided by the embodiment of the disclosure, when the electronic device is the first terminal, the processing pressure of the server is greatly reduced and the performance of the server is improved by detecting the sensitive information at the anchor user side. In addition, no matter the electronic device is the first terminal or the server, when the sensitive information is detected, the embodiment of the disclosure can construct a sensitive information detection model with different detection functions according to detection requirements, so as to meet the requirements of different detection scenes. Particularly, in a live broadcast scene, a sensitive information detection model for detecting the sensitive information of the specified type is constructed, so that the sensitive information does not need to be detected comprehensively, the detection speed of the sensitive information is increased, and the detection time is shortened.
303. When any frame of audio and video data is detected to contain sensitive information, the electronic equipment processes the sensitive information in the audio and video data to obtain processed audio and video data.
In an embodiment of the disclosure, when it is detected that any frame of audio/video data contains sensitive information, the electronic device processes the sensitive information in the audio/video data to obtain processed audio/video data. The specific treatment process comprises the following steps:
3031. the electronic equipment acquires the sensitivity level and the sensitivity score of the sensitive information.
Based on the sensitive information detection result obtained by the electronic device detecting the sensitive information in step 302, the electronic device may obtain the sensitivity level and the sensitivity score of the sensitive information from the sensitive information detection result.
3032. And the electronic equipment determines the risk degree of the sensitive information according to the sensitivity grade and the sensitivity score.
The electronic equipment carries out comprehensive judgment based on the sensitivity level and the sensitivity score of the sensitive information, and if the sensitivity level and the sensitivity score of the sensitive information are both higher, the risk degree of the sensitive information can be determined to be higher; if the sensitivity level and the sensitivity score of the sensitive information are both low, it may be determined that the sensitive information is at a lower level of risk.
3033. And the electronic equipment processes the sensitive information according to a processing mode corresponding to the risk degree to obtain processed audio and video data.
Considering that the sensitive information includes at least one of a sensitive image and a sensitive audio, the form of the sensitive information is different, and the processing mode of the electronic device for the sensitive information is also different. For different forms of sensitive information, when the electronic device processes the sensitive information according to a processing mode corresponding to the risk degree, the processing method includes, but is not limited to, the following cases:
the first case, sensitive information, is a sensitive image
When the sensitive information is a sensitive image, the electronic equipment can perform fuzzification or noise adding processing on the sensitive image, so that the sensitive image cannot be normally displayed on the terminal of each audience user, and the sensitive image is prevented from being spread on the network. Specifically, the electronic device may determine the blurring or noise adding degree corresponding to the risk degree according to the risk degree corresponding to the sensitive information, and further perform blurring or noise adding processing on the sensitive image according to the blurring or noise adding degree corresponding to the risk degree to obtain a processed image. For example, if the risk degree corresponding to the sensitive image is higher, determining that the fuzzification or noise adding degree corresponding to the risk degree is higher, and when the fuzzification or noise adding degree corresponding to the risk degree is adopted, fuzzification or noise adding processing is carried out on the sensitive image, the sensitive area in the processed image is completely shielded, and the content of the original image cannot be seen by the audience user; and when the fuzzification or noise addition degree corresponding to the risk degree is lower, the fuzzification or noise addition processing is carried out on the sensitive image, the sensitive image area in the processed image is only blurred, and the content of the original image can be seen by the hidden of the audience user.
In the second case, the sensitive information is sensitive audio
When the sensitive information is the sensitive audio, the electronic device can add noise or cut the sensitive audio, so that the sensitive audio cannot be normally played at the terminals of all audience users, and the sensitive audio is prevented from being transmitted on the network. Specifically, the electronic device may determine a noise adding or clipping degree corresponding to the risk degree according to the risk degree corresponding to the sensitive information, and then perform noise adding or clipping on the sensitive audio according to the noise adding or clipping degree corresponding to the risk degree to obtain a processed audio. For example, if the risk degree corresponding to the sensitive audio is high, determining that the noise adding or cutting degree corresponding to the risk degree is high, and when the noise adding or cutting degree corresponding to the risk degree is adopted, performing noise adding or cutting processing on the sensitive audio, wherein all sensitive segments in the processed audio are interfered or cut by noise, and audience users cannot hear the content of the original audio completely; and if the risk degree corresponding to the sensitive audio is low, determining that the noise adding or cutting degree corresponding to the risk degree is low, and if the noise adding or cutting degree corresponding to the risk degree is adopted, performing noise adding or cutting processing on the sensitive audio, and interfering or cutting the sensitive fragment part in the processed audio by noise, so that the audience user can listen to the content of the original audio intermittently.
In another embodiment of the present disclosure, when it is detected that the audio data contains the sensitive information, the electronic device may replace the audio data related to the sensitive information, so as to prevent the audio-video data containing the sensitive information from being transmitted on the network. Specifically, the electronic device can acquire target audio and video data with the same duration as the audio and video data from historical audio and video data of the anchor user, and further replace the audio and video data with the target audio and video data to obtain processed audio and video data. The historical audio and video data of the anchor user can be historical audio and video data of the anchor user in the current live broadcast process, and can also be historical audio data of the anchor user in the historical live broadcast process. In order to attract audience users to continuously watch the live broadcast of the anchor user, the electronic equipment can acquire a wonderful audio and video clip from historical audio and video data according to the number of the audience users, the presentation number of virtual articles and other indexes, and further acquire audio and video data with the same duration as the audio and video data from the wonderful audio and video clip as target audio and video data.
304. And the electronic equipment sends the processed audio and video data to audience users.
When the method provided by the embodiment of the disclosure is adopted to process the audio and video data containing the sensitive information, the obtained processed audio and video data does not contain the sensitive information any more, the electronic device sends the processed audio and video data to the audience users, and on the premise that the audience users watch the live broadcast of the anchor users, the transmission of the sensitive information is avoided, and the network environment is effectively maintained.
Because the electronic device can be the first terminal and also can be the server, the paths for transmitting the processed audio and video data to the audience users are different for different types of electronic devices. When the electronic equipment is a first terminal, the first terminal firstly sends the processed audio and video data to a server, and then the server sends the processed audio and video data to audience users; when the electronic equipment is a server, the server can directly send the processed audio and video data to audience users.
Further, in order to facilitate follow-up responsibility for the anchor user who issues sensitive information, the electronic device also stores unprocessed original audio and video data.
305. And the electronic equipment reports the detection result of the sensitive information aiming at the audio and video data.
Considering that the detection of the electronic device has certain limitation, the content and the form of the sensitive information are various, and in order to avoid the misjudgment of the electronic device and improve the accuracy of the detection result of the sensitive information, the electronic device also reports the detection result of the sensitive information aiming at the audio and video data, and a backstage technician manually rechecks the detection result of the sensitive information.
In another embodiment of the disclosure, when the rechecking result of the technician indicates that the audio/video data does not contain sensitive information, the technician can perform recovery processing on the processed audio/video data to obtain the restored data of the audio/video data. In order to enable the audience to be able to view the real live content of the anchor user, the electronic device may send a prompt message to the audience user to prompt the audience user to play the restored data of the processed audio and video data. Furthermore, in order to facilitate the audience user to play the audio and video data, the electronic device may carry a viewing link of the processed audio and video data restoration data in the prompt message, and the audience user may play the processed audio and video data restoration data by clicking the viewing link.
In another embodiment of the disclosure, when the rechecking result of the technician indicates that the audio/video data does not contain sensitive information, in order to improve the detection accuracy of the sensitive information detection model, the electronic device further optimizes the sensitive information detection model according to the audio/video data and the rechecking result to obtain an optimized sensitive information detection model, so that sensitive information detection can be performed on the audio/video data based on the optimized sensitive information detection model in a subsequent live broadcast scene.
The method provided by the embodiment of the disclosure detects the audio and video data of the anchor user in real time in the live broadcasting process, and timely processes the sensitive information in the audio and video data when the audio and video data is detected to contain the sensitive information, thereby avoiding adverse effects caused by the transmission of the audio and video data containing the sensitive information on the network and effectively maintaining the network environment.
Referring to fig. 4, an embodiment of the present disclosure provides an apparatus for processing sensitive information, where the apparatus includes:
the acquisition module 401 is configured to acquire audio and video data of a anchor user in a live broadcast process;
the detection module 402 is used for detecting sensitive information aiming at each frame of audio/video data;
the processing module 403 is configured to, when it is detected that any frame of audio/video data includes sensitive information, process the sensitive information in the audio/video data to obtain processed audio/video data, where the processed audio/video data does not include the sensitive information;
and a sending module 404, configured to send the processed audio/video data to the viewer user.
In another embodiment of the present disclosure, the detection module 402 is configured to input each frame of audio/video data into a sensitive information detection model, and output a sensitive information detection result, where the sensitive information detection model is configured to detect all types of sensitive information or specific types of sensitive information in the audio/video data.
In another embodiment of the present disclosure, the processing module 403 is configured to obtain a sensitivity level and a sensitivity score of the sensitive information; determining the risk degree of the sensitive information according to the sensitivity grade and the sensitivity score; and processing the sensitive information according to a processing mode corresponding to the risk degree to obtain processed audio and video data.
In another embodiment of the present disclosure, the processing module 403 is configured to, when the sensitive information is a sensitive image, perform blurring or denoising processing on the sensitive image to different degrees to obtain a processed image; and/or the presence of a gas in the gas,
the processing module 403 is configured to, when the sensitive information is a sensitive audio, perform noise addition or clipping processing on the sensitive audio to different degrees to obtain a processed audio.
In another embodiment of the present disclosure, the processing module 403 is configured to obtain target audio/video data with the same duration as the audio/video data from the historical audio/video data of the anchor user; and replacing the audio and video data with target audio and video data to obtain processed audio and video data.
In another embodiment of the present disclosure, the apparatus further comprises:
and the reporting module is used for reporting the sensitive information detection result aiming at the audio and video data, and the sensitive information detection result is used for carrying out manual recheck.
In another embodiment of the present disclosure, the sending module 404 is further configured to send a prompt message to the audience user when the review result indicates that the audio/video data does not contain the sensitive information, where the prompt message is used to prompt the audience user to play the restored data of the processed audio/video data.
In another embodiment of the present disclosure, the apparatus further comprises:
and the optimization module is used for optimizing the sensitive information detection model according to the audio and video data and the rechecking result to obtain the optimized sensitive information detection model when the rechecking result indicates that the audio and video data does not contain sensitive information.
In summary, the apparatus provided in the embodiment of the present disclosure detects audio and video data of a anchor user in real time during a live broadcast process, and timely processes the sensitive information in the audio and video data when detecting that the audio and video data includes the sensitive information, thereby avoiding adverse effects caused by propagation of the audio and video data including the sensitive information on a network, and effectively maintaining a network environment.
Fig. 5 shows a block diagram of an electronic device 500 according to an exemplary embodiment of the present disclosure. In general, the apparatus 500 includes: a processor 501 and a memory 502.
The processor 501 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and so on. The processor 501 may be implemented in at least one hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array). The processor 501 may also include a main processor and a coprocessor, where the main processor is a processor for Processing data in an awake state, and is also called a Central Processing Unit (CPU); a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor 501 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the display screen. In some embodiments, processor 501 may also include an AI (Artificial Intelligence) processor for processing computational operations related to machine learning.
In some embodiments, the electronic device 500 may further optionally include: a peripheral interface 503 and at least one peripheral. The processor 501, memory 502 and peripheral interface 503 may be connected by a bus or signal lines. Each peripheral may be connected to the peripheral interface 503 by a bus, signal line, or circuit board. Specifically, the peripheral device includes: a power supply 504.
The peripheral interface 503 may be used to connect at least one peripheral related to I/O (Input/Output) to the processor 501 and the memory 502. In some embodiments, the processor 501, memory 502, and peripheral interface 503 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 501, the memory 502, and the peripheral interface 503 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.
The power supply 504 is used to power various components in the electronic device 50. The power source 504 may be alternating current, direct current, disposable batteries, or rechargeable batteries. When the power supply 504 includes a rechargeable battery, the rechargeable battery may be a wired rechargeable battery or a wireless rechargeable battery. The wired rechargeable battery is a battery charged through a wired line, and the wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery may also be used to support fast charge technology.
Those skilled in the art will appreciate that the configuration shown in fig. 5 is not intended to be limiting of the electronic device 500 and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components may be used.
In an exemplary embodiment, a computer-readable storage medium comprising instructions, such as a memory comprising instructions, executable by a processor of the electronic device 500 to perform the video processing method described above is also provided. Alternatively, the storage medium may be a non-transitory computer readable storage medium, which may be, for example, a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
The electronic equipment provided by the embodiment of the disclosure detects the audio and video data of the anchor user in real time in the live broadcasting process, and timely processes the sensitive information in the audio and video data when the audio and video data is detected to contain the sensitive information, so that adverse effects caused by the propagation of the audio and video data containing the sensitive information on the network are avoided, and the network environment is effectively maintained.
The disclosed embodiments provide a computer readable storage medium having at least one program code stored therein, the at least one program code being loaded and executed by a processor to implement a method for processing sensitive information. The computer readable storage medium may be non-transitory. For example, the computer-readable storage medium may be a Read-Only Memory (ROM), a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
The computer-readable storage medium provided by the embodiment of the disclosure detects audio and video data of a anchor user in a live broadcasting process in real time, and timely processes the sensitive information in the audio and video data when the audio and video data is detected to contain the sensitive information, so that adverse effects caused by the propagation of the audio and video data containing the sensitive information on a network are avoided, and the network environment is effectively maintained.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
The above description is intended to be exemplary only and not to limit the present disclosure, and any modification, equivalent replacement, or improvement made without departing from the spirit and scope of the present disclosure is to be considered as the same as the present disclosure.
Claims (11)
1. A method for processing sensitive information, the method comprising:
acquiring audio and video data of a main broadcast user in a live broadcast process;
sensitive information detection is carried out on each frame of audio and video data;
when any frame of audio and video data is detected to contain sensitive information, processing the sensitive information in the audio and video data to obtain processed audio and video data, wherein the processed audio and video data does not contain sensitive information;
and sending the processed audio and video data to audience users.
2. The method of claim 1, wherein the performing sensitive information detection for each frame of audio-video data comprises:
inputting each frame of audio and video data into a sensitive information detection model, and outputting a sensitive information detection result, wherein the sensitive information detection model is used for detecting all types of sensitive information or specified types of sensitive information in the audio and video data.
3. The method according to claim 1, wherein the processing sensitive information in the audio/video data to obtain processed audio/video data comprises:
acquiring the sensitivity level and the sensitivity score of the sensitive information;
determining the risk degree of the sensitive information according to the sensitivity grade and the sensitivity score;
and processing the sensitive information according to a processing mode corresponding to the risk degree to obtain the processed audio and video data.
4. The method according to claim 3, wherein the processing the sensitive information according to the processing mode corresponding to the risk degree to obtain the processed audio/video data comprises:
when the sensitive information is a sensitive image, performing fuzzification or noise addition processing on the sensitive image to different degrees to obtain a processed image; and/or the presence of a gas in the gas,
and when the sensitive information is sensitive audio, performing noise addition or shearing processing of different degrees on the sensitive audio to obtain processed audio.
5. The method according to claim 1, wherein the processing sensitive information in the audio/video data to obtain processed audio/video data comprises:
acquiring target audio and video data with the same time length as the audio and video data from historical audio and video data of the anchor user;
and replacing the audio and video data with the target audio and video data to obtain processed audio and video data.
6. The method according to any one of claims 1 to 5, further comprising:
and reporting a sensitive information detection result aiming at the audio and video data, wherein the sensitive information detection result is used for manual rechecking.
7. The method according to claim 6, wherein after reporting the sensitive information detection result for the audio/video data, the method further comprises:
and when the rechecking result shows that the audio and video data does not contain sensitive information, sending a prompt message to the audience user, wherein the prompt message is used for prompting the audience user to play the restored data of the processed audio and video data.
8. The method according to claim 6, wherein after reporting the sensitive information detection result for the audio/video data, the method further comprises:
and when the rechecking result indicates that the audio and video data does not contain sensitive information, optimizing the sensitive information detection model according to the audio and video data and the rechecking result to obtain the optimized sensitive information detection model.
9. An apparatus for processing sensitive information, the apparatus comprising:
the acquisition module is used for acquiring audio and video data of a main broadcast user in a live broadcast process;
the detection module is used for detecting sensitive information aiming at each frame of audio and video data;
the processing module is used for processing the sensitive information in the audio and video data to obtain processed audio and video data when detecting that any frame of audio and video data contains sensitive information, wherein the processed audio and video data does not contain sensitive information;
and the sending module is used for sending the processed audio and video data to audience users.
10. An electronic device, characterized in that the electronic device comprises a processor and a memory, wherein at least one program code is stored in the memory, and the at least one program code is loaded and executed by the processor to implement the method for processing sensitive information according to any one of claims 1 to 8.
11. A computer-readable storage medium, in which at least one program code is stored, which is loaded and executed by a processor to implement the method of processing sensitive information according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110872578.6A CN113613035A (en) | 2021-07-30 | 2021-07-30 | Sensitive information processing method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110872578.6A CN113613035A (en) | 2021-07-30 | 2021-07-30 | Sensitive information processing method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113613035A true CN113613035A (en) | 2021-11-05 |
Family
ID=78338716
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110872578.6A Pending CN113613035A (en) | 2021-07-30 | 2021-07-30 | Sensitive information processing method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113613035A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114979787A (en) * | 2022-05-17 | 2022-08-30 | 北京量子之歌科技有限公司 | Live broadcast playback management method, device, equipment and storage medium |
| CN115665438A (en) * | 2022-12-26 | 2023-01-31 | 共道网络科技有限公司 | Audio and video processing method, device, equipment and storage medium for court remote court trial |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150163379A1 (en) * | 2013-12-11 | 2015-06-11 | Cellco Partnership D/B/A Verizon Wireless | Time synchronization of video and data inside a mobile device |
| CN106101819A (en) * | 2016-06-21 | 2016-11-09 | 武汉斗鱼网络科技有限公司 | A kind of live video sensitive content filter method based on speech recognition and device |
| CN106973317A (en) * | 2016-12-30 | 2017-07-21 | 华为软件技术有限公司 | Multimedia data processing method, multimedia data providing method, apparatus and system |
| CN108419091A (en) * | 2018-03-02 | 2018-08-17 | 北京未来媒体科技股份有限公司 | A kind of verifying video content method and device based on machine learning |
| CN111031329A (en) * | 2018-10-10 | 2020-04-17 | 北京默契破冰科技有限公司 | Method, apparatus and computer storage medium for managing audio data |
| CN111382623A (en) * | 2018-12-28 | 2020-07-07 | 广州市百果园信息技术有限公司 | Live broadcast auditing method, device, server and storage medium |
| CN111726648A (en) * | 2020-06-28 | 2020-09-29 | 百度在线网络技术（北京）有限公司 | Method, device and equipment for detecting image data and computer readable storage medium |
| CN112492343A (en) * | 2020-12-16 | 2021-03-12 | 浙江大华技术股份有限公司 | Video live broadcast monitoring method and related device |
-
2021
- 2021-07-30 CN CN202110872578.6A patent/CN113613035A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150163379A1 (en) * | 2013-12-11 | 2015-06-11 | Cellco Partnership D/B/A Verizon Wireless | Time synchronization of video and data inside a mobile device |
| CN106101819A (en) * | 2016-06-21 | 2016-11-09 | 武汉斗鱼网络科技有限公司 | A kind of live video sensitive content filter method based on speech recognition and device |
| CN106973317A (en) * | 2016-12-30 | 2017-07-21 | 华为软件技术有限公司 | Multimedia data processing method, multimedia data providing method, apparatus and system |
| CN108419091A (en) * | 2018-03-02 | 2018-08-17 | 北京未来媒体科技股份有限公司 | A kind of verifying video content method and device based on machine learning |
| CN111031329A (en) * | 2018-10-10 | 2020-04-17 | 北京默契破冰科技有限公司 | Method, apparatus and computer storage medium for managing audio data |
| CN111382623A (en) * | 2018-12-28 | 2020-07-07 | 广州市百果园信息技术有限公司 | Live broadcast auditing method, device, server and storage medium |
| CN111726648A (en) * | 2020-06-28 | 2020-09-29 | 百度在线网络技术（北京）有限公司 | Method, device and equipment for detecting image data and computer readable storage medium |
| CN112492343A (en) * | 2020-12-16 | 2021-03-12 | 浙江大华技术股份有限公司 | Video live broadcast monitoring method and related device |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114979787A (en) * | 2022-05-17 | 2022-08-30 | 北京量子之歌科技有限公司 | Live broadcast playback management method, device, equipment and storage medium |
| CN115665438A (en) * | 2022-12-26 | 2023-01-31 | 共道网络科技有限公司 | Audio and video processing method, device, equipment and storage medium for court remote court trial |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110942031B (en) | Game picture abnormality detection method and device, electronic equipment and storage medium | |
| CN113613035A (en) | Sensitive information processing method and device, electronic equipment and storage medium | |
| US20220261217A1 (en) | Audio data processing method, server, and storage medium | |
| US20210295015A1 (en) | Method and apparatus for processing information, device, and medium | |
| CN108989864B (en) | Wheat connecting method, device, equipment and storage medium | |
| JP7467644B2 (en) | Message processing method, device and electronic device | |
| CN111782470A (en) | Distributed container log data processing method and device | |
| CN106921865A (en) | Method for processing video frequency and device | |
| CN110909241B (en) | Information recommendation method, user identification recommendation method, device and equipment | |
| WO2020124454A1 (en) | Font switching method and related product | |
| CN112672202B (en) | Bullet screen processing method, equipment and storage medium | |
| CN115858556A (en) | Data processing method and device, storage medium and electronic equipment | |
| CN111680754B (en) | Image classification method, device, electronic equipment and computer readable storage medium | |
| CN114840421A (en) | Log data processing method and device | |
| CN113852835A (en) | Live broadcast audio processing method and device, electronic equipment and storage medium | |
| CN114390452A (en) | Message sending method and related equipment | |
| CN110516043B (en) | Answer generation method and device for question-answering system | |
| CN111507870A (en) | Lecture display method, system, equipment and storage medium based on online education | |
| CN111026911A (en) | Method and device for self-adaptive adjustment of search engine in live broadcast room and readable storage medium | |
| CN111831953B (en) | Data processing method, device, equipment and storage medium | |
| CN110708592B (en) | Video content complementing and cutting method and system based on user quality evaluation | |
| CN117593596B (en) | Sensitive information detection method, system, electronic equipment and medium | |
| CN115134666B (en) | Live broadcast card pause detection method, system, equipment and storage medium | |
| CN111862248B (en) | Method and device for outputting information | |
| CN112507884A (en) | Live content detection method and device, readable medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |