CN111835739A

CN111835739A - Video playing method and device and computer readable storage medium

Info

Publication number: CN111835739A
Application number: CN202010622057.0A
Authority: CN
Inventors: 陈帅; 林形省
Original assignee: Beijing Xiaomi Pinecone Electronic Co Ltd
Current assignee: Beijing Xiaomi Pinecone Electronic Co Ltd
Priority date: 2020-06-30
Filing date: 2020-06-30
Publication date: 2020-10-27

Abstract

The present disclosure relates to a video playing method, apparatus and computer readable storage medium, the method comprising: caching the streaming media data received in real time; acquiring first target streaming media data to be audited from the cached streaming media data; auditing data to be audited to obtain audited second target streaming media data, wherein the data to be audited comprises audio data and/or image data in the first target streaming media data; determining time axis information corresponding to the second target streaming media data; and playing the second target streaming media data according to the time axis information. Therefore, the video auditing can be completed in the real-time video playing process, and the real-time performance and efficiency of the video auditing are improved, so that the real-time performance and efficiency of the video playing are improved, and the video uploading auditing process is effectively simplified. Meanwhile, the comprehensiveness of video audit can be effectively ensured. In addition, the content reasonability and the safety of the played video can be ensured.

Description

Video playing method and device and computer readable storage medium

Technical Field

The present disclosure relates to the field of video technologies, and in particular, to a video playing method and apparatus, and a computer-readable storage medium.

Background

Nowadays, various video playing software are widely applied, and users make video uploading and sharing gradually increase. For such video data, it is usually required to be audited to be able to be played out to other users for viewing. In the related art, it is generally required that a live video of a user is first audited and delayed for a period of time before being played to other users for watching. The auditing time for the live video is too long, and the watching delay of the user is large.

Disclosure of Invention

To overcome the problems in the related art, the present disclosure provides a video playing method, apparatus, and computer-readable storage medium.

According to a first aspect of the embodiments of the present disclosure, there is provided a video playing method, including:

caching the streaming media data received in real time;

acquiring first target streaming media data to be audited from the cached streaming media data;

auditing data to be audited to obtain audited second target streaming media data, wherein the audited data comprises audio data and/or image data in the first target streaming media data;

determining time axis information corresponding to the second target streaming media data;

and playing the second target streaming media data according to the time axis information.

Optionally, the auditing the data to be audited to obtain the audited second target streaming media data includes:

determining data to be adjusted in the data to be checked;

and performing adjustment operation on the data to be adjusted to obtain the second target streaming media data, wherein the adjustment operation comprises replacing the data to be adjusted with target data or deleting the data to be adjusted.

Optionally, the data to be audited includes audio data in the first target streaming media data;

the determining the data to be adjusted in the data to be checked includes:

performing voice recognition on the audio data to obtain text information corresponding to the audio data;

and determining the audio data corresponding to the text segment which accords with the preset text content in the text information as the data to be adjusted.

Optionally, the data to be audited includes image data in the first target streaming media data;

the determining the data to be adjusted in the data to be checked includes:

inputting image data into a picture detection model, and determining the data to be adjusted according to a detection result of the picture detection model, wherein the picture detection model is used for detecting whether preset picture content exists in the image data.

Optionally, the adjusting operation includes deleting the data to be adjusted;

the determining of the time axis information corresponding to the second target streaming media data includes:

deleting image data corresponding to the audio data of the text clip in the second target streaming media data, and determining time axis information of the remaining audio data and image data in the second target streaming media data according to the time period information corresponding to the audio data of the text clip.

Optionally, the adjusting operation includes deleting the data to be adjusted;

if the text segment is a part of content in a complete sentence, determining an audio frame contained in the audio data of the sentence after deleting the data to be adjusted under the condition that the sentence has corresponding image data;

determining time axis information of each audio frame according to the time period information corresponding to each audio frame;

and determining time axis information of each video frame according to the time period information of the statement and the number of video frames in the image data corresponding to the audio data of the statement in the second target streaming media data, wherein the time period information of the statement is the sum of the time period information corresponding to each audio frame.

Optionally, the method further comprises:

receiving an audit setting instruction set by a user, wherein the audit setting instruction is used for indicating an audit rule set by the user;

the auditing of the data to be audited comprises the following steps:

and auditing the data to be audited according to the auditing rule.

Optionally, the method further comprises:

acquiring a user image of a user watching a video;

carrying out face recognition on the user image to obtain the characteristic information of the user;

determining an auditing rule according to the characteristic information of the user;

the auditing of the data to be audited comprises the following steps:

and auditing the data to be audited according to the auditing rule.

Optionally, the method further comprises:

and storing progress information corresponding to the first target streaming media data to indicate that the first target streaming media data is checked.

According to a second aspect of the embodiments of the present disclosure, there is provided a video playback apparatus including:

the caching module is configured to cache the streaming media data received in real time;

the first acquisition module is configured to acquire first target streaming media data to be audited from the cached streaming media data;

the auditing module is configured to audit data to be audited to obtain audited second target streaming media data, wherein the audited data to be audited comprises audio data and/or image data in the first target streaming media data;

a first determining module configured to determine timeline information corresponding to the second target streaming media data;

and the playing module is configured to play the second target streaming media data according to the time axis information.

Optionally, the auditing module includes:

the first determining submodule is configured to determine data to be adjusted in the data to be checked;

and the execution submodule is configured to execute an adjustment operation on the data to be adjusted to obtain the second target streaming media data, wherein the adjustment operation comprises replacing the data to be adjusted with target data or deleting the data to be adjusted.

the first determination submodule includes:

the recognition submodule is configured to perform voice recognition on the audio data to obtain text information corresponding to the audio data;

and the second determining submodule is configured to determine audio data corresponding to a text segment which accords with preset text content in the text information as the data to be adjusted.

the first determination submodule includes:

the input sub-module is configured to input image data into a picture detection model, and determine the data to be adjusted according to a detection result of the picture detection model, wherein the picture detection model is used for detecting whether preset picture content exists in the image data.

Optionally, the adjusting operation includes deleting the data to be adjusted;

the first determining module includes:

and the third determining submodule is configured to delete the image data corresponding to the audio data of the text clip in the second target streaming media data, and determine time axis information of the audio data and the image data remaining in the second target streaming media data according to the time period information corresponding to the audio data of the text clip.

Optionally, the adjusting operation includes deleting the data to be adjusted;

the first determining module includes:

the fourth determining submodule is configured to determine an audio frame contained in the audio data of the sentence after the data to be adjusted is deleted, when the text segment is a part of content in a complete sentence and the sentence has corresponding image data;

a fifth determining submodule configured to determine time axis information of each of the audio frames according to the period information corresponding to each of the audio frames;

a sixth determining sub-module, configured to determine, according to the time period information of the sentence and the number of video frames in the image data corresponding to the audio data of the sentence in the second target streaming media data, the time axis information of each of the video frames, where the time period information of the sentence is a sum of the time period information corresponding to each of the audio frames.

Optionally, the apparatus further comprises:

the receiving module is configured to receive an audit setting instruction set by a user, wherein the audit setting instruction is used for indicating an audit rule set by the user;

the auditing module comprises:

and the first auditing sub-module is configured to audit the data to be audited according to the auditing rule.

Optionally, the apparatus further comprises:

a second acquisition module configured to acquire a user image of a user viewing the video;

the recognition module is configured to perform face recognition on the user image to obtain feature information of the user;

a second determination module configured to determine an audit rule according to the characteristic information of the user;

the auditing module comprises:

and the second auditing sub-module is configured to audit the data to be audited according to the auditing rule.

Optionally, the apparatus further comprises:

the storage module is configured to store progress information corresponding to the first target streaming media data to indicate that the first target streaming media data has been audited.

According to a third aspect of the embodiments of the present disclosure, there is provided a video playback apparatus including:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to:

caching the streaming media data received in real time;

According to a fourth aspect of embodiments of the present disclosure, there is provided a computer-readable storage medium having stored thereon computer program instructions, which when executed by a processor, implement the steps of the video playback method provided by the first aspect of the present disclosure.

The technical scheme provided by the embodiment of the disclosure can have the following beneficial effects:

in the technical scheme, streaming media data received in real time are cached; acquiring first target streaming media data to be audited from the cached streaming media data; and auditing the data to be audited to obtain audited second target streaming media data, wherein the audited data comprises audio data and/or image data in the first target streaming media data. And then, determining time axis information corresponding to the second target streaming media data, and playing the second target streaming media data according to the time axis information. Therefore, by the technical scheme, the first target streaming media data in the cached streaming media data are audited, so that the auditing of the video can be completed in the real-time video playing process, the real-time performance and efficiency of the video auditing are improved, the real-time performance and efficiency of the video playing are improved, and the process of the video uploading auditing is effectively simplified. Meanwhile, the comprehensiveness of video audit can be effectively ensured by auditing the audio data, the image data and the like in the first target streaming media data. In addition, the time axis information corresponding to the second target streaming media data after being audited is re-determined, so that the content reasonability and the safety of the played video can be guaranteed, and meanwhile, the synchronization and the fluency of the sound and the picture of the played video can be guaranteed, and the use requirements of users can be met.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure.

Fig. 1 is a flow chart illustrating a video playback method according to an exemplary embodiment.

Fig. 2 is a flowchart illustrating an exemplary implementation manner of auditing to-be-audited data to obtain audited second target streaming media data according to an exemplary embodiment.

Fig. 3 is a block diagram illustrating a video playback device according to an example embodiment.

Fig. 4 is a block diagram illustrating a video playback device in accordance with an exemplary embodiment.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

Fig. 1 is a flow chart illustrating a video playback method according to an exemplary embodiment, which may include the following steps, as shown in fig. 1.

In step 11, streaming media data received in real time is buffered.

As an example, in a live scenario, the streaming media data may be video data loaded by the terminal from the server, and the terminal may cache the received streaming media data at the terminal, so that the cached streaming media data may be processed at the terminal.

In step 12, first target streaming media data to be audited is obtained from the buffered streaming media data.

Optionally, in an initial situation of loading the streaming media data, the streaming media data does not have the audited streaming media data, a data amount corresponding to the cached streaming media data may be determined in real time or periodically, and when the data amount is greater than or equal to the target threshold, the cached streaming media data is determined as the first target streaming media data. The target threshold may be set according to an actual usage scenario, which is not limited by this disclosure. Illustratively, the target threshold may be set to 100M.

In a possible embodiment, a data amount corresponding to the cached streaming media data may be determined in real time, and when it is determined that the data amount reaches a target threshold, the streaming media data corresponding to the data amount is determined as the first target streaming media data. For example, when streaming media data received in real time is buffered, the data amount corresponding to the streaming media data is determined at the same time. For example, when the target threshold is 100M, every time it is determined that the data size reaches 100M, the streaming media data corresponding to the data size in the cache is determined as the first target streaming media data. It should be noted that the data volume of the streaming media data determined in real time may exceed 100M, and if the data volume is 110M, the streaming media data portion corresponding to 110M is determined as the first target streaming media data at this time. It should be noted that, when determining the first target streaming media data to be audited, it needs to be ensured that the streaming media data that has been audited before is not completely played, that is, the first target streaming media data to be audited is determined from the streaming media data that has been cached and has not been played yet.

In another possible embodiment, the method may further include: and storing progress information corresponding to the first target streaming media data to indicate that the first target streaming media data is checked. For example, the determined data amount of the first target streaming media data may be stored as progress information corresponding to the first target streaming media data. For example, it may be determined, at each interval of the target time, whether a data amount of the streaming media data after the progress information in the cached streaming media data reaches a target threshold, and if the data amount reaches the target threshold, the streaming media data after the progress information in the cached streaming media data may be determined as the first target streaming media data. If the data volume corresponding to the determined first target streaming media data is 100M in the initial situation, it may be determined that the progress information corresponding to the first target streaming media data is 100M. As an example, it may be determined whether the data amount of the streaming media data after the progress information reaches 100M every 5 seconds, for example, the determination may be made by a difference between the total data amount corresponding to the streaming media data and the progress information. For example, if the total data amount corresponding to the streaming media data is determined to be 230M, it indicates that the data amount (130M) of the streaming media data after the progress information reaches the target threshold, and at this time, the streaming media data after the progress information may be determined to be the first target streaming media data.

Therefore, the first target streaming media data can be quickly determined through the method, and therefore subsequent segmented auditing processing of the first target streaming media data is facilitated. Meanwhile, the integrity and the accuracy of the auditing treatment of the streaming media data can be ensured.

In step 13, the data to be audited is audited to obtain the audited second target streaming media data, where the audited data includes audio data and/or image data in the first target streaming media data.

The streaming media data may include audio data and image data, where the audio data is used to play audio and the image data is used to play images, so as to implement playing of video according to the streaming media data.

Therefore, in this embodiment, the audio data can be audited, so that auditing of the voice of the played video can be realized; and auditing the image data, thereby realizing auditing the picture displayed in the video. For example, in the embodiment of the present disclosure, streaming media data played in real time is audited, and therefore, in order to ensure normal playing of a video, the first target streaming media data may be modified in the auditing process, so as to obtain second target streaming media data after auditing, which not only can ensure playing of the video, but also can avoid inappropriate content in the played video.

In step 14, time axis information corresponding to the second target streaming media data is determined. For example, time axis information corresponding to the audio data and the image data in the second target streaming media data can be determined, so that sound and picture synchronization during video playing can be ensured.

In step 15, the second target streaming media data is played according to the time axis information.

The audio data and the image data corresponding to each moment in the second target streaming media data can be played according to the time axis information, so that the second target streaming media data can be played, and video playing is realized.

In order to make the technical solutions provided by the embodiments of the present invention easier to understand for those skilled in the art, the following detailed description is provided for each step in the methods provided by the present disclosure.

Optionally, in step 13, an exemplary implementation manner of auditing the data to be audited to obtain the audited second target streaming media data is as follows, as shown in fig. 2, the step may include:

in step 21, data to be adjusted in the data to be checked is determined. The data to be adjusted is the data which is determined in the process of auditing the data to be audited, cannot be directly played and needs to be adjusted.

In a possible embodiment, the data to be audited includes audio data in the first target streaming media data;

accordingly, in step 21, an exemplary implementation manner of determining data to be adjusted in the data to be reviewed is as follows, and the step may include:

and performing voice recognition on the audio data to obtain text information corresponding to the audio data.

For example, the audio data may be subjected to Speech Recognition through an Automatic Speech Recognition (ASR) technology and a Natural Language Processing (NLP) technology, so that text information corresponding to the audio data may be obtained.

For example, the preset text content may be preset according to an audit standard of the audio data. For example, when the dirty words in the video need to be reviewed, a corresponding template may be preset for the dirty words that need to be reviewed, where the template may include keywords or expressions. And then, when the audio data is checked, matching the text information corresponding to the audio data with a template, and determining the text segment matched with the template as the data to be adjusted.

In one possible embodiment, the text information matched with the template can be directly determined as the text segment. In another possible embodiment, after the text information matched with the template is determined, the sentence to which the text information belongs is determined as the text segment.

Therefore, according to the technical scheme, the audio data are converted into the text information, so that the data to be adjusted in the audio data can be quickly and accurately determined based on the text information, and accurate data support is provided for auditing the audio data.

Exemplarily, the data to be audited includes image data in the first target streaming media data;

then in step 21, an exemplary implementation manner of determining data to be adjusted in the data to be checked may be as follows, and this step may include:

Illustratively, a picture detection model may be pre-trained based on audit criteria for video pictures. For example, if the preset picture content is a violent and bloody picture in a video, the violent and bloody picture in a part of the video can be acquired in advance as training data, and the picture detection model is obtained by training a neural network model. Therefore, when the image data is input into the picture detection model, whether each video frame corresponding to the image data contains a picture with violence and bloody smell can be quickly determined through the picture detection model, so that the data to be adjusted is determined. The preset picture content is only an exemplary illustration, and may be set according to an audit requirement in an actual application scenario, which is not limited in this disclosure.

Therefore, according to the technical scheme, whether the image data contain the preset image content can be directly determined through the image detection model, so that the data to be adjusted in the image data can be rapidly and accurately determined, and accurate data support is provided for auditing the image data.

In step 22, an adjustment operation is performed on the data to be adjusted to obtain second target streaming media data, where the adjustment operation includes replacing the data to be adjusted with target data or deleting the data to be adjusted.

In a possible embodiment, the data to be adjusted is audio data, such as audio data of a text segment corresponding to the preset text content matched in the above manner. In one case, the audio data may be replaced with the target audio data, for example, the audio data may be silenced and replaced with the target audio data. If the duration corresponding to the target audio data is different from the duration corresponding to the audio data of the text segment, replacing the audio data of the text segment with the target audio data by intercepting or circularly playing the target audio data, so as to obtain the audio data in the second target streaming media data after the auditing. In another case, the audio data of the text segment in the audio data of the first target streaming media data may be directly deleted, so as to obtain the audio data in the second target streaming media data.

In another possible embodiment, the data to be adjusted is image data, such as the image data with the preset picture content determined in the above manner, in one case, the image data may be replaced by target image data, and the target image data may be image data preset by the user. Optionally, in order to ensure the playing continuity of the video frames, the target image data may be an image of the last video frame before the data to be adjusted, so that when the data to be adjusted is replaced by the image of the video frame, the user still views the image in the video, and thus the fluency of video playing can be ensured. In another case, the data to be adjusted in the image data of the first target streaming media data may be directly deleted, so as to obtain the image data in the second target streaming media data.

Therefore, according to the technical scheme, the data to be adjusted in the first target streaming media data can be determined, so that the data to be adjusted is adjusted, the second target streaming media data after being audited does not contain the data to be adjusted, the data to be adjusted determined in the auditing process is shielded or deleted, the content corresponding to the data to be adjusted does not appear in the played video, the safety and the auditing and filtering accuracy of the played video are guaranteed, and the use experience of a user is improved.

For example, when the adjustment operation includes replacing the data to be adjusted with the target data, in this embodiment, the content of the partial video frames in the image data is changed, or the content of the partial audio frames in the audio data is changed, and the number of video frames in the image data and the number of audio frames in the audio data are not modified. Therefore, in this embodiment, the time axis information of each video frame of the image data in the second target streaming media data may correspond to the time axis information of each video frame of the image data in the first target streaming media data one to one, and the time axis information of each audio frame of the audio data in the second target streaming media data may correspond to the time axis information of each audio frame of the audio data in the first target streaming media data one to one, so that the time axis information corresponding to the second target streaming media data may be determined.

Optionally, the adjusting operation includes deleting the data to be adjusted, in an embodiment, when image data of second target streaming media data is obtained, the number of video frames in the image data of the first target streaming media data may be reduced, or when audio data of the second target streaming media data is obtained, the number of audio frames in the audio data of the first target streaming media data is reduced, or both the number of video frames and the number of audio frames may be reduced. Therefore, in this embodiment, there may be a case where the video frame of the image data and the video frame of the audio data in the second target streaming media data obtained after the auditing do not correspond to each other, and therefore, the present disclosure provides the following embodiment to determine the time axis information corresponding to the second target streaming media data, so as to implement synchronization between the audio data and the image data of the second target streaming media data.

Accordingly, an exemplary implementation manner of determining the time axis information corresponding to the second target streaming media data is as follows, and this step may include:

For example, the time of each data to be adjusted corresponding to the audio data of the second target streaming media data may be determined from the order of early to late. In this embodiment, after deleting the audio data of the text segment, in order to ensure the synchronization between the picture and the audio, the image data corresponding to the audio data of the text segment may be directly deleted. Therefore, when video playing is performed on the streaming media data obtained in the above manner, neither audio nor a picture corresponding to the text segment is played.

The time interval information corresponding to the audio data of the text segment is a time interval corresponding to the audio data from the start time to the end time, and can be determined by time axis information of the audio data in the first target streaming media data. Illustratively, the start time of the audio data of the text segment is 1s, and the end time is 3s, then the period information corresponding to the audio data of the text segment is (1, 3).

In a possible embodiment, when the time axis information of the remaining audio data and image data in the second target streaming media data is determined according to the time period information corresponding to the audio data of the text clip, the time axis information of the remaining audio data and image data in the second target streaming media data may be moved forward by a time period corresponding to the time period information, that is, the time axis information of the remaining audio data and image data in the second target streaming media data becomes (Ti-2, Ti + t-2), where Ti is the current starting time of the ith audio frame in the remaining audio data, and t is the duration of the ith audio frame. The determination method of the time axis information of the video frame is similar to that described above, and is not described herein again.

If the second target streaming media data corresponds to the text clip of the unadjusted time axis information, the determined time axis information of the audio data and the image data remaining in the second target streaming media data may be used as the current time axis information, and the above process may be executed again until the second target streaming media data does not have the corresponding text clip of the unadjusted time axis information.

The following takes an audio frame as an example to specifically describe the above process. The audio data in the first target streaming media data includes 5 pieces of audio data T1, T2, T3, T4, and T5, and the time axis information thereof is (0,1), (1,3), (3,5), (5,6), (6,8), respectively, and through the above embodiment, it is determined that the data to be adjusted is T2 and T4, that is, the audio data of T2 and T4 in the second target streaming media data has been deleted, and then the time axis information corresponding to the second target streaming media data is first determined according to the time period information corresponding to T2 in the order from early to late in time.

For example, after T2 is deleted, the time axis information of the remaining audio data in the second target streaming media data may be shifted forward by the time length (2s) corresponding to the time period information of T2, where the current time axis information is the time axis information corresponding to the audio data of the first target streaming media data, and then the second target streaming media data after the adjustment operation is performed includes 3 pieces of audio data T1, T3, and T5, where the time axis information corresponding to the pieces of audio data T1, T3, and T5 are (0,1), (1,3), (4,6), respectively. At this time, if the second target streaming media data further corresponds to the text segment T4 with unadjusted time axis information, the above process may be executed again by using the determined time axis information 0,1), (1,3), (4,6) of the 3 segments of audio data T1, T3, and T5 as the current time axis information, and the time axis information corresponding to the 3 segments of audio data T1, T3, and T5 included in the adjusted second target streaming media data are (0,1), (1,3), (3,5), respectively. At this time, if there is no text segment of the second target streaming media data with unadjusted time axis information, the final time axis information corresponding to the second target streaming media data is determined. The determination method of the time axis information of the image frame is similar to that described above, and is not described herein again.

After the time axis information of the second target streaming media data is determined, the end time corresponding to the second previous target streaming media data can be directly used as the start time corresponding to the next second target streaming media data, so that the time axis information of the next second target streaming media data is determined. The specific determination method is similar to that described above, and is not described herein again.

Therefore, the technical scheme can ensure the synchronism of the image data and the audio data in the second target streaming media data to a certain degree, and ensure the fluency of video playing while realizing the auditing and filtering of the video content.

When the determined text segment is a part of the content of the complete sentence, after the audio data of the text segment is deleted in the sentence, the audio frame in the audio data of the sentence and the video frame in the image data may not correspond to each other, and at this time, the synchronous playing of the audio and the picture after the sentence may be affected. Therefore, the present disclosure further provides the following embodiments, when a text segment is a part of content in a complete sentence, determining time axis information of an audio frame and a video frame corresponding to the text segment by using the sentence as a basic unit, so as to ensure that audio data and video data corresponding to each sentence are synchronized, thereby ensuring that the whole sound and picture of a played video are synchronized.

Optionally, in a case that the adjusting operation includes deleting the data to be adjusted, as another example, an exemplary implementation manner of determining the timeline information corresponding to the second target streaming media data is as follows, and this step may include:

if the text segment is a part of content in a complete sentence, determining an audio frame contained in the audio data of the sentence after deleting the data to be adjusted under the condition that the sentence has corresponding image data.

Illustratively, the audio frames P1, P2, P3, P4, and P5 correspond to a complete sentence, and the corresponding time axis information is (0,1), (1,3), (3,5), (5,6), (6,8), respectively, wherein the audio frames P2 and P4 are data to be adjusted, and the audio frames included in the audio data of the sentence after the data to be adjusted is deleted are P1, P3, and P5.

And then, determining the time axis information of each audio frame according to the time interval information corresponding to each audio frame.

In a manner similar to the above, the determined time axis information of each audio frame is (0,1), (1,3), (3,5), respectively.

And determining the time axis information of each video frame according to the time period information of the sentence and the number of video frames in the image data corresponding to the audio data of the sentence in the second target streaming media data, wherein the time period information of the sentence is the sum of the time period information corresponding to each audio frame, and in this example, the time period information of the sentence is the sum of the time period information corresponding to the audio frames P1, P3 and P5, which is 5s (1+2+ 2).

In this embodiment, if it is determined that the number of video frames in the image data corresponding to the audio data of the sentence in the second target streaming media data is 5, each video frame may be equally allocated with the period information to determine the duration, i.e., 1s, corresponding to each video frame. Since it is necessary to ensure synchronization between the audio data corresponding to the sentence and the image data, the time axis information of each image frame can be determined according to the start time of the audio frame corresponding to the sentence, which is (0,1), (1,2), (2,3), (3,4), (4,5), respectively.

Therefore, the technical scheme can ensure the synchronism of the image data and the audio data of each statement in the second target streaming media data in a fine-grained manner, and ensure the fluency of video playing while realizing the auditing and filtering of the video content.

Optionally, when the audio data corresponding to the sentence is not deleted, but the video data corresponding to the sentence is deleted, the time axis information of each video frame may be determined according to the time period information of the sentence and the number of video frames in the image data corresponding to the audio data of the sentence in the second target streaming media data, so as to ensure audio and video synchronization when video is played based on the second target streaming media data. The detailed implementation thereof has been described above, and will not be described herein.

Optionally, if the text segment is a part of content in a complete sentence, and the sentence does not have corresponding image data, the audio data corresponding to the sentence may be directly deleted, and the time axis information of the remaining audio data and image data in the second target streaming media data is determined according to the time period information corresponding to the audio data of the sentence. The specific determination method of the time axis information is described in detail above, and is not described herein again.

In an actual use scenario, for some movies or videos edited by users, the videos need to be uploaded to a sharing platform (such as various video APPs), and then, a worker in the sharing platform audits the uploaded video data, and after the audits are passed, the videos can be made online, so that other users can watch the videos. Although the video passes the review of the shared platform, it may still contain portions that are not suitable for viewing by minors. Accordingly, the present disclosure also provides the following embodiments.

Optionally, the method further comprises:

and receiving an audit setting instruction set by a user, wherein the audit setting instruction is used for indicating an audit rule set by the user.

For example, the user may set a voice rule and/or a screen rule that needs to be audited in the rule setting interface.

Accordingly, an exemplary implementation of auditing data to be audited is as follows, which may include:

and auditing the data to be audited according to the auditing rule. The corresponding audit text content and/or audit picture content can be determined according to the audit rule, so that the audit text content can be used as the preset text content, and the audit picture content can be determined as the preset picture content, so that the audit of the data to be audited is realized. Specific embodiments in which the audit data is to be audited have been described above in detail.

Therefore, according to the technical scheme, the user can set the corresponding auditing rule by himself and perform content auditing and filtering of the video in the user terminal equipment, so that secondary auditing can be performed on the video content, the personalized requirements of the user are met, and the application range of the video playing method can be widened.

Optionally, the method may further include:

and acquiring a user image of a user watching the video, wherein the user image can be acquired through a camera device on the terminal.

And performing face recognition on the user image to obtain feature information of the user, wherein the feature information can be age information.

The face image in the user image can be extracted based on the existing face detection algorithm, for example, the detection can be performed through the face detection algorithms such as setaface and mtcnn. Then, key points can be extracted based on the existing key point detection algorithm, for example, the human face key point detection methods such as the ert (ensemble of Regression tress) algorithm, the mdm (mnemonic description method) and the like, so that key points corresponding to the human face image can be obtained, and then the age information corresponding to the human image can be determined through the key points. As another example, the fingerprint information may be acquired by a fingerprint acquisition device on the terminal, so that the age information is determined by analyzing the fingerprint information. The manner of determining the age information is the prior art, and is not described herein again.

And determining an auditing rule according to the characteristic information of the user.

For example, corresponding audit rules may be set for users of different ages according to video audit requirements, for example, the rule R1 is set for users of 5-14 years old, the rule R2 is set for users of 15-18 years old, and the rule R3 is set for users over 18 years old, where a specific audit rule corresponding to age information may be set according to an actual usage scenario, which is not limited by the present disclosure. Therefore, after the characteristic information of the user is determined, the corresponding auditing rule can be determined according to the characteristic information. Illustratively, if the determined age information of the user indicates 13 years of age, rule R2 may be used to determine the audit rule.

Accordingly, an exemplary implementation of auditing data to be audited is as follows, which may include: and auditing the data to be audited according to the auditing rule. The specific implementation of auditing the to-be-audited data according to the auditing rule is described in detail above, and is not described herein again.

Therefore, by the technical scheme, the corresponding auditing rule during playing of the user can be automatically determined by acquiring the characteristic information of the user, so that the auditing rule corresponding to the user can be automatically determined for different users, the safety of watching videos by minors can be effectively ensured, the use requirements of the user are met, and the use experience of the user is further improved.

Optionally, the method may further include: receiving a color distinguishing instruction of a user, wherein the color distinguishing instruction is used for indicating colors which are difficult to distinguish by the user, and if the color distinguishing instruction indicates that the colors which are difficult to distinguish by the user are red and green, performing color extraction on the image data of the cached streaming media data in response to the color distinguishing instruction, and replacing the green close to the red with a target color, illustratively, the target color is blue, so that when video playing is performed, the user can distinguish the outlines of different objects or persons therein through the comparison of the red and the blue.

As another example, the color distinguishing instruction indicates that the colors that are difficult for the user to distinguish are blue and purple, and similarly, purple near blue in the image data may be replaced with other colors so as to facilitate the user to distinguish the outlines of different objects or characters therein.

Therefore, according to the technical scheme, the color of the image data in the streaming media data cached by the terminal can be modified according to different users, so that the video suitable for the user to watch is played, the color weakness optimization of the video can be realized at the terminal according to the individual difference of the user, the user with the color weakness can watch the video conveniently, the user use experience is further improved, and the application range of the video playing method can be effectively widened.

The present disclosure also provides a video playing apparatus, as shown in fig. 3, the apparatus 10 includes:

a caching module 100 configured to cache streaming media data received in real time;

a first obtaining module 200, configured to obtain first target streaming media data to be audited from the cached streaming media data;

the auditing module 300 is configured to audit data to be audited to obtain audited second target streaming media data, where the audited data includes audio data and/or image data in the first target streaming media data;

a first determining module 400 configured to determine timeline information corresponding to the second target streaming media data;

a playing module 500 configured to play the second target streaming media data according to the time axis information.

Optionally, the auditing module includes:

the first determination submodule includes:

Optionally, the adjusting operation includes deleting the data to be adjusted;

the first determining module includes:

Optionally, the adjusting operation includes deleting the data to be adjusted;

the first determining module includes:

Optionally, the apparatus further comprises:

the auditing module comprises:

Optionally, the apparatus further comprises:

the auditing module comprises:

Optionally, the apparatus further comprises:

With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

The present disclosure also provides a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the steps of the video playback method provided by the present disclosure.

Fig. 4 is a block diagram illustrating a video playback device 800 according to an example embodiment. For example, the apparatus 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, an exercise device, a personal digital assistant, and the like.

Referring to fig. 4, the apparatus 800 may include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input/output (I/O) interface 812, a sensor component 814, and a communication component 816.

The processing component 802 generally controls overall operation of the device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing components 802 may include one or more processors 820 to execute instructions to perform all or a portion of the steps of the video playback method described above. Further, the processing component 802 can include one or more modules that facilitate interaction between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.

The memory 804 is configured to store various types of data to support operations at the apparatus 800. Examples of such data include instructions for any application or method operating on device 800, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 804 may be implemented by any type or combination of volatile or non-volatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.

Power component 806 provides power to the various components of device 800. The power components 806 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the device 800.

The multimedia component 808 includes a screen that provides an output interface between the device 800 and a user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front facing camera and/or a rear facing camera. The front camera and/or the rear camera may receive external multimedia data when the device 800 is in an operating mode, such as a shooting mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a Microphone (MIC) configured to receive external audio signals when the apparatus 800 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may further be stored in the memory 804 or transmitted via the communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

The I/O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.

The sensor assembly 814 includes one or more sensors for providing various aspects of state assessment for the device 800. For example, the sensor assembly 814 may detect the open/closed status of the device 800, the relative positioning of components, such as a display and keypad of the device 800, the sensor assembly 814 may also detect a change in the position of the device 800 or a component of the device 800, the presence or absence of user contact with the device 800, the orientation or acceleration/deceleration of the device 800, and a change in the temperature of the device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 816 is configured to facilitate communications between the apparatus 800 and other devices in a wired or wireless manner. The device 800 may access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, the apparatus 800 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components for performing the above-described video playing method.

In an exemplary embodiment, a non-transitory computer-readable storage medium comprising instructions, such as the memory 804 comprising instructions, executable by the processor 820 of the device 800 to perform the video playback method described above is also provided. For example, the non-transitory computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

In another exemplary embodiment, a computer program product is also provided, which comprises a computer program executable by a programmable apparatus, the computer program having code portions for performing the video playback method described above when executed by the programmable apparatus.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A video playback method, comprising:

caching the streaming media data received in real time;

2. The method according to claim 1, wherein the auditing the data to be audited to obtain the audited second target streaming media data includes:

determining data to be adjusted in the data to be checked;

3. The method of claim 2, wherein the pending data comprises audio data in the first target streaming media data;

the determining the data to be adjusted in the data to be checked includes:

4. The method of claim 3, wherein the adjustment operation comprises deleting the data to be adjusted;

5. The method of claim 3, wherein the adjustment operation comprises deleting the data to be adjusted;

6. The method of claim 2, wherein the pending data comprises image data in the first target streaming media data;

the determining the data to be adjusted in the data to be checked includes:

7. The method of claim 1, further comprising:

the auditing of the data to be audited comprises the following steps:

and auditing the data to be audited according to the auditing rule.

8. The method of claim 1, further comprising:

acquiring a user image of a user watching a video;

the auditing of the data to be audited comprises the following steps:

and auditing the data to be audited according to the auditing rule.

9. The method of claim 1, further comprising:

10. A video playback apparatus, comprising:

11. A video playback apparatus, comprising:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to:

caching the streaming media data received in real time;

12. A computer-readable storage medium, on which computer program instructions are stored, which program instructions, when executed by a processor, carry out the steps of the method according to any one of claims 1 to 9.