CN115396691A - Data stream processing method and device and electronic equipment - Google Patents
Data stream processing method and device and electronic equipment Download PDFInfo
- Publication number
- CN115396691A CN115396691A CN202110555814.1A CN202110555814A CN115396691A CN 115396691 A CN115396691 A CN 115396691A CN 202110555814 A CN202110555814 A CN 202110555814A CN 115396691 A CN115396691 A CN 115396691A
- Authority
- CN
- China
- Prior art keywords
- processing
- task
- time point
- tasks
- processing time
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title abstract description 19
- 238000004140 cleaning Methods 0.000 claims abstract description 64
- 238000000034 method Methods 0.000 claims abstract description 63
- 238000009877 rendering Methods 0.000 claims description 48
- 238000004590 computer program Methods 0.000 claims description 8
- 230000015654 memory Effects 0.000 claims description 5
- 239000000126 substance Substances 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 5
- 230000006978 adaptation Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44012—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The disclosure relates to a data stream processing method, a data stream processing device and electronic equipment, and relates to the technical field of data processing, wherein the method comprises the following steps: acquiring a task sequence, wherein the task sequence comprises the following steps: processing tasks of a plurality of data in the data stream; determining predicted ending processing time points of the plurality of processing tasks according to the starting processing time points and the predicted processing time of the plurality of processing tasks; in the execution process of the multiple processing tasks, when the cleaning time point is reached, cleaning a target processing task and corresponding task execution data, wherein the target processing task is a processing task of which the corresponding predicted ending processing time point is not later than the cleaning time point, so that the processing task which is not finished at the predicted ending processing time point and the corresponding task execution data can be timely cleaned in the execution process of the multiple processing tasks, the subsequent processing tasks can be timely executed, the execution speed of the processing tasks can be improved, and the processing delay of the processing tasks can be reduced.
Description
Technical Field
The present disclosure relates to the field of data processing technologies, and in particular, to a data stream processing method and apparatus, and an electronic device.
Background
In the related art, in a panoramic scene, the Omaf-dash protocol divides a panoramic video into a plurality of blocks (tiles) in a space domain, and the video of each block corresponding to a certain duration is coded into a fragmented mp4 file according to a certain time interval in a time domain. In the current data stream mode, when the main view angle is switched, the high definition stream of the block corresponding to the new main view angle and the low definition streams of the blocks corresponding to other non-main view angles are pulled again, and the high definition streams and the low definition streams are decoded and rendered again for playing.
In the above scheme, when the main view angle is frequently switched, the data stream to be pulled is too much, and the processing delay for the data stream is too long.
Disclosure of Invention
The present disclosure provides a data stream processing method, an apparatus and an electronic device, so as to at least solve the problem in the related art that, when a main field angle is frequently switched, a pulled data stream is too much and processing delay is too long. The technical scheme of the disclosure is as follows:
according to a first aspect of the embodiments of the present disclosure, there is provided a data stream processing method, including: acquiring a task sequence, wherein the task sequence comprises: processing tasks of a plurality of data in the data stream; determining predicted ending processing time points of the processing tasks according to the starting processing time points and the predicted processing time of the processing tasks; in the execution process of the plurality of processing tasks, when a cleaning time point is reached, cleaning a target processing task and corresponding task execution data, wherein the target processing task is a processing task of which the corresponding predicted ending processing time point is not later than the cleaning time point.
According to an embodiment of the present disclosure, the determining an expected end processing time point of a plurality of the processing tasks according to a start processing time point and an expected processing time of the plurality of the processing tasks includes:
aiming at a first processing task in the task sequence, determining an estimated ending processing time point of the first processing task according to the initial processing time point and the estimated processing time of the first processing task;
alternatively, the first and second electrodes may be,
aiming at a non-first processing task in the task sequence, determining an initial processing time point of the non-first processing task according to a serial number of the non-first processing task, the serial number of the first processing task and the initial processing time point of the first processing task in the task sequence; and determining the predicted ending processing time point of the non-first processing task according to the starting processing time point and the predicted processing time of the non-first processing task.
According to an embodiment of the present disclosure, the method further comprises: and stopping the execution process of the processing task when the predicted ending processing time point of the processing task is reached for each processing task in the execution process of the plurality of processing tasks.
According to an embodiment of the present disclosure, the data stream is a video stream, or a data stream obtained by processing the video stream at a preset task node; the images in the video stream are images in a non-panoramic scene, or image blocks in a partial field angle in a panoramic scene.
According to an embodiment of the present disclosure, the preset task node is at least one of the following task nodes: downloading task nodes, decoding task nodes and rendering task nodes.
According to one embodiment of the disclosure, for a partial image in the video stream, a difference value of sequence numbers between a first sequence number of the partial image and a second sequence number of a reference image corresponding to the partial image is greater than a preset value.
The technical scheme provided by the embodiment of the disclosure at least brings the following beneficial effects:
the present disclosure obtains a task sequence, where the task sequence includes: processing tasks of a plurality of data in the data stream; determining predicted ending processing time points of the plurality of processing tasks according to the starting processing time points and the predicted processing time of the plurality of processing tasks; in the execution process of the multiple processing tasks, when the cleaning time point is reached, cleaning a target processing task and corresponding task execution data, wherein the target processing task is a processing task of which the corresponding predicted ending processing time point is not later than the cleaning time point, so that the processing task which is not finished at the predicted ending processing time point and the corresponding task execution data can be timely cleaned in the execution process of the multiple processing tasks, the subsequent processing tasks can be timely executed, the execution speed of the processing tasks can be improved, and the processing delay of the processing tasks can be reduced.
According to a second aspect of the embodiments of the present disclosure, there is provided a data stream processing apparatus including:
an acquisition module for acquiring the task sequence, wherein the task sequence comprises: processing tasks of a plurality of data in the data stream; the determining module is used for determining the predicted ending processing time points of the processing tasks according to the starting processing time points and the predicted processing time of the processing tasks; and the processing module is used for cleaning a target processing task and corresponding task execution data when a cleaning time point is reached in the execution process of the plurality of processing tasks, wherein the target processing task is a processing task of which the corresponding predicted ending processing time point is not later than the cleaning time point.
According to an embodiment of the present disclosure, the determining module is specifically configured to,
aiming at a first processing task in the task sequence, determining an estimated ending processing time point of the first processing task according to the initial processing time point and the estimated processing time of the first processing task;
alternatively, the first and second electrodes may be,
aiming at a non-first processing task in the task sequence, determining an initial processing time point of the non-first processing task according to a serial number of the non-first processing task, the serial number of the first processing task and the initial processing time point of the first processing task in the task sequence; and determining the predicted ending processing time point of the non-first processing task according to the starting processing time point and the predicted processing time of the non-first processing task.
According to one embodiment of the present disclosure, the processing module is further configured to,
and stopping the execution process of the processing task when the predicted ending processing time point of the processing task is reached for each processing task in the execution process of the plurality of processing tasks.
According to an embodiment of the present disclosure, the data stream is a video stream, or a data stream obtained by processing the video stream at a preset task node; the images in the video stream are images in a non-panoramic scene, or image blocks in a partial field angle in a panoramic scene.
According to an embodiment of the present disclosure, the preset task node is at least one of the following task nodes: downloading task nodes, decoding task nodes and rendering task nodes.
According to one embodiment of the disclosure, for a partial image in the video stream, a difference value of sequence numbers between a first sequence number of the partial image and a second sequence number of a reference image corresponding to the partial image is greater than a preset value.
According to a third aspect of the embodiments of the present disclosure, there is provided an electronic apparatus including: a processor; a memory for storing the processor-executable instructions; wherein the processor is configured to execute the instructions to implement the data stream processing method provided by the embodiment of the first aspect of the disclosure.
According to a fourth aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium, wherein instructions of the computer-readable storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the data stream processing method as provided in the first aspect of the present disclosure.
According to a fifth aspect of embodiments of the present disclosure, there is provided a computer program product comprising a computer program, wherein the computer program is configured to implement the data stream processing method as provided in the first aspect of the present disclosure when executed by a processor.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure and are not to be construed as limiting the disclosure.
Fig. 1 is a flow chart illustrating a method of data stream processing according to an example embodiment.
FIG. 2 is a diagram illustrating multiple images referencing the same image, according to an example embodiment.
Fig. 3 is a flow chart illustrating a method of data stream processing according to an example embodiment.
Fig. 4 is a flow chart illustrating a method of data stream processing in accordance with an exemplary embodiment.
Fig. 5 is a flow chart illustrating a method of data stream processing according to an example embodiment.
Fig. 6 is a flow chart illustrating a method of data stream processing according to an example embodiment.
Fig. 7 is a block diagram illustrating a data stream processing apparatus according to an example embodiment.
FIG. 8 is a block diagram illustrating an electronic device in accordance with an example embodiment.
Detailed Description
In order to make the technical solutions of the present disclosure better understood by those of ordinary skill in the art, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.
It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the disclosure described herein are capable of operation in sequences other than those illustrated or otherwise described herein. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.
FIG. 1 is a flow chart illustrating a method of data stream processing in accordance with an exemplary embodiment. It should be noted that the main execution body of the data stream processing method of the present disclosure is a data stream processing apparatus. The data stream processing method according to the embodiment of the present disclosure may be executed by the data stream processing apparatus according to the embodiment of the present disclosure, and the data stream processing apparatus may specifically be a hardware device, or software in a hardware device, or the like. The hardware devices are, for example, terminal devices, servers, and the like.
As shown in fig. 1, the data stream processing method provided in this embodiment includes the following steps:
in step 101, a task sequence is obtained, wherein the task sequence includes: and processing a plurality of data in the data stream.
In the embodiment of the present disclosure, the data stream may be a video stream, or a data stream processed by the video stream at a preset task node. The images in the video stream are images in a non-panoramic scene, or image blocks in a partial field angle in a panoramic scene.
In this embodiment of the present disclosure, the preset task node may be at least one of the following task nodes: downloading task nodes, decoding task nodes and rendering task nodes.
In the embodiment of the present disclosure, in the case that the data stream is a video stream, the plurality of data in the data stream refers to a plurality of images in the video stream; the processing task of the plurality of data may refer to a task of downloading a plurality of images, or a task of downloading and decoding a plurality of images, or a task of downloading, decoding, and rendering a plurality of images.
Under the condition that the data stream is obtained by processing the video stream at a preset task node, if the preset task node is a downloading task node, the data stream can be the video stream obtained by downloading; in this case, the processing task of the plurality of data may be a task of decoding a plurality of images in the downloaded video stream, or a task of decoding and rendering a plurality of images in the downloaded video stream.
When the data stream is a data stream obtained by processing a video stream at a preset task node, if the preset task node is a decoding task node, the data stream may be a data stream obtained by decoding the downloaded video stream; in this case, the processing task of the plurality of data may be a rendering task of the plurality of data in the data stream obtained after the decoding process.
When the data stream is a data stream obtained by processing a video stream at a preset task node, if the preset task node is a rendering task node, the data stream may be a data stream obtained by rendering the decoded data stream; in this case, the processing task of the plurality of data may be a playback task of the plurality of data in the data stream obtained after the rendering processing.
In the embodiment of the present disclosure, in a case that an image in a video stream is an image block at a partial field angle in a panoramic scene, the image in the video stream may be a high-definition image block on a block (tile) corresponding to a main field angle; alternatively, the first and second electrodes may be, the images in the video stream may be low-definition image blocks on blocks corresponding to other angles of view.
In the embodiment of the present disclosure, in the case where the image in the video stream is an image block at a partial field angle in the panoramic scene, the number of the video streams may be plural. For example, video streams over blocks corresponding to multiple angles of view.
In the embodiment of the present disclosure, it should be further noted that, when the data stream is a video stream, the image information of each frame of image in the video stream is the variation information of the frame of image with respect to the corresponding reference image, and therefore, the video stream needs to include an integer number of groups of pictures. Wherein each group of pictures contains an intra-coded reference picture I having complete picture information.
In addition, when the data stream is a video stream, in order to reduce the amount of processing at the time of decoding the video stream, a difference in sequence number between a first sequence number of a partial image and a second sequence number of a reference image corresponding to the partial image in the video stream is greater than a preset value. That is, the number of reference pictures to which each frame of picture refers is reduced. For example, in a case where a picture a, a picture B, a picture C, and a picture D are included in a video stream, a reference picture of the picture B is the picture a, a reference picture of the picture C is the picture B, and a reference picture of the picture D is the picture C in the related art; therefore, when decoding the image D, the actual reference images include the image a, the image B, and the image C, and the image a, the image B, and the image C need to be decoded to decode the image D. In the present application, for example, an image B, an image C, and an image D may all refer to an image a directly, and when decoding an image D, the image D may be decoded only by completing decoding the image a, as shown in fig. 2, which is a schematic diagram in which a plurality of images refer to the same image, and in fig. 2, a plurality of p represent a plurality of images; i denotes the same picture to which multiple pictures are referenced.
In step 102, predicted end processing time points of the plurality of processing tasks are determined based on the start processing time points and the predicted processing times of the plurality of processing tasks.
In the embodiment of the present disclosure, the process of the data stream processing apparatus to execute step 102 may be, for example, to determine, for a first processing task in the task sequence, an expected ending processing time point of the first processing task according to a starting processing time point and an expected processing time of the first processing task;
alternatively, the first and second electrodes may be,
aiming at a non-first processing task in a task sequence, determining an initial processing time point of the non-first processing task according to a serial number of the non-first processing task, a serial number of the first processing task and an initial processing time point of the first processing task in the task sequence; and determining the predicted ending processing time point of the non-first processing task according to the starting processing time point of the non-first processing task and the predicted processing time.
Wherein the content of the first and second substances, the type of processing task may be, for example, a download task type, a decode task type, or a render task type. Different types of processing tasks, having different expected processing times. The predicted processing time of a certain type of processing task is processing time determined after processing time statistics and analysis are performed on a plurality of processing tasks of the type.
In the embodiment of the present disclosure, a difference value between starting processing time points of two adjacent processing tasks in the task sequence is fixed, and therefore, for a non-first processing task in the task sequence, a sequence number of the non-first processing task and a sequence number of the first processing task are determined, and a difference value between the sequence number of the non-first processing task and the starting processing time point between the first processing tasks can be determined; in combination with the starting processing time point difference and the starting processing time point of the first processing task, the starting processing time point of the non-first processing task can be determined.
In step 103, in the execution process of the plurality of processing tasks, when the cleaning time point is reached, cleaning a target processing task and corresponding task execution data, wherein the target processing task is a processing task whose corresponding predicted ending processing time point is not later than the cleaning time point.
In the embodiment of the present disclosure, the number of the cleaning time points may be multiple, and the time difference value between adjacent cleaning time points may be smaller than a preset time difference value threshold; alternatively, the cleaning time point may be an expected end processing time point of each processing task.
In the embodiment of the present disclosure, in the execution process of a plurality of processing tasks, before reaching the cleaning time point, if a certain processing task is processed before the corresponding predicted ending processing time point, or the processing task is processed before the cleaning time point, according to the current processing mechanism, the processing result of the processing task is transferred to the next task node, and the processing task and the corresponding task execution data are cleaned. Thus, the target processing task to be actually cleaned up when the cleaning time point is reached may be a processing task whose corresponding predicted end processing time point is not later than the cleaning time point and has not yet been processed at the cleaning time point.
In an embodiment of the present disclosure, the method further includes: in the execution of the plurality of processing tasks, the execution of the processing task is stopped when the processing time point of the predicted end of the processing task is reached for each processing task. When the predicted processing ending time point of the processing task is reached, the execution process of the processing task is stopped, so that resources can be saved for processing other processing tasks, and the processing delay of other processing tasks can be reduced. In this case, the target processing task to be actually cleaned up when the cleaning time point is reached may be a processing task whose corresponding scheduled end processing time point is not later than the cleaning time point and whose completion is not processed at the corresponding scheduled end processing time point.
The present disclosure obtains a task sequence, wherein the task sequence includes: processing tasks of a plurality of data in the data stream; determining predicted ending processing time points of the plurality of processing tasks according to the starting processing time points and the predicted processing time of the plurality of processing tasks; in the execution process of the multiple processing tasks, when the cleaning time point is reached, cleaning a target processing task and corresponding task execution data, wherein the target processing task is a processing task of which the corresponding predicted ending processing time point is not later than the cleaning time point, so that the processing task which is not finished at the predicted ending processing time point and the corresponding task execution data can be timely cleaned in the execution process of the multiple processing tasks, the subsequent processing tasks can be timely executed, the execution speed of the processing tasks can be improved, and the processing delay of the processing tasks can be reduced.
As a possible implementation manner, as shown in fig. 3, in the case that the data stream is a video stream, the plurality of data in the data stream refers to a plurality of images in the video stream; the processing task of a plurality of data may refer to a downloading task of a plurality of images. The data stream processing method provided by the embodiment includes the following steps:
in step 301, a download task sequence is obtained, where the download task sequence includes: and downloading a plurality of images in the video stream.
In the embodiment of the present disclosure, the image in the video stream is an image in a non-panoramic scene, or an image block in a partial field angle in a panoramic scene.
In the embodiment of the present disclosure, it should be further noted that, since the image information of each frame of image in the video stream is the variation information of the frame of image with respect to the corresponding reference image, the video stream needs to include an integer number of groups of pictures. Wherein each group of pictures contains an intra-coded reference picture I having complete picture information.
In addition, in order to reduce the processing amount in decoding the video stream, the difference between the sequence numbers of the partial images in the video stream between the first sequence number of the partial image and the second sequence number of the reference image corresponding to the partial image is greater than a preset value. That is, the number of reference pictures to which each frame of picture refers is reduced.
In step 302, predicted end processing time points of the plurality of download tasks are determined according to the start processing time points and the predicted processing times of the plurality of download tasks.
In the embodiment of the present disclosure, the manner of determining the predicted ending processing time points of the plurality of downloading tasks may refer to the manner of determining the predicted ending processing time points of the processing tasks in the embodiment shown in fig. 1, and will not be described in detail here.
In step 303, in the execution process of the plurality of download tasks, before the cleaning time point is reached, when the execution of the first download task is completed, the execution result of the first download task is transferred to the next processing node, and the first download task and the corresponding task execution data are cleaned.
The task execution data corresponding to the first downloading task comprises: the execution result and temporary data in the execution process.
In step 304, in the execution process of the plurality of download tasks, when the cleaning time point is reached, cleaning the target download task and the corresponding task execution data, wherein the target download task is a download task whose corresponding predicted ending processing time point is not later than the cleaning time point.
The present disclosure obtains a download task sequence, where the download task sequence includes: downloading tasks of a plurality of images in a video stream; determining predicted ending processing time points of the plurality of downloading tasks according to the starting processing time points and the predicted processing time of the plurality of downloading tasks; in the execution process of a plurality of downloading tasks, before the cleaning time point is reached, when the execution of the first downloading task is finished, the execution result of the first downloading task is transmitted to the next processing node, and the first downloading task and the corresponding task execution data are cleaned; when the cleaning time point is reached, the target downloading task and the corresponding task execution data are cleaned, wherein the target downloading task is the downloading task of which the corresponding predicted ending processing time point is not later than the cleaning time point, so that the downloading task which is not finished at the predicted ending processing time point and the corresponding task execution data can be cleaned in time in the execution process of a plurality of downloading tasks, the subsequent downloading tasks can be executed in time, the execution speed of the downloading tasks can be improved, and the processing delay of the downloading tasks is reduced.
As a possible implementation manner, as shown in fig. 4, in the case that the data stream is a video stream downloaded at the download node, the plurality of data in the data stream refers to a plurality of images in the video stream; the processing task of the plurality of data may refer to a decoding task of the plurality of pictures. The data stream processing method provided by the embodiment includes the following steps:
in step 401, a decoding task sequence is obtained, where the decoding task sequence includes: and downloading a plurality of decoding tasks of the images in the obtained video stream.
In the embodiment of the present disclosure, the image in the video stream is an image in a non-panoramic scene, or an image block in a partial field angle in a panoramic scene.
In the embodiment of the present disclosure, it should be further noted that, since the image information of each frame of image in the video stream is the variation information of the frame of image with respect to the corresponding reference image, the video stream needs to include an integer number of groups of pictures. Wherein each group of pictures contains an intra-coded reference picture I having complete picture information.
In addition, in order to reduce the processing amount in decoding the video stream, the difference between the sequence numbers of the partial images in the video stream between the first sequence number of the partial image and the second sequence number of the reference image corresponding to the partial image is greater than a preset value. That is, the number of reference pictures to which each frame of picture refers is reduced.
In step 402, predicted end processing time points of the plurality of decoding tasks are determined according to the start processing time points and the predicted processing times of the plurality of decoding tasks.
In the embodiment of the present disclosure, the manner of determining the predicted ending processing time points of the plurality of decoding tasks may refer to the manner of determining the predicted ending processing time points of the processing tasks in the embodiment shown in fig. 1, and will not be described in detail here.
In step 403, in the execution process of the multiple decoding tasks, before the cleaning time point is reached, when the execution of the first decoding task is completed, the execution result of the first decoding task is transmitted to the next processing node, and the first decoding task and the corresponding task execution data are cleaned.
Wherein, the task execution data corresponding to the first decoding task comprises: the execution result and temporary data in the execution process.
In step 404, in the execution process of the plurality of decoding tasks, when the clearing time point is reached, clearing the target decoding task and the corresponding task execution data, wherein the target decoding task is a decoding task whose corresponding predicted ending processing time point is not later than the clearing time point.
The present disclosure provides for obtaining a sequence of decoding tasks, wherein the decoding task sequence comprises: decoding tasks of a plurality of images in a video stream; determining predicted ending processing time points of the plurality of decoding tasks according to the starting processing time points and the predicted processing time of the plurality of decoding tasks; in the execution process of a plurality of decoding tasks, before the cleaning time point is reached, when the execution of a first decoding task is completed, the execution result of the first decoding task is transmitted to a next processing node, and the first decoding task and the corresponding task execution data are cleaned; when the clearing time point is reached, clearing the target decoding task and the corresponding task execution data, wherein the target decoding task is a decoding task of which the corresponding predicted ending processing time point is not later than the clearing time point, so that the decoding task which is not finished at the predicted ending processing time point and the corresponding task execution data can be cleared away in time in the execution process of a plurality of decoding tasks, the subsequent decoding tasks can be executed in time, the execution speed of the decoding tasks can be improved, and the processing delay of the decoding tasks is reduced.
As a possible implementation manner, as shown in fig. 5, in the case that the data stream is decoded by the decoding node, the processing task of the plurality of data may refer to a rendering task of the plurality of data. The data stream processing method provided by the embodiment includes the following steps:
in step 501, a rendering task sequence is obtained, where the rendering task sequence includes: and decoding a plurality of data rendering tasks in the obtained data stream.
In step 502, predicted end processing time points of the plurality of rendering tasks are determined according to the start processing time points and the predicted processing times of the plurality of rendering tasks.
In the embodiment of the present disclosure, the manner of determining the processing time point for the predicted end of the plurality of rendering tasks may refer to the manner of determining the processing time point for the predicted end of the processing task in the embodiment shown in fig. 1, and will not be described in detail here.
In step 503, in the execution process of the rendering tasks, before the cleaning time point is reached, when the execution of the first rendering task is completed, the execution result of the first rendering task is transmitted to the next processing node, and the first rendering task and the corresponding task execution data are cleaned.
Wherein the task execution data corresponding to the first rendering task comprises: the execution result and temporary data in the execution process.
In step 504, during the execution of the plurality of rendering tasks, when the cleaning time point is reached, cleaning a target rendering task and corresponding task execution data, wherein the target rendering task is a rendering task whose corresponding predicted ending processing time point is not later than the cleaning time point.
The present disclosure obtains a sequence of rendering tasks by obtaining, wherein the rendering task sequence comprises: decoding rendering tasks of a plurality of data in the obtained data stream; determining predicted ending processing time points of the rendering tasks according to the starting processing time points and the predicted processing time of the rendering tasks; in the execution process of a plurality of rendering tasks, before the cleaning time point is reached, when the execution of a first rendering task is completed, transmitting the execution result of the first rendering task to a next processing node, and cleaning the first rendering task and the corresponding task execution data; when the cleaning time point is reached, the target rendering tasks and the corresponding task execution data are cleaned, wherein the target rendering tasks are rendering tasks of which the corresponding predicted ending processing time point is not later than the cleaning time point, so that the rendering tasks which are not processed at the predicted ending processing time point and the corresponding task execution data can be cleaned in time in the execution process of the plurality of rendering tasks, the subsequent rendering tasks can be executed in time, the execution speed of the rendering tasks can be improved, and the processing delay of the rendering tasks is reduced.
As a possible implementation manner, as shown in fig. 6, in the case that the data stream is a data stream rendered at the rendering node, the processing task of the plurality of data may refer to a playing task of the plurality of data. The data stream processing method provided by the embodiment includes the following steps:
in step 601, a play task sequence is obtained, where the play task sequence includes: and rendering the plurality of data in the obtained data stream.
In step 602, predicted end processing time points of the plurality of playback tasks are determined according to the start processing time points and the predicted processing times of the plurality of playback tasks.
In the embodiment of the present disclosure, the manner of determining the processing time points of the predicted ending of the multiple play tasks may refer to the manner of determining the processing time points of the predicted ending of the processing tasks in the embodiment shown in fig. 1, and will not be described in detail here.
In step 603, during the execution of the plurality of playback tasks, before the cleaning time point is reached, the first playback task and the corresponding task execution data are cleaned when the first playback task is completely executed.
The task execution data corresponding to the first playing task comprises: the execution result and temporary data in the execution process.
In step 604, in the execution process of the plurality of playback tasks, when the cleaning time point is reached, cleaning the target playback task and the corresponding task execution data, wherein the target playback task is a playback task whose corresponding predicted ending processing time point is not later than the cleaning time point.
The present disclosure obtains a play task sequence, where the play task sequence includes: rendering a plurality of data playing tasks in the obtained data stream; determining predicted ending processing time points of the plurality of playing tasks according to the starting processing time points and the predicted processing time of the plurality of playing tasks; in the execution process of a plurality of rendering tasks, before the cleaning time point is reached, when the execution of the first playing task is completed, cleaning the first playing task and the corresponding task execution data; when the cleaning time point is reached, the target playing task and the corresponding task execution data are cleaned, wherein the target playing task is the playing task of which the corresponding predicted ending processing time point is not later than the cleaning time point, so that the playing tasks which are not processed at the predicted ending processing time point and the corresponding task execution data can be cleaned in time in the execution process of a plurality of playing tasks, the subsequent playing tasks can be executed in time, the execution speed of the playing tasks can be improved, and the processing delay of the playing tasks is reduced.
Fig. 7 is a block diagram illustrating a data stream processing apparatus according to an example embodiment.
As shown in fig. 7, the data stream processing apparatus 1000, the apparatus 1000 includes an obtaining module 110, a determining module 120, and a processing module 130. The obtaining module 110 is configured to obtain a task sequence, where the task sequence includes: processing tasks of a plurality of data in the data stream;
a determining module 120, configured to determine predicted ending processing time points of a plurality of processing tasks according to starting processing time points and predicted processing times of the plurality of processing tasks;
a processing module 130, configured to, in the execution process of the multiple processing tasks, when a cleaning time point is reached, clean a target processing task and a corresponding task execution data, where the target processing task is a processing task whose corresponding predicted ending processing time point is not later than the cleaning time point.
In an embodiment of the present disclosure, the determining module 120 is specifically configured to,
aiming at a first processing task in the task sequence, determining an estimated ending processing time point of the first processing task according to the initial processing time point and the estimated processing time of the first processing task;
alternatively, the first and second electrodes may be,
aiming at a non-first processing task in the task sequence, determining an initial processing time point of the non-first processing task according to a serial number of the non-first processing task, the serial number of the first processing task and the initial processing time point of the first processing task in the task sequence;
and determining the predicted ending processing time point of the non-first processing task according to the starting processing time point and the predicted processing time of the non-first processing task.
In an embodiment of the present disclosure, the processing module 130 is further configured to,
and stopping the execution process of the processing task when the predicted ending processing time point of the processing task is reached for each processing task in the execution process of the plurality of processing tasks.
In the embodiment of the present disclosure, the data stream is a video stream, or a data stream obtained by processing the video stream at a preset task node;
the images in the video stream are images in a non-panoramic scene, or image blocks in a partial field angle in a panoramic scene.
In an embodiment of the present disclosure, the preset task node is at least one of the following task nodes: downloading task nodes, decoding task nodes and rendering task nodes.
In an embodiment of the present disclosure, for a partial image in the video stream, a difference value of sequence numbers between a first sequence number of the partial image and a second sequence number of a reference image corresponding to the partial image is greater than a preset value.
The present disclosure obtains a task sequence, wherein the task sequence includes: processing tasks of a plurality of data in the data stream; determining predicted ending processing time points of the plurality of processing tasks according to the starting processing time points and the predicted processing time of the plurality of processing tasks; in the execution process of the multiple processing tasks, when the cleaning time point is reached, cleaning a target processing task and corresponding task execution data, wherein the target processing task is a processing task of which the corresponding predicted ending processing time point is not later than the cleaning time point, so that the processing task which is not finished at the predicted ending processing time point and the corresponding task execution data can be timely cleaned in the execution process of the multiple processing tasks, the subsequent processing tasks can be timely executed, the execution speed of the processing tasks can be improved, and the processing delay of the processing tasks can be reduced.
In order to implement the foregoing embodiment, the present disclosure further provides an electronic device, as shown in fig. 8, where the electronic device 8000 includes: a processor 801; one or more memories 802 for storing instructions executable by the processor 801; wherein the processor 801 is configured to execute the data stream processing method according to the above embodiment. The processor 801 and the memory 802 are connected by a communication bus.
To implement the above embodiments, the present disclosure also provides a computer-readable storage medium, and when instructions in the computer-readable storage medium are executed by the processor 801 of the electronic device 8000, the electronic device 8000 is enabled to execute to complete the data stream processing method described in the above embodiments. Alternatively, the storage medium may be a non-transitory computer readable storage medium, which may be, for example, a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
In order to implement the foregoing embodiments, the present disclosure further provides a computer program product including a computer program, where the computer program is executed by a processor to implement the data stream processing method described in the foregoing embodiments.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims (10)
1. A method for processing a data stream, comprising:
acquiring a task sequence, wherein the task sequence comprises: processing tasks of a plurality of data in the data stream;
determining predicted ending processing time points of the processing tasks according to the starting processing time points and the predicted processing time of the processing tasks;
in the execution process of a plurality of processing tasks, when a cleaning time point is reached, cleaning a target processing task and corresponding task execution data, wherein the target processing task is a processing task of which the corresponding predicted ending processing time point is not later than the cleaning time point.
2. The method of claim 1, wherein determining an expected end processing time point for a plurality of the processing tasks based on a starting processing time point and an expected processing time for the plurality of the processing tasks comprises:
aiming at a first processing task in the task sequence, determining an estimated ending processing time point of the first processing task according to the initial processing time point and the estimated processing time of the first processing task;
alternatively, the first and second electrodes may be,
aiming at a non-first processing task in the task sequence, determining an initial processing time point of the non-first processing task according to a serial number of the non-first processing task, the serial number of the first processing task and the initial processing time point of the first processing task in the task sequence;
and determining the predicted ending processing time point of the non-first processing task according to the starting processing time point and the predicted processing time of the non-first processing task.
3. The method of claim 1, further comprising:
and stopping the execution process of the processing task when the predicted ending processing time point of the processing task is reached for each processing task in the execution process of the plurality of processing tasks.
4. The method according to claim 1, wherein the data stream is a video stream, or a data stream obtained by processing the video stream at a preset task node;
the images in the video stream are images in a non-panoramic scene, or image blocks in a partial field angle in a panoramic scene.
5. The method according to claim 4, wherein the preset task node is at least one of the following task nodes: downloading task nodes, decoding task nodes and rendering task nodes.
6. The method according to claim 4, wherein for a partial image in the video stream, a difference between sequence numbers of a first sequence number of the partial image and a second sequence number of a reference image corresponding to the partial image is greater than a preset value.
7. A data stream processing apparatus, comprising:
an obtaining module, configured to obtain a task sequence, where the task sequence includes: processing tasks of a plurality of data in the data stream;
the determining module is used for determining the predicted ending processing time points of the processing tasks according to the starting processing time points and the predicted processing time of the processing tasks;
and the processing module is used for cleaning a target processing task and corresponding task execution data when a cleaning time point is reached in the execution process of the plurality of processing tasks, wherein the target processing task is a processing task of which the corresponding predicted ending processing time point is not later than the cleaning time point.
8. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.
9. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-6.
10. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110555814.1A CN115396691A (en) | 2021-05-21 | 2021-05-21 | Data stream processing method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110555814.1A CN115396691A (en) | 2021-05-21 | 2021-05-21 | Data stream processing method and device and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115396691A true CN115396691A (en) | 2022-11-25 |
Family
ID=84113868
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110555814.1A Pending CN115396691A (en) | 2021-05-21 | 2021-05-21 | Data stream processing method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115396691A (en) |
Citations (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2004104704A (en) * | 2002-09-12 | 2004-04-02 | Toshiba Solutions Corp | Video reproducing apparatus, video reproducing method, and program |
| CN1684512A (en) * | 2004-04-12 | 2005-10-19 | 联想(北京)有限公司 | Recording method based on set top box |
| JP2008129709A (en) * | 2006-11-17 | 2008-06-05 | Shinko Electric Ind Co Ltd | Task control method, data processing system, task control program, and its storage medium |
| CN102279759A (en) * | 2010-06-11 | 2011-12-14 | 任天堂株式会社 | Information processing terminal, information processing system, and information processing method |
| US20120066683A1 (en) * | 2010-09-09 | 2012-03-15 | Srinath Nadig S | Balanced thread creation and task allocation |
| JP2012142988A (en) * | 2006-09-15 | 2012-07-26 | Jvc Kenwood Corp | Digital broadcast receiving device and image information display method in program guide |
| US20150089509A1 (en) * | 2013-09-26 | 2015-03-26 | International Business Machines Corporation | Data processing resource management |
| CN107066398A (en) * | 2016-01-12 | 2017-08-18 | 船井电机株式会社 | Dispensing device and massaging device |
| CN107102894A (en) * | 2017-04-07 | 2017-08-29 | 百度在线网络技术（北京）有限公司 | Method for scheduling task, device and system |
| CN107634928A (en) * | 2016-07-18 | 2018-01-26 | 华为技术有限公司 | A kind of processing method and processing device of bit stream data |
| CN108574851A (en) * | 2017-12-11 | 2018-09-25 | 北京金山云网络技术有限公司 | Method for pushing, device, streaming media server and computer readable storage medium |
| US20190220201A1 (en) * | 2018-01-18 | 2019-07-18 | EMC IP Holding Company LLC | Methods, apparatuses, and computer program products for controlling write requests in storage system |
| CN110992993A (en) * | 2019-12-17 | 2020-04-10 | Oppo广东移动通信有限公司 | Video editing method, video editing device, terminal and readable storage medium |
| JP2020113922A (en) * | 2019-01-15 | 2020-07-27 | キヤノン株式会社 | Video distribution device, video distribution method, and program |
| CN111708625A (en) * | 2020-06-16 | 2020-09-25 | 中国建设银行股份有限公司 | Multi-user task execution method, device, equipment and storage medium |
| CN112131280A (en) * | 2020-09-29 | 2020-12-25 | 杭州东方通信软件技术有限公司 | Data acquisition method based on time scheduling |
| CN112148493A (en) * | 2020-09-30 | 2020-12-29 | 武汉中科通达高新技术股份有限公司 | Streaming media task management method and device and data server |
| CN112231087A (en) * | 2020-10-27 | 2021-01-15 | 中国平安人寿保险股份有限公司 | Task scheduling method, device, equipment and storage medium |
| CN112379982A (en) * | 2020-11-12 | 2021-02-19 | 北京字跳网络技术有限公司 | Task processing method and device, electronic equipment and computer readable storage medium |
| WO2021068617A1 (en) * | 2019-10-12 | 2021-04-15 | 平安国际智慧城市科技股份有限公司 | Method and apparatus for automatically predicting task processing time, electronic device and medium |
| CN112702625A (en) * | 2020-12-23 | 2021-04-23 | Oppo广东移动通信有限公司 | Video processing method and device, electronic equipment and storage medium |
-
2021
- 2021-05-21 CN CN202110555814.1A patent/CN115396691A/en active Pending
Patent Citations (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2004104704A (en) * | 2002-09-12 | 2004-04-02 | Toshiba Solutions Corp | Video reproducing apparatus, video reproducing method, and program |
| CN1684512A (en) * | 2004-04-12 | 2005-10-19 | 联想(北京)有限公司 | Recording method based on set top box |
| JP2012142988A (en) * | 2006-09-15 | 2012-07-26 | Jvc Kenwood Corp | Digital broadcast receiving device and image information display method in program guide |
| JP2008129709A (en) * | 2006-11-17 | 2008-06-05 | Shinko Electric Ind Co Ltd | Task control method, data processing system, task control program, and its storage medium |
| CN102279759A (en) * | 2010-06-11 | 2011-12-14 | 任天堂株式会社 | Information processing terminal, information processing system, and information processing method |
| US20120066683A1 (en) * | 2010-09-09 | 2012-03-15 | Srinath Nadig S | Balanced thread creation and task allocation |
| US20150089509A1 (en) * | 2013-09-26 | 2015-03-26 | International Business Machines Corporation | Data processing resource management |
| CN107066398A (en) * | 2016-01-12 | 2017-08-18 | 船井电机株式会社 | Dispensing device and massaging device |
| CN107634928A (en) * | 2016-07-18 | 2018-01-26 | 华为技术有限公司 | A kind of processing method and processing device of bit stream data |
| CN107102894A (en) * | 2017-04-07 | 2017-08-29 | 百度在线网络技术（北京）有限公司 | Method for scheduling task, device and system |
| CN108574851A (en) * | 2017-12-11 | 2018-09-25 | 北京金山云网络技术有限公司 | Method for pushing, device, streaming media server and computer readable storage medium |
| US20190220201A1 (en) * | 2018-01-18 | 2019-07-18 | EMC IP Holding Company LLC | Methods, apparatuses, and computer program products for controlling write requests in storage system |
| JP2020113922A (en) * | 2019-01-15 | 2020-07-27 | キヤノン株式会社 | Video distribution device, video distribution method, and program |
| WO2021068617A1 (en) * | 2019-10-12 | 2021-04-15 | 平安国际智慧城市科技股份有限公司 | Method and apparatus for automatically predicting task processing time, electronic device and medium |
| CN110992993A (en) * | 2019-12-17 | 2020-04-10 | Oppo广东移动通信有限公司 | Video editing method, video editing device, terminal and readable storage medium |
| CN111708625A (en) * | 2020-06-16 | 2020-09-25 | 中国建设银行股份有限公司 | Multi-user task execution method, device, equipment and storage medium |
| CN112131280A (en) * | 2020-09-29 | 2020-12-25 | 杭州东方通信软件技术有限公司 | Data acquisition method based on time scheduling |
| CN112148493A (en) * | 2020-09-30 | 2020-12-29 | 武汉中科通达高新技术股份有限公司 | Streaming media task management method and device and data server |
| CN112231087A (en) * | 2020-10-27 | 2021-01-15 | 中国平安人寿保险股份有限公司 | Task scheduling method, device, equipment and storage medium |
| CN112379982A (en) * | 2020-11-12 | 2021-02-19 | 北京字跳网络技术有限公司 | Task processing method and device, electronic equipment and computer readable storage medium |
| CN112702625A (en) * | 2020-12-23 | 2021-04-23 | Oppo广东移动通信有限公司 | Video processing method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110557670B (en) | Method, device, terminal and storage medium for playing video in webpage | |
| EP1803095B1 (en) | A method and apparatus for reading film grain patterns in a raster order in film grain simulation | |
| CN111277895B (en) | Video frame interpolation method and device | |
| CN111447455A (en) | Live video stream playback processing method and device and computing equipment | |
| CN110996126A (en) | Video streaming method, device, client device and computer readable medium | |
| CN111225171B (en) | Video recording method, device, terminal equipment and computer storage medium | |
| CN110267100A (en) | Code rate switching method, device, electronic equipment and the storage medium of FLV video | |
| CN109218745B (en) | Live broadcast method, server, client and readable storage medium | |
| EP3249652A1 (en) | Method and apparatus for playing back recorded video | |
| CN108471548B (en) | Live video quick playing method and device | |
| CN111131874B (en) | Method, equipment and computer storage medium for solving problem of playing jam of H.265 code stream random access point | |
| CN107872669A (en) | Video code rate treating method and apparatus | |
| EP3648059A1 (en) | Video processing device and method for determining motion metadata for an encoded video | |
| CN111726657A (en) | Live video playing processing method and device and server | |
| JP2012070190A (en) | Encoding method, decoding method, encoder, and decoder | |
| CN109120995B (en) | Video cache analysis method, device, equipment and medium | |
| CN108924574B (en) | Packet loss processing method, device, equipment and storage medium in recording and broadcasting system | |
| CN111010603A (en) | Video caching and forwarding processing method and device | |
| CN107613302B (en) | Decoding method and device, storage medium and processor | |
| US20120106632A1 (en) | Method and apparatus for error resilient long term referencing block refresh | |
| CN113259779B (en) | Video processing method, device, equipment and storage medium | |
| CN115396691A (en) | Data stream processing method and device and electronic equipment | |
| CN109144613B (en) | Android definition switching method and device, terminal and readable medium | |
| US10992946B2 (en) | Coding of video and audio with initialization fragments | |
| CN115086472B (en) | Mobile phone APP management system based on key frame information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |