CN115174578B - Kubernetes-based on-demand parallel transcoding method, device and storage medium - Google Patents
Kubernetes-based on-demand parallel transcoding method, device and storage medium Download PDFInfo
- Publication number
- CN115174578B CN115174578B CN202210884368.3A CN202210884368A CN115174578B CN 115174578 B CN115174578 B CN 115174578B CN 202210884368 A CN202210884368 A CN 202210884368A CN 115174578 B CN115174578 B CN 115174578B
- Authority
- CN
- China
- Prior art keywords
- virtual
- kubernetes
- transcoding
- slice
- file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
Abstract
The invention discloses a Kubernetes-based on-demand parallel transcoding method, which comprises the following steps of S1: acquiring cluster Node information through an API of Kubernetes; s2: receiving a source file for scanning analysis, and obtaining the type and the total duration of the source file; s3: calculating the slicing time length of the pre-slice based on the cluster Node information and the total time length of the source file; s4: sequentially slicing the source file according to the slicing time length to sequentially obtain a plurality of virtual slices and corresponding virtual slice information, and synchronously transcoding the virtual slices according to the virtual slice information; s5: and splicing the plurality of virtual slices after synchronous transcoding in sequence to generate a video, thereby obtaining the target file. The invention shortens the time consumed by the virtual slice by using the Kubernetes transcoding system, improves the utilization rate of cluster resources, accelerates the transcoding process and efficiency of audio and video files, and provides a steady and efficient distributed file transcoding system for clients.
Description
Technical Field
The invention belongs to the technical field of multimedia, and particularly relates to a Kubernetes-based on-demand parallel transcoding method, a Kubernetes-based on-demand parallel transcoding device and a storage medium.
Background
In recent years, the 5G technology has been widely used in commerce, and high-resolution and high-frame-rate videos such as 4K and 8K are becoming mainstream, and long, medium and short video platforms in China are developing audio and video coding and transcoding technologies continuously, so as to improve the quality of terminal videos and the multimedia transcoding efficiency, thereby processing tens of thousands of daily massive videos and actively meeting the watching demands of clients.
At present, the on-demand transcoding technology is quite mature in application, and continues to merge new coding standards, such as an AV1 coding format with higher compression efficiency, while supporting the existing audio and video coding standards. On-demand transcoding essentially involves the efficiency of file transcoding by passing an input source file through a decode-then-encode process and finally outputting a new stream file. In order to solve the problem, a physical slice transcoding method appears firstly, namely, a physical slice is firstly carried out on a file, and the slices are respectively transcoded and then output in a combined way; then, a virtual dotting and slicing method is presented, namely, an actual key frame is searched through a preset cutting point, the information of the cutting point is recorded in an index file, and a source file is transcoded and output according to the information of the index file.
The above methods all improve the transcoding efficiency to a certain extent, but in practical use, if the source file is a high-resolution and high-frame-rate video, and the duration is long, the following problems will be encountered if a transcoding method of virtual slices is used:
(1) The time length of each section of slice is to search a corresponding key frame according to the preset cutting time length to record the cutting point information. However, the load conditions of the resources of different cluster machines are different, the unified preset cutting time length cannot well utilize the resources of each Node, and the transcoding efficiency is also discounted;
(2) The essence of the virtual slice is to decode the source file, the process involves searching key frames, for the video with 4K and 8K and time length longer than 1 hour, the data analysis amount is relatively large, if the cut point index of the source video with primary resolution of 8K, frame rate of 50fps and time length of 1 hour is completed, more than 15 minutes is needed, and then the video can be transcoded and combined according to the index file information and output. Thus, there is an initial intention to raise the efficiency of virtual slicing.
Disclosure of Invention
The invention aims to provide a Kubernetes-based on-demand parallel transcoding method, a Kubernetes-based on-demand parallel transcoding device and a Kubernetes-based storage medium, so as to solve the problem of low transcoding efficiency in the prior art.
In order to solve the problems, the technical scheme of the invention is as follows:
an on-demand parallel transcoding method based on Kubernetes comprises the following steps of
S1: acquiring cluster Node information through an API of Kubernetes;
s2: receiving a source file for scanning analysis, and obtaining the type and the total duration of the source file;
s3: calculating the slicing time length of the pre-slice based on the cluster Node information and the total time length of the source file;
s4: sequentially slicing the source file according to the slicing time length to sequentially obtain a plurality of virtual slices and corresponding virtual slice information, and synchronously transcoding the virtual slices according to the virtual slice information;
s5: and splicing the plurality of virtual slices after synchronous transcoding in sequence to generate a video, thereby obtaining the target file.
The cluster Node information is load information of cluster Nodes, and the cluster Node information comprises pods, services, replicationcontrollers running in the Node.
The source file is a video file, if an audio file exists, the audio file needs to be separated and removed, and the audio file is incorporated into the target file in step S5.
Further preferably, step S4 further comprises the steps of:
s41: reading in a source file after a preset interval time is reserved;
s42: sequentially searching the end of the I frame backwards and cutting to obtain a virtual slice of the first section, taking the first frame of the virtual slice as a key frame, and writing the position of the virtual slice and the key frame as virtual slice information into redis;
s43: and repeating the step S42 for the rest source files until all the source files are cut.
Further preferably, step S4 further comprises the steps of:
s44: after the schedulers of Kubernetes acquire the positions of the virtual slices and the corresponding key frames, the scheduling resources create Pod and transcode the corresponding virtual slices, wherein each Pod corresponds to each virtual slice.
Specifically, step S43 and step S44 are synchronized with each other, and step S44 is performed immediately after each virtual slice information write in redis is received by schedulers of Kubernetes.
The on-demand parallel transcoding device based on the Kubernetes comprises a memory and a processor, wherein the memory stores computer instructions capable of being run on the processor, and the processor executes the steps of any one of the methods when executing the computer instructions.
A computer readable storage medium having stored thereon computer instructions which, when executed, perform the steps of any of the methods described above.
By adopting the technical scheme, the invention has the following advantages and positive effects compared with the prior art:
the invention shortens the time consumed by the virtual slice by using the Kubernetes transcoding system, improves the utilization rate of cluster resources, accelerates the transcoding process and efficiency of audio and video files, and provides a steady and efficient distributed file transcoding system for clients.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention.
Fig. 1 is a schematic flow chart of a Kubernetes-based on-demand parallel transcoding method of the present invention;
FIG. 2 is a block diagram of a Kubernetes architecture according to an embodiment of the present invention;
fig. 3 is a block diagram of transcoding software according to an embodiment of the present invention.
Detailed Description
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following description will explain the specific embodiments of the present invention with reference to the accompanying drawings. It is evident that the drawings in the following description are only examples of the invention, from which other drawings and other embodiments can be obtained by a person skilled in the art without inventive effort.
For the sake of simplicity of the drawing, the parts relevant to the present invention are shown only schematically in the figures, which do not represent the actual structure thereof as a product. Additionally, in order to simplify the drawing for ease of understanding, components having the same structure or function in some of the drawings are shown schematically with only one of them, or only one of them is labeled. Herein, "a" means not only "only this one" but also "more than one" case.
The invention provides a method, a device and a storage medium for on-demand parallel transcoding based on Kubernetes, which are further described in detail below with reference to the accompanying drawings and specific embodiments. Advantages and features of the invention will become more apparent from the following description and from the claims.
Example 1
Referring to fig. 1 to 3, in this embodiment, a Kubernetes-based on-demand parallel transcoding method uses a Kubernetes cluster scheduling management technology to obtain resource load information of a Node, combines the total duration of a source video file, calculates an optimal virtual slice time interval, obtains a slice time period by searching a key frame, and simultaneously allocates a Pod for synchronous transcoding, and performs Pod recovery by the Kubernetes after completing transcoding, thereby realizing an efficient on-demand parallel transcoding technology of transcoding while slicing.
First, referring to fig. 1 and 2, in step S1, cluster Node information is acquired through an API service of Kubernetes. The cluster Node information is load information of cluster Nodes, the cluster Node information comprises pods, services, replicationcontrollers running in the Node, and then the load information of the cluster Node is transmitted to a task receiving module in transcoding software.
Then, referring to fig. 1 and 3, in step S2, after receiving the source file, the transcoding software starts scanning analysis on the audio/video file, and obtains the type and the total duration of the source file. Since this embodiment is directed to video files only, if audio files exist at the same time, the audio files need to be separated and removed first, and the audio files are transcoded separately.
Next, in step S3, the slicing duration of the pre-slice is calculated according to the cluster Node load information acquired in step S1 and the total duration of the video file acquired in step S2. Assuming that a Node exists in the cluster, the number of idle CPUs is 32, the number of idle memories is 50G, each transcoding Pod default resource request is configured to be 14 CPU memories and 20G memories, the duration of video files in a transcoding task can be known to be 120 seconds through scanning, and the virtual segmentation duration is obtained through calculation to be 60 seconds.
Then, in step S4, the slicing time length of the virtual slice is set, so that in order to ensure the slicing accuracy of the video file, the video file is not necessarily sliced according to the absolute slicing time length, but the first frame of each virtual slice must be ensured to be a key frame.
The virtual slicing method specifically comprises the following steps of firstly reading in a video file after a preset interval time is set. Assuming that the duration of the video file is T, the duration of the virtual slice is T n The first segment of the virtual slice starts at t 0 =0, sequentially searching the end of the I frame backwards and cutting to obtain a virtual slice of the first section, and taking the first frame of the virtual slice as a key frame, wherein the position of the first key frame is t 1 The segmentation of the first segment of virtual slice is completed. And then the position information of the first section of virtual slice and the key frame are used as virtual slice information to be written into redis. And then repeating the cutting mode as the first section for the rest video files until all the cutting is finished. And when the software cutting is converted, the schedulers of the Kubernetes can acquire virtual slice information at the same time, schedule resources to create Pod and transcode corresponding virtual slices, wherein each Pod corresponds to each virtual slice. Therefore, in this embodiment, when each piece of virtual slice information is received by the Scheduler of Kubernetes after being written into redis, the transcoding is performed immediately, and compared with the prior art that the transcoding is performed after all the pieces of virtual slice information are cut, the transcoding efficiency is greatly improved.
Finally, in step S5, the virtual slices after synchronous transcoding are spliced in sequence in a sum-slice module, and are connected to generate a video, so as to obtain the target file required by us. Preferably, the previously separate individually transcoded audio file is incorporated into the target file.
By applying the transcoding method as shown in table 1, a part of 8K movie files with the size of 10 minutes and 18 seconds takes 776 seconds in the common transcoding method, while 538 seconds are taken in the parallel transcoding method in the embodiment method, so that the transcoding time is 248 seconds. If the large-traffic transcoding task is performed simultaneously, the method of the embodiment can effectively improve the transcoding speed.
Table 1 comparison of transcoding time
Example 2
The present embodiment provides a Kubernetes-based on-demand parallel transcoding device, including a memory and a processor, where the memory stores computer instructions executable on the processor, and the processor executes the steps of the method as in embodiment 1 when executing the computer instructions.
Example 3
A computer readable storage medium having stored thereon computer instructions which when run perform the steps of the method as in embodiment 1.
The embodiments of the present invention have been described in detail with reference to the drawings, but the present invention is not limited to the above embodiments. Even if various changes are made to the present invention, it is within the scope of the appended claims and their equivalents to fall within the scope of the invention.
Claims (5)
1. The on-demand parallel transcoding method based on the Kubernetes is characterized by comprising the following steps of
S1: acquiring cluster Node information through the API of the Kubernetes;
s2: receiving a source file for scanning analysis, and obtaining the type and the total duration of the source file;
s3: calculating the slice time length of the pre-slice based on the cluster Node information and the total time length of the source file;
s4: sequentially slicing the source file according to the slicing time length to sequentially obtain a plurality of virtual slices and corresponding virtual slice information, and synchronously transcoding the virtual slices according to the virtual slice information;
s5: sequentially splicing the plurality of virtual slices subjected to synchronous transcoding to generate a video, and obtaining a target file;
wherein, the step S4 further comprises the following steps:
s41: reading in the source file after a preset interval time is reserved;
s42: sequentially searching the end of the I frame backwards and cutting to obtain the virtual slice of the first section, taking the first frame of the virtual slice as a key frame, and writing the position of the virtual slice and the key frame as virtual slice information into redis;
s43: repeating the step S42 for the rest source files until all the source files are cut;
s44: scheduling resources to create Pod and transcode the corresponding virtual slice after the schedulers of the Kubernetes acquire the positions of the virtual slices and the corresponding key frames, wherein each Pod corresponds to each virtual slice;
the step S43 and the step S44 are synchronized with each other, and the step S44 is performed immediately after each of the virtual slice information writes in redis is received by the schedulers of the Kubernetes.
2. The Kubernetes-based on-demand parallel transcoding method of claim 1, wherein the cluster Node information is load information of cluster Nodes, and the cluster Node information includes pods, services, replicationcontrollers running in the Node.
3. The Kubernetes-based on-demand parallel transcoding method of claim 1, wherein the source file is a video file, and if an audio file exists, the audio file needs to be separated and removed, and the audio file is incorporated into the target file in step S5.
4. A Kubernetes-based on-demand parallel transcoding device comprising a memory and a processor, the memory having stored thereon computer instructions executable on the processor, wherein the processor executes the computer instructions to perform the steps of the method of any of claims 1-3.
5. A computer readable storage medium having stored thereon computer instructions, which when run perform the steps of the method of any of claims 1-3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210884368.3A CN115174578B (en) | 2022-07-25 | 2022-07-25 | Kubernetes-based on-demand parallel transcoding method, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210884368.3A CN115174578B (en) | 2022-07-25 | 2022-07-25 | Kubernetes-based on-demand parallel transcoding method, device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115174578A CN115174578A (en) | 2022-10-11 |
| CN115174578B true CN115174578B (en) | 2023-10-20 |
Family
ID=83496985
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210884368.3A Active CN115174578B (en) | 2022-07-25 | 2022-07-25 | Kubernetes-based on-demand parallel transcoding method, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115174578B (en) |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1434386A (en) * | 2003-01-25 | 2003-08-06 | 华中科技大学 | Division distributed storage method of program source based on cluster video server |
| CA2848455A1 (en) * | 2011-09-16 | 2013-03-21 | Tencent Technology (Shenzhen) Company Limited | Mobile multimedia real-time transcoding system, apparatus, storage medium and method |
| CN105227968A (en) * | 2014-06-30 | 2016-01-06 | 惠州市伟乐科技股份有限公司 | Based on the video code conversion accelerated method of section |
| CN105245880A (en) * | 2015-09-21 | 2016-01-13 | 北京暴风科技股份有限公司 | Video file cutting method and distributed transcoding method |
| WO2016011823A1 (en) * | 2014-07-22 | 2016-01-28 | 中兴通讯股份有限公司 | Method for acquiring live video slice, server, and storage medium |
| CN107370726A (en) * | 2017-06-22 | 2017-11-21 | 上海网达软件股份有限公司 | A kind of virtual sliced sheet method and system for disributed media file trans-coding system |
| CN110769278A (en) * | 2019-10-29 | 2020-02-07 | 鹏城实验室 | Distributed video transcoding method and system |
| CN110830805A (en) * | 2019-10-24 | 2020-02-21 | 上海网达软件股份有限公司 | Multi-resolution output distributed file transcoding method and device |
| CN112165622A (en) * | 2020-09-23 | 2021-01-01 | 上海网达软件股份有限公司 | Single-Pod multi-protocol video file transcoding method and system |
| WO2021254375A1 (en) * | 2020-06-18 | 2021-12-23 | 中兴通讯股份有限公司 | Video partitioning method, transfer method, server, adaptor and storage medium |
| CN114302254A (en) * | 2021-12-28 | 2022-04-08 | 天翼云科技有限公司 | Method and device for transcoding and slicing live video, storage medium and electronic equipment |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9756361B2 (en) * | 2014-07-25 | 2017-09-05 | Verizon Patent And Licensing Inc. | On-demand load balancer and virtual live slicer server farm for program ingest |
-
2022
- 2022-07-25 CN CN202210884368.3A patent/CN115174578B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1434386A (en) * | 2003-01-25 | 2003-08-06 | 华中科技大学 | Division distributed storage method of program source based on cluster video server |
| CA2848455A1 (en) * | 2011-09-16 | 2013-03-21 | Tencent Technology (Shenzhen) Company Limited | Mobile multimedia real-time transcoding system, apparatus, storage medium and method |
| CN105227968A (en) * | 2014-06-30 | 2016-01-06 | 惠州市伟乐科技股份有限公司 | Based on the video code conversion accelerated method of section |
| WO2016011823A1 (en) * | 2014-07-22 | 2016-01-28 | 中兴通讯股份有限公司 | Method for acquiring live video slice, server, and storage medium |
| CN105245880A (en) * | 2015-09-21 | 2016-01-13 | 北京暴风科技股份有限公司 | Video file cutting method and distributed transcoding method |
| CN107370726A (en) * | 2017-06-22 | 2017-11-21 | 上海网达软件股份有限公司 | A kind of virtual sliced sheet method and system for disributed media file trans-coding system |
| CN110830805A (en) * | 2019-10-24 | 2020-02-21 | 上海网达软件股份有限公司 | Multi-resolution output distributed file transcoding method and device |
| CN110769278A (en) * | 2019-10-29 | 2020-02-07 | 鹏城实验室 | Distributed video transcoding method and system |
| WO2021254375A1 (en) * | 2020-06-18 | 2021-12-23 | 中兴通讯股份有限公司 | Video partitioning method, transfer method, server, adaptor and storage medium |
| CN112165622A (en) * | 2020-09-23 | 2021-01-01 | 上海网达软件股份有限公司 | Single-Pod multi-protocol video file transcoding method and system |
| CN114302254A (en) * | 2021-12-28 | 2022-04-08 | 天翼云科技有限公司 | Method and device for transcoding and slicing live video, storage medium and electronic equipment |
Non-Patent Citations (2)
| Title |
|---|
| Ike Pertiwi Windasari ; Luqman Setyo Nugroho ; Adian Fatchur Rochim ; Risma Septiana.An-SPf: as an Alternative Architecture of no Single Point Failure of Scalable Transcoding System Based on Kubernetes.《2021 International Conference on Computer System, Information Technology, and Electrical Engineering (COSITE)》.2021,18-23页. * |
| 基于无服务器计算的支持GPU的视频处理任务编排系统的设计与实现;莫佳骏;《中国优秀硕士学位论文全文数据库 信息科技辑》;I138-1994页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115174578A (en) | 2022-10-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9961398B2 (en) | Method and device for switching video streams | |
| CN105451031B (en) | Video transcoding method and system | |
| CN106982356B (en) | Distributed large-scale video stream processing system | |
| CN103297807A (en) | Hadoop-platform-based method for improving video transcoding efficiency | |
| CN102855133B (en) | A kind of computer processing unit interactive system | |
| CN103838779A (en) | Idle computing resource multiplexing type cloud transcoding method and system and distributed file device | |
| CN107370726B (en) | Virtual slicing method and system for distributed media file transcoding system | |
| CN104469396A (en) | Distributed transcoding system and method | |
| CN102857531B (en) | A kind of long-distance interactive system based on cloud computing | |
| CN110830805A (en) | Multi-resolution output distributed file transcoding method and device | |
| CN112423140A (en) | Video playing method and device, electronic equipment and storage medium | |
| CN105262957A (en) | Video image processing method and device | |
| CN114827663A (en) | Distributed live broadcast frame insertion system and method | |
| CN107229516B (en) | Data processing method and device | |
| CN115174578B (en) | Kubernetes-based on-demand parallel transcoding method, device and storage medium | |
| CN113906764A (en) | Interruptible video transcoding | |
| CN111083408B (en) | Method, system and equipment for processing video storage service | |
| CN111611479B (en) | Data processing method and related device for network resource recommendation | |
| US11315606B1 (en) | Timecode generation and assignment | |
| CN112312164A (en) | Video transcoding system based on distributed transcoding server | |
| CN112188285A (en) | Video transcoding method, device, system and storage medium | |
| CN112911332A (en) | Method, apparatus, device and storage medium for clipping video from live video stream | |
| CN116567170A (en) | Video super-resolution parallel scheduling method and device for cloud platform and cloud platform | |
| CN115866331A (en) | Video frame extraction analysis method, device, equipment and storage medium | |
| CN114157895A (en) | Video processing method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |