CN114007109B - Mixed stream transcoding processing method and device, equipment, medium and product thereof - Google Patents

Mixed stream transcoding processing method and device, equipment, medium and product thereof Download PDF

Info

Publication number
CN114007109B
CN114007109B CN202111266099.6A CN202111266099A CN114007109B CN 114007109 B CN114007109 B CN 114007109B CN 202111266099 A CN202111266099 A CN 202111266099A CN 114007109 B CN114007109 B CN 114007109B
Authority
CN
China
Prior art keywords
transcoding
video stream
service
code rate
stream
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202111266099.6A
Other languages
Chinese (zh)
Other versions
CN114007109A (en
Inventor
陈宇辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangzhou Huaduo Network Technology Co Ltd
Original Assignee
Guangzhou Huaduo Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou Huaduo Network Technology Co Ltd filed Critical Guangzhou Huaduo Network Technology Co Ltd
Priority to CN202111266099.6A priority Critical patent/CN114007109B/en
Publication of CN114007109A publication Critical patent/CN114007109A/en
Application granted granted Critical
Publication of CN114007109B publication Critical patent/CN114007109B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/236Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
    • H04N21/2365Multiplexing of several video streams
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21Server components or server architectures
    • H04N21/218Source of audio or video content, e.g. local disk arrays
    • H04N21/2187Live feed
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234309Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4 or from Quicktime to Realvideo
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234381Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/242Synchronization processes, e.g. processing of PCR [Program Clock References]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/266Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
    • H04N21/2662Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85Assembly of content; Generation of multimedia applications
    • H04N21/854Content authoring
    • H04N21/8547Content authoring involving timestamps for synchronizing content

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Computer Security & Cryptography (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

The application discloses a mixed stream transcoding processing method and a device, equipment, medium and product thereof, wherein the method comprises the following steps: receiving a video stream to be processed by a current mixed stream transcoding service; decoding the video stream to be processed by the current mixed stream transcoding service to obtain an intermediate video stream; distributing the intermediate video stream to a plurality of internal transcoding services for encoding by the current mixed stream transcoding service, wherein each internal transcoding service encodes according to a corresponding fixed code rate so as to output a corresponding target video stream; and pushing the target video stream obtained by transcoding the target video stream to user terminal equipment matched with the corresponding fixed code rate by each internal transcoding service. According to the method and the device, the mixed stream and the transcoding process of the video stream are integrated by saving the coding and decoding links of the mixed stream and the transcoding, so that the mixed stream and the transcoding efficiency of the video stream are higher, the transcoding time delay and the transcoding cost can be reduced, and the image quality of the low-code-rate video stream can be improved.

Description

Mixed stream transcoding processing method and device, equipment, medium and product thereof
Technical Field
The present disclosure relates to streaming media processing technologies, and in particular, to a mixed-stream transcoding processing method, and corresponding apparatus, computer device, computer readable storage medium, and computer program product thereof.
Background
In a live video scene, a server often needs to transcode an original video stream of an upstream of a host user into a plurality of audience streams with different code rates so as to adapt to the streaming requirements of different network environments and playing terminals of audience terminals. The general practice is that the server decodes the original video stream of the anchor into YUV or RGB data, scales or mixes with other video streams, encodes the YUV or RGB data into the audience stream with the highest code rate, and then sequentially pulls the audience stream with the highest code rate by another independent transcoding service to decode, scale and encode the audience stream with other low code rate, as shown in fig. 1. Therefore, the resource allocation and the scheduling are easier, but for the audience streams beyond the highest code rate, the audience streams undergo two decoding and encoding processes, so that the video image quality is greatly lost, and the transcoding delay and the transcoding cost are increased.
In addition, in the prior art, because the mixed stream service and the independent transcoding service are mutually independent, and the problem of load balance does not exist between the mixed stream service and the independent transcoding service, the mixed stream service cannot grasp the load condition of the whole mixed stream transcoding process, so that the mixed stream service cannot be effectively responsible for maintaining the running resources of a computer.
Disclosure of Invention
It is a primary object of the present application to solve at least one of the above problems and provide a mixed stream transcoding method and corresponding apparatus, computer device, computer readable storage medium, computer program product.
In order to meet the purposes of the application, the application adopts the following technical scheme:
one of the objects of the present application is to provide a mixed stream transcoding method, comprising the steps of:
receiving a video stream to be processed by a current mixed stream transcoding service;
decoding the video stream to be processed by the current mixed stream transcoding service to obtain an intermediate video stream;
distributing the intermediate video stream to a plurality of internal transcoding services for encoding by the current mixed stream transcoding service, wherein each internal transcoding service encodes according to a corresponding fixed code rate so as to output a corresponding target video stream;
and pushing the target video stream obtained by transcoding the target video stream to user terminal equipment matched with the corresponding fixed code rate by each internal transcoding service.
In a further embodiment, the receiving, by the current hybrid stream transcoding service, the video stream to be processed includes the steps of:
the current mixed stream transcoding service adds the data frames of the video stream to be processed to the corresponding jitter buffer queues for time sequence reformation, so that the data frames in the jitter buffer queues are dequeued in time sequence;
sequentially passing the data frames dequeued from the jitter buffer queue through the to-be-mixed flow queues corresponding to the jitter buffer queue;
And using the data frames dequeued from the to-be-mixed stream queue for the decoding mixed stream.
In a further embodiment, before the intermediate video stream is distributed by the current mixed stream transcoding service to a plurality of internal transcoding services for encoding, the method comprises the following steps:
acquiring code rate information of user terminal equipment in network live broadcast service;
judging whether an internal transcoding service corresponding to the fixed code rate specified by the code rate information exists or not, and creating the internal transcoding service corresponding to the fixed code rate when the internal transcoding service does not exist;
and establishing a corresponding relation of the user terminal equipment and the internal transcoding service corresponding to the fixed code rate in the code rate information of the user terminal equipment so as to push a target video stream generated by the internal transcoding service to the user terminal equipment.
In a further embodiment, each internal transcoding service encodes according to its corresponding fixed code rate to output a corresponding target video stream, comprising the steps of:
each internal transcoding service judges whether the code rate of the video stream to be processed is a fixed code rate corresponding to the internal transcoding service;
when the former judgment is established, directly encoding the intermediate video stream into a target video stream by using a fixed code rate corresponding to the intermediate video stream, and outputting the target video stream;
And when the former judgment is not established, performing image scaling on the intermediate video stream according to the fixed code rate corresponding to the intermediate video stream, and then encoding the intermediate video stream into a target video stream and outputting the target video stream.
In an extended embodiment, the method further comprises the steps of:
monitoring an alarm event that the usage amount of computer operation resources occupied by the current mixed stream transcoding service exceeds a preset threshold value;
responding to the alarm event, and determining at least one relatively lower target code rate in the corresponding fixed code rates in the plurality of internal transcoding services;
invoking an independent transcoding service corresponding to the target code rate in another computer device, and transmitting the target video stream which is obtained by the corresponding internal transcoding service code and is output with the highest code rate to the independent transcoding service;
and decoding, scaling and encoding the target video stream by each independent transcoding service according to the corresponding target code rate to obtain the target video stream corresponding to the target code rate so as to push the target video stream to the user terminal equipment matched with the target code rate.
In a further embodiment, invoking an independent transcoding service in another computer device corresponding to the target code rate comprises the steps of:
inquiring the idle rate of the running resources of the computer equipment in the service cluster;
Creating an independent transcoding service which is suitable for transcoding by the target code rate in the computer equipment with the highest running resource idle rate;
and transmitting the target video stream output with the highest code rate to the independent transcoding service for transcoding.
A mixed stream transcoding processing device according to one of the objects of the present application, comprising: the system comprises a video stream receiving module, a decoding mixed stream module, a distributing transcoding module and a video stream pushing module, wherein the video stream receiving module is configured to receive a video stream to be processed by a current mixed stream transcoding service; the decoding mixed stream module is configured to decode the video stream to be processed into a mixed stream by the current mixed stream transcoding service to obtain an intermediate video stream; the distribution transcoding module is configured to distribute the intermediate video stream to a plurality of internal transcoding services by the current mixed stream transcoding service to encode, and each internal transcoding service encodes according to a corresponding fixed code rate so as to output a corresponding target video stream; the video stream pushing module is configured to push the target video stream obtained by transcoding the target video stream to user terminal equipment matched with a corresponding fixed code rate by each internal transcoding service.
In a further embodiment, the video stream receiving module includes: the time sequence reforming sub-module is used for adding the data frames of the video stream to be processed to the corresponding jitter buffer queue by the current mixed stream transcoding service to perform time sequence reforming, so that the data frames in the jitter buffer queue are dequeued in time sequence; the queue transfer sub-module is used for sequentially passing the data frames dequeued from the jitter buffer queue through the to-be-mixed flow queues corresponding to the jitter buffer queue; and the mixed stream output sub-module is used for using the data frames output from the to-be-mixed stream queue for the decoding mixed stream.
In a further embodiment, the decoding mixed stream module includes: the code rate searching sub-module is used for acquiring code rate information of user terminal equipment in the network live broadcast service; the service configuration sub-module is used for judging whether an internal transcoding service corresponding to the fixed code rate specified by the code rate information exists or not, and creating the internal transcoding service corresponding to the fixed code rate when the internal transcoding service does not exist; and the service association sub-module is used for establishing the corresponding relation of the user terminal equipment and the internal transcoding service corresponding to the fixed code rate in the code rate information thereof so as to push the target video stream generated by the internal transcoding service to the user terminal equipment.
In a further embodiment, the decoding mixed stream module includes the following structure for implementing each internal transcoding service: the code rate self-checking sub-module is configured to judge whether the code rate of the video stream to be processed is a fixed code rate corresponding to each internal transcoding service; the direct-conversion sub-module is used for directly encoding the intermediate video stream into a target video stream by using a fixed code rate corresponding to the direct-conversion sub-module when the previous judgment is established and then outputting the target video stream; and the scaling and converting sub-module is used for performing image scaling on the intermediate video stream according to the fixed code rate corresponding to the intermediate video stream when the previous judgment is not established, and then encoding the intermediate video stream into a target video stream and outputting the target video stream.
In an extended embodiment, the mixed stream transcoding processing device further includes: the alarm monitoring module is used for monitoring an alarm event that the usage amount of the computer operation resources occupied by the current mixed stream transcoding service exceeds a preset threshold value; the target determining module is used for responding to the alarm event and determining at least one relatively lower target code rate in the corresponding fixed code rates in the plurality of internal transcoding services; the response scheduling module is used for calling an independent transcoding service corresponding to the target code rate in another computer device and transmitting the target video stream which is obtained by the corresponding internal transcoding service code and is output at the highest code rate to the response scheduling module; and the independent transcoding module is configured to decode, scale and encode the target video stream according to the corresponding target code rate by each independent transcoding service, obtain the target video stream corresponding to the target code rate and push the target video stream to the user terminal equipment matching the target code rate.
In a further embodiment, the response scheduling module includes: the resource query sub-module is used for querying the idle rate of the running resources of the computer equipment in the service cluster; the service creation sub-module is used for creating an independent transcoding service which is suitable for transcoding with the target code rate in the computer equipment with the highest running resource idle rate; and the video transmission sub-module is used for transmitting the target video stream output at the highest code rate to the independent transcoding service for transcoding.
A computer device provided in accordance with one of the objects of the present application includes a central processor and a memory for invoking a computer program stored in the memory for performing the steps of the hybrid stream transcoding method described herein.
A computer readable storage medium adapted to another object of the present application stores a computer program implemented according to the hybrid stream transcoding method in the form of computer readable instructions, which when invoked by a computer, performs the steps comprised by the method.
A computer program product is provided adapted for another object of the present application, comprising a computer program/instruction which, when executed by a processor, carries out the steps of the method described in any of the embodiments of the present application.
Compared with the prior art, the method has the following advantages:
firstly, after decoding mixed stream of video stream to be processed, the method of coding before code rate conversion output is abandoned, intermediate video stream obtained by mixed stream is directly distributed to each internal transcoding service for transcoding, each internal transcoding service controls the coding output of one path of intermediate video stream obtained by the corresponding fixed code rate, and target video stream output by corresponding fixed code rate is pushed to relevant user terminal equipment matched with the fixed code rate.
And secondly, the internal transcoding service and the mixed stream service are integrated into the mixed stream transcoding service, so that the mixed stream transcoding service can intensively realize load detection and load balancing in the mixed stream transcoding process, thereby carrying out on-demand scheduling on computer operation resources when needed, further ensuring the robust operation of the service and ensuring the provision of the robust service for the mixed stream transcoding.
In addition, the advantages obtained by the application make the technical scheme of the application be particularly suitable for running resource intensive application scenes, especially CPU intensive running tasks, for example, mixed stream transcoding is often needed to be carried out on multiple paths of audio and video streams in network live broadcast service, and the mixed stream transcoding is the CPU intensive task.
Drawings
The foregoing and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:
FIG. 1 is a schematic diagram of a logic architecture between a mixed stream service and a transcoding service employed in the prior art;
FIG. 2 is a schematic diagram of a logic architecture of a service cluster to which the hybrid stream transcoding method of the present application is applied;
FIG. 3 is a flow chart of an exemplary embodiment of a hybrid stream transcoding method of the present application;
FIG. 4 is a schematic diagram of a logic architecture of a mixed stream transcoding service used in the present application;
Fig. 5 is a flow chart illustrating a process of receiving a video to be processed by the mixed stream transcoding service of the present application;
fig. 6 is a flow chart illustrating a process of establishing a correspondence between internal transcoding service and fixed code rate in the present application;
fig. 7 is a flow chart illustrating a process of adaptive matching of a fixed code rate by an internal transcoding service in the present application;
fig. 8 is a flow chart illustrating a process of transferring a transcoding task corresponding to a fixed code rate by using a degradation mechanism in the present application;
FIG. 9 is a schematic diagram of a logic architecture for loading independent transcoding services by a hybrid stream transcoding service according to the present application;
FIG. 10 is a flow chart illustrating a process for scheduling computer devices for independent transcoding services in the present application;
FIG. 11 is a schematic block diagram of a hybrid stream transcoding device of the present application;
fig. 12 is a schematic structural diagram of a computer device used in the present application.
Detailed Description
Embodiments of the present application are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary only for the purpose of illustrating the present application and are not to be construed as limiting the present application.
As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless expressly stated otherwise, as understood by those skilled in the art. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. The term "and/or" as used herein includes all or any element and all combination of one or more of the associated listed items.
It will be understood by those skilled in the art that all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs unless defined otherwise. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
As used herein, "client," "terminal device," and "terminal device" are understood by those skilled in the art to include both devices that include only wireless signal receivers without transmitting capabilities and devices that include receiving and transmitting hardware capable of two-way communication over a two-way communication link. Such a device may include: a cellular or other communication device such as a personal computer, tablet, or the like, having a single-line display or a multi-line display or a cellular or other communication device without a multi-line display; a PCS (Personal Communications Service, personal communication system) that may combine voice, data processing, facsimile and/or data communication capabilities; a PDA (Personal Digital Assistant ) that can include a radio frequency receiver, pager, internet/intranet access, web browser, notepad, calendar and/or GPS (Global Positioning System ) receiver; a conventional laptop and/or palmtop computer or other appliance that has and/or includes a radio frequency receiver. As used herein, "client," "terminal device" may be portable, transportable, installed in a vehicle (aeronautical, maritime, and/or land-based), or adapted and/or configured to operate locally and/or in a distributed fashion, at any other location(s) on earth and/or in space. As used herein, a "client," "terminal device," or "terminal device" may also be a communication terminal, an internet terminal, or a music/video playing terminal, for example, a PDA, a MID (Mobile Internet Device ), and/or a mobile phone with music/video playing function, or may also be a device such as a smart tv, a set top box, or the like.
The hardware referred to by the names "server", "client", "service node" and the like in the present application is essentially an electronic device having the performance of a personal computer, and is a hardware device having necessary components disclosed by von neumann's principle, such as a central processing unit (including an arithmetic unit and a controller), a memory, an input device, and an output device, and a computer program is stored in the memory, and the central processing unit calls the program stored in the external memory to run in the memory, executes instructions in the program, and interacts with the input/output device, thereby completing a specific function.
It should be noted that the concept of "server" as referred to in this application is equally applicable to the case of a server farm. The servers should be logically partitioned, physically separate from each other but interface-callable, or integrated into a physical computer or group of computers, according to network deployment principles understood by those skilled in the art. Those skilled in the art will appreciate this variation and should not be construed as limiting the implementation of the network deployment approach of the present application.
One or several technical features of the present application, unless specified in the plain text, may be deployed either on a server to implement access by remotely invoking an online service interface provided by the acquisition server by a client, or directly deployed and run on the client to implement access.
The neural network model cited or possibly cited in the application can be deployed on a remote server and used for implementing remote call on a client, or can be deployed on a client with sufficient equipment capability for direct call unless specified in a clear text, and in some embodiments, when the neural network model runs on the client, the corresponding intelligence can be obtained through migration learning so as to reduce the requirement on the running resources of the hardware of the client and avoid excessively occupying the running resources of the hardware of the client.
The various data referred to in the present application, unless specified in the plain text, may be stored either remotely in a server or in a local terminal device, as long as it is suitable for being invoked by the technical solution of the present application.
Those skilled in the art will appreciate that: although the various methods of the present application are described based on the same concepts so as to be common to each other, the methods may be performed independently, unless otherwise indicated. Similarly, for each of the embodiments disclosed herein, the concepts presented are based on the same inventive concept, and thus, the concepts presented for the same description, and concepts that are merely convenient and appropriately altered although they are different, should be equally understood.
The various embodiments to be disclosed herein, unless the plain text indicates a mutually exclusive relationship with each other, the technical features related to the various embodiments may be cross-combined to flexibly construct a new embodiment, so long as such combination does not depart from the inventive spirit of the present application and can satisfy the needs in the art or solve the deficiencies in the prior art. This variant will be known to the person skilled in the art.
A mixed stream transcoding processing method can be programmed into a computer program product and deployed in a service cluster to be operated so as to be realized by accessing an interface opened after the computer program product is operated and performing the method through man-machine interaction with the computer program product through a graphical user interface.
Referring to fig. 2, in the service cluster including a plurality of computer devices acting as servers, the service cluster may use a micro service architecture to perform containerization management, so as to obtain a plurality of service containers, where each service container is essentially a service process for executing one or more running tasks, and thus, each service process may concurrently process a plurality of running tasks. In the exemplary network live broadcast application scenario of the present application, the running task is mainly used for performing mixed stream transcoding on multiple paths of original media streams, and necessary transcoding processing can be performed together after mixed streams. According to the scheduling mechanism involved in one embodiment of the present application, the running task may also be an independent transcoding service extended by the mixed-stream transcoding service of the present application, and may be stored in a different computer device of the same service cluster as the mixed-stream transcoding service.
Each of the service processes generally runs in a computer device, and the running tasks processed by different service processes can also be running tasks with the same function, for example, the running tasks in the network live broadcast scene in the application are computer tasks for mixing and transcoding multiple paths of original media streams.
The running task in the service cluster can also comprise an access service responsible for external output and input, and through the access service, the input of one or more external to-be-processed video streams can be received and forwarded to the mixed stream transcoding service for mixed stream transcoding processing, and in the process, the access service can be responsible for various necessary scheduling to ensure that the provided network live broadcast service can run stably.
Referring to fig. 3, in an exemplary embodiment, the hybrid stream transcoding method of the present application includes the following steps:
step S1100, receiving, by the current mixed stream transcoding service, a video stream to be processed:
as shown in fig. 4, in the service cluster, the access service may be responsible for forwarding multiple paths of pending video streams in, for example, an exemplary network live scene to various mixed stream transcoding services of the present application, which is described later for ease of understanding only with the business logic of a single mixed stream transcoding service.
Taking the network live broadcast application scenario as an example, the corresponding mixed stream transcoding service is deployed in a service cluster shown in fig. 2, and the mixed stream transcoding service in a computer device serving as a server in the service cluster is responsible for carrying out mixed stream transcoding processing on video streams uploaded by a host user in each party of the online communication or on multiple paths of video streams designated by the same host user so as to obtain corresponding target video streams. Wherein the video stream for the mixed stream is referred to as a video stream to be processed. The technical scheme of the method and the device are mainly implemented in a mixed stream transcoding service, and the mixed stream transcoding service can communicate with an access service through an interface.
Each video stream can be an independent video stream or an audio-video stream in content, the video streams provided by the same anchor user can be one or more paths, and finally all paths of video streams needing mixed streams are submitted to an access service to be scheduled to a corresponding mixed stream transcoding service for processing, and the video streams provided to the mixed stream transcoding service are the video streams to be processed which are called as the mixed stream transcoding service in the application. Similarly, the video stream formed by transcoding the mixed streams of the video streams to be processed is the target video stream called as the application.
The video stream comprises a plurality of data frames, also called data packets, which for the video stream are mainly image frames.
The mixed stream transcoding service is specially responsible for mixed stream transcoding operation of multiple paths of to-be-processed video streams scheduled by the access service under the default condition, and the multiple paths of to-be-processed video streams are transcoded into target video streams and then delivered to the access service for output. The mixed stream transcoding service can monitor the load condition of the computer equipment in which the mixed stream transcoding service is located in the life cycle of the mixed stream transcoding service, and when the mixed stream transcoding service monitors that the running resource of the computer equipment in which the mixed stream transcoding service is located is overloaded or the running resource of the computer equipment occupied by the mixed stream transcoding service exceeds a preset threshold value, the mixed stream transcoding service can trigger a corresponding alarm event.
The operation resources may include CPU operation resources, memory operation resources, hard disk operation resources, etc., and in this embodiment, for convenience of understanding, detection of the CPU operation resources is simplified to be described. Thus, for example, an empirical load threshold may be preset, and the mixed stream transcoding service detects whether its current CPU usage exceeds the load threshold, and when the current CPU usage exceeds the load threshold, the alert event is triggered, otherwise, the mixed stream transcoding service is continuously responsible for the mixed stream transcoding operation.
In one embodiment, in order to prevent data such as packet loss of data frames and out-of-order data frames of the video stream to be processed, each path of the video stream to be processed received by the mixed stream transcoding service may be buffered in the mixed stream transcoding service, and each received data frame may be subjected to time sequence arrangement within a buffering duration range, which will be further disclosed later.
Step S1200, decoding the video stream to be processed by the current mixed stream transcoding service to obtain an intermediate video stream:
as shown in fig. 4, according to the video encoding and decoding principle, after the video stream to be processed enters the current mixed stream transcoding service, the decoding logic unit of the current mixed stream transcoding service can decode the video stream, and according to the encoding protocol corresponding to the video stream to be processed, the image data corresponding to each path of video stream to be processed is decoded.
On the basis, the mixed stream transcoding service performs image mixed stream on image data corresponding to each path of video stream to be processed based on the same time coordinate system, and combines multiple paths of video streams to be processed into one path of intermediate video stream. Specifically, in order to achieve synchronization of data frames of different paths of to-be-processed video streams in time, the mixed stream transcoding service maps time stamps of all paths of to-be-processed video streams to the same time coordinate system to form a unified time stamp, so that the paths of to-be-processed video streams are mixed based on the unified time stamp, and output time stamps of the data frames in a target video stream formed by mixing the paths of to-be-processed video streams are marked by referring to the unified time stamp.
In the mixed stream transcoding service, in the mixed stream process, the mixed stream transcoding service adopts a time sequence pointer to indicate the current time stamp of the intermediate video stream, and the current time stamp is used for marking the data frame in the intermediate video stream formed by mixed streams each time, and then the pre-protocol duration specified by the coding protocol is overlapped for being used as the current time stamp of the data frame formed by the next mixed stream. In order to avoid the phenomenon that the time stamps of the multipath video streams to be processed are uneven due to possible reasons such as packet loss, unaligned time stamps, messy codes and the like, the phenomenon can be detected, after the corresponding phenomenon is detected, whether the current time stamp exceeds the time zone range defined by the minimum time stamp and the maximum time stamp in the data frames formed by mixing the streams is specifically judged, and when the current time stamp exceeds the range, the current time stamp is finely adjusted by a preset amount, so that the time stamp of each data frame in the intermediate video stream keeps relatively uniform change, and the terminal equipment is ensured to be smoother and more natural when playing the target video stream.
Since the video stream to be processed is itself transmitted to the current mixed stream transcoding service at the default bitrate, which is actually the highest bitrate in the mixed stream transcoding process, it can be understood that the intermediate video stream will be output at the highest bitrate.
Step S1300, distributing, by the current mixed stream transcoding service, the intermediate video stream to a plurality of internal transcoding services for encoding, where each internal transcoding service encodes according to its corresponding fixed code rate to output a corresponding target video stream:
as shown in fig. 4, unlike the prior art, each mixed stream transcoding service is built in the service logic responsible for transcoding as its own internal transcoding service, so that the service logic and the mixed stream transcoding service are executed in the same running task, that is, in the same computer device, so that interface call between devices can be avoided, and the execution efficiency is improved.
Taking a network live broadcast scene as an example, the code rates of target video streams which can be processed by different users in a live broadcast room may be different due to the bandwidth, hardware and other reasons of user terminal equipment, so that the target video streams corresponding to one live broadcast room are usually required to be converted into output with different fixed code rates, so that the target video streams with different fixed code rates are adapted to different user terminal equipment. In order to obtain the output of different fixed code rates, the mixed stream transcoding service firstly directly distributes the intermediate video stream formed by mixing the mixed stream into a plurality of internal transcoding services corresponding to different fixed code rates for encoding processing, and it can be understood that each internal transcoding service actually forms a corresponding encoding channel.
Each internal transcoding service works on a fixed code rate which is adaptive to the internal transcoding service, the intermediate video stream received by the internal transcoding service is encoded to obtain a corresponding target video stream at the fixed code rate, and the target video stream is controlled to be pushed to corresponding user terminal equipment through the access service at the corresponding fixed code rate.
It is not easy to understand that, for the internal transcoding service corresponding to the highest code rate, the intermediate video stream is directly encoded according to the encoding protocol without performing code rate conversion on the intermediate video stream, and a corresponding target video stream is output.
For the internal transcoding service corresponding to the fixed code rate lower than the highest code rate, necessary processing needs to be performed on the intermediate video stream before encoding, for example, image scaling, frame loss and the like are performed on the intermediate video stream, so that the target video stream obtained by final encoding can be transmitted at the fixed code rate.
So far, it can be understood that after the mixed stream transcoding service accesses the intermediate video stream obtained by mixing the mixed stream into the encoding channels corresponding to different fixed code rates, the internal transcoding service in each encoding channel can encode and generate corresponding target video stream corresponding to each fixed code rate.
Step S1400, pushing the target video stream obtained by transcoding the target video stream to a user terminal device matching a corresponding fixed code rate by each internal transcoding service:
after each internal transcoding service transcodes the intermediate video stream according to the corresponding fixed code rate to generate the target video stream, the target video stream can be correspondingly pushed to the user terminal equipment matched with the corresponding fixed code rate through the access service. And the user terminal equipment receives the corresponding target video stream according to the corresponding fixed code rate and decodes and plays the target video stream. The target video stream obtained by the user terminal equipment is output according to the fixed code rate code which is adaptive to the user terminal equipment, so that the target video stream can obtain a smoother effect when being played, and the target video stream is transcoded after being directly mixed in the mixed stream transcoding service without cross-service or cross-equipment calling, so that part of decoding and encoding links in the middle are saved, and the smoothness when the user terminal equipment is played is further ensured.
As can be seen from the above exemplary embodiments, after decoding the video stream to be processed, the method of encoding before performing rate conversion output is abandoned, the intermediate video stream obtained by the mixed stream is directly distributed to each internal transcoding service for transcoding, each internal transcoding service controls the encoded output of one path of intermediate video stream obtained by the separation according to its own corresponding fixed code rate, and the target video stream output with the corresponding fixed code rate is pushed to the relevant user terminal device matching the fixed code rate.
And secondly, the internal transcoding service and the mixed stream service are integrated into the mixed stream transcoding service, so that the mixed stream transcoding service can intensively realize load detection and load balancing in the mixed stream transcoding process, thereby carrying out on-demand scheduling on computer operation resources when needed, further ensuring the robust operation of the service and ensuring the provision of the robust service for the mixed stream transcoding.
In addition, the advantages obtained by the application make the technical scheme of the application especially suitable for running resource intensive application scenes, especially CPU intensive running tasks, for example, mixed stream transcoding is often needed to be carried out on multiple audio and video streams in network live broadcast service, and the mixed stream transcoding is the CPU intensive task, under the scenes, the application of the technical scheme of the application can obtain more obvious advantages, so that the method has obvious assistance for ensuring the robustness and fluency of the network live broadcast service, and can ensure the network live broadcast service
Referring to fig. 5, in a deepened embodiment, the step S1100 of receiving, by the current mixed stream transcoding service, a video stream to be processed includes the following steps:
step S1110, adding the data frames of the video stream to be processed to the corresponding jitter buffer queue by the current mixed stream transcoding service for time sequence reformation, so as to realize time sequence dequeuing of the data frames in the jitter buffer queue:
In this embodiment, in order to implement time sequence reformation of data frames of a video stream received by a mixed stream transcoding service, a jitter buffer mechanism is adopted, and a jitter buffer queue and a to-be-mixed stream queue are correspondingly set for each path of to-be-processed video stream.
Each channel of video stream to be processed received by the mixed stream transcoding service is successively added into the corresponding jitter buffer queues. And each jitter buffer queue is responsible for carrying out time sequence rearrangement on each data frame in the queue according to the time stamp of the data frame in the jitter buffer queue, so that the data frames of each path of video stream to be processed are sequentially dequeued and transferred to the corresponding queue to be mixed according to the time sequence of the time stamp of the data frames.
Step S1120, the data frames dequeued from the jitter buffer queue sequentially pass through the waiting mixed stream queue corresponding to the jitter buffer queue;
each jitter buffer queue controls the length of the queue according to a certain buffer time length so as to drive the jitter buffer queue to dequeue the data frame at the head of the jitter buffer queue when the number of received data frames exceeds the buffer length corresponding to the buffer time length. Therefore, the number of the data frames stored in the jitter buffer queue is determined by a preset buffer duration, and the buffer duration can be evaluated and determined by the mixed stream transcoding service according to the arrival of the data frames of all the paths of video streams to be processed, so as to ensure that the jitter buffer queues of all the paths of video streams to be processed can effectively receive enough data frames as much as possible.
It will be appreciated that since the data frames dequeued by each jitter buffer queue have been previously time-sequentially re-queued, the data frames output by each jitter buffer queue are dequeued into the to-be-mixed queue in the order of their time stamps, and the data frames in the to-be-mixed queue are ordered regularly.
Step S1130, using the data frame dequeued from the pending mixed stream queue for the decoded mixed stream:
the data frames of the to-be-mixed flow queue are used for mixed flow, can be dequeued according to a certain strategy, do not affect the inventive spirit of the application, and can be flexibly realized by a person skilled in the art. As long as a sufficient number of data frames can be obtained from the to-be-mixed flow queues of each path of to-be-processed video flow for mixed flows when the business logic of mixed flows is executed.
According to the method, the device and the system, the mixed stream transcoding service is used for respectively carrying out time sequence reforming on the received data frames of each path of video stream to be processed, and a buffer mechanism of the data frames is established by controlling each jitter buffer queue through the buffer time length, so that the video to be processed can be prepared with enough data frames before mixed stream, the situation that enough data frames can be collected from the video stream to be processed for effective mixed stream is ensured, the phenomena of packet loss, disorder and the like are avoided, and the image quality of finally obtained target video audio is ensured.
Referring to fig. 6, in a deepened embodiment, the step S1300, before the intermediate video stream is distributed to a plurality of internal transcoding services for encoding by the current mixed stream transcoding service, includes the following steps:
step S2100, obtaining code rate information of a user terminal device in a live network service:
in the network live broadcast service of the embodiment of the present application, for audience users in the same live broadcast room or in a plurality of live broadcast rooms, different user terminal devices are correspondingly matched with different fixed code rates, that is, different user terminal devices are only suitable for receiving a target video stream with a certain fixed code rate, and the fixed code rate can be detected and determined by a live broadcast room application program on the user terminal device supporting the live broadcast room operation, and then is promoted to the access service of the present application. After the access service obtains the code rate information of each user terminal device, clustering can be performed to determine a plurality of corresponding fixed code rates so as to match the corresponding code channels required by transcoding for each fixed code rate.
Step S2200, judging whether an internal transcoding service corresponding to the fixed code rate specified by the code rate information exists, and creating the internal transcoding service corresponding to the fixed code rate when the internal transcoding service does not exist:
When the access service needs to add a coding channel corresponding to a designated fixed code rate, the mixed stream transcoding service can be informed to execute. The mixed stream transcoding service detects whether the internal transcoding service corresponding to the fixed code rate exists in each internal transcoding service currently running according to the mixed stream transcoding service, if the internal transcoding service exists, the mixed stream transcoding service does not need to be re-established, and the access service is directly replied and informed; if there is no internal transcoding service corresponding to the fixed code rate, a new internal transcoding service can be created directly, which is set for transcoding the intermediate video stream at the fixed code rate.
Step S2300, establishing a correspondence between the ue device and an internal transcoding service corresponding to a fixed code rate in the code rate information thereof, so as to push, to the ue device, a target video stream generated by the internal transcoding service:
in order to facilitate maintenance, the access service may establish a correspondence between different user terminal devices in different live broadcasting rooms and the fixed code rates adapted to the user terminal devices, thereby also establishing a correspondence between different audience users and different internal transcoding services, and a correspondence between different user terminal devices and different internal transcoding services.
In this embodiment, by maintaining the mapping relation data between the user terminal device and its fixed code rate and the internal service of transcoding, the mixed stream transcoding service is convenient to dynamically adapt to the actual requirement and add or delete the internal service of transcoding, so as to maximize the utilization efficiency of the operating resources of the computer device.
Referring to fig. 7, in a deepened embodiment, in the step S1300, each internal transcoding service encodes according to its corresponding fixed code rate to output a corresponding target video stream, and includes the following steps:
step S3100, each internal transcoding service determines whether the code rate of the video stream to be processed is a fixed code rate corresponding to itself:
in this embodiment, in order to facilitate implementation of the internal transcoding service by adopting a uniform class definition, a determination mechanism may be implemented in the service logic of the internal transcoding service by using the same class definition. After creating a corresponding instance through the class of the internal transcoding service, the internal transcoding service instance makes a subsequent step of processing according to different judgment conditions, wherein whether the code rate of the video stream to be processed is consistent with the fixed code rate set by the internal transcoding service itself or not.
Step S3200, when the previous judgment is true, directly encoding the intermediate video stream into a target video stream by using a fixed code rate corresponding to the intermediate video stream, and then outputting the target video stream:
When the fixed code rate of the current internal transcoding service is judged to be consistent with the transmission code rate of the video stream to be processed, the current internal transcoding service is indicated to be unnecessary to transcode the video stream to be processed, so that the intermediate video stream is only required to be directly transcoded into the target video stream, the transmission code rate of the video stream to be processed is the highest code rate, and the output of the target video stream is controlled.
Step S3300, when the previous judgment is not satisfied, performing image scaling on the intermediate video stream according to the fixed code rate corresponding to the intermediate video stream, and then encoding the intermediate video stream into a target video stream and outputting the target video stream:
when it is determined that the fixed code rate of the current internal transcoding service is inconsistent with the transmission code rate of the video stream to be processed, the fixed code rate is generally a lower code rate because the transmission code rate of the video stream to be processed is regarded as the highest code rate, in this case, it is indicated that the current internal transcoding service needs to transcode the video stream to be processed, and therefore, the intermediate video stream is processed in two steps, the first step is to perform image scaling processing on the intermediate video stream, including, but not limited to, reducing the size of each frame of image in the intermediate video stream, discarding technical means that are commonly used for image compression by those skilled in the art, such as individual non-key frames, so as to enable the scaled image to adapt to the fixed code rate. The second step is to encode the scaled image according to the inherent coding logic of the coding protocol to obtain the corresponding target video stream based on the image scaling, and then output the target video stream with the corresponding fixed code rate.
In this embodiment, by implementing a judging mechanism in the internal transcoding service, it is convenient to perform standardized class definition on the internal transcoding service and to implement standardized call by the mixed stream transcoding service, so as to implement that the internal transcoding service adaptively executes corresponding service logic according to its own corresponding fixed code rate, and ensure that each internal transcoding service can correspondingly output a target video stream with a preset fixed code rate.
Referring to fig. 8 and fig. 9, in an extended embodiment, an automatic scheduling mechanism for a cross-computer device and a cross-service process is further implemented for the hybrid stream transcoding method of the present application, so that the method further includes the following steps:
step S4100, monitoring an alarm event that the usage amount of the computer operation resources occupied by the current mixed stream transcoding service exceeds a preset threshold value:
for the current running mixed stream transcoding service, the current running mixed stream transcoding service is responsible for monitoring the usage of the computer running resources occupied in the process of executing the mixed stream transcoding by itself in real time, and comparing the current running mixed stream transcoding service with a preset threshold, wherein the preset threshold can be an empirical threshold or an adaptive threshold which is determined by adapting to the usage rate of the running resources of the computer equipment where the mixed stream transcoding service is located, for example, the lower the usage rate of the running resources of the computer equipment is, the higher the usage rate of the running resources of the computer equipment is, and conversely, the higher the adaptive threshold is, so that the current mixed stream transcoding service is diversified and can be flexibly set by a person skilled in the art.
When the usage exceeds the preset threshold, an alarm event is triggered for indicating that the current service process and even the current computer equipment are overloaded.
Step S4200, in response to the alarm event, determining at least one target code rate that is relatively lower among the fixed code rates corresponding to the plurality of internal transcoding services:
in order to avoid system crash, one or a plurality of internal transcoding services in the current mixed stream transcoding service can be transferred to a similar service process running on other computer equipment to run, and in this embodiment, the one or a plurality of internal transcoding services with lower corresponding fixed code rate are preferably tuned away to the other computer equipment to process, so as to realize the degradation transcoding service with lower fixed code rate. The selected fixed code rate to be transferred is called the target code rate. Those skilled in the art may determine the target code rate and the number thereof using various scheduling strategies, for example, in this embodiment, only the lowest two fixed code rates may be determined as the target code rate in response to the alarm event, and then the internal transcoding service of the two fixed code rates may be transferred to the external two independent transcoding services for processing.
Step S4300, invoking an independent transcoding service corresponding to the target bitrate in another computer device, and transmitting the target video stream output at the highest bitrate obtained by encoding the corresponding internal transcoding service to the independent transcoding service:
in order to implement the degraded transcoding service, corresponding to each fixed code rate, it is necessary to invoke and run its corresponding independent transcoding service in another computer device, as shown in fig. 9, where the independent transcoding service has a corresponding decoding unit as an independent service compared to the internal transcoding service, so as to perform corresponding independent decoding, scaling, and encoding processes on the target video stream received by the independent transcoding service. Since the independent transcoding service has complete transcoding service logic, the independent transcoding service can be directly implemented by adopting a traditional corresponding transcoding component.
Accordingly, in response to the alert event, the current hybrid stream transcoding service may transcode the target video stream that it outputs at the highest rate to the independent transcoding service that it invokes and that runs in another computer device.
Step S4400, each independent transcoding service decodes, scales and encodes the target video stream according to its corresponding target code rate, and obtains a target video stream corresponding to the target code rate so as to push the target video stream to a user terminal device matching the target code rate:
For any independent transcoding service, after receiving a target video stream transmitted at the highest code rate and output by an internal transcoding service of the mixed stream transcoding service, the target video stream is correspondingly decoded, scaled and encoded to obtain a corresponding final target video stream, and then the final target video stream can be pushed to user terminal equipment corresponding to the target code rate through the access service.
It can be appreciated that for the internal transcoding service being diverted in the hybrid stream transcoding service, the operation can be stopped, so as to release the operation resources of the computer device where the hybrid stream transcoding service is located, and make the operation smoother and more robust.
The embodiment realizes a dispatching mechanism of the cross-computer equipment, the mixed-flow transcoding service monitors the overload condition of the self, when the mixed-flow transcoding service finds the overload of the running environment where the self is located, the mixed-flow transcoding service can transfer the lower fixed code rate to the independent transcoding service of other computer equipment to run, and the transcoding tasks corresponding to the fixed code rates are transferred to the independent transcoding service of other computer equipment to run, so that the degradation processing of the transcoding services corresponding to the fixed code rates is realized, the occupation of the running resources of the computer equipment is reduced, the running environment where the self is located is more robust, and meanwhile, the effective processing of the mixed-flow transcoding tasks is not influenced.
Referring to fig. 10, in a further embodiment, in the step S4300, an independent transcoding service corresponding to the target code rate in another computer device is invoked, and the method includes the following steps:
step S5100, querying an idle rate of operating resources of computer devices in the service cluster:
in order to realize optimal scheduling, when the mixed stream transcoding service needs to transfer part of transcoding service corresponding to a fixed code rate, the computer equipment can be optimized by inquiring the micro-service architecture of the service cluster. The running resource allowance in each computer device can be known by inquiring the running resource idle rate set by each computer in each cluster from the service cluster, so as to determine which computer device needs to be transferred with the transcoding task according to the running resource allowance.
Step S5200, creating an independent transcoding service adapted to transcode with the target bitrate in a computer device with the highest idle rate of running resources:
in order to make the operation of the independent transcoding service more robust, according to the idle rate information obtained by inquiring the service cluster, a computer device with the lowest idle rate is selected to be used as a target server for operating the independent transcoding service, then the independent transcoding service is created in the target server, and is set to perform transcoding output according to a certain target code rate, so that the target transcoding service has the capability of controlling the output of a target video stream according to the target code rate.
Step S5300, transmitting the target video stream output with the highest code rate to the independent transcoding service for transcoding:
after the configuration of the independent transcoding service is completed, the mixed stream transcoding service can start to transmit a path of target video stream output by the highest code rate in the mixed stream transcoding service to each independent transcoding service for independent transcoding, and each independent transcoding service receives the target video stream output by the highest code rate, decodes the target video stream again, then performs scaling, finally encodes the target video stream into a target video stream output by the corresponding target code rate, and pushes the target video stream to corresponding user terminal equipment.
The process of transferring from the internal transcoding service to the external independent transcoding service can be controlled by the mixed stream transcoding service implementing service logic, or can be controlled by the access service implementing service logic, so long as the switching time of the two is coordinated to ensure smooth switching of the target video stream finally reaching the user terminal equipment in the time domain.
In this embodiment, when the scheduling of the independent transcoding service is served, it is ensured that the computer device preferably having the optimal idle running resource implements the independent transcoding service, so that the running of the independent transcoding service is healthier and more stable, and the stable and smooth mixed stream transcoding service is implemented as a whole.
Referring to fig. 11, a mixed stream transcoding processing device according to one of the objects of the present application includes: a video stream receiving module 1100, a decoding mixed stream module 1200, a distributing transcoding module 1300, and a video stream pushing module 1400, wherein the video stream receiving module 1100 is configured to receive a video stream to be processed by a current mixed stream transcoding service; the decoding mixed stream module 1200 is configured to decode the video stream to be processed by the current mixed stream transcoding service to obtain an intermediate video stream; the distributing transcoding module 1300 is configured to distribute, by the current mixed stream transcoding service, the intermediate video stream to a plurality of internal transcoding services for encoding, each internal transcoding service encoding according to its corresponding fixed code rate to output a corresponding target video stream; the video stream pushing module 1400 is configured to push the target video stream obtained by transcoding the target video stream to a user terminal device matching a corresponding fixed code rate by each internal transcoding service.
In a further embodiment, the video stream receiving module 1100 includes: the time sequence reforming sub-module is used for adding the data frames of the video stream to be processed to the corresponding jitter buffer queue by the current mixed stream transcoding service to perform time sequence reforming, so that the data frames in the jitter buffer queue are dequeued in time sequence; the queue transfer sub-module is used for sequentially passing the data frames dequeued from the jitter buffer queue through the to-be-mixed flow queues corresponding to the jitter buffer queue; and the mixed stream output sub-module is used for using the data frames output from the to-be-mixed stream queue for the decoding mixed stream.
In a further embodiment, the decoding mixed stream module 1200 includes: the code rate searching sub-module is used for acquiring code rate information of user terminal equipment in the network live broadcast service; the service configuration sub-module is used for judging whether an internal transcoding service corresponding to the fixed code rate specified by the code rate information exists or not, and creating the internal transcoding service corresponding to the fixed code rate when the internal transcoding service does not exist; and the service association sub-module is used for establishing the corresponding relation of the user terminal equipment and the internal transcoding service corresponding to the fixed code rate in the code rate information thereof so as to push the target video stream generated by the internal transcoding service to the user terminal equipment.
In a further embodiment, the decoding mixed stream module 1200 includes the following structures for implementing each internal transcoding service: the code rate self-checking sub-module is configured to judge whether the code rate of the video stream to be processed is a fixed code rate corresponding to each internal transcoding service; the direct-conversion sub-module is used for directly encoding the intermediate video stream into a target video stream by using a fixed code rate corresponding to the direct-conversion sub-module when the previous judgment is established and then outputting the target video stream; and the scaling and converting sub-module is used for performing image scaling on the intermediate video stream according to the fixed code rate corresponding to the intermediate video stream when the previous judgment is not established, and then encoding the intermediate video stream into a target video stream and outputting the target video stream.
In an extended embodiment, the mixed stream transcoding processing device further includes: the alarm monitoring module is used for monitoring an alarm event that the usage amount of the computer operation resources occupied by the current mixed stream transcoding service exceeds a preset threshold value; the target determining module is used for responding to the alarm event and determining at least one relatively lower target code rate in the corresponding fixed code rates in the plurality of internal transcoding services; the response scheduling module is used for calling an independent transcoding service corresponding to the target code rate in another computer device and transmitting the target video stream which is obtained by the corresponding internal transcoding service code and is output at the highest code rate to the response scheduling module; and the independent transcoding module is configured to decode, scale and encode the target video stream according to the corresponding target code rate by each independent transcoding service, obtain the target video stream corresponding to the target code rate and push the target video stream to the user terminal equipment matching the target code rate.
In a further embodiment, the response scheduling module includes: the resource query sub-module is used for querying the idle rate of the running resources of the computer equipment in the service cluster; the service creation sub-module is used for creating an independent transcoding service which is suitable for transcoding with the target code rate in the computer equipment with the highest running resource idle rate; and the video transmission sub-module is used for transmitting the target video stream output at the highest code rate to the independent transcoding service for transcoding.
In order to solve the technical problems, the embodiment of the application also provides computer equipment. As shown in fig. 12, the internal structure of the computer device is schematically shown. The computer device includes a processor, a computer readable storage medium, a memory, and a network interface connected by a system bus. The computer readable storage medium of the computer device stores an operating system, a database and computer readable instructions, the database can store a control information sequence, and when the computer readable instructions are executed by a processor, the processor can realize a mixed stream transcoding processing method. The processor of the computer device is used to provide computing and control capabilities, supporting the operation of the entire computer device. The memory of the computer device may store computer readable instructions that, when executed by the processor, cause the processor to perform the hybrid stream transcoding method of the present application. The network interface of the computer device is for communicating with a terminal connection. It will be appreciated by those skilled in the art that the structure shown in fig. 12 is merely a block diagram of some of the structures associated with the present application and is not limiting of the computer device to which the present application may be applied, and that a particular computer device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.
The processor in this embodiment is configured to execute specific functions of each module and its sub-module in fig. 11, and the memory stores program codes and various data required for executing the above-mentioned modules or sub-modules. The network interface is used for data transmission between the user terminal or the server. The memory in this embodiment stores program codes and data required for executing all modules/sub-modules in the mixed stream transcoding processing device of the present application, and the server can call the program codes and data of the server to execute the functions of all sub-modules.
The present application also provides a storage medium storing computer readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the hybrid stream transcoding method of any of the embodiments of the present application.
The present application also provides a computer program product comprising computer programs/instructions which when executed by one or more processors implement the steps of the method described in any of the embodiments of the present application.
Those skilled in the art will appreciate that implementing all or part of the above-described methods of embodiments of the present application may be accomplished by way of a computer program stored on a computer readable storage medium, which when executed, may comprise the steps of embodiments of the methods described above. The storage medium may be a computer readable storage medium such as a magnetic disk, an optical disk, a Read-Only Memory (ROM), or a random access Memory (Random Access Memory, RAM).
In summary, the present application integrates the mixed stream and the transcoding process of the video stream by saving the encoding and decoding links of the mixed stream and the transcoding, so that the mixed stream and the transcoding efficiency of the video stream are higher, the transcoding delay and the transcoding cost can be reduced, and the image quality of the low-bitrate video stream can be improved.
Those of skill in the art will appreciate that the various operations, methods, steps in the flow, actions, schemes, and alternatives discussed in the present application may be alternated, altered, combined, or eliminated. Further, other steps, means, or steps in a process having various operations, methods, or procedures discussed in this application may be alternated, altered, rearranged, split, combined, or eliminated. Further, steps, measures, schemes in the prior art with various operations, methods, flows disclosed in the present application may also be alternated, altered, rearranged, decomposed, combined, or deleted.
The foregoing is only a partial embodiment of the present application, and it should be noted that, for a person skilled in the art, several improvements and modifications can be made without departing from the principle of the present application, and these improvements and modifications should also be considered as the protection scope of the present application.

Claims (9)

1. The mixed stream transcoding processing method is characterized by comprising the following steps of:
receiving a video stream to be processed by a current mixed stream transcoding service;
decoding the video stream to be processed by the current mixed stream transcoding service to obtain an intermediate video stream;
distributing the intermediate video stream to a plurality of internal transcoding services for encoding by the current mixed stream transcoding service, wherein each internal transcoding service encodes according to a corresponding fixed code rate so as to output a corresponding target video stream;
and pushing the target video stream obtained by transcoding the target video stream to user terminal equipment matched with the corresponding fixed code rate by each internal transcoding service.
2. The method of mixed stream transcoding according to claim 1, wherein the reception of the video stream to be processed by the current mixed stream transcoding service comprises the steps of:
the current mixed stream transcoding service adds the data frames of the video stream to be processed to the corresponding jitter buffer queues for time sequence reformation, so that the data frames in the jitter buffer queues are dequeued in time sequence;
sequentially passing the data frames dequeued from the jitter buffer queue through the to-be-mixed flow queues corresponding to the jitter buffer queue;
and using the data frames dequeued from the to-be-mixed stream queue for the decoding mixed stream.
3. The method of mixed stream transcoding according to claim 1, characterized in that before the intermediate video stream is distributed by the current mixed stream transcoding service to a plurality of internal transcoding services for encoding, it comprises the steps of:
acquiring code rate information of user terminal equipment in network live broadcast service;
judging whether an internal transcoding service corresponding to the fixed code rate specified by the code rate information exists or not, and creating the internal transcoding service corresponding to the fixed code rate when the internal transcoding service does not exist;
and establishing a corresponding relation of the user terminal equipment and the internal transcoding service corresponding to the fixed code rate in the code rate information of the user terminal equipment so as to push a target video stream generated by the internal transcoding service to the user terminal equipment.
4. The method of mixed stream transcoding according to claim 1, wherein each internal transcoding service encodes according to its corresponding fixed code rate to output a corresponding target video stream, comprising the steps of:
each internal transcoding service judges whether the code rate of the video stream to be processed is a fixed code rate corresponding to the internal transcoding service;
when the judgment is established, directly encoding the intermediate video stream into a target video stream by using a fixed code rate corresponding to the intermediate video stream, and outputting the target video stream;
And when the judgment is not established, performing image scaling on the intermediate video stream according to the fixed code rate corresponding to the intermediate video stream, and then encoding the intermediate video stream into a target video stream and outputting the target video stream.
5. The method for mixed stream transcoding according to any one of claims 1 to 4, further comprising the step of:
monitoring an alarm event that the usage amount of computer operation resources occupied by the current mixed stream transcoding service exceeds a preset threshold value;
responding to the alarm event, and determining at least one relatively lower target code rate in the corresponding fixed code rates in the plurality of internal transcoding services;
invoking an independent transcoding service corresponding to the target code rate in another computer device, and transmitting the target video stream which is obtained by the corresponding internal transcoding service code and is output with the highest code rate to the independent transcoding service;
and decoding, scaling and encoding the target video stream by each independent transcoding service according to the corresponding target code rate to obtain the target video stream corresponding to the target code rate so as to push the target video stream to the user terminal equipment matched with the target code rate.
6. The method of mixed stream transcoding according to claim 5, wherein invoking an independent transcoding service in another computer device corresponding to said target code rate comprises the steps of:
Inquiring the idle rate of the running resources of the computer equipment in the service cluster;
creating an independent transcoding service which is suitable for transcoding by the target code rate in the computer equipment with the highest running resource idle rate;
and transmitting the target video stream output with the highest code rate to the independent transcoding service for transcoding.
7. A mixed stream transcoding processing device, comprising:
a video stream receiving module configured to receive a video stream to be processed by a current mixed stream transcoding service;
the decoding mixed stream module is configured to decode the mixed stream of the video stream to be processed by the current mixed stream transcoding service to obtain an intermediate video stream;
a distribution transcoding module configured to distribute the intermediate video stream by a current mixed stream transcoding service to a plurality of internal transcoding services for encoding, each internal transcoding service encoding according to its corresponding fixed code rate to output a corresponding target video stream;
and the video stream pushing module is configured to push the target video stream obtained by transcoding the video stream to user terminal equipment matched with a corresponding fixed code rate by each internal transcoding service.
8. A computer device comprising a central processor and a memory, characterized in that the central processor is arranged to invoke a computer program stored in the memory for performing the steps of the method according to any of claims 1 to 6.
9. A computer-readable storage medium, characterized in that it stores in the form of computer-readable instructions a computer program implemented according to the method of any one of claims 1 to 6, which, when invoked by a computer, performs the steps comprised by the corresponding method.
CN202111266099.6A 2021-10-28 2021-10-28 Mixed stream transcoding processing method and device, equipment, medium and product thereof Active CN114007109B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111266099.6A CN114007109B (en) 2021-10-28 2021-10-28 Mixed stream transcoding processing method and device, equipment, medium and product thereof

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111266099.6A CN114007109B (en) 2021-10-28 2021-10-28 Mixed stream transcoding processing method and device, equipment, medium and product thereof

Publications (2)

Publication Number Publication Date
CN114007109A CN114007109A (en) 2022-02-01
CN114007109B true CN114007109B (en) 2023-05-12

Family

ID=79924832

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111266099.6A Active CN114007109B (en) 2021-10-28 2021-10-28 Mixed stream transcoding processing method and device, equipment, medium and product thereof

Country Status (1)

Country Link
CN (1) CN114007109B (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107332830A (en) * 2017-06-19 2017-11-07 腾讯科技(深圳)有限公司 Video code conversion, video broadcasting method and device, computer equipment, storage medium
CN108200358A (en) * 2018-03-20 2018-06-22 网宿科技股份有限公司 A kind of audio and video director method, system and control server
CN110233844A (en) * 2019-06-13 2019-09-13 杭州雅顾科技有限公司 A kind of multimedia live broadcast method, apparatus, equipment and medium
CN111107387A (en) * 2019-12-30 2020-05-05 广州酷狗计算机科技有限公司 Video transcoding method and device and computer storage medium
CN113423018A (en) * 2021-08-24 2021-09-21 腾讯科技(深圳)有限公司 Game data processing method, device and storage medium

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030028386A1 (en) * 2001-04-02 2003-02-06 Zinser Richard L. Compressed domain universal transcoder
US10298969B2 (en) * 2016-11-10 2019-05-21 University Of Louisiana At Lafayette Architecture and method for high performance on demand video transcoding

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107332830A (en) * 2017-06-19 2017-11-07 腾讯科技(深圳)有限公司 Video code conversion, video broadcasting method and device, computer equipment, storage medium
CN108200358A (en) * 2018-03-20 2018-06-22 网宿科技股份有限公司 A kind of audio and video director method, system and control server
CN110233844A (en) * 2019-06-13 2019-09-13 杭州雅顾科技有限公司 A kind of multimedia live broadcast method, apparatus, equipment and medium
CN111107387A (en) * 2019-12-30 2020-05-05 广州酷狗计算机科技有限公司 Video transcoding method and device and computer storage medium
CN113423018A (en) * 2021-08-24 2021-09-21 腾讯科技(深圳)有限公司 Game data processing method, device and storage medium

Also Published As

Publication number Publication date
CN114007109A (en) 2022-02-01

Similar Documents

Publication Publication Date Title
US9537667B2 (en) Duplicating digital streams for digital conferencing using switching technologies
US8832751B2 (en) Enhanced video streaming to mobile clients
CN101895718B (en) Video conference system multi-image broadcast method, and device and system thereof
CN101658000A (en) Method of transmitting data in a communication system
CN101088294A (en) Method for processing multimedia streams
KR20070039944A (en) System for layering content for scheduled delivery in a data network
CN104067593A (en) Packet streaming service capability exchange for enhanced peripheral device support
CN109068187A (en) Real-time traffic conveyer system and method
WO2013188150A1 (en) Technique for adapting device tasks based on the available device resources
CN103442391A (en) Data transmission method and system as well as user terminal
WO2023035894A1 (en) Data processing method, device, readable storage medium, and program product
US9270937B2 (en) Real time stream provisioning infrastructure
CN105187360A (en) Document conference sharing method and device
US20230231787A1 (en) Communication method and an apparatus
CN113992949B (en) Mixed flow service switching method and device, equipment, medium and product thereof
CN105338549A (en) Method and equipment for determining multimedia data sending mode
US20030153265A1 (en) Video and graphics distribution system for mobile users
CN100446562C (en) Mutimedium stream system for wireless manual apparatus
GB2494411A (en) Selecting a hardware processing module or a software processing module for processing a packet based call at a mobile device
CN114007109B (en) Mixed stream transcoding processing method and device, equipment, medium and product thereof
CN110784731B (en) Data stream transcoding method, device, equipment and medium
US11431779B2 (en) Network controlled uplink media transmission for a collaborative media production in network capacity constrained scenarios
CN101815073A (en) Embedded Bluetooth-Ethernet server
CN104429089A (en) Method of and apparatus for determining composite video services stream
US20150149638A1 (en) Resource Allocation

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant