WO2019228078A1 - 一种视频转码系统、方法、装置及存储介质 - Google Patents

一种视频转码系统、方法、装置及存储介质 Download PDF

Info

Publication number
WO2019228078A1
WO2019228078A1 PCT/CN2019/082227 CN2019082227W WO2019228078A1 WO 2019228078 A1 WO2019228078 A1 WO 2019228078A1 CN 2019082227 W CN2019082227 W CN 2019082227W WO 2019228078 A1 WO2019228078 A1 WO 2019228078A1
Authority
WO
WIPO (PCT)
Prior art keywords
video
transcoder
encoder
transcoding
encoding
Prior art date
Application number
PCT/CN2019/082227
Other languages
English (en)
French (fr)
Inventor
张宏顺
程曦铭
侯慧慧
Original Assignee
腾讯科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技（深圳）有限公司 filed Critical 腾讯科技（深圳）有限公司
Priority to EP19810652.8A priority Critical patent/EP3806477B1/en
Publication of WO2019228078A1 publication Critical patent/WO2019228078A1/zh
Priority to US16/996,132 priority patent/US11184627B2/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/40—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/107—Selection of coding mode or of prediction mode between spatial and temporal predictive coding, e.g. picture refresh
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/109—Selection of coding mode or of prediction mode among a plurality of temporal predictive coding modes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/127—Prioritisation of hardware or computational resources
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/177—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a group of pictures [GOP]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/2187—Live feed
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/231—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion
    • H04N21/23103—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion using load balancing strategies, e.g. by placing or distributing content on different disks, different memories or different servers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/231—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion
    • H04N21/23106—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion involving caching operations
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/23418—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234309—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4 or from Quicktime to Realvideo
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/24—Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests
    • H04N21/2404—Monitoring of server processing errors or hardware failure
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/24—Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests
    • H04N21/2405—Monitoring of the internal components or processes of the server, e.g. server load
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440218—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4

Definitions

  • the embodiments of the present application relate to the field of computer technology, and in particular, to a video transcoding system, method, device, and storage medium.
  • Video transcoding refers to converting a video code stream into another video code stream to meet different network bandwidths, different terminal processing capabilities, and different user needs. Video transcoding is essentially a re-encoding process, so the bitstream before and after the conversion may or may not follow the same video encoding standard.
  • the embodiments of the present application disclose a video transcoding system, method and related products, which are used to improve the reliability of video transcoding.
  • An embodiment of the present application provides a video transcoding system, including:
  • the main transcoder obtains an input video, transcodes the input video, and sends the transcoded video to a receiving device;
  • the backup transcoder obtains the input video and caches the input video, and obtains the transcoding position information of the main transcoder when the main transcoder is down, and according to the transcoding position information Transcoding the buffered input video to obtain an output video, and sending the output video to the receiving device.
  • an embodiment of the present application further provides a video transcoding method, which is executed by a computing device.
  • the method includes:
  • an embodiment of the present application further provides a video transcoding device, including a memory and a processor, where the memory is configured to store program instructions, and the program instructions are adapted to be loaded by the processor;
  • the processor is configured to load the program instructions and execute the video transcoding method according to any one of the embodiments provided in this application.
  • an embodiment of the present application further provides a storage medium, where a plurality of program instructions are stored in the storage medium, and the program instructions are adapted to be loaded by a processor and executed as described in any one of the embodiments provided in this application.
  • the video transcoding method is described.
  • an embodiment of the present application further provides a computer program product, where the computer program product includes multiple program instructions, and the program instructions are adapted to be loaded and executed by a processor as described in any one of the embodiments of the present application.
  • the video transcoding method is described.
  • FIG. 1 is a schematic structural diagram of a system according to an embodiment of the present application.
  • FIG. 2 is a schematic flowchart of a method according to an embodiment of the present application.
  • FIG. 3 is a schematic structural diagram of a standby transcoder according to an embodiment of the present application.
  • FIG. 4 is a schematic diagram of transcoding correspondence and splicing according to an embodiment of the present application.
  • FIG. 5 is a schematic diagram of a FIFO queue structure according to an embodiment of the present application.
  • FIG. 6 is a schematic flowchart of a method according to an embodiment of the present application.
  • FIG. 7 is a schematic structural diagram of a system according to an embodiment of the present application.
  • FIG. 8 is a schematic diagram of transcoding correspondence and splicing according to an embodiment of the present application.
  • FIG. 9 is a schematic diagram of packaging after transcoding according to an embodiment of the present application.
  • FIG. 10 is a schematic diagram of packaging after transcoding according to an embodiment of the present application.
  • FIG. 11 is a schematic diagram of a start encoding position after a handover according to an embodiment of the present application.
  • FIG. 12 is a schematic diagram of concurrent encoding and stitching video according to an embodiment of the present application.
  • FIG. 13 is a schematic diagram of a video transcoding apparatus according to an embodiment of the present application.
  • the term “if” can be construed as “when” or “once” or “in response to a determination” or “in response to a detection” depending on the context .
  • the phrase “if determined” or “if [the described condition or event] is detected” may be interpreted, depending on the context, to mean “once determined” or “in response to the determination” or “once [the condition or event described ] “Or” In response to [Description of condition or event] detected ".
  • a large-scale live video application scenario is taken as an example: a special person collects high-definition original video on the scene, and then uploads it to the background according to high-bandwidth encoding, and then transcodes into a stream suitable for different bandwidths and different resolutions. Sent to the user side for users to choose to watch. The entire process requires higher real-time and on-site synchronization; therefore, the requirements for the encoding process are correspondingly higher. Once the background transcoding is down, the video will not be viewed, and black screens and jams will occur.
  • the main processing method is: notify the receiving end to keep displaying the previous normal video, at the same time restart the background encoder, apply for an I frame to the sending end, and the encoder starts to pull the video stream again from the I frame. After decoding and re-encoding, it is sent to the receiving end.
  • the above solution will not be black, but it will be stuck for a long time, so the reliability is still very low.
  • FIG. 1 is a schematic diagram of a network system architecture provided by an embodiment of the present application, which includes a video acquisition device, a video transcoding system (also referred to as a video transcoding device), a transmission network, and a receiving device.
  • the video capture device collects the video source code and sends it to the video transcoding device. After the video transcoding device transcodes it, it sends it to the receiving device via the transmission network.
  • the video transcoding system includes a main transcoder and a backup transcoder.
  • the input on the left is the input video. Since the video is usually transmitted in the form of a stream, it can also be called an input stream or an input video stream. In the live broadcast industry, videos are generated on the user side.
  • the video After the video is transcoded, it is sent to the server or forwarded to the receiving device via the network. Therefore, the video needs to be uploaded, so the input stream can also be called upload stream, upstream stream, etc .;
  • the video stream input on the left will be sent to the main transcoder and the backup transcoder at the same time; when the main transcoder works normally, the backup transcoder does not need to perform the transcoding operation, and only needs to buffer the input. Video; in case the main transcoder goes down, the standby transcoder performs transcoding operations.
  • an embodiment of the present application provides a video transcoding system including a main transcoder and a backup transcoder.
  • the specific workflow of the video transcoding system is as follows:
  • the main transcoder and the backup transcoder obtain input video; the input video is transmitted to a receiving device after being transcoded by the main transcoder; the backup transcoder buffers the input video;
  • the above input video refers to a video input to a video transcoder, and generally can be a code stream of an original video collected by a video capture device, for example, a video code stream collected by a camera in a video live broadcast industry.
  • the process of transcoding the input data by the main transcoder can follow the general transcoding method.
  • the decoder decodes the input video, and then transcodes the decoded result into a stream suitable for different bandwidths and different resolutions.
  • the above main transcoder and standby transcoder can be independent hardware; if soft transcoding is used, then the above main transcoder and standby transcoder can respectively correspond to a transcoding process. Or a virtual transcoder. This embodiment of the present application is not uniquely limited.
  • the backup transcoder obtains the transcoding position information of the main transcoder; and in a case where the main transcoder is down, the backup transcoder performs buffering on the cached information according to the transcoding position information.
  • the input video is transcoded to obtain the output video and sent to the receiving device.
  • the above transcoding position information is used to locate the transcoding position when the main transcoder is down. In this way, the output video obtained by the transcoder that continues to transcode can be connected to the output video that has been transcoded.
  • the frame is a unit, so the transcoding position information here may include the frame serial number of the main transcoder during the downtime. In addition, the transcoding position will also correspond to the serial number of the transmitted data packet and other information.
  • the standby transcoder may determine that the primary transcoder is down, or it may be other hardware, virtual device, or process that informs the standby transcoder after learning that the primary transcoder is down.
  • the above output video is video data relative to the output video generated by the main transcoder.
  • the main transcoder is used in conjunction with the backup transcoder. Since the main transcoder will have a certain amount of untranscoded data during the process of transcoding the input video data, the backup transcoder is When the main transcoder is down, the untranscoded data can be continuously transcoded to achieve seamless migration. Therefore, the receiver does not experience stutters and black screens, and has higher reliability.
  • a more detailed structure diagram of a standby transcoder is provided, and the standby transcoder includes a decoder, a first encoder, and a second encoder;
  • the block diagram also contains memory for buffering input video.
  • the memory can be located outside the backup transcoder, such as the common storage of the main transcoder and the backup transcoder.
  • the buffered input video can be the original video or the The video decoded by the decoder; the former saves energy because it is not pre-decoded, and the latter has pre-decoding, so subsequent switching is faster. You can choose any of these two methods to cache the video according to actual needs.
  • the transcoding the buffered input video according to the transcoding position information by the backup transcoder includes:
  • the decoder decodes the cached input video according to the location information, and sends the cached decoded data to the first encoder, where the first encoder encodes the cached decoded data;
  • the decoder is further configured to decode the obtained new input video into newly decoded data, and send the newly decoded data to the second encoder, where the second encoder encodes the newly decoded data;
  • the new input video is an input video received after the main transcoder is down.
  • the first encoder is used for the connection of the video missing due to the main transcoder's downtime
  • the second encoder is used for the encoding of the new input video
  • the output video of the second encoder is connected after the first encoder completes the encoding.
  • the output of the first encoder; the two encoders included in the standby decoder are independently encoded; as shown in Figure 4 is a schematic diagram of the connection of video data.
  • the first two lines of the top row are assumed to be input videos, where the separation position is the video frame corresponding to the transcoding position information, then the first input video is transcoded by the main transcoder, and the second input video is converted by the backup.
  • Codec transcoding in which the second segment of video is decoded by the decoder of the backup transcoder, and the corresponding output video is encoded by the first encoder; the new input video is transcoded by the backup transcoder, and specifically by the backup transcoder
  • the decoder's decoder decodes and encodes the corresponding output video by the second encoder.
  • the last three videos can be seamlessly connected together.
  • the first encoder is used for the missing connection of the video caused by the main transcoder being down, and the second encoder is used for the encoding of the new input video; in order to adapt to the requirements of different encoders, the embodiments of this application may use The following scheme: the encoding delay of the first encoder is lower than the encoding delay of the second encoder; the encoding quality of the second encoder is higher than the encoding quality of the first encoder.
  • the above encoding quality may refer to a relatively complicated compression performance of an algorithm or an image encoding quality.
  • the first encoder can use the IPP frame mode encoding.
  • the IPP frame mode encoding refers to that the frame type in the code stream is only I-frames and P-frames. The delay is extremely small and the compression performance is slightly weak.
  • the second encoder can use the IPB frame mode encoding.
  • the IPB frame mode encoding refers to that the frame type in the code stream is I frame, B frame, and P frame.
  • the IPB frame mode has a slightly longer encoding delay, larger encoding complexity, and compression performance. higher.
  • I-frame is also called intra-coded frame, which is an independent frame with all the information. It can be decoded independently without referring to other images. It can be simply understood as a static picture.
  • the first frame in a video sequence is always an I frame because it is a key frame.
  • P-frames are also called inter-frame predictive coding frames, which need to refer to the previous I-frames for coding. Represents the difference between the current frame picture and the previous frame (the previous frame may be an I frame or a P frame). When decoding, you need to superimpose the differences defined in this frame with the previously buffered picture to generate the final picture.
  • P-frames usually occupy fewer data bits, but the disadvantage is that because P-frames have complex tolerances to previous P and I reference frames, they are very sensitive to transmission errors.
  • the B frame is also called a bidirectional predictive coding frame, that is, the B frame records the difference between the current frame and the previous and subsequent frames.
  • the B frame compression rate is high, but the decoding performance requirements are high.
  • the embodiment of the present application also provides an implementation solution for video frame packing:
  • the encoding result of the two encoders includes B frames; the second encoder is further configured to encode the video frame at the last non-B frame position of the buffered input video to obtain a group of pictures that can be continuously decoded, and group the pictures Cache
  • the picture group is sent to the output buffer, and then the first I frame or P frame of the video frame obtained by encoding the newly decoded data is sent to the output buffer.
  • the use of B frames by the second encoder can improve the encoding efficiency, and the picture group of the previous P frame is packed at the time of packing, so it can be decoded correctly and continuously when the video receiving end is decoded, thereby achieving different encoding quality Video seamless stitching.
  • a solution for avoiding conflicts in encoding results that may be caused by synchronous encoding of the first encoder and the second encoder is also provided: the first encoder is further configured to encode the video obtained by the first encoder The frame is sent to an output buffer; the second encoder is further configured to send a video frame obtained by encoding by the second encoder to a buffer;
  • the video frames in the buffer are sent to the output buffer in the order of first in, first out.
  • the video frames obtained by the second encoder are sent to the buffer first, instead of being sent directly to the output buffer. Therefore, the asynchronous method is used to avoid conflicts in the output buffer between the video frames obtained by the first encoder and the second encoder. , Resulting in the problem of dropped frames; correspondingly, the problem of displaying fancy screens at the receiving end is also avoided.
  • a specific implementation manner of buffering the input video is also provided, and the backup transcoder to buffer the input video includes:
  • the standby transcoder stores the input video stream in units of video frames through a first-in-first-out queue, and discards video frames that first enter the queue when the queue is full.
  • Figure 5 shows a schematic diagram of a first-in-first-out cache queue. Assume that each rectangle corresponds to the storage space of a frame, and the sequence number in the rectangle is the frame sequence number.
  • the first line indicates that the empty queue starts to be cached. When it is full, the third line continues to store after the queue is full, and the video frame with frame number 1 is discarded.
  • This queue can store a total of N video frames.
  • N can be set arbitrarily, and this embodiment of the present application does not limit this uniquely.
  • the transcoder obtaining the transcoding position information of the main transcoder includes:
  • the standby transcoder receives the hash information and frame number of the last video frame, the last audio frame number, and the last data packet number generated by the main transcoder and fed back by the monitoring device.
  • this embodiment provides a solution in this case, as follows:
  • the standby transcoder from The starting position of the buffer starts to transcode the input video to obtain the output video and send it to the receiving device.
  • the backup transcoder needs to continue transcoding, then it needs to know the frame number of "N-4"; the video will usually There are corresponding sounds.
  • the image frame corresponds to the audio frame.
  • the serial number of the audio frame must be obtained accordingly.
  • the encoding result will be sent out.
  • the transmission method is usually sent in the form of a data packet, and the packet number of the data packet needs to be obtained.
  • the standby transcoder knows from which video frame to start transcoding, which audio frame number is corresponding, and how to number the data packets when sending.
  • An embodiment of the present application further provides a video transcoding method, which can be executed by a computing device.
  • a video transcoding method which can be executed by a computing device.
  • the method includes:
  • 201 Send an input video to a main transcoder and a backup transcoder, so that the main transcoder transcodes the input video, so that the backup transcoder buffers the input video;
  • the execution subject of the method provided in this embodiment may be hardware having a transcoding control function, such as a computing device, or software or a virtual device having a transcoding control function.
  • a transcoding control function such as a computing device, or software or a virtual device having a transcoding control function.
  • the above main transcoder and standby transcoder can be independent hardware; if soft transcoding is used, then the above main transcoder and standby transcoder can respectively correspond to a transcoding process. Or a virtual transcoder.
  • This embodiment of the present application is not uniquely limited.
  • the above transcoding position information is used to locate the transcoding position when the main transcoder is down. In this way, the backup transcoder can continue to transcode and can continue to the sent video. Because transcoding is usually based on frames, The transcoding position information here may include the frame serial number of the main transcoder during the downtime. In addition, the transcoding position may correspond to the serial number of the transmitted data packet and other information.
  • the obtaining the transcoding position information of the main transcoder includes:
  • the main transcoder is used in conjunction with the backup transcoder. Since the main transcoder will have a certain amount of untranscoded data during the process of transcoding the input video data, the backup transcoder is When the main transcoder is down, the untranscoded data can be continuously transcoded to achieve seamless migration. Therefore, the receiver does not experience stutters and black screens, and has higher reliability.
  • an implementation manner of separately transcoding the cached input video and the new input video is also provided, as follows:
  • the controlling the transcoder performs buffering on the input video according to the transcoding position information.
  • Transcoding includes:
  • a first encoder controlling the standby decoder to transcode a buffered input video according to the transcoding position information
  • the method further includes:
  • the new input video is an input video received after the main transcoder is down.
  • the first encoder is used for the connection of the video missing due to the main transcoder's downtime
  • the second encoder is used for the encoding of the new input video
  • the output video of the second encoder is connected after the first encoder completes the encoding.
  • the output of the first encoder; the two encoders included in the standby decoder are independently encoded; as shown in Figure 4 is a schematic diagram of the connection of video data. This embodiment is particularly suitable for hard decoding and hard coding application scenarios.
  • this embodiment provides a solution that the encoding delay of the first encoder is lower than that of the second encoder. Encoding delay; the encoding quality of the second encoder is higher than the encoding quality of the first encoder.
  • the embodiment of the present application also provides an implementation solution for video frame packing:
  • the encoding result of the two encoders includes B frames; using the second encoder to encode the video frame at the last non-B frame position of the buffered input video to obtain picture groups that can be continuously decoded, and buffering the picture groups;
  • the picture group is sent to the output buffer, and then the first I frame or P frame of the video frame obtained by encoding the newly decoded data is sent to the output buffer.
  • the use of B frames by the second encoder can improve the encoding efficiency, and the picture group of the previous P frame is packed at the time of packing, so it can be decoded correctly and continuously when the video receiving end is decoded, thereby achieving different encoding qualities.
  • Video seamless stitching
  • a solution for avoiding conflicts in encoding results that may be caused by synchronous encoding of the first encoder and the second encoder is provided, that is, the method further includes:
  • the video frames in the buffer are sent to the output buffer in the order of first in, first out.
  • the video frames obtained by the second encoder are sent to the buffer first, instead of being sent directly to the output buffer. Therefore, the asynchronous method is used to avoid conflicts in the output buffer between the video frames obtained by the first encoder and the second encoder. , Resulting in the problem of dropped frames; correspondingly, the problem of displaying fancy screens at the receiving end is also avoided.
  • the two transcoders may be arranged symmetrically, that is, the structure of the main transcoder and the standby transcoder are the same, and the transcoder currently performing transcoding is used as the main transcoder, and the other transcoder As a standby transcoder, the method further includes:
  • the backup transcoder is set as the main transcoder
  • the downed main transcoder is set as the standby transcoder
  • the main transcoder that is down is restarted. Device.
  • two channels of codes can be created, one of which is the primary code and the other of which is the standby code. After the primary code is down, switch to the standby code.
  • the main encoding can include a decoder and an encoder, where the decoder is used to decode the upload stream and input the video source to the encoder, the encoder uses IPB frame mode encoding; and the other encoding process can include a decoder and Two encoders.
  • One of the two encoders encodes a low-latency IPP frame mode and the other encodes a high-quality IPB frame mode.
  • the backup encoding and the main encoding obtain the upload stream synchronously.
  • the decoder buffers certain decoded images.
  • the two encoders that are ready to encode do not encode after the creation, and start encoding when the main encoding is down. Then the code streams are stitched together to achieve Seamless connection of live streams after the main code is down.
  • this embodiment provides a solution in this case, that is, the method further includes:
  • the standby transcoder In the case of failure to obtain the transcoding position information, or in the case that the transcoding position information does not successfully determine the position of the correct transcoding when the main transcoder is down, the standby transcoder is controlled Transcoding the input video from the starting position of the buffer to obtain the output video and sending it to the receiving device.
  • FIG. 6 is a schematic flowchart of a video transcoding method according to an embodiment of the present application. As shown in Figure 6, the method includes the following steps:
  • the upload stream is sent to the main transcoder and the standby transcoder synchronously; the main transcoder performs transcoding, and the standby transcoder only caches a certain number of upload streams.
  • the cache method refer to the cache queue shown in FIG. 5.
  • Step 601 The proxy pulls the video stream from the main transcoder and finds that the main transcoder is disconnected. After the proxy obtains the interrupted status, the proxy pulls the video stream to the backup transcoder.
  • the above agent may be an interface function module with a receiving device.
  • the agent can send the transcoding position information to the standby transcoder when pulling the video stream (referred to as "pull stream").
  • Step 602 After receiving the proxy pull request, the standby transcoder performs two operations:
  • the low-latency encoder can output streams with a delay of nearly 0, high-quality encoding buffers new video frames and initializes them; when the low-latency encoder runs out of streams, the high-quality encoder's The initialization has been completed so the flow can continue.
  • the low-delay encoder stops, and the high-quality encoder continues to encode the output stream until the standby pull stream ends.
  • the low-delay encoder is used to offset the pre-processing delay of the high-quality encoder, and the zero-delay transition of the seamless switchover between the master and the backup is completed.
  • FIG 7 it is a schematic diagram of the system architecture, including: the master (master), the main transcoding server (Main transcoding server), backup transcoding server (Backup transcoding server) ); A main proxy (Main proxy), a backup proxy (Backup proxy), a data center (Data Center, DC), where there can be multiple data centers, Figure 2 illustrates DC0 and DC1 two.
  • the backup transcoding server and the main transcoding server correspond to the standby transcoder and the main transcoder respectively.
  • the source is the data of the original video frame, which corresponds to the upload stream mentioned above.
  • the source code is transcoded by the main transcoding server or backup transcoding server, it is sent to the data center by the master agent or backup agent.
  • the video stream can be sent to the receiving device.
  • the above video live broadcast system is initialized before the live broadcast is started.
  • the transcoding machine is divided into a main transcoding server and a backup transcoding server, and they all pull video data (that is, source code) from the video source server.
  • the main transcoding server is in the state of transcoding output stream.
  • the backup transcoder can only decode and not encode the input stream.
  • a first-in, first-out queue can be maintained inside the backup transcoding server, and the input video and audio data are buffered to a certain extent. Video frames and audio frames.
  • the main transcoding server packs the output, it can insert the original video frame decoding YUV (a color coding method) corresponding hash information into the data packet.
  • first-in-first-out queues which respectively cache peer-to-peer (P2P) headers, audio data packets, and decoded video frames.
  • P2P peer-to-peer
  • the DC triggers the proxy to pull the stream.
  • the proxy queries the status information of the video live broadcast from the master and performs its own initialization operation.
  • the proxy is divided into a primary proxy and a standby proxy. If a live user starts watching the video stream, the DC pulls the stream toward the proxy, and the proxy pulls the stream to the main transcoding server, maintaining a heartbeat connection with the main transcoding server. If the heartbeat connection times out and the proxy cannot pull the video stream, the proxy can record the hash information in the last data packet received, determine the last video frame sequence number, the last audio frame sequence number, and the last data packet number. And send this information to the backup transcoding server.
  • the standby transcoding server After receiving the information sent by the proxy, the standby transcoding server uses the hash information to find the corresponding video frame in the cache of the child thread. The main thread of the standby transcoding server starts to start encoding the new video stream. The child thread After the corresponding video frame is found, the low-latency mode is used to encode the video frame in the buffer; when outputting, the data packet corresponding to the low-latency frame is preferentially packed and output, and then the data packet corresponding to the normal-coded frame is output.
  • the DC pull stream is switched from the main transcoding server to the backup transcoding server.
  • the live video stream is continuous, so the picture and sound the user sees are continuous. .
  • the function of the proxy in the above live broadcast system is to proxy network users to obtain network information, and it is a relay station for network information.
  • the technical solution provided in the embodiment of the present application mainly includes three parts: a two-way transcoding design, an active / standby switchover, and a code stream splicing.
  • the following embodiments will describe these three aspects respectively.
  • the two transcodings correspond to the main transcoding and the backup transcoding respectively.
  • the main transcoding is completed by the main transcoder and the backup transcoding is completed by the backup transcoder.
  • the main transcoding and the backup transcoding can be hard or It is a soft transcoding. If it is a soft transcoding, the encoder and decoder included in the transcoder in the subsequent embodiments can be implemented by processes or sub-processes with corresponding functions, which will not be repeated in subsequent embodiments.
  • the main transcoder includes a decoder and an encoder, where the decoder is used to decode the upload stream, the decoding result is used as the input video source of the encoder, and the encoder uses IPB frame mode encoding for the video source from the decoder;
  • the backup encoder includes one decoder and two encoders, where the decoder is used to pull the stream in synchronization with the main transcoder to obtain the upload stream; cache a certain number of video frames of the upload stream; if the main transcoder is down, the backup
  • the encoder decodes the buffered video frame and passes the decoded result to one of the encoders.
  • the encoder uses low-latency IPP frame mode encoding. At the same time, the decoder also pulls new video frames and sends them to the other after decoding. Decoder using high-quality IPB frame mode encoding.
  • the standby transcoder will be immediately started for transcoding. You can use the standby transcoder as the main transcoder, and then restart the main transcoder that has been down, and restart the The main transcoder acts as a backup transcoder in preparation for the next switch.
  • the main transcoder uses IPB frame mode encoding, in order to further increase the compression rate, some pre-processing is generally performed by lookahead, so the real encoding process will be delayed by dozens of frames, and the number of delayed frames is determined by lookahead. The length is determined, so when the main transcoder is down, there will still be dozens of frames of data that are not encoded. Therefore, when starting the standby transcoder, two encoders can be started at the same time, that is, low-latency IPP frame mode coding and high-quality IPB frame mode coding.
  • the main transcoder or the backup transcoder after decoding the video frames of the uploaded stream, the hash value of each video frame can be calculated and recorded.
  • the main transcoder only needs to record the last frame, and the standby transcoder can record the latest N image frames, as shown in Figure 5, where N is greater than the length of the lookahead and buffers the latest N frames of decoded YUV data.
  • the first-in-first-out queue shown in FIG. 5 can be used to keep the video frames in the cache updated and correspond to the hash value of the video frames.
  • the main transcoder is down, the video frames buffered in the standby transcoder may not need to be continuously updated.
  • the two encoders included in the standby transcoder start encoding at the same time. Proceed as follows.
  • Low-latency encoder The first encoder shown in Figure 4, only encodes the YUV data in the cache when the main transcoder is down, and quickly encodes it in a non-delay manner.
  • the cache is judged by the hash value From which video frame to start encoding, the role of the hash value is positioning. After encoding the YUV data in the buffer, close the low-latency encoder.
  • Low-latency encoding can be implemented using x265. No B frame is generated. The encoder will output video frame data immediately after sending YUV data. There is no pre-processing process in the middle.
  • the I-frame interval does not need to be adjusted too much, and because the low-latency encoding efficiency is relatively low, the output bit rate does not need to be adjusted too high, so the output video frame The picture quality is relatively poor. Because there are not many buffered pending video frames and the proxy switching time is relatively short, low-latency encoding will not process too many video frames. After the main transcoder is down, users will see the time when the picture quality deteriorates. It was short and the image sound was continuous, and then the image quality returned to normal.
  • the second encoder shown in FIG. 4 can be configured the same as the encoder in the original main transcoder, and only encodes the YUV image decoded by the new video frame and continues encoding.
  • the entire upload stream has been transcoded, and the transcoded stream is a continuous process, with no frame loss and no peak code stream mutation.
  • This part mainly combines the video stream generated by the original main transcoder, the video stream generated by the low-delay encoder in the standby transcoder, and the video stream generated by the high-quality encoder into a complete video stream.
  • the low-latency encoder of the standby transcoder is the frame number of the image frame when the main transcoder goes down. Start encoding to obtain low-latency IPP image frames. High-quality encoders encode new video frames to obtain IPB image frames, which are connected in sequence.
  • the code stream splicing depends on the packaging of the image frames.
  • the embodiments of the present application also provide a specific packaging method of the image frames.
  • the encoder packing methods can be divided into two types, namely: the packing of low-delay encoder output and the main and standby machine encoder packing methods to ensure seamless connection of B-frame output. Corresponds to IPP image frame packing and IPB image frame packing respectively.
  • the output packing of the low-latency encoding is shown in FIG. 9, and the video data output by the encoder is shown in the upper part of the left I frame (the leftmost consecutive P frame) in the upper line of FIG.
  • the video data output by the encoder is based on the private protocol of the video application plus the private data header of the protocol header to obtain the hash information of the original YUV corresponding to the encoded output.
  • the fifth version of the message digest algorithm is used (message digest algorithm MD5, MD5) hash, put this hash data at the end of the entire single frame packet. In this way, each frame of video data is individually packaged and sent to the output buffer.
  • the output buffer is used to buffer the data to be sent.
  • Low-latency encoding does not have B-frame encoding.
  • Each frame, whether it is an I-frame or a P-frame, is encoded and packaged as described above. In this way, after the position where the proxy sends the interruption, the standby transcoder can always find the next video frame.
  • the main transcoder and the backup transcoder both perform hashing according to the data decoded by the original stream, so the corresponding hash value of the same video frame must be the same.
  • high-quality encoding includes B-frame packing:
  • the encoder's B-frame encoding will cause the encoding order of the image and the display order of the image, that is, the picture sequence (POC) is inconsistent, the downtime of the main encoder during the B-frame encoding will use the single frame packing Causes disordered output picture. Therefore, the video stream output from the main encoder cannot be combined with the output stream of the backup encoder.
  • the picture sequence POC
  • the solution provided by this embodiment is: buffering a group of pictures (GOP) that can be continuously decoded during packaging.
  • the GOP is called mini-GOP in x265.
  • the schematic diagram of packing is shown in Figure 10. The dashed boxes are added to represent that these frames of data must be processed together and sent to the output buffer together.
  • the encoder outputs I frames
  • the mini-GOP video frames in the buffer are packed together, and then the individual I frames are packed
  • the encoder outputs P frames the mini-GOP in the buffer is also packed.
  • Video frames are packed together, and then individual P frames are packed. In this way, each overall output can be used as a separate decoding sequence.
  • the continuous B frames output by the encoder can be set smaller to improve the encoding and packing efficiency.
  • the first non-B frame video hash information of the mini-GOP is obtained (the first video frame of each mini-GOP must be a non-B frame).
  • the corresponding private protocol header is added according to the private protocol of the video application.
  • Non-B frames directly put their own hash information at the end of the packet.
  • B frames copy a mini-GOP with a non-B at the beginning.
  • the hash information of the frame is also placed at the end of its own packet.
  • the previous mini-GOP can form a decodable video sequence, and the encoder's Open-GOP (Open-GOP) function can be closed.
  • the repeat-header can be enabled to integrate the picture parameter set (picture paramater set, PPS) in H.264 when each key frame is output. , Sequence parameter set (sequence paramater set, SPS) data packet.
  • the above packaging method can ensure that the proxy always breaks at the position of the dashed box every time the stream is interrupted, so that you know the hash information of the first non-B frame of the mini-GOP (this frame happens to be the mini-GOP playback order The last frame), you can connect the video streams at the transcoding standby machine.
  • the standby transcoder can maintain three first-in-first-out (FIFO) queues, which respectively buffer a certain number of P2P headers, audio packets, and decoded YUV video packets.
  • FIFO first-in-first-out
  • the status of the standby transcoder is stable. Once the stream request is received, it will immediately change to the status of the standby transcoder transcoding.
  • the main contents are as follows:
  • Part I Finding the video frame to start encoding after switching:
  • the proxy will send hash information to the standby transcoder, and it needs to find the corresponding video frame from the FIFO buffering the YUV video data packet.
  • the standby transcoder compares the hash value in the cache, finds the YUV frame with the same hash value, and starts the low-latency encoding from the next frame with the same hash value.
  • the original YUV of the new video frame after decoding is no longer Buffered YUV video data packets are stored in the FIFO, but sent directly to a high-quality encoder. At this point, two encoders start to encode concurrently.
  • the two encoders are on different threads.
  • the output of the low-latency encoding is shown in the previous part (all P frames) that is not enclosed by a dashed frame in Figure 9 or 10 above. This part of the encoded output is sent directly to the output buffer. Because the output encoded by a high-quality encoder needs to wait for the encoder with low latency to finish encoding before it can be output, it is temporarily stored in a buffer. After the low-latency encoder finishes encoding, the high-quality encoder starts to send the video frames in the buffer cache to the output buffer at a certain speed, and continues to encode the newly input video frames.
  • single-threaded control is used to output buffers to the output buffer and continue encoding, and the video on the receiving end has a certain probability of screen display. This is because there may be a large amount of data in the buffer, which may cause blocking of high-quality encoders, and the output video frames may be dropped.
  • This embodiment uses asynchronous buffer output, that is, the output of a high-quality encoder adopts an asynchronous output strategy; a high-quality encoder encodes normally, puts the result of the encoding in the buffer cache, and in the sub-thread from the buffer at a certain speed Fetch the data in the buffer, and then send the obtained video frame to the output buffer.
  • asynchronous buffer output that is, the output of a high-quality encoder adopts an asynchronous output strategy
  • a high-quality encoder encodes normally, puts the result of the encoding in the buffer cache, and in the sub-thread from the buffer at a certain speed Fetch the data in the buffer, and then send the obtained video frame to the output buffer.
  • the first transcoding result of the main transcoder is encoded using high-quality IPB frame mode, where the main transcoder fails before the second dashed box (mini-GOP) in the first line.
  • the transcoder backup and transcode machine
  • the part before the dotted line in the second line is the low-latency encoder encoding; the part in the dotted line in the second line is the high-quality encoder encoding.
  • the standby transcoder will use the mini-GOP and hash information when packaging. For details, refer to the description of the previous packaging, and will not be described here.
  • FIG. 13 is a schematic block diagram of a video transcoding device according to another embodiment of the present application.
  • the video transcoding apparatus in this embodiment as shown in FIG. 13 may include: one or more processors 1301; one or more input devices 1302, one or more output devices 1303, and a memory 1304.
  • the processor 1301, the input device 1302, the output device 1303, and the memory 1304 are connected through a bus 1305.
  • the memory 1302 is configured to store a computer program.
  • the computer program includes program instructions.
  • the processor 1301 is configured to execute the program instructions stored in the memory 1302.
  • the processor 1301 is configured to call the program instruction to perform the following operations:
  • the processor 1301 may be a central processing unit (CPU), and the processor may also be another general-purpose processor or a digital signal processor (DSP). , Application Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
  • a general-purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
  • the input device 1302 may include a touchpad, a fingerprint sensor (for collecting user's fingerprint information and fingerprint orientation information), a microphone, and the like, and the output device 1303 may include a display (LCD, etc.), a speaker, and the like.
  • a touchpad for collecting user's fingerprint information and fingerprint orientation information
  • a microphone for collecting user's fingerprint information and fingerprint orientation information
  • the output device 1303 may include a display (LCD, etc.), a speaker, and the like.
  • the memory 1304 may include a read-only memory and a random access memory, and provide instructions and data to the processor 1301. A part of the memory 1304 may further include a non-volatile random access memory. For example, the memory 1304 may also store device type information.
  • the processor 1301, the input device 1302, and the output device 1303 described in the embodiments of the present application may perform the video transcoding method provided by any embodiment of the present application, and details are not described herein again.
  • the processor 1301 in this embodiment may execute the method flow of the foregoing method embodiment. If the encoder and decoder are both soft encoders and decoders, the functions of the above encoders and decoders are implemented by program instructions executed by the processor 1301.
  • an embodiment of the present application further provides a video transcoding device, including a memory and a processor, where the memory is configured to store program instructions, and the program instructions are adapted to be loaded by the processor;
  • the processor is configured to load the program instructions and execute the video transcoding method according to any one of the embodiments provided in this application.
  • the embodiment of the present application further provides a storage medium, where a plurality of program instructions are stored in the storage medium, and the program instructions are suitable for being loaded by a processor and executing any of the program instructions provided in the embodiments of the present application.
  • the video transcoding method is described.
  • an embodiment of the present application further provides a computer program product, where the computer program product includes a plurality of program instructions, and the program instructions are suitable for being loaded by a processor and executing any one of the services provided by the embodiment of the application.
  • the video transcoding method is described.
  • the main transcoder is used in conjunction with the backup transcoder. Since the main transcoder will have a certain amount of untranscoded data during the process of transcoding the input video data, the backup transcoder is in the main transcoder. When the encoder is down, the untranscoded data can be continuously transcoded to achieve seamless migration. Therefore, the receiver will not experience stuttering and black screen, and has higher reliability.
  • the disclosed systems, methods, and devices may be implemented in other ways.
  • the device embodiments described above are only schematic.
  • the division of the above units is only a logical function division.
  • multiple units or components may be combined or may be combined. Integration into another system, or some features can be ignored or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, which may be electrical, mechanical or other forms.
  • the units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, may be located in one place, or may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objective of the solution of this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, or each of the units may exist separately physically, or two or more units may be integrated into one unit.
  • the above embodiments it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof.
  • software it may be implemented in whole or in part in the form of a computer program product.
  • the computer program product described above includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described above according to the embodiments of the present application are generated.
  • the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.
  • the computer instructions may be stored in a computer-readable storage medium, or transmitted through the computer-readable storage medium.
  • the above computer instructions can be transmitted from one website site, computer, server, or data center to another through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.).
  • the computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, a data center, and the like that includes one or more available medium integration.
  • the available medium may be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a digital versatile disc (DVD)), or a semiconductor medium (for example, a solid state disk (SSD)) Wait.
  • the processes may be completed by a computer program instructing related hardware.
  • the program may be stored in a computer-readable storage medium. When the program is executed, It may include the processes of the method embodiments described above.
  • the foregoing storage medium includes: a read-only memory (ROM) or a random access memory (RAM), a magnetic disk, or an optical disk, which can store various program codes.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

本申请实施例提供了一种视频转码系统、方法及相关产品，其中系统包括：主转码器和备转码器；所述主转码器获得输入视频，对所述输入视频进行转码，将转码后的视频发往接收设备；所述备转码器获得所述输入视频并缓存所述输入视频，在所述主转码器宕机的情况下获得所述主转码器的转码位置信息，依据所述转码位置信息对缓存的输入视频进行转码获得输出视频，将所述输出视频发往所述接收设备。

Description

一种视频转码系统、方法、装置及存储介质
本申请要求于2018年5月31日提交中国专利局、申请号为201810548780.1，申请名称为“一种视频转码系统、方法及相关产品”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及计算机技术领域，特别涉及一种视频转码系统、方法、装置及存储介质。
背景技术
视频转码(video transcoding)是指将视频码流转换成另一种视频码流，以适应不同的网络带宽、不同的终端处理能力和不同的用户需求。视频转码本质上是再编码的过程，因此转换前后的码流可能遵循相同的视频编码标准，也可能不遵循相同的视频编码标准。
发明内容
本申请实施例公开一种视频转码系统、方法及相关产品，用于提高视频转码的可靠性。
第一方面本申请实施例提供了一种视频转码系统，包括：
主转码器和备转码器；
所述主转码器获得输入视频，对所述输入视频进行转码，将转码后的视频发往接收设备；
所述备转码器获得所述输入视频并缓存所述输入视频，在所述主转码器宕机的情况下获得所述主转码器的转码位置信息，依据所述转码位置信息对缓存的输入视频进行转码获得输出视频，将所述输出视频发往所述接收设备。
第二方面本申请实施例还提供了一种视频转码方法，由计算设备执行，该方法包括：
将输入视频发往主转码器和备转码器，控制所述主转码器对所述输 入视频进行转码并将转码后的视频发往接收设备，控制所述备转码器缓存所述输入视频；
在所述主转码器宕机的情况下，获得所述主转码器的转码位置信息；
向所述备转码器发送所述转码位置信息，控制所述备转码器依据所述转码位置信息对缓存的输入视频进行转码获得输出视频，将所述输出视频发往所述接收设备。
第三方面本申请实施例还提供了一种视频转码装置，包括存储器和处理器，所述存储器用于存储程序指令，所述程序指令适于由所述处理器加载；
所述处理器，用于加载所述程序指令并执行如本申请实施例提供的任一项所述视频转码方法。
第四方面本申请实施例还提供了一种存储介质，所述存储介质中存储有多条程序指令，所述程序指令适于由处理器加载并执行如本申请实施例提供的任一项所述视频转码方法。
第五方面本申请实施例还提供了一种计算机程序产品，所述计算机程序产品包含多条程序指令，所述程序指令适于由处理器加载并执行如本申请实施例提供的任一项所述视频转码方法。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对本申请实施例中所需要使用的附图进行说明。
图1为本申请实施例系统结构示意图；
图2为本申请实施例方法流程示意图；
图3为本申请实施例备转码器结构示意图；
图4为本申请实施例转码对应及拼接示意图；
图5为本申请实施例先进先出队列结构示意图；
图6为本申请实施例方法流程示意图；
图7为本申请实施例系统结构示意图；
图8为本申请实施例转码对应及拼接示意图；
图9为本申请实施例转码后打包示意图；
图10为本申请实施例转码后打包示意图；
图11为本申请实施例切换后起始编码位置示意图；
图12为本申请实施例并发编码拼接视频的示意图；
图13为本申请实施例视频转码装置示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本申请一部分实施例，而不是全部的实施例。
应当理解，当在本说明书和所附权利要求书中使用时，术语“包括”和“包含”指示所描述特征、整体、步骤、操作、元素和/或组件的存在，但并不排除一个或多个其它特征、整体、步骤、操作、元素、组件和/或其集合的存在或添加。
还应当理解，在此本申请说明书中所使用的术语仅仅是出于描述特定实施例的目的而并不意在限制本申请。如在本申请说明书和所附权利要求书中所使用的那样，除非上下文清楚地指明其它情况，否则单数形式的“一”、“一个”及“该”意在包括复数形式。
还应当进一步理解，在本申请说明书和所附权利要求书中使用的术语“和/或”是指相关联列出的项中的一个或多个的任何组合以及所有可能组合，并且包括这些组合。
如在本说明书和所附权利要求书中所使用的那样，术语“如果”可以依据上下文被解释为“当...时”或“一旦”或“响应于确定”或“响应于检测到”。类似地，短语“如果确定”或“如果检测到[所描述条件或事件]”可以依据上下文被解释为意指“一旦确定”或“响应于确定”或“一旦检测到[所描述条件或事件]”或“响应于检测到[所描述条件或事件]”。
本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别不同对象，而不是用于描述特定顺序。
在本申请一实施例中，以大型视频直播应用场景为例：专人在现场 采集高清原始视频，然后按高带宽编码上传到后台，经后台转码成适应于不同带宽、不同分辨率的码流发送给用户侧，供用户选择观看。整个过程对实时和现场同步要求较高；因此，对编码过程要求也相应较高。一旦后台转码宕机，会导致观看不到视频，出现黑屏、卡死等现象。
目前针对转码宕机的问题，主要处理方式为：通知接收端保持显示上一个正常视频，同时重启后台的编码器，向发送端申请I帧，编码器从I帧开始重新拉取视频流，经解码和重新编码后发往接收端。但是以上方案虽然不会黑屏，但是会有长时间卡死，因此可靠性仍然很低。
如图1所示为本申请实施例提供的网络系统架构示意图，其中包含有视频采集设备、视频转码系统(也可称为视频转码设备)、传输网络以及接收设备。视频采集设备采集视频源码，发往视频转码设备，视频转码设备转码后经传输网络发往接收设备。其中视频转码系统中包含有主转码器和备转码器，左侧输入为输入视频，由于视频通常是以流形式传输的，因此也可以称为输入流，或者输入视频流等；另外，在直播行业，视频产生于用户侧，视频转码后会发往服务器或者经网络转发到接收设备，因此视频需要上传，所以输入流也可以称为上传流，上行流等；在该视频转码系统中，左侧输入的视频流会同时发给主转码器和备转码器；在主转码器正常工作的情况下，备转码器可以不必执行转码操作，仅需缓存输入视频；在主转码器宕机的情况下，备转码器执行转码操作。
如图2所示，本申请实施例提供了一种视频转码系统，包含主转码器和备转码器。视频转码系统的具体工作流程如下：
201：所述主转码器和所述备转码器获得输入视频；所述输入视频经所述主转码器转码后发往接收设备；所述备转码器缓存所述输入视频；
上述输入视频是指输入视频转码器的视频，通常来说可以是视频采集设备所采集的原始视频的码流，例如：视频直播行业中，由摄像头所采集的视频码流。
主转码器对输入数据进行转码的过程，可以遵循一般转码方法，先使用解码器对输入视频进行解码，然后将解码结果转码为适应于不同带宽、不同分辨率的码流。
如果采用硬转码，那么上述主转码器和备转码器可以是相互独立的硬件；如果使用软转码，那么上述主转码器和备转码器可以分别对应到一个转码进程，或者虚拟的转码器。对此本申请实施例不作唯一性限定。
202：所述备转码器获得所述主转码器的转码位置信息；在所述主转码器宕机的情况下，所述备转码器依据所述转码位置信息对缓存的输入视频进行转码获得输出视频发往所述接收设备。
以上转码位置信息用于定位主转码器宕机时转码的位置，这样备转码器继续转码得到的输出视频则可以接续到已转码得到的输出视频；由于转码通常是以帧为单位的，因此这里的转码位置信息可以包含宕机时主转码器转码的帧序号，另外相应地转码位置还会对应有已发送的数据包的序号等信息。
获得主转码器是否宕机的方式有很多，例如对主转码器的输出视频进行监测，如果没有输出视频则可以确定宕机了；本申请实施例对如何确定主转码器是否宕机的具体手段不作唯一性限定。对于备转码器而言，可以是备转码器自己确定主转码器宕机，也可以是其他硬件、虚拟设备或进程获知主转码器宕机后告知备转码器。
由于备转码器输出的视频可以接续主转码器宕机导致的视频中断，因此上述输出视频是相对于主转码器生成的输出视频而言的视频数据。
基于本实施例的说明，使用主转码器和备转码器配合，由于主转码器在对输入视频数据进行转码过程中会存在一定数量的未转码的数据，备转码器在主转码器宕机的情况下继续对这些未转码的数据进行转码，可以实现无缝迁移，因此接收端不会出现卡顿和黑屏现象，具有更高的可靠性。
在一个实施例中，如图3所示，提供了备转码器更详细的结构图，所述备转码器包括解码器、第一编码器和第二编码器；在图3所示的结构图中还包含了存储器用于缓存输入视频。需说明的是，该存储器可以位于备转码器之外，例如主转码器和备转码器的公共存储器；另外，对于存储器而言，缓存的输入视频可以是原始视频，也可以是经过解码器解码后的视频；前者由于未预解码会更节省能源，后者有预解码因此后 续切换更快，可以根据实际需求选择这两种方式中的任意方式缓存视频。
所述备转码器依据所述转码位置信息对缓存的输入视频进行转码包括：
所述解码器依据所述位置信息对缓存的输入视频进行解码后将缓存解码数据发往所述第一编码器，所述第一编码器对所述缓存解码数据进行编码；
所述解码器还用于将获得的新输入视频解码为新解码数据，将所述新解码数据发往所述第二编码器，所述第二编码器对所述新解码数据进行编码；所述新输入视频为所述主转码器宕机后接收到的输入视频。
基于本实施例，第一编码器用于主转码器宕机导致的视频缺失的接续，第二编码器用于新输入视频的编码，在第一编码器完成编码后第二编码器的输出视频接续第一编码器的输出；备解码器包含的两个编码器分别独立编码；如图4所示为视频数据的接续示意图。
如图4中，最上一行前两格假定为输入视频，其中分隔位置为转码位置信息对应的视频帧，那么第一段输入视频由主转码器转码，第二段输入视频由备转码器转码，其中第二段视频具体由备转码器的解码器解码后经第一编码器编码得到相应的输出视频；新输入视频则由备转码器转码，具体由备转码器的解码器解码后经第二编码器编码得到相应的输出视频。最后输出的三段视频可以无缝接续在一起。
在一个实施例中，第一编码器用于主转码器宕机导致的视频缺失的接续，第二编码器用于新输入视频的编码；为了适于不同编码器的需求，本申请实施例可以采用如下方案：所述第一编码器的编码时延低于所述第二编码器的编码时延；所述第二编码器的编码质量高于所述第一编码器的编码质量。
上述编码质量可以指算法相对复杂压缩性能或者图像编码质量。
其中第一编码器可以采用IPP帧模式编码，IPP帧模式编码是指码流中帧类型只有I帧和P帧，时延极小，压缩性能略弱。第二编码器可以采用IPB帧模式编码，IPB帧模式编码是指码流中帧类型有I帧、B帧及P帧，该IPB帧模式编码时延略大，编码复杂度较大，压缩性能更 高。
其中，I帧又称帧内编码帧，是一种自带全部信息的独立帧，无需参考其他图像便可独立进行解码，可以简单理解为一张静态画面。视频序列中的第一个帧始终都是I帧，因为它是关键帧。
P帧又称帧间预测编码帧，需要参考前面的I帧才能进行编码。表示的是当前帧画面与前一帧(前一帧可能是I帧也可能是P帧)的差别。解码时需要用之前缓存的画面叠加上本帧定义的差别，生成最终画面。与I帧相比，P帧通常占用更少的数据位，但不足是，由于P帧对前面的P和I参考帧有着复杂的依耐性，因此对传输错误非常敏感。
B帧又称双向预测编码帧，也就是B帧记录的是本帧与前后帧的差别。也就是说要解码B帧，不仅要取得之前的缓存画面，还要解码之后的画面，通过前后画面的与本帧数据的叠加取得最终的画面。B帧压缩率高，但是对解码性能要求较高。
在一个实施例中，由于第二编码器编码效率更高，使用B帧，该B帧需要前后的P帧才能正确解码，因此本申请实施例还提供了视频帧打包的实现方案：所述第二编码器的编码结果包含B帧；所述第二编码器还用于对所述缓存输入视频最后一个非B帧位置的视频帧进行编码得到能被连续解码的画面组，将所述画面组缓存；
将所述画面组打包发往输出缓存，然后打包对新解码数据编码得到的视频帧的第一个I帧或P帧发往所述输出缓存。
在本实施例中，第二编码器使用B帧可以提高编码效率，在打包时打包了前面一个P帧的画面组，因此在视频接收端解码时可以正确地连续解码，从而实现了不同编码质量的视频无缝拼接。
在一个实施例中，还提供了避免第一编码器和第二编码器同步编码可能导致的编码结果冲突的方案：所述第一编码器还用于将所述第一编码器编码获得的视频帧发往输出缓存；所述第二编码器还用于将所述第二编码器编码获得的视频帧发往缓冲区；
在所述第一编码器编码结束后，按照先入先出的顺序将所述缓冲区内的视频帧发往所述输出缓存。
本实施例将第二编码器获得的视频帧先发往缓冲区，而不是直接发往输出缓存，因此采用异步方式避免了第一编码器和第二编码器分别获得的视频帧在输出缓存冲突，导致丢帧的问题；相应地也避免了在接收端出现显示花屏的问题。
在一个实施例中，还提供了缓存输入视频的具体实现方式，所述备转码器缓存所述输入视频包括：
所述备转码器通过先进先出的队列以视频帧为单位存储所述输入视频流，在所述队列满时丢弃先进入所述队列的视频帧。
如图5所示为先进先出缓存队列的示意图，假定每个矩形格对应一个帧的存储空间，矩形格内的序号为帧序号；第一行示意为空队列开始缓存，第二行为队列刚好存满，第三行为队列满后继续存储，丢弃了帧序号为1的视频帧。该队列总共可以存储N个视频帧。这里的N可以任意设定，本申请实施例对此不作唯一性限定。
在一个实施例中，还提供了转码位置信息的具体可选参数：所述备转码器获得所述主转码器的转码位置信息包括：
所述备转码器接收由监控设备反馈的所述主转码器生成的最后一个视频帧的哈希信息和帧序号、最后一个音频帧序号，以及最后一个数据包包号。
在一个实施例中，由于转码位置信息可能获取失败，本实施例提供了此种情况下的解决方案，具体如下：
在获取所述转码位置信息失败的情况下，或者，在所述转码位置信息未成功确定所述主转码器宕机时正确转码的位置的情况下，所述备转码器从所述缓存的起始位置开始对输入视频进行转码获得输出视频发往所述接收设备。
如图5所示，假定在帧序号为N-4的视频帧主转码器宕机了，那么备转码器需要继续转码，那么需要获知“N-4”这个帧序号；视频通常会有相应的声音，图像帧与音频帧是对应的，相应地要获得音频帧序号；另外，编码结果会发送出去，发送方式通常是以数据包形式发送的，还需获得数据包的包号。由此，备转码器就知道从哪一个视频帧开始继续 转码，对应的音频帧号是哪一个，发送时如何对数据包进行编号。
本申请实施例还提供了一种视频转码方法，该方法可以由计算设备执行。该方法的具体流程可参考前一实施例中视频转码系统的说明，如图2所示，该方法包括：
201：将输入视频发往主转码器和备转码器，使所述主转码器对所述输入视频进行转码，使所述备转码器缓存所述输入视频；
特别地，本实施例提供的方法的执行主体可以是具有转码控制功能的硬件，如计算设备，也可以是具有转码控制功能的软件或者虚拟设备。如果采用硬转码，那么上述主转码器和备转码器可以是相互独立的硬件；如果使用软转码，那么上述主转码器和备转码器可以分别对应到一个转码进程，或者虚拟的转码器。对此本申请实施例不作唯一性限定。
202：在所述主转码器宕机的情况下，获得所述主转码器的转码位置信息；
以上转码位置信息是用于定位主转码器宕机时转码的位置，这样备转码器继续转码则可以接续到已发送的视频；由于转码通常是以帧为单位的，因此这里的转码位置信息可以包含宕机时主转码器转码的帧序号，另外相应地转码位置还会对应有已发送的数据包的序号等信息。
在一个实施例中，所述获得所述主转码器的转码位置信息包括：
获得所述主转码器生成的最后一个视频帧的哈希信息和帧序号、最后一个音频帧序号，以及最后一个数据包包号。
203：向所述备转码器发送所述转码位置信息，控制所述备转码器依据所述转码位置信息对缓存的输入视频进行转码。
基于本实施例的说明，使用主转码器和备转码器配合，由于主转码器在对输入视频数据进行转码过程中会存在一定数量的未转码的数据，备转码器在主转码器宕机的情况下继续对这些未转码的数据进行转码，可以实现无缝迁移，因此接收端不会出现卡顿和黑屏现象，具有更高的可靠性。
在一个实施例中，还提供了缓存的输入视频和新输入视频分别独立转码的实现方式，具体如下：所述控制所述备转码器依据所述转码位置 信息对缓存的输入视频进行转码包括：
控制所述备解码器的第一编码器依据所述转码位置信息对缓存的输入视频进行转码；
所述方法还包括：
将新输入视频发往所述第二编码器，使所述第二编码器对所述新解码数据进行编码；所述新输入视频为所述主转码器宕机后接收到的输入视频。
基于本实施例，第一编码器用于主转码器宕机导致的视频缺失的接续，第二编码器用于新输入视频的编码，在第一编码器完成编码后第二编码器的输出视频接续第一编码器的输出；备解码器包含的两个编码器分别独立编码；如图4所示为视频数据的接续示意图。本实施例特别适应于硬解码和硬编码的应用场景。
在一个实施例中，为了适应于缓存的输入视频和新输入视频不同的转码需求，本实施例提供了如下方案：所述第一编码器的编码时延低于所述第二编码器的编码时延；所述第二编码器的编码质量高于所述第一编码器的编码质量。
在一个实施例中，由于第二编码器编码效率更高，使用B帧，该B帧需要前后的P帧才能正确解码，因此本申请实施例还提供了视频帧打包的实现方案：所述第二编码器的编码结果包含B帧；使用所述第二编码器对所述缓存输入视频最后一个非B帧位置的视频帧进行编码得到能被连续解码的画面组，将所述画面组缓存；
将所述画面组打包发往输出缓存，然后打包对新解码数据编码得到的视频帧的第一个I帧或P帧发往所述输出缓存。
在本实施例中，第二编码器使用B帧可以提高编码效率，在打包时打包了前面一个P帧的画面组，因此在视频接收端解码时可以正确地连续解码，从而实现了不同编码质量的视频无缝拼接。
在一个实施例中，还提供了避免第一编码器和第二编码器同步编码可能导致的编码结果冲突的方案，即所述方法还包括：
将所述第一编码器编码获得的视频帧发往输出缓存；将所述第二编 码器编码获得的视频帧发往缓冲区；
在所述第一编码器编码结束后，按照先入先出的顺序将所述缓冲区内的视频帧发往所述输出缓存。
本实施例将第二编码器获得的视频帧先发往缓冲区，而不是直接发往输出缓存，因此采用异步方式避免了第一编码器和第二编码器分别获得的视频帧在输出缓存冲突，导致丢帧的问题；相应地也避免了在接收端出现显示花屏的问题。
在一个实施例中，两个转码器可以是对称设置的，即：主转码器和备转码器结构相同，以当前执行转码的转码器作为主转码器，另一转码器作为备转码器，那么所述方法还包括：
在所述主转码器宕机后将所述备转码器设置为主转码器，将已宕机的主转码器设置为备转码器，重启已宕机的所述主转码器。
基于前文说明，本申请实施例可以创建两路编码，一路为主编码，另一路为备编码，主编码宕机后，切换到备编码。主编码可以包含一个解码器和一个编码器，其中的解码器用于上传流的解码，并给编码器输入视频源，编码器采用IPB帧模式编码；而另一路备编码过程可以包括一个解码器和两个编码器，其中，两个编码器中一个为低时延的IPP帧模式编码，另一个为高质量的IPB帧模式编码。备编码和主编码同步获取上传流，解码器解码后缓存一定解码图像，备编码的两个编码器创建后并不编码，当主编码宕机后才开始编码；然后将码流拼接在一起，实现主编码宕机后直播码流的无缝衔接。
在一个实施例中，由于转码位置信息可能获取失败，本实施例提供了此种情况下的解决方案，即所述方法还包括：
在获取所述转码位置信息失败的情况下，或者，在所述转码位置信息未成功确定所述主转码器宕机时正确转码的位置的情况下，控制所述备转码器从所述缓存的起始位置开始对输入视频进行转码获得输出视频发往所述接收设备。
图6为本申请实施例提供的视频转码方法的流程示意图。如图6所示，该方法包括如下步骤：
上传流同步发往主转码器和备转码器；其中主转码器会执行转码，备转码器仅缓存一定数量的上传流，缓存方式可以参考图5所示的缓存队列。
步骤601：代理(proxy)从主转码器拉视频流，发现主转码器断流，那么proxy获取断流状态后，代理向备转码器拉视频流。
上述代理可以是与接收设备之间的接口功能模块。代理可以在拉视频流(简称“拉流”)时将转码位置信息发送给备转码器。
步骤602：备转码器在收到proxy的拉流请求后，执行两部分操作：
A、低时延拉流：
将缓存(buffer)中的缓存图像帧放到低时延的编码器中编码，同时可以启动高质量的编码器，将新视频帧发往高质量的编码器；
在以上A部分中，低时延的编码器可以近0时延的出流，高质量编码缓存新的视频帧并初始化；低时延的编码器出流耗尽时，高质量的编码器的初始化已经完成故可以继续出流。
B、高质量拉流：
在低时延的编码器出流耗尽时，低时延的编码器停机，高质量的编码器继续编码出流，直至备拉流结束。
在以上流程中，依靠低时延的编码器来抵消高质量编码器的前处理时延，完成主备无缝切换的0时延过渡。
以大型直播系统的视频转码为例，如图7所示，为系统架构示意图，包括：主机(master)，其下的主转码服务器(Main transcoding server)、备转码服务器(Backup transcoding server)；主代理(Main proxy)、备代理(Backup proxy)、数据中心(Data Center，DC)，其中数据中心可以有多个，在图7中示意了DC0和DC1两个。其中备转码服务器和主转码服务器分别对应到前文中的备转码器和主转码器。
源码(source)即原始的视频帧的数据，对应到前文中的上传流；源码经主转码服务器或备转码服务器转码后被主代理或备代理拉流发往数据中心，数据中心的视频流可以发往接收设备。
以上视频直播系统在开始直播前会初始化，在完成初始化之后，转 码机器分成主转码服务器和备份转码服务器，它们都会从视频源服务器拉取视频数据(即源码)。其中主转码服务器处于转码输出流的状态，备转码器对输入流可以只解码不编码，在备份转码服务器内部可以维护一个先入先出队列，对输入的视频和音频数据缓存一定的视频帧和音频帧。主转码服务器在打包输出的时候可以在数据包中插入原始视频帧解码YUV(一种颜色编码方法)对应的哈希(hash)信息。其中先入先出队列的数量可以有多个，分别缓存对等网络(peer to peer，P2P)包头，音频数据包以及解码之后的视频帧。P2P包头、音频数据包与视频帧之间具有对应关系，在时间上相关联。
具体流程如下：
在Proxy启动之后，由DC触发proxy拉流；DC触发proxy拉流时，如果proxy处于未初始化的状态，proxy从master查询视频直播的状态信息，执行自身的初始化操作。
Proxy分成主proxy和备proxy，如果有直播用户开始观看视频流，DC便朝proxy拉流，proxy向主转码服务器拉流，与主转码服务器之间维持心跳连接。如果心跳连接超时，并且proxy拉取不到视频流，那么proxy可以记录收到的最后一个数据包中的hash信息，确定最后一个视频帧序号，最后一个音频帧序号，最后一个数据包包号，并把这些信息发送给备转码服务器。
备转码服务器在收到proxy发送的信息之后，利用其中的hash信息在子线程中的缓存中寻找到对应的视频帧，备转码服务器的主线程开始启动对新视频流的编码，子线程找到对应视频帧之后，使用低时延模式编码缓存内的视频帧；在输出时优先打包输出低时延的帧对应的数据包，然后输出正常编码帧对应的数据包。
基于以上流程，主转码服务器宕机后，DC拉流从主转码服务器切换到备份转码服务器，直播的视频流是连续的，因此用户看到的画面和听到的声音都是连续的。
在以上直播系统中代理(Proxy)的功能是代理网络用户去取得网络信息，它是网络信息的中转站。
如图6所示，本申请实施例提供的技术方案主要包含：双路转码设计、主备切换、以及码流拼接3个部分，以下实施例将分别就这三个方面进行说明。
(1)双路编码设计：
两路转码分别对应主转码和备转码，其中主转码由主转码器完成，备转码由备转码器完成，其中主转码和备转码可以是硬转码也可以是软转码，如果是软转码后续实施例中的转码器所包含的编码器和解码器都可以由相应功能的进程或子进程实现，后续实施例将不再赘述。
主转码器包含一个解码器和一个编码器，其中的解码器用于对上传流解码，将解码结果作为编码器的输入视频源，编码器对来自解码器的视频源采用IPB帧模式编码；
备编码器包含一个解码器和两个编码器，其中，解码器用于与主转码器同步拉流获得上传流；缓存上传流的一定数量的视频帧；如果主转码器宕机，那么备编码器对缓存的视频帧进行解码，将解码结果传递给其中一个编码器，该编码器采用低时延的IPP帧模式编码，同时解码器还拉取新视频帧，在解码后发送给另一个采用高质量的IPB帧模式编码的解码器。
(2)主备切换：
基于前文介绍，一旦主转码器宕机，会立即启动备转码器进行转码，可以将备转码器作为主转码器，然后重启已经宕机的主转码器，并将重启的主转码器作为备转码器以备下次切换。
由于在主转码器中采用是IPB帧模式编码，为了进一步提高压缩率，一般会通过提前量(lookahead)做一些前处理，因此真正编码过程会延迟几十帧，延迟的帧数由lookahead的长度决定，因此在主转码器宕机时，还会存在几十帧的数据没有编码。因此，启动备转码器时，可以同时启动2个编码器，即低时延的IPP帧模式编码和高质量的IPB帧模式编码。
不管是主转码器还是备转码器，在对上传流的视频帧解码后，均可以计算每视频帧的hash值并记录下来。主转码器只需记录最后一帧，备 转码器可以记录最新的N个图像帧，如图5所示，其中N大于lookahead的长度，缓存最新的N帧解码的YUV数据。使用如图5所示的先进先出队列可以保持缓存内的视频帧保持更新，并与视频帧的hash值对应。但主转码器宕机时，备转码器内缓存的视频帧可以不必继续更新。
当启动备转码器时，备转码器所包含的两个编码器同时开始编码。按下列方式操作。
低时延编码器：如图4中所示的第一编码器，只编码从主转码器宕机时，缓存内的YUV数据，以无时延的方式快速编码，通过hash值来判断缓存内从哪一个视频帧开始编码，hash值的作用就是定位。对缓存内的YUV数据编码完成后，关闭该低时延编码器。低时延编码可以使用x265实现，不产生B帧，编码器送入YUV数据后会立刻输出视频帧的数据，中间没有预处理过程。由于此时低时延编码处理的帧比较少，这里I帧间隔不需要调节太大，同时由于低时延编码效率比较低，此时输出码率也不需调节过高，所以输出的视频帧的画面质量相对较差。由于缓存的待处理视频帧不多，proxy切换的时间也比较短，因此低时延编码不会处理太多的视频帧，在主转码器宕机后用户看到画面质量变差的时间会很短，并且图像声音是连续的，随后图像质量恢复正常。
高质量编码器：如图4中所示的第二编码器，和原先主转码器中的编码器的配置可以一样，只编码新视频帧解码后的YUV图像，并一直编码下去。
这样，整个上传流都经过了转码，转码后的码流是个连续过程，没有帧丢失，也没有峰值码流突变。
(3)码流拼接：
该部分主要是将原主转码器生成的视频流、备转码器中低时延编码器生成的视频流以及高质量编码器生成的视频流拼接成一个完整的视频流。如图8所示，在原主转码器宕机前会有第一段主转码器的视频流，备转码器的低时延编码器从主转码器宕机时图像帧的帧序号开始编码获得低时延的IPP的图像帧，高质量编码器对新视频帧进行编码得到IPB的图像帧，依次衔接。
码流拼接依赖于图像帧的打包，本申请实施例还提供了图像帧的具体打包方式。编码器打包方式可以分成两类，即：低时延编码器输出的打包和保证B帧输出无缝衔接的主备机编码器打包方式。分别对应到前文中IPP的图像帧打包和IPB的图像帧的打包。
一、低时延编码的打包：
低时延编码的输出打包参考图9所示，编码器输出的视频数据如图9上面一行左侧I帧前面的部分(最左边的连续的P帧)。首先编码器输出的视频数据按照视频应用的私有协议加上协议头(header)的私有数据头，获得对应编码输出的原始YUV的哈希(hash)信息，目前使用的是消息摘要算法第五版(message digest algorithm MD5，MD5)hash，把这个hash数据放在了整个单帧数据包的结尾。这样每一帧视频数据单独打包发往输出缓存。输出缓存是用来缓存待发送数据的缓存。
低时延编码没有B帧编码，每一个帧无论是I帧还是P帧编码打包都按照上面的方式来操作。这样就可以保证proxy发送断流的位置之后，备转码器总是可以找到下一个视频帧。而且主转码器和备转码器都是按照原始流解码之后的数据进行hash，所以相同的视频帧对应的hash值一定相同。
二、高质量编码包含B帧的打包方式：
由于编码器的B帧编码会导致图像的编码顺序和图像的显示顺序，即图像序列(picture order count，POC)不一致，主编码器宕机位置在B帧编码的时候如果采用单帧打包方式会导致输出画面紊乱。因此主编码器输出的视频流没有办法和备编编码器的输出流合并在一起。
本实施例提供的解决方案是：在打包时缓存可以连续解码的画面组(group of pictures，GOP)，该GOP在x265中叫做迷你GOP(mini-GOP)。打包的示意图如图10所示，其中添加了虚线框代表这几帧数据要一起处理好之后整体一起发往输出缓存。当遇到编码器输出I帧时，把缓存中的mini-GOP的视频帧一起打包，然后再把单独的I帧打包；遇到编码器输出P帧时，也把缓存中的mini-GOP的视频帧一起打包，然后把单独的P帧打包。这样每一个整体的输出都可以作为一个单独的解码序 列。在本实施例中，编码器输出的连续B帧可以设置得小一些，提高编码打包效率。
具体到某一个虚线框内视频帧打包时，首先会获得mini-GOP的第一个非B帧的视频hash信息(每个mini-GOP的第一个视频帧一定是一个非B帧)，虚线框内部每一个视频帧打包的时候先按照视频应用的私有协议加上对应的私有协议头，非B帧直接把自己的hash信息放在包的末尾，B帧复制一份mini-GOP开头非B帧的hash信息也放在自己包的末尾。
为了保证编码器输出P帧时，上一个mini-GOP能够形成一个可解码的视频序列，可以关闭编码器的开放GOP(Open-GOP)的功能。另外，为了使编码器输出的码流可以让用户随机接入，可以开启重复头(repeat-header)在每个关键帧输出的时候整合H.264中的图像参数集(picture paramater set，PPS)，序列参数集(sequence paramater set，SPS)数据包。
上面的打包方式就可以保证proxy每次断流的时候总是会断在虚线框的位置，这样知道mini-GOP的第一个非B帧的hash信息(这一帧恰好是mini-GOP播放顺序的最后一帧)，就可以在转码备机把视频流连接起来。
具体如果在转码的过程中知道输出的帧是I/P/B帧，知道对应的输入YUV的hash信息，修改ffmpeg(快进动态图像专家组，fast forward moving picture experts group，是一种可以用来记录、转换数字音频、视频，并能将其转化为流的开源计算机程序)的代码，添加对应的和x265连接的相关接口即可。
在前述实施例的备机切换的策略中，主转码器宕机之后，需要切换到备转码器。备转码器为了能实现和主转码器断流后无缝拼接，备转码器在断流之前需要完成一些初始化工作，具体补充说明如下：
启动备转码器成功之后，立刻初始化备转码器的两个编码器，一个编码器在主线程初始化，设置高质量编码的编码参数，另一个编码器在子线程初始化，设置成低时延的编码参数。子线程自动进入等待状态。
备转码器可以维护三个先入先出(first input first output，FIFO)队列，分别缓存一定数目的P2P包头，音频数据包，解码之后的YUV视频数据包。
这些初始化工作完成之后，备转码器状态稳定，一旦收到拉流请求，立刻转变成备转码器转码的状态，主要执行的内容包括如下两个部分：
第一部分：寻找切换后起始编码的视频帧：
Proxy朝备转码器拉流会发送hash信息，需要从缓存YUV视频数据包的FIFO中寻找对应的视频帧。备转码器在缓存中对比hash值，找到相同的hash值的YUV帧，从相同的hash值的下一帧开始启动低时延编码，新来的视频帧在解码之后的原始YUV不再向缓存YUV视频数据包的FIFO中存储，而是直接送给高质量的编码器。此时开始两个编码器并发编码。
如图11所示：包含主编码器宕机前主编码器的编码输出(output)，即从x到y之前的视频帧；在Proxy输入目标框架(target frame)告知FIFO中对应的视频帧y后，开始低时延视频编码(low-latency video encoder)并输出，新的视频输入(input)后，使用高质量编码，即普通视频编码(normal video encoder)。
第二部分：低时延编码直接输出，高质量编码输出在缓冲区：
在备转码器内两个编码器并发编码时，两个编码器处于不同的线程。低时延编码输出如上图9或10的前面未被虚线框框住的部分(全部为P帧)，此部分编码输出会直接发往输出缓存中。由于高质量的编码器编码的输出需要等到低时延的编码器编码结束之后才能真正的输出，所以暂时放在一个缓冲区(buffer)缓存起来。低时延编码器编码结束之后，高质量的编码器开始按照一定的速度将buffer缓存中的视频帧发往输出缓存，继续编码新输入的视频帧。
实际应用中，使用单线程控制，把buffer输出到输出缓存并继续编码，接收端的视频有一定概率出现花屏。这是由于buffer中的数据量有可能比较多，可能会使高质量的编码器出现阻塞，输出的视频帧可能会丢帧。
本实施例使用异步buffer输出，也就是高质量的编码器的输出采用异步输出策略；高质量的编码器正常编码，将编码的结果放在buffer缓存中，在子线程中按照一定的速度从buffer缓存中取数据，然后把取到的视频帧发往输出缓存。采用本实施例的方案，接收端的视频不会出现花屏的现象。
如图12所示，第一行为主转码器转码结果，使用高质量IPB帧模式编码，其中在第一行第二个虚线框(mini-GOP)之前主转码器发生宕机，备转码器(backup transcode machine)开始工作，第二行虚线框前的部分是低时延的编码器编码；第二行虚线框中的部分是高质量编码器编码。获得mini-GOP的哈希信息，传递到备转码器，备转码器在打包时会使用该mini-GOP和哈希信息，具体参考前文打包的说明，在此不再赘述。
参见图13，图13是本申请另一实施例提供的一种视频转码装置的示意框图。如图13所示的本实施例中的视频转码装置可以包括：一个或多个处理器1301；一个或多个输入设备1302，一个或多个输出设备1303和存储器1304。上述处理器1301、输入设备1302、输出设备1303和存储器1304通过总线1305连接。存储器1302用于存储计算机程序，该计算机程序包括程序指令，处理器1301用于执行存储器1302存储的程序指令。其中，处理器1301被配置用于调用该程序指令执行以下操作：
将输入视频发往主转码器和备转码器，控制所述主转码器对所述输入视频进行转码并将转码后的视频发往接收设备，控制所述备转码器缓存所述输入视频；
在所述主转码器宕机的情况下，获得所述主转码器的转码位置信息；
向所述备转码器发送所述转码位置信息，控制所述备转码器依据所述转码位置信息对缓存的输入视频进行转码获得输出视频，将所述输出视频发往所述接收设备。
应当理解，在本申请实施例中，所称处理器1301可以是中央处理单元(Central Processing Unit，CPU)，该处理器还可以是其他通用处理器、 数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现成可编程门阵列(Field-Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。
输入设备1302可以包括触控板、指纹采传感器(用于采集用户的指纹信息和指纹的方向信息)、麦克风等，输出设备1303可以包括显示器(LCD等)、扬声器等。
该存储器1304可以包括只读存储器和随机存取存储器，并向处理器1301提供指令和数据。存储器1304的一部分还可以包括非易失性随机存取存储器。例如，存储器1304还可以存储设备类型的信息。
具体实现中，本申请实施例中所描述的处理器1301、输入设备1302、输出设备1303可执行本申请任一实施例提供的视频转码方法，在此不再赘述。
本实施例中的处理器1301可以执行前述方法实施例的方法流程。如果编码器和解码器均为软编码器和解码器，那么以上编码器和解码器的功能由程序指令被处理器1301执行实现。
基于前文说明本申请实施例还提供了一种视频转码装置，包括存储器和处理器，所述存储器用于存储程序指令，所述程序指令适于由所述处理器加载；
所述处理器，用于加载所述程序指令并执行如本申请实施例提供的任一项所述视频转码方法。
基于前文说明本申请实施例还提供了一种存储介质，所述存储介质中存储有多条程序指令，所述程序指令适于由处理器加载并执行如本申请实施例提供的任一项所述视频转码方法。
基于前文说明本申请实施例还提供了一种计算机程序产品，所述计算机程序产品包含多条程序指令，所述程序指令适于由处理器加载并执行如本申请实施例提供的任一项所述视频转码方法。
基于以上实施例，使用主转码器和备转码器配合，由于主转码器在 对输入视频数据进行转码过程中会存在一定数量的未转码的数据，备转码器在主转码器宕机的情况下继续对这些未转码的数据进行转码，可以实现无缝迁移，因此接收端不会出现卡顿和黑屏现象，具有更高的可靠性。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及方法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的通信装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统、方法和装置，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，上述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
上述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其 任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。上述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行上述计算机程序指令时，全部或部分地产生按照本申请实施例上述的流程或功能。上述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。上述计算机指令可以存储在计算机可读存储介质中，或者通过上述计算机可读存储介质进行传输。上述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(digital subscriber line，DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。上述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。上述可用介质可以是磁性介质，(例如，软盘、硬盘、磁带)、光介质(例如，数字通用光盘(digital versatile disc，DVD))、或者半导体介质(例如固态硬盘(solid state disk，SSD))等。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，该流程可以由计算机程序来指令相关的硬件完成，该程序可存储于计算机可读取存储介质中，该程序在执行时，可包括如上述各方法实施例的流程。而前述的存储介质包括：只读存储器(read-only memory，ROM)或随机存储存储器(random access memory，RAM)、磁碟或者光盘等各种可存储程序代码的介质。

Claims (14)

  1. 一种视频转码系统，包括：
    主转码器和备转码器；
    所述主转码器获得输入视频，对所述输入视频进行转码，将转码后的视频发往接收设备；
    所述备转码器获得所述输入视频并缓存所述输入视频，在所述主转码器宕机的情况下获得所述主转码器的转码位置信息，依据所述转码位置信息对缓存的输入视频进行转码获得输出视频，将所述输出视频发往所述接收设备。
  2. 根据权利要求1所述视频转码系统，所述备转码器包括解码器、第一编码器和第二编码器；
    所述解码器依据所述转码位置信息对缓存的输入视频进行解码后得到缓存解码数据，将所述缓存解码数据发往所述第一编码器；
    所述第一编码器接收所述缓存解码数据并对所述缓存解码数据进行编码；
    所述解码器还用于将获得的新输入视频解码为新解码数据，将所述新解码数据发往所述第二编码器，所述新输入视频为所述主转码器宕机后接收到的输入视频；
    所述第二编码器接收所述新解码数据并对所述新解码数据进行编码。
  3. 根据权利要求2所述视频转码系统，所述第一编码器的编码时延低于所述第二编码器的编码时延；所述第二编码器的编码质量高于所述第一编码器的编码质量。
  4. 根据权利要求2所述视频转码系统，所述第二编码器的编码结果包含B帧；
    所述第二编码器还用于对所述缓存的输入视频最后一个非B帧位置的视频帧进行编码得到能被连续解码的画面组，将所述画面组缓存；
    将所述画面组打包发往输出缓存，然后打包对所述新解码数据编码得到的视频帧的第一个I帧或P帧发往所述输出缓存。
  5. 根据权利要求2所述视频转码系统，
    所述第一编码器还用于将所述第一编码器编码获得的视频帧发往输出缓存；
    所述第二编码器还用于将所述第二编码器编码获得的视频帧发往缓冲区，以便在所述第一编码器编码结束后，按照先入先出的顺序将所述缓冲区内的视频帧发往所述输出缓存。
  6. 根据权利要求1至5任意一项所述视频转码系统，
    所述备转码器通过先进先出的队列以视频帧为单位存储所述输入视频，在所述队列满时丢弃先进入所述队列的视频帧。
  7. 一种视频转码方法，由计算设备执行，包括：
    将输入视频发往主转码器和备转码器，控制所述主转码器对所述输入视频进行转码并将转码后的视频发往接收设备，控制所述备转码器缓存所述输入视频；
    在所述主转码器宕机的情况下，获得所述主转码器的转码位置信息；
    向所述备转码器发送所述转码位置信息，控制所述备转码器依据所述转码位置信息对缓存的输入视频进行转码获得输出视频，将所述输出视频发往所述接收设备。
  8. 根据权利要求7所述方法，所述控制所述备转码器依据所述转码位置信息对缓存的输入视频进行转码包括：
    控制所述备转码器的解码器依据所述转码位置信息对缓存的输入视频进行解码后得到缓存解码数据，将所述缓存解码数据发往所述备转码器的第一编码器；
    控制所述备转码器的第一编码器依据所述转码位置信息对所述缓存解码数据进行编码；
    所述方法还包括：
    将新输入视频发往所述解码器，控制所述解码器将所述新输入视频解码为新解码数据，将所述新解码数据发往所述第二编码器，控制所述第二编码器对所述新解码数据进行编码；所述新输入视频为所述主转码器宕机后接收到的输入视频。
  9. 根据权利要求8所述方法，所述第一编码器的编码时延低于所述第二编码器的编码时延；所述第二编码器的编码质量高于所述第一编码器的编码质量。
  10. 根据权利要求8所述方法，所述第二编码器的编码结果包含B帧；
    该方法还包括：
    控制所述第二编码器对所述缓存的输入视频最后一个非B帧位置的视频帧进行编码得到能被连续解码的画面组，将所述画面组缓存；
    将所述画面组打包发往输出缓存，然后打包对所述新解码数据编码得到的视频帧的第一个I帧或P帧发往所述输出缓存。
  11. 根据权利要求8所述方法，还包括：
    将所述第一编码器编码获得的视频帧发往输出缓存；将所述第二编码器编码获得的视频帧发往缓冲区；
    在所述第一编码器编码结束后，按照先入先出的顺序将所述缓冲区内的视频帧发往所述输出缓存。
  12. 根据权利要求7至11任意一项所述方法，还包括：
    在所述主转码器宕机后将所述备转码器设置为主转码器，将已宕机的所述主转码器设置为备转码器，重启已宕机的所述主转码器。
  13. 一种视频转码装置，包括存储器和处理器，所述存储器用于存储程序指令，所述程序指令适于由所述处理器加载；
    所述处理器，用于加载所述程序指令并执行如权利要求7至12任一项所述视频转码方法。
  14. 一种存储介质，所述存储介质中存储有多条程序指令，所述程序指令适于由处理器加载并执行如权利要求7至12任一项所述视频转码方法。
PCT/CN2019/082227 2018-05-31 2019-04-11 一种视频转码系统、方法、装置及存储介质 WO2019228078A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP19810652.8A EP3806477B1 (en) 2018-05-31 2019-04-11 Video transcoding system and method, apparatus, and storage medium
US16/996,132 US11184627B2 (en) 2018-05-31 2020-08-18 Video transcoding system, method, apparatus, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810548780.1A CN110213598B (zh) 2018-05-31 2018-05-31 一种视频转码系统、方法及相关产品
CN201810548780.1 2018-05-31

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/996,132 Continuation US11184627B2 (en) 2018-05-31 2020-08-18 Video transcoding system, method, apparatus, and storage medium

Publications (1)

Publication Number Publication Date
WO2019228078A1 true WO2019228078A1 (zh) 2019-12-05

Family

ID=67778848

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/082227 WO2019228078A1 (zh) 2018-05-31 2019-04-11 一种视频转码系统、方法、装置及存储介质

Country Status (4)

Country Link
US (1) US11184627B2 (zh)
EP (1) EP3806477B1 (zh)
CN (1) CN110213598B (zh)
WO (1) WO2019228078A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113645490A (zh) * 2021-06-23 2021-11-12 天津津航计算技术研究所 一种软硬结合的多通道视频同步解码方法

Families Citing this family (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102612528B1 (ko) 2019-09-23 2023-12-12 구글 엘엘씨 인터럽트 가능한 비디오 트랜스코딩
CN110677687B (zh) * 2019-10-21 2023-01-24 上海网达软件股份有限公司 一种支持延时的直播转码方法及系统
CN111083521B (zh) * 2019-12-27 2022-02-01 杭州当虹科技股份有限公司 一种rtmp发布主备快速切换方法
CN111343503B (zh) * 2020-03-31 2022-03-04 北京金山云网络技术有限公司 视频的转码方法、装置、电子设备及存储介质
CN111541941B (zh) * 2020-05-07 2021-10-29 杭州小影创新科技股份有限公司 一种移动端多编码器加速编码的方法
CN111757123A (zh) * 2020-07-07 2020-10-09 杭州海康威视数字技术股份有限公司 一种分布式视频解码系统、视频解码输出方法及设备
CN112333447A (zh) * 2020-11-02 2021-02-05 上海网达软件股份有限公司 音视频转码方法、装置、设备及存储介质
CN112601097B (zh) * 2021-03-02 2021-04-30 中国传媒大学 双编码云导播方法及系统
CN113518259B (zh) * 2021-05-25 2023-06-09 龙迅半导体（合肥）股份有限公司 一种数据处理方法及装置
CN113438470B (zh) * 2021-06-18 2022-12-09 苏州浪潮智能科技有限公司 视频转码器掉卡模拟测试方法、装置、终端及存储介质
CN114173183B (zh) * 2021-09-26 2023-01-24 荣耀终端有限公司 投屏方法和电子设备
EP4203472A1 (en) * 2021-12-21 2023-06-28 Axis AB Method and image processing device for encoding a video
US12052447B1 (en) * 2022-06-27 2024-07-30 Amazon Technologies, Inc. Dynamically moving transcoding of content between servers
US11910044B1 (en) 2022-06-30 2024-02-20 Amazon Technologies, Inc. Systems and methods for switching the processing of a live content stream to another datacenter
CN115426503B (zh) * 2022-08-30 2024-05-17 百果园技术(新加坡)有限公司 一种转码服务切换方法、装置、设备及存储介质
WO2024050192A1 (en) * 2022-09-02 2024-03-07 Qualcomm Incorporated Data reconstruction using machine-learning predictive coding
CN115866248B (zh) * 2022-11-08 2024-01-19 格兰菲智能科技有限公司 视频转码方法、装置、计算机设备、存储介质

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101960852A (zh) * 2008-03-05 2011-01-26 阿尔卡特朗讯公司 用于在媒体递送系统中提供冗余的系统、方法和计算机可读介质
CN105049889A (zh) * 2015-06-26 2015-11-11 安徽广行通信科技股份有限公司 一种视频在线转码系统及其转码方法
US20150381978A1 (en) * 2014-06-30 2015-12-31 Richard Dunphy Techniques for parallel video transcoding

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7159234B1 (en) * 2003-06-27 2007-01-02 Craig Murphy System and method for streaming media server single frame failover
US7330509B2 (en) * 2003-09-12 2008-02-12 International Business Machines Corporation Method for video transcoding with adaptive frame rate control
US9432704B2 (en) * 2011-11-06 2016-08-30 Akamai Technologies Inc. Segmented parallel encoding with frame-aware, variable-size chunking
US9137151B2 (en) * 2012-05-18 2015-09-15 Google Technology Holdings LLC Redundant architecture for failover of transcoder instances
US8928804B2 (en) * 2012-11-19 2015-01-06 Broadcom Corporation Managing encoder parameters for parallel transcoding
US8955027B1 (en) * 2013-11-21 2015-02-10 Google Inc. Transcoding media streams using subchunking
US9414100B2 (en) * 2014-03-31 2016-08-09 Arris Enterprises, Inc. Adaptive streaming transcoder synchronization
US9479720B1 (en) * 2014-09-16 2016-10-25 Amazon Technologies, Inc. Method and/or apparatus for frame accurate hot failover
US9532099B2 (en) * 2015-03-24 2016-12-27 Intel Corporation Distributed media stream synchronization control
US9973769B2 (en) * 2015-08-12 2018-05-15 Time Warner Cable Enterprises Llc Methods and apparatus of encoding real time media content
US10735783B1 (en) * 2017-09-28 2020-08-04 Twitch Interactive, Inc. Intra-rendition latency variation

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101960852A (zh) * 2008-03-05 2011-01-26 阿尔卡特朗讯公司 用于在媒体递送系统中提供冗余的系统、方法和计算机可读介质
US20150381978A1 (en) * 2014-06-30 2015-12-31 Richard Dunphy Techniques for parallel video transcoding
CN105049889A (zh) * 2015-06-26 2015-11-11 安徽广行通信科技股份有限公司 一种视频在线转码系统及其转码方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3806477A4 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113645490A (zh) * 2021-06-23 2021-11-12 天津津航计算技术研究所 一种软硬结合的多通道视频同步解码方法
CN113645490B (zh) * 2021-06-23 2023-05-09 天津津航计算技术研究所 一种软硬结合的多通道视频同步解码方法

Also Published As

Publication number Publication date
CN110213598B (zh) 2021-10-15
CN110213598A (zh) 2019-09-06
EP3806477A1 (en) 2021-04-14
EP3806477B1 (en) 2023-03-01
EP3806477A4 (en) 2021-08-04
US20200382803A1 (en) 2020-12-03
US11184627B2 (en) 2021-11-23

Similar Documents

Publication Publication Date Title
WO2019228078A1 (zh) 一种视频转码系统、方法、装置及存储介质
KR102324326B1 (ko) 상이한 인코딩 파라미터를 이용해 인코딩되는 복수의 인코딩 스트리밍
WO2018196790A1 (zh) 视频播放方法、设备及系统
US10154320B2 (en) Dynamic time synchronization
US11109092B2 (en) Synchronizing processing between streams
US20170280167A1 (en) Recovery From Packet Loss During Transmission Of Compressed Video Streams
WO2021208321A1 (zh) 一种多视点视频的视点切换方法和系统
JP2010514346A (ja) Ｉｐｔｖの低ビットレートのストリームを使用したビデオデータ損失回復システム
US9661351B2 (en) Client side frame prediction for video streams with skipped frames
JP6157651B2 (ja) 圧縮されたビデオストリームを切り替えるためのスイッチング装置、当該スイッチング装置を備えた会議システムおよび圧縮されたビデオストリームを切り替える方法
US10142644B2 (en) Decoding frames
WO2024027768A1 (zh) 多屏视频显示方法、系统、播放端及存储介质
CN112672163B (zh) 用于分段流动性的代码转换器调节
US20030128765A1 (en) Receiving apparatus
US20140294080A1 (en) Image processing device, image processing method, and computer program
KR20130109969A (ko) 단일 디코더로 채널 변경을 가능하게 하기 위한 수신기 및 수신기에서의 방법
TWI554083B (zh) 影像處理方法及其攝影機
JP2007166022A (ja) 映像表示システム及び映像表示装置
WO2020048617A1 (en) Latency efficient streaming of video frames for machine vision over an ip network
KR101231009B1 (ko) 네트워크 비디오 서버 및 이의 비디오 제어 방법
JP2014017607A (ja) 映像配信装置及び映像配信方法
JP2009049855A (ja) コンテンツ再生装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19810652

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2019810652

Country of ref document: EP

Effective date: 20210111