KR101823321B1 - Systems and methods of encoding multiple video streams with adaptive quantization for adaptive bitrate streaming - Google Patents

Abstract

A method for encoding source video as alternative video streams includes collecting statistics for source video data in a first pass through received multimedia content and recording statistics into a shared memory, Collecting and recording statistics for the source video data, including measurements, determining initial encoding information for the source video data during the first pass and writing initial encoding information to the shared memory, And encoding the source video data in parallel using collected statistics and initial encoding information to generate alternative video streams, wherein the parallel encoding processes reuse additional encoding information already determined for a portion of the video Video Generate encoded additional information that is not already determined for the portion, and the encoded additional information comprises a quantization parameter for a block of pixels.

Description

[0001] SYSTEMS AND METHODS FOR ENCODING MULTIPLE VIDEO STREAMS WITH ADAPTIVE QUANTIZATION FOR ADAPTIVE BITRATE STREAMING [0002]

The present invention relates generally to video encoding and more particularly to systems and methods for efficiently encoding multiple streams of video content for adaptive bitrate streaming from a source video stream.

The term streaming media describes playback of media on a playback device, wherein the media is stored on a server and is continuously transferred to the playback device over the network during playback. Typically, the playback device is configured to allow a sufficient amount of media in the buffer at any given time during playback to prevent interruption of playback due to completing playback of all of the buffered media prior to receipt of the next portion of media by the playback device. . Adaptive bitrate streaming, or adaptive streaming, involves detecting current streaming states (e.g., the user's network bandwidth and CPU capacity) in real time and adjusting the quality of the streamed media accordingly. Typically, the source media is encoded at multiple bit rates and the playback device or client switches between streaming different encodings depending on the available resources. When the playback device begins adaptive bitrate streaming, the playback device typically begins by requesting portions of the media from the lowest bitrate streams (alternate streams available). As the playback device downloads the requested media, the playback device can measure the available bandwidth. If there is additional bandwidth available, the playback device may switch to higher bitrate streams.

In adaptive streaming systems, the source media is typically stored on the media server as a top-level index file representing a plurality of alternative streams including actual video and audio data. Each stream is typically stored in one or more container files. Different adaptive streaming solutions typically use different indexes and media containers. The Matroska container is a media container developed as an open standard project by the Martros-Kavby-for-profit organization in Aussonne, France. Martroska Container is based on the Extensible Binary Meta Language (EBML), which is a binary derivative of the Extensible Markup Language (XML). The decoding of the Martroska container is supported by many consumer electronics (CE) devices. The DivX Plus file format, developed by DivX, LLC of San Diego, Calif., Utilizes an extension of the Martroska Container format, including elements not specified in the Martrosika format. Other commonly used media container formats are the MP4 container formats specified in MPEG-4 Part 14 (i.e., ISO / IEC 14496-14) and MPEG-2 Part 1 (i.e. ISO / IEC Standard 13818-1) MPEG transport stream (TS) container. The MP4 container format is used for IIS Smooth streaming and Flash dynamic streaming. The TS container is used in HTTP adaptive bitrate streaming. In alternate streams, video is encoded in a high efficiency video coding (HEVC / H.265) jointly specified by the ISO / IEC Moving Picture Experts Group (MPEG) and the International Telecommunication Union Telecommunication Standardization Sector (ITU-T) Can be encoded according to various block-oriented video compression standards (or codecs) such as the H.264 / MPEG-4 AVC (Enhanced Video Coding) standard specified by ITU-T.

The present invention relates to systems and methods for efficiently encoding multiple streams of video content for adaptive bitrate streaming from a source video stream.

Systems and methods are disclosed for encoding a plurality of video streams for adaptive bitrate streaming in accordance with embodiments of the present invention. In one embodiment, a source encoder configured to encode source video as a plurality of alternative video streams comprises a source encoder application, a shared memory, and a parallel processing system, wherein the parallel processing system is operable to encode multimedia content Wherein the multimedia content comprises source video data having a primary resolution, collects statistics on the source video data in a first pass through the received multimedia content, The statistics collecting and recording statistics for the source video data, including statistics of complexity of blocks of pixels, during the first pass through the received multimedia content, To determine initial encoding information for the source video data and to write the initial encoding information to a shared memory and to generate a plurality of alternative video streams during the second pass through the received multimedia content with a plurality of parallel encoding processes Encoding the source video data in parallel using the collected statistics and initial encoding information, wherein the encoding of the source video utilizes additional encoding information and the parallel encoding processes are performed by another parallel encoding process And wherein the parallel encoding processes are not already determined for a portion of the video by another parallel encoding process and the shared memory It is configured to store the additional information encoding the generated, the additional information encoding is configured to encode said source video data including the quantization parameters for blocks of pixels in parallel.

In a further implementation, statistics for source video data include statistics selected from the group of average quantization parameters, size of header bits, size of texture bits, number of intra blocks, number of interblocks, and number of skip blocks.

In another embodiment, the parallel processing system configured to determine initial encoding information for source video data also includes a parallel processing system configured to calculate a frame complexity measure.

In a further embodiment, the parallel encoding processes configured to generate additional encoding information that have not been previously determined for a portion of the video by another parallel encoding process include a coding tree for encoding a portion of a frame of video in the source video data, And determining a size of a Coding Tree Unit (CTU).

In another embodiment, the step of determining a CTU size for encoding a portion of a frame of video in the source video data comprises selecting a portion of a frame of video for encoding as at least one output CTU in the first output stream, Selecting a CTU size if the size is not determined for the similar CTU, selecting a previously determined CTU size determined for the second output stream, and determining for a similar CTU size Scaling the CTU size if the resolution of the first output stream is not the same resolution as the second output stream, comparing the resolution of the second output stream with the resolution of the first output stream if the resolution of the second output stream is determined; Determining whether the selected CTU size is acceptable for an output CTU, Selecting a smaller CTU size when the selected CTU size is not acceptable and applying the selected CTU size to a portion of a frame of video if the selected CTU size is acceptable for the output CTU.

In a further embodiment, a parallel processing system configured to determine initial encoding information for source video data is further configured to determine a mode distribution for at least one frame of video in at least one of the plurality of alternative video streams. Parallel processing system.

In another embodiment, the parallel processing system configured to encode the source video data in parallel using collected statistics and initial encoding information to generate a plurality of alternative video streams may also be used to generate video To determine a threshold number of blocks based on the mode distribution, and to adjust criteria for block type decisions if the count of blocks meets a threshold number of blocks. ≪ RTI ID = 0.0 > Parallel processing system.

In a further embodiment, the parallel encoding processes, which have been previously determined for a portion of the video by another encoding process and are configured to reuse additional encoding information stored in the shared memory, may also be included in the video frame in the first alternative stream And determining whether the first alternative stream and the second alternative stream are of the same resolution, determining whether the first alternative stream and the second alternative stream are of the same resolution, determining whether the motion vector is present for the second corresponding block in the second alternative stream, Scaling the motion vector and refining the motion vector if the first alternative stream and the second alternative stream are not of the same resolution, and when encoding the first block in the video frame in the first alternative stream, The parallel encoding processes configured to apply It should.

In yet another embodiment, the initial encoding information also includes a header size, a macroblock size, and a relative ratio of header size to macro size.

In a further additional embodiment, the initial encoding information also includes virtual reference decoder data.

In yet another additional embodiment, each of the parallel encoding processes encodes at different resolutions.

In a further embodiment, each of the parallel encoding processes encodes one or more alternative video streams and each of the alternative video streams encoded by the parallel encoding process is at a different bit rate.

In another embodiment, each of the parallel encoding processes sequentially encodes from the source video data sequentially to each stream in a subset of a plurality of alternative video streams.

Again in a further embodiment, the additional encoding information includes rate distortion information and quantization parameters.

Yet another embodiment is a method for receiving multimedia content, the multimedia content including source video data having a primary resolution using a source encoder, the method comprising the steps of: receiving the multimedia content; Collecting statistics on the source video data in a first pass through the multimedia content and recording the statistics in a shared memory, using the source encoder to determine, for the source video data during the first pass, Determining initial encoding information and writing the initial encoding information to a shared memory, wherein the statistics include determining and recording the initial encoding information, including complexity measures of blocks of pixels, Parallel Encoding Pro Encoding the source video data in parallel using the collected statistics, initial encoding information, and additional encoding information to generate a plurality of alternative video streams during a second pass through the received multimedia content Wherein the encoding of the source video is further determined for a portion of the video by another parallel encoding process and reusing additional encoding information stored in the shared memory using at least one of a plurality of parallel encoding processes And generating additional encoding information that has not been previously determined for a portion of the video by another of the plurality of parallel encoding processes, encoding the source video data in parallel and using the parallel encoder process The ball A step of storing the additional information encoding said generated in the memory, the additional encoding information comprises the step of storing, encoding additional information, the generated including quantization parameters for blocks of pixels.

In a further additional embodiment, the statistics for the source video data include statistics selected from the group consisting of: average quantization parameter, size of header bits, size of texture bits, number of intra blocks, number of interblocks, .

In yet another additional embodiment, determining the initial encoding information for the source video data also includes calculating a frame complexity measure.

Again in a further embodiment, generating additional encoding information that has not been previously determined for a portion of the video by another parallel encoding process may also include generating a coding tree unit (CTU) for encoding a portion of the video frame in the source video data ) ≪ / RTI > size.

In yet another embodiment, determining the CTU size for encoding a portion of a frame of video in the source video data may also include selecting a portion of the frame of video to encode as at least one output CTU in the first output stream Selecting a CTU size and selecting a previously determined CTU size determined for the second output stream if the size is not determined for the similar CTU, Scaling the CTU size if the resolution of the first output stream is not the same resolution as the second output stream, comparing the resolution of the second output stream with the resolution of the first output stream if the resolution of the second output stream is determined to To determine if the selected CTU size is acceptable for the output CTU Selecting a smaller CTU size if the selected CTU size is unacceptable and applying the selected CTU size to a portion of a frame of video if the selected CTU size is acceptable for the output CTU do.

In a further additional embodiment, determining the initial encoding information for the source video data also includes determining a mode distribution for at least one frame of video in at least one of a plurality of alternative video streams .

In yet another additional embodiment, the step of encoding the source video data in parallel using collected statistics, initial encoding information, and additional encoding information to generate a plurality of alternative video streams, The method comprising: maintaining a count of blocks processed in a frame of video in a stream; determining a threshold number of blocks based on the mode distribution; and if the count of blocks meets a threshold number of blocks, And adjusting the criteria.

In yet another additional embodiment, reusing additional encoding information already determined for a portion of the video by another parallel encoding process and stored in the shared memory may also include re-encoding the first encoded video information in the first alternative stream, Determining whether a motion vector is present for a second corresponding block in a second alternative stream when encoding the block, determining whether the first alternative stream and the second alternative stream are at the same resolution, Scaling the motion vector if the first alternative stream and the second alternative stream are not of the same resolution, refining the motion vector, and encoding the first block in the video frame in the first alternative stream ≪ / RTI > applying the motion vector.

In yet another additional embodiment, the initial encoding information also includes a header size, a macroblock size, and a relative ratio of header size to macroblock size.

Again in a further embodiment, the initial encoding information also includes virtual reference decoder data.

Again in a further embodiment, each of the parallel encoding processes encodes at different resolutions.

In a further additional embodiment, each of the parallel encoding processes encodes one or more alternative video streams and each of the alternative video streams encoded by the parallel encoding process is at a different bit rate.

In yet another additional embodiment, each of the parallel encoding processes sequentially encodes blocks from the source video data sequentially into respective streams in a subset of the plurality of alternative video streams.

In yet a further additional embodiment, the additional encoding information includes rate distortion information and quantization parameters.

In yet a further additional embodiment, the blocks of pixels are coding tree units.

In yet another additional embodiment, the blocks of pixels are coding units.

In yet another further embodiment, the blocks of pixels are transform units.

In yet another further embodiment, quantization parameters for blocks of pixels are generated using complexity measures for blocks of pixels.

In yet a further embodiment, quantization parameters for blocks of pixels are generated using previously generated quantization parameters.

In yet another additional embodiment, the quantization parameters for blocks of pixels are generated using a distortion rate and a bit rate.

1 is a system diagram of an adaptive streaming system in accordance with embodiments of the present invention.
2 conceptually illustrates a media server configured to encode streams of video data for use in adaptive streaming systems in accordance with embodiments of the present invention.
3 is a flow diagram illustrating a process for reusing block size information when encoding streams of video in accordance with embodiments of the present invention;
4 is a flow diagram illustrating a process for adjusting block type decisions when encoding streams of video in accordance with embodiments of the present invention.
5 is a flow diagram illustrating a process for reusing motion vectors when encoding streams of video in accordance with embodiments of the present invention;
6 is a flow diagram illustrating a process for sharing statistics and encoding information in the encoding of alternative streams of video data in accordance with embodiments of the present invention.
7 is a flow diagram illustrating a process for generating a complexity measure for a block of pixels in accordance with embodiments of the present invention.
8 is a flow diagram illustrating a process for generating quantization parameters for a block of pixels in accordance with embodiments of the present invention.
9 is a flow chart illustrating a process for generating quantization parameters for a block of pixels using quantization parameters for similar blocks of pixels according to embodiments of the present invention.
10A is a graph showing plots of complexity measurements of CTU blocks.
10B is a graph showing plots of complexity measures and quantization parameters of CTU blocks.

Turning now to the drawings, there are shown systems and methods for encoding a plurality of video streams for adaptive bitrate streaming in accordance with embodiments of the present invention. In accordance with embodiments of the present invention, encoders analyze media content for statistics on content, determine encoding information used to encode the content, and provide multiple video streams at different resolutions and bit rates Lt; RTI ID = 0.0 > content. ≪ / RTI > Although the present invention is described below with respect to adaptive streaming systems and block-based video encoding techniques such as HEVC / H.265 and H.265 AVC, the described systems and methods may be applied to different streams of video data, The present invention is equally applicable to conventional streaming systems that are selected based on non-based video encoding techniques and connection quality of network clients.

In adaptive streaming systems, the multimedia content is encoded as a set of alternative streams of video data. Since each alternative stream of video data is encoded using the same source multimedia content, similar encoding information is determined in the encoding of each alternate stream of video data. The encoding information may include, but is not limited to, frame complexity measurements, selection of block sizes, block mode distribution, and motion estimation results. Systems and methods in accordance with many embodiments of the present invention reuse encoding information determined in the encoding of one alternate stream of video data in the encoding of at least one other alternate stream of video data. By reusing the encoding information in the encoding of several alternative streams of video data, significant improvements in the encoding of alternative streams of video data can be achieved and particularly significant time savings can be realized according to embodiments of the present invention.

Adaptive streaming systems are configured to stream encoded multimedia content at different maximum bit rates and resolutions over the network, such as the Internet. Adaptive streaming systems stream the highest quality multimedia content that can be supported based on current streaming states. Multimedia content typically includes video and audio data, captions, and other related metadata. To provide a top quality video experience that is independent of the network data rate, the adaptive streaming systems include, but are not limited to, available network data rates and video decoder capabilities, And to switch between the available sources of video data. When streaming conditions deteriorate, adaptive streaming systems typically attempt to switch to encoded multimedia streams at lower maximum bit rates. If the available network data rate can not support streaming of the encoded stream at the lowest maximum bit rate, then playback is often interrupted until a sufficient amount of content can be buffered to restart playback. Systems and methods for switching between video streams during playback that may be used in an adaptive streaming system in accordance with embodiments of the present invention are disclosed in Braness et al., "The Hypertext Transfer Protocol, filed on August 30, Systems and Methods for Adaptive Bitrate Streaming of Media Stored in Martoscir Container Files Using US Patent Application Serial No. < RTI ID = 0.0 > 13 / 221,682, < / RTI > .

To generate multiple sources of video data used in adaptive streaming systems, a source encoder may be configured to encode a plurality of alternative streams of video data from source video included in a piece of multimedia content. Systems and methods for encoding source video for use in adaptive streaming systems are described in Braness et al., "Hypertext Transfer Protocol, filed on August 30, 2011, Systems and Methods for Encoding Source Media in Illustrator Container Files, " which is incorporated herein by reference in its entirety. According to embodiments of the present invention, the source encoder may be implemented using a media source and / or a media server.

As described above, alternative streams of video data based on the same source video include similar content, and thus the encoding information determined for one alternate stream of video data and statistics determined from the source content is stored in another alternative stream of video data Lt; / RTI > and / or < / RTI > According to embodiments of the present invention, the set of alternative streams of video data based on the same source video may include video data at the same resolution but at different bit rates. In many embodiments of the present invention, the motion estimation results produced for encoding a particular alternative stream of video data may be reused among other alternative streams of video data. As discussed below, various statistics and encoding information determined in the encoding of alternative streams of video data may be reused among alternative streams of video data. Systems and methods for sharing statistics and encoding information when encoding alternative streams of video data in accordance with embodiments of the present invention are discussed further below.

Adaptive  Streaming System Architecture

Adaptive streaming systems in accordance with embodiments of the present invention are configured to generate multiple streams of video to be available for streaming to user devices. In many embodiments of the present invention, an adaptive streaming system includes a source encoding server that performs encoding of multiple streams of video from a source media. An adaptive streaming system according to embodiments of the present invention is shown in FIG. The illustrated adaptive streaming system 10 includes a source encoding server 12 configured to encode the source media as a plurality of alternative streams. The source media can be stored on the encoding server 12 or retrieved from the media server 13. As discussed further below, source encoding server 12 generates container files containing encoded streams, at least a plurality of which are alternative streams of encoded video. The encoding server creates a first pass to collect statistics for the content at each output resolution and a second pass to encode the content to multiple output streams where the streams have various resolutions and bit rates . In some embodiments, the first pass is completed before the second pass begins. In other embodiments, the second pass may be started before the first pass is completed. In other words, if the frames are processed by the first pass process (s) before they are processed by the second pass process (s), the calculation processes for the first and second passes can be performed simultaneously. These files are uploaded to the content server 14, which may be an HTTP server. The various playback devices 18,20, and 22 may then request portions of the encoded streams from the content server 14 over the network 16, such as the Internet.

Although a particular adaptive streaming system for delivering media content streams is discussed above with respect to FIG. 1, any of a variety of streaming systems may be used to convey media content streams in accordance with embodiments of the present invention.

Source encoders

In the illustrated embodiment, the adaptive bitrate streaming system includes one or more source encoders capable of encoding the source stream of video content into alternative streams of encoded video with different resolutions and / or bitrates. In many embodiments, the source encoder may be implemented using any device capable of encoding multimedia streams, wherein the streams are encoded at different resolutions, sampling rates, and / or maximum bit rates . The basic architecture of an adaptive streaming system source encoder according to an embodiment of the present invention is shown in FIG. Source encoder 200 includes a processor 210 that communicates with memory 230 and network interface 240. In the illustrated embodiment, the volatile memory 230 includes a source encoding application 250. The processor is configured to encode the plurality of streams of video data from the source video data 260, also in the volatile memory, by the source encoding application 250. The source video data 260 may already be in memory or may be received via the network interface 240.

In multiple embodiments, the source encoder includes a plurality of processors and the encoding process may be distributed among the plurality of processors. In many embodiments, the source encoding application 250 may launch multiple processes executing on one or more processors, each processor being an encoder controller 270 that encodes one or more output streams. In further embodiments, each encoder controller encodes multiple output streams at the same resolution and at different bit rates. In various embodiments, the encoder controller for each of the three output resolutions is launched to run on one or more processors, where the output resolutions are 768 x 432, 1280 x 720, and 1920 x 1080. In some embodiments, the encoder controller 270 encodes 768x422 output streams at two different bit rates, and the encoder controller 280 decodes at 1280x720 at three different bit rates, and the encoder controller 290 encode at 1920 x 1080 at three different bit rates. Encoder controllers 270, 280, and 290 typically reside in memory 230 as they run. In accordance with many embodiments of the present invention, encoder controllers 270,280, and 290 have shared data buffers 295 in memory for exchanging statistics, encoding information, and other information between controllers .

Although a particular architecture for the source encoder is shown in FIG. 2, any of a variety of architectures including architectures in which video encoder 250 is located on a disk or some other type of storage device and loaded into memory 230 at runtime May be used to encode multimedia content according to embodiments of the present invention. Systems and methods for re-use of statistics and encoding information in the encoding of alternative streams of video data in accordance with embodiments of the present invention are discussed further below.

Steps to collect and use statistics and encoding information

In many embodiments of the invention, statistics and encoding information are determined for a piece of media content before encoding the content into a plurality of output streams. As will be discussed in further detail below, statistics and encoding information may be stored and shared between encoding processes to speed up the decision on how to encode the content. The statistics for collecting in the first pass include, but are not limited to: average quantization parameters, size of header bits, size of texture bits, number of intra macroblocks / CTUs, number of inter macroblocks / CTUs, Blocks / CTUs. The encoding information may include, but is not limited to: frame complexity measurements, a coding tree unit (CTU) structure, a mode distribution, and motion information. The collection and use of encoding information in the encoding of alternative streams of video in accordance with embodiments of the present invention is discussed below.

Frame complexity measure

Prior to encoding the source content, the frame complexity measure may be assigned to each frame of content. The frame complexity measure represents an indication of the level of complexity of the visual information in the frame and thereby the data that will cause the output content stream from encoding the frame (i.e., in bits). Algorithms, including those known in the art, may be used in accordance with embodiments of the present invention to yield a frame complexity measure. Such algorithms may be based on variations in values, such as color and brightness, from average values across corresponding pixels in pixels or in multiple frames, and / or pixels in frames and ones in previous frames, (I. E., Between) and intra (i. E.) Frame measurements as well as similarities between blocks of pixels. Moreover, the algorithms can yield measurements similar to those used to encode the content when performing motion estimation.

As will be discussed further below, the frame complexity measure may be used to select parameters from the input stream to encode the frame into frames in the output stream. In many embodiments of the present invention, the frame complexity measure is assigned to each frame of content and is represented as an integer, where a larger value indicates greater complexity within the frame. In further embodiments of the present invention, the frame complexity measure is assigned to a frame in the input stream and the same measure is used to select parameters for encoding corresponding frames in a plurality of alternative output streams.

Bit rate control algorithms, including those known in the art, are used to determine how many bits to allocate to each encoded frame of the video and buffer levels of the output stream and to determine the quantization level to apply when encoding each frame of video May be utilized in accordance with embodiments of the present invention to select a plurality of < RTI ID = 0.0 > According to embodiments of the present invention, these algorithms use a once computed frame complexity measure for an input content stream and reuse the measure to determine a quantization level for a number of output streams encoded from the input stream can do.

Block size decisions

In block-oriented encoding standards, pixels are taken as partitions or "blocks" in the encoding process. In some current standards, the block size is variable, where partitions in a frame may have different sizes. The HEVC standard uses partitions known as CTUs whose block size of the coding tree unit (CTU) is typically 64x64, 32x32, 16x16, or 8x8 pixels. A frame of video may comprise a mixture of CTUs of different sizes and arrangements. Often, the block sizes of the CTUs in the frame (or other "block" partitions of pixels in accordance with the relevant video compression standard) are selected based on the resulting size and efficiency in the bits of the encoded frame. An encoder in accordance with embodiments of the present invention may be configured for use with block sizes for use in encoding frames of content for an output stream when it has similar determinations already made for corresponding portions of frames in yet another alternative output stream It is possible to accelerate crystals.

The encoder typically chooses block sizes based at least in part on the size (in bits) of the resulting encoded output frame while maintaining the image quality of the frame. In general, smaller output frames (i.e., more efficient) are required and can be achieved with higher compression using larger block sizes in encoding. The encoder may try to use larger block sizes when encoding a portion of the frame and attempt smaller block sizes incrementally for that portion of the frame if the resulting image quality or accuracy does not meet the threshold. For example, a 32x32 pixel portion of a frame may be examined to encode as four 16x16 pixel CTUs. If any of the 16x16 pixel CTUs are undesirable, it can be further divided into four 8x8 pixel CTUs. In other video standards, larger and smaller block sizes specified in a particular standard may be examined in a similar manner.

According to embodiments of the present invention, specific data may be held and shared among the encoder controllers encoding the alternative output streams from the same input stream. The shared data may include block sizes that are checked for portions of frames. For example, if an encoder controller is selected to process a 32 x 32 pixel portion of a frame as 16 x 16 CTUs (or any other combination of CTU sizes), then the encoder controller or another encoder controller will have the same resolution but different bits When encoding the same portion of a frame at rate, it may omit the previously made decision (s) and use the same size CTUs and / or check other CTU sizes. At higher bit rates, the step of checking for larger block sizes may be unnecessary because a higher bit rate can accommodate more stored information and therefore no higher compression due to larger block sizes is required have. Therefore, when the encoder controller encodes the same portion of a frame at a higher bit rate, it omits the previous check / decision to not encode with 32 x 32 pixel CTUs and can simply use 16 x 16 CTUs and / X 8 pixel CTUs meet the target bit rate while achieving the same or better quality.

Past CTU size selections may also be useful when selecting CTU sizes in streams of different resolutions. For example, suppose a CTU size of 32 x 32 is examined and rejected for a portion of a frame in a stream with a resolution of 1920 x 1080 for 16 x 16 or smaller partitions. The same portion of the frame in the stream of resolution 3840x2160 will cover a size of 64x64. The encoder may skip the step of verifying the 64 x 64 CTU size for this portion in the higher resolution frame because the size of 32 x 32 is rejected in the lower resolution stream. The encoder can start by examining CTU sizes of 32 x 32 or less.

A process for reusing block size information when encoding a stream of video in accordance with embodiments of the present invention is shown in FIG. The process includes ascertaining (310) whether the size has been determined for a similar CTU (i. E., The corresponding CTU in the corresponding frame in another stream). If the CTU size is not selected to be checked (312) then it typically begins with a larger size. If the size is determined for a similar CTU, the resolution of the source stream of a similar CTU is compared to the current stream (312). If the resolution is different, the CTU size is scaled (314) to the current stream. The predetermined CTU size (if predetermined) or the selected CTU size (if not already predetermined) is checked 316 for the current CTU being encoded. If the size is not acceptable, a smaller CTU size is attempted (318). If the size is acceptable, the CTU size is applied to the current CTU (320).

Although a particular process for reusing CTU size decisions when encoding a stream of videos is discussed above with respect to FIG. 3, any of a variety of processes may be used when encoding a plurality of media content streams in accordance with embodiments of the present invention, (Or other "block" partitions of pixels in accordance with the relevant video compression standard) size decisions. Furthermore, it should be appreciated by those skilled in the art that, in other video standards other than HEVC, "block " partitions may have available sizes other than those discussed herein and that techniques for selecting block sizes discussed herein are equally applicable do. As will be discussed further below, decisions by one encoder controller (i. E., A software process that encodes multiple streams at the same resolution and different bit rates) are performed by the encoder controller or by another encoding It can be stored and reused by the controller.

mode  Distribution and mode  Choices

The mode distribution represents the overall ratio of intra, inter, and skip macroblocks (or other "block" partitions of pixels according to the relevant video compression standard) within a frame of video. Predictive frames, such as P frames and B frames, may include intra (encoded with information only within the frame) and / or inter (co-encoded with reference to information in another frame) frame macroblocks . Under different video compression standards, block pixel partitions can be variously disadvantaged as macroblocks or coding tree units (CTUs). The mode distribution may represent block types regardless of whether the block itself is referred to as a macroblock or CTU. Tests and / or algorithms, including those known in the art, may be used to analyze each macroblock in a frame to know which type of block is most efficient when encoding the block while maintaining image quality . For example, measurements of encoding efficiency may be the number of bits required to encode a particular macroblock.

For encoding efficiency and image quality, it is often desirable to keep as close as possible the same mode distribution across the corresponding frames in the alternative streams. For example, considering the frame p with 30% intra blocks, 60% inter blocks, and 10% skip blocks, the corresponding frame p1 in the first alternative stream at the same resolution and different bit rate, Must have a similar distribution of intra, inter, and skip blocks, i.e., target mode distribution. The corresponding frame p2 in the second alternative stream at different resolutions and different bit rates will ideally also have a similar distribution of block types.

The mode distribution can be used to accelerate decisions made by the encoder about which block type to use when processing blocks in a frame. By trying to comply with the determined mode distribution, the number of crystals can be reduced. For example, towards the end of processing a particular frame, the mode distribution of the frame may deviate from the target mode distribution. Thus, the blocks may be selected to be of a specific type (e.g., intra, inter, skip) to encode these blocks closer to the target mode distribution while maintaining good quality and integrity of the image, . In various embodiments of the present invention, when choosing a block type, decisions are made between streams at different bit rates (e.g., more intra-coded blocks) because they are accommodated at higher bit rates Lt; / RTI >

A process for adjusting block type decisions in accordance with embodiments of the present invention is illustrated in FIG. Process 400 may be used when blocks in a frame are encoded or when a frame is analyzed and block types are selected without being encoded. The process includes maintaining (410) a count of blocks to be processed (e.g., analyzed or encoded). If the threshold number of blocks, or alternatively the ratio of the total blocks in the frame, is reached, then the criteria for selecting the block types may be further adjusted to select the block types to maintain the target mode distribution of the blocks in the frame ). If the end of the frame is not reached (416), processing continues for blocks in the frame and updates the count of processed blocks (410).

Although a particular process for adjusting block type decisions when encoding streams of video is discussed above with respect to FIG. 4, any of the various processes may be used when encoding a plurality of media content streams in accordance with embodiments of the present invention, Type determinations. ≪ RTI ID = 0.0 >

motion  calculation

In video compression, information is extracted from reference frames (which may be I, P, or B frames), which are predictive frames such as P frames and B frames, and information such as motion vectors and residual data used in motion estimation . A motion vector for a block of pixels (e.g., a macroblock, CTU, or similar partition) typically describes where to place the indicated block of pixels from the reference frame in the predicted frame. The residual data describes any differences between a block of pixels and a reference block of pixels. In many embodiments of the present invention, motion estimation information, such as motion vectors, can be reused from one output stream to another output stream at different resolutions and / or bit rates.

In several embodiments of the present invention, motion vectors are generated for at least one output stream at a given resolution and bit rate. The motion vectors are then applied and refined when encoding in at least one other output stream at the same resolution at different bit rates. In further embodiments of the present invention, the motion vectors generated for the output stream are applied and refined when encoding at least one other output stream at a higher resolution. The motion vectors are scaled proportionally to the difference in resolution between the first stream and the second stream and then refined to take advantage of the higher precision available at higher resolutions. For example, motion vectors generated for a stream of resolution 768x422 may be used to encode a 1280x720 resolution stream by multiplying the horizontal component of the vector by 1280/768 and the vertical component of the vector by 720/432 Can be used. The multipliers 1280/768 and 720/432 can be considered as scale factors for this change in resolution. Furthermore, the encoder can determine that the encoding cost of the residual data and the motion vector for the macroblock is very low and can skip the encoding cost for the macroblock in another output stream. In this way, motion vectors can be reused across multiple output streams and motion estimation is accelerated by not having to generate new motion vectors. Other techniques for sharing and / or reusing motion estimation information across multiple output streams may be utilized in accordance with embodiments of the present invention. Systems and methods for reusing motion estimation information when encoding alternative streams are described in Zurpal et al., Filed May 31, 2012, entitled "Systems for Reuse of Encoding Information When Encoding Alternative Streams of Video Data & Methods, " in U. S. Patent Application Serial No. < RTI ID = 0.0 > 13 / 485,609, < / RTI >

The process for reusing motion vectors in encoding a stream of video in accordance with embodiments of the present invention is illustrated in FIG. When encoding a frame in a stream of video, the encoder may verify 510 that the motion vector has already been generated for the corresponding macroblock / CTU in a different stream of video that may be reused when encoding the current frame . If not, a new motion vector is generated and stored (512). If a motion vector is present, the encoder may determine 514 whether another stream of video is at the same resolution as the current stream. If it is the same resolution, the motion vector may be applied directly (516). The motion vector may also be refined with a search, such as with a small diamond search over one or two neighboring pixels, or with sub-pixel refinement. Various other search patterns may alternatively be used to refine the motion vector. If it is a different resolution, the motion vector may be scaled (518) and refined (520) to the current resolution before being applied.

Although a particular process for reusing motion vectors in encoding a stream of video is discussed above with respect to FIG. 5, any of the various processes may be used to encode motion vector streams when encoding multiple media content streams in accordance with embodiments of the present invention. Lt; / RTI >

Adaptive Bit rate  2-pass encoding for streaming Content

Encoding content using a two-pass process can provide efficiencies through sharing and reuse of information. In many embodiments of the invention, the encoder receives the content, parses the content in the first pass, and encodes the content in the second pass. Statistics and other information collected in the first pass may be used to speed up encoding in the second pass and statistics and encoding information may be shared between encoding processes in the second pass. In some embodiments, the first pass is completed before the second pass begins. In other embodiments, the second pass may be started before the first pass is completed. In other words, the computational processes for the first and second passes may be performed concurrently if the frames are processed by the first pass process (s) before being processed by the second pass process (s).

A process for encoding content in accordance with an embodiment of the present invention is illustrated in FIG. The encoding process 600 includes receiving media content (610). The content may be contained in a single input file (e.g., a multimedia file or container format file) or a collection of media files. The content may also be an input stream of video received by the encoder. In various embodiments of the invention, the encoding process is implemented on the encoding server as an encoding application and the disk read / write operations are reduced since each input frame needs to be read only once.

The encoding process also includes making a first pass to collect statistics for the media stream (612). The first pass is an initial analysis of the content in the source input video stream and is intended to collect statistics for each resolution to be encoded. In many embodiments of the invention, the input stream is transformed into a stream for each desired output resolution and statistics are collected for each stream. The transform may include color transform (e.g., RGB to YUV, ITU 601 to 709, or format specific transforms) and resizing the dimensions / resolution of each frame. Encoding information, such as a frame complexity measure, may be assigned to each frame in the stream based on the level of complexity of the image data in the frame using techniques as discussed above. The mode distribution may also be determined for each frame. The complexity measures and mode distributions may be used as inputs to the encoding of the final streams.

Other encoding information may include the size of the macroblock data to be used when encoding the streams and the size of the header data. Considering the header size, the encoder can estimate how many bits are allocated to the overhead data and how many bits are available to encode the content. The macroblock size may also contribute to better estimates of bit allocation when encoding the content. For effective rate control, the relative ratio of header size to macroblock size must be consistent across the output streams. The relative size of the header size, the macroblock size, and / or the header size to the macroblock size may be provided as encoding parameters for the encoding process.

The encoding information may also include hypothetical reference decoder (HRD) data. The HRD model is specified in several video compression standards to assist in calculating the frame sizes and buffer fullness. The HRD recovers the decoder behavior on the encoder to avoid buffer overflow in the playback device buffer. HRD data is determined in the first analysis pass and may be stored in shared buffers or local caches for use in the second encoding pass.

Other statistics collected for one resolution may be used to accelerate determinations for encoding streams at various bit rates at the same resolution and / or another resolution. The statistics for collecting in the first pass include, but are not limited to: average quantization parameters, size of header bits, size of texture bits, number of intra macroblocks / CTUs, number of inter macroblocks / CTUs, Blocks / CTUs. In various embodiments of the present invention, statistics may be stored internally (e.g., output as a single file) and / or shared buffers in memory accessible to each encoder controller when encoding in the second pass Lt; / RTI >

In the second pass, the content is encoded 616 as output streams. In many embodiments of the invention, a separate thread or process is launched (614) by an encoding application for each output resolution, each thread or process being referred to as an encoder controller. In further embodiments of the present invention, each encoder controller encodes multiple output streams at the same resolution but at different bit rates. In various embodiments of the invention, the encoding server may have multiple processors, and the encoder controllers may each be running on different processors or may be distributed across other processors. The output streams may be encoded at a fixed bit rate (CBR) or a variable bit rate (VBR).

Until the encoding of the streams is complete (618), the encoder controllers may concurrently execute using the information and statistics stored in internal caches and / or shared buffers in memory. The encoder controller can identify (620) the memory as to whether encoding information is present for blocks and / or frames that it encodes (622) and use the information when processing the current block or frame (622). As discussed further above, different types of encoding information may be used to allow the encoder to omit decisions, or may be used as a starting point to determine the encoding information for processing the current block or frame. If the information does not yet exist, it may be generated 624 and the encoding 616 of the stream continues.

The shared information may include motion estimation information such as motion vectors. The motion vectors generated for the stream can be reused when encoding another stream at the same resolution or at different resolutions. As discussed further above, reused motion vectors may be scaled and refined to compensate for differences in resolution to encode the stream at higher resolutions. Some statistics may be specific to the resolution of the stream being encoded. As discussed above, CTU size selections at a particular resolution and bit rate are retained and can be applied to encode the same resolution at a higher bit rate. When the encoder controller makes a determination on the CTU size in a portion of a frame, it may re-use the CTU size or try smaller CTU sizes, omitting the determination when encoding the same portion of the frame at a higher bit rate.

In many embodiments of the present invention, the CTU at a given portion of the frame is sequentially encoded by the same encoder controller for a plurality of output streams at different bit rates. If the encoder controller encodes the CTU for the output stream and thereafter encodes the CTU for the different output stream, then the data required to encode the CTU may still be in the processor cache or other local memory. According to embodiments of the present invention, the encoder controller sequentially encodes CTUs for each output stream in turn. The corresponding CTUs for each output stream may be encoded using different quantization parameters for the streams at different bit rates.

Encoding with block-oriented video compression typically involves considering quantization parameters that contribute to the compression of image information within the frame. The quantization parameter for a frame is typically selected based on factors such as the frame's mode distribution and a measure of frame complexity and is selected such that the encoded frame is at the target size (in bits) while maintaining the highest possible image quality. As discussed further above, values for frame complexity measurements and mode distributions are determined for a frame of the input stream and can be used to process corresponding output frames in multiple output streams. The quantization parameter (s) from the previous frame can be used as a starting value and can be refined using the mode distribution and frame complexity measurements for the current frame. For example, if the next frame has a lower complexity measure than the previous frame, then the quantization parameters can be increased for that frame as less complex images can be compressed without significant loss of fidelity. The quantization parameters may be reduced when encoding the same frame for a higher bitrate stream due to the higher capacity for data at a higher bitrate. In many embodiments of the invention, the quantization parameters may be independently determined for corresponding frames in each output stream using the same frame complexity measure and mode distribution while adjusting for the target bit rate of each output stream have. The quantization parameters determined for the CTUs in the frame in the lower bitrate output stream are referred to as corresponding CTUs in the higher bitrate output stream and can be reduced therefrom. Encoders in accordance with embodiments of the present invention may generate rate-distortion information when encoding output streams and reuse rate-distortion information to select quantization parameters when encoding an output stream at a higher bit rate. In various embodiments of the present invention, each frame in each output stream is encoded such that its mode distribution is approximately equal to the target mode distribution determined using the source frame from the input stream. Other types of shared information may be accessed and used by the further different encoder controllers.

In various embodiments of the present invention, one or more frames of video are divided into blocks that encode the remaining motion prediction. These blocks are called conversion units (TU) in the HEVC standard and can have sizes of 4x4, 8x8, 16x16, or 32x32. Other types of block units for prediction residues may be used depending on the relevant standard or video codec. Encoders in accordance with embodiments of the present invention may use quantization parameters when encoding CTUs, TUs, coding units (CUs), and / or other types of encoding blocks defined by a video codec or standard for which the encoder is configured Can be used. The process of selecting quantization parameters for blocks and / or previously calculated quantization parameters in a frame of video based on statistics of the stream may be referred to as adaptive quantization. The quantization parameters for the blocks of pixels are based on the complexity measures for these blocks of pixels (similar to the further discussed frame complexity measures) and / or the quantization parameters already calculated for similar blocks of pixels And can be calculated in the first pass and / or the second pass. The quantization parameters and / or complexity measurements produced in the first pass may be stored in, for example, shared buffers accessible by the encoder controllers and utilized in the second pass. For example, values computed for an input stream at one resolution can be used when encoding output streams at the same resolution at different bit rates.

In the process described above with respect to FIG. 6, statistics are collected or calculated for an input video stream (612). The computed statistics may include a measure of complexity for a block of pixels in a frame within the input video stream. In many embodiments of the invention, the complexity measure for a block of pixels may be stored and used when calculating the quantization parameter for encoding the block in the second pass. A process for generating complexity measures in accordance with embodiments of the present invention is shown in FIG. The process includes selecting a block of pixels in a frame of video (710). The process analyzes the complexity of the block (712). The complexity measure can be calculated, for example, from the histogram of luminance and / or the contrast level distribution, in the AVC standard. However, any of a number of techniques under any video encoding standard may be used to produce a complexity measure in accordance with embodiments of the present invention. The complexity measure may be stored 714 for use in generating the quantization parameter (e.g., in the second pass of encoding). 10A shows a plot of the complexity measures of the CTUs as calculated in the first pass of the input stream. As shown in the figure, the complexity measurements may vary between different CTUs.

Referring again to FIG. 6, the further described process includes generating (624) encoding information if it is not present and reusing (622) encoding information if it exists. The encoding information may include quantization parameters for a particular block of pixels (such as CTU or TU). A process for generating quantization parameters (such as when generating 624 encoding information) according to embodiments of the present invention is shown in FIG. The process includes a step 810 of selecting a block of pixels. The stored complexity measure for the block is retrieved (812). The quantization parameter for encoding the block (such as CTU or TU) in the second pass may be determined 814 using statistics such as the complexity measure computed for the block in the first pass. The output can also take into account the deviation from the mean based image quantization parameter.

The quantization parameters may use the determined quantization parameters for the blocks of complexity measurements determined in the first pass and / or blocks in other output streams that have already been encoded (e.g., in a lower or higher bitrate stream). A process (such as when reusing encoding information 622) for using already generated quantization parameters for similar blocks of pixels is shown in FIG. The process includes selecting (block 910) a block of pixels. The stored quantization parameters are retrieved for similar blocks (912). A similar block is a block in a frame in the output stream generated from the same block in the selected block and the frame in the input stream. For example, a similar block may be a corresponding block that is already encoded in an output stream of the same resolution but at a different bit rate. The quantization parameter is calculated for the selected block using the retrieved quantization parameter. The quantization parameter may be further adjusted 916 for the statistics computed for the input streams and / or already encoded frames of the output streams such as, but not limited to, distortion rate and bit rate. The quantization parameters can be computed to achieve a minimum distortion and a target bit rate. FIG. 10B shows a plot of quantization parameters calculated for CTUs (at the same resolution and at different bit rates) in three different output streams with a plot of the complexity measurements shown in FIG. 10A. As shown in the figure, complexity measure values were used in determining the quantization parameter values, and therefore the plots have a similar general shape due to their relationship.

To facilitate switching of streams during playback, the set of alternative streams of video content should have intra frames (I frames) at the same location in each stream. If there is an I frame in one stream, corresponding frames in the same position (i.e., time) in each of the other alternative streams must be I frames. When an I-frame contains all of the information needed to render a frame, when the playback device needs to switch streams, it can switch in an I-frame such that playback does not lose frames and is seamless. In many embodiments of the present invention, I frames are encoded for frames at the same location in a plurality of alternative output streams. When a P frame or a B frame is encoded for an output stream, the encoder may skip checking to encode the corresponding frame at the same position as the I frame when encoding another output stream, and as a B frame or a P frame Frames can be encoded.

Although a particular process for encoding a plurality of media content streams using shared statistics is discussed above with respect to FIG. 6, any of a variety of processes may be implemented on a number of media for adaptive bitrate streaming May be used to encode the content streams.

While the invention has been described in terms of specific embodiments, many additional modifications and variations will be apparent to those skilled in the art. Therefore, the present invention is not intended to be exhaustive or to limit the invention to the precise form disclosed, including various modifications in implementations, such as using encoders and decoders that support features beyond those specified within the specific standard to which they are subjected, It will be understood that the invention may be practiced otherwise than as described. Accordingly, the embodiments of the present invention should be considered in all respects as illustrative and not restrictive.

10: adaptive streaming system 12: source encoding server
13: media server 14: content server
16: Network 18, 20, 22: Playback device
200: Source encoder 210: Processor
230: memory 240: network interface
250: source encoding application 260: source video data
270: Encoder controller 280: Encoder controller
295: Shared data buffer

Claims (40)

A source encoder configured to encode source video as a plurality of alternative video streams, the source encoder comprising:
A memory including a source encoding application;
Shared memory; And
By the source encoding application:
Receiving multimedia content, the multimedia content including source video data having a primary resolution; receiving the multimedia content;
Collecting statistics on source video data in a first pass through the received multimedia content and recording the statistics in a shared memory, wherein the statistics include complexity measures of blocks of pixels, and a complexity measure of blocks of pixels is Collecting and recording statistics for the source video data, the statistics representing a level of complexity of visual information within the block of pixels;
Determine initial encoding information for the source video data during the first pass through the received multimedia content and write the initial encoding information to a shared memory;
Encoding the source video data in parallel using collected statistics and initial encoding information to generate a plurality of alternative video streams during a second pass through the received multimedia content with a plurality of parallel encoding processes, Wherein the encoding of the source video utilizes additional encoding information and the parallel encoding processes are configured to reuse additional encoding information already determined for a portion of video by another parallel encoding process and stored in the shared memory, The parallel encoding processes are configured to generate additional encoding information that has not been previously determined for a portion of the video by another parallel encoding process and to store the generated additional encoding information in the shared memory, Wherein the encoding information comprises a parallel processing system configured to encode the source video data in parallel, the quantization parameters including a quantization parameter for blocks of pixels,
The parallel encoding processes are configured to generate additional encoding information that has not yet been determined for a portion of the video by another parallel encoding process is a coding tree unit for encoding a portion of the video frame in the source video data Tree Unit (CTU) size. The method according to claim 1,
The statistics for the source video data comprising statistics selected from the group consisting of an average quantization parameter, a size of header bits, a size of texture bits, a number of intra blocks, a number of interblocks, and a number of skip blocks. The method according to claim 1,
Wherein the parallel processing system configured to determine initial encoding information for source video data further comprises the parallel processing system configured to calculate a frame complexity measurement. delete The method according to claim 1,
Wherein determining a CTU size for encoding a portion of a frame of video in the source video data comprises:
Selecting a portion of a frame of video to encode as at least one output CTU in a first output stream;
Confirming that the size is determined for a similar CTU;
Selecting a CTU size if the size is not determined for the similar CTU;
Selecting a previously determined CTU size determined for the second output stream and comparing the resolution of the second output stream to the resolution of the second output stream if the magnitude is determined for a similar CTU;
Scaling the CTU size if the resolution of the first output stream is not the same resolution as the second output stream;
Determining if the selected CTU size is acceptable for the output CTU;
Selecting a smaller CTU size when the selected CTU size is unacceptable; And
And applying the selected CTU size to a portion of a frame of the video if the selected CTU size is acceptable for the output CTU. The method according to claim 1,
Wherein the parallel processing system configured to determine initial encoding information for source video data further comprises the parallel processing system configured to determine a mode distribution for at least one frame of video in at least one of the plurality of alternative video streams Includes a source encoder. The method according to claim 6,
Wherein the parallel processing system is configured to encode the source video data in parallel using collected statistics and initial encoding information to produce a plurality of alternative video streams comprising:
Maintaining a count of blocks processed in a frame of video in an alternate video stream;
Determine a threshold number of blocks based on the mode distribution;
Wherein the parallel processing system is further configured to adjust criteria for block type decisions if the count of blocks satisfies a threshold number of blocks. The method according to claim 1,
The parallel encoding processes configured to reuse additional encoding information already determined for a portion of video by another parallel encoding process and stored in the shared memory include:
Determining whether a motion vector exists for a second corresponding block in a second alternative stream when encoding a first block in a video frame in a first alternative stream;
Determine whether the first alternative stream and the second alternative stream are of the same resolution;
Scale the motion vector if the first alternative stream and the second alternative stream are not of the same resolution;
Refine the motion vector;
The parallel encoding processes being further configured to apply the motion vector when encoding the first block in the video frame in the first alternative stream. The method according to claim 1,
Wherein the initial encoding information further comprises a header size, a macroblock size, and a relative ratio of header size to macroblock size. The method according to claim 1,
Wherein the initial encoding information further comprises hypothetical reference decoder data. The method according to claim 1,
Wherein each of the parallel encoding processes encodes at different resolutions. 12. The method of claim 11,
Wherein each of the parallel encoding processes encodes one or more alternative video streams and wherein each of the alternative video streams encoded by the parallel encoding process is at a different bit rate. The method according to claim 1,
Wherein each of the parallel encoding processes sequentially encodes blocks from the source video data into respective streams in a subset of the plurality of alternative video streams. The method according to claim 1,
Wherein the additional encoding information comprises rate distortion information and quantization parameters. The method according to claim 1,
Wherein the blocks of pixels are coding tree units. The method according to claim 1,
Wherein the blocks of pixels are coding units. The method according to claim 1,
Wherein the blocks of pixels are transform units. The method according to claim 1,
Wherein the quantization parameters for the blocks of pixels are generated using complexity measures for blocks of pixels. The method according to claim 1,
Wherein the quantization parameters for blocks of pixels are generated using previously generated quantization parameters. The method according to claim 1,
Wherein the quantization parameters for blocks of pixels are generated using a distortion rate and a bit rate. A method for encoding source video as a plurality of alternative video streams, the method comprising:
A method comprising: receiving multimedia content, the multimedia content comprising source video data having a primary resolution using a source encoder; receiving the multimedia content;
Collecting statistics for source video data in a first pass through the received multimedia content using a source encoder and writing the statistics to a shared memory, the statistics comprising complexity measures of blocks of pixels, Collecting and recording statistics for the source video data, the complexity measure of the block of pixels representing the level of complexity of visual information within the block of pixels;
Determining initial encoding information for the source video data during the first pass through the received multimedia content using a source encoder and writing the initial encoding information to a shared memory; And
Initial encode information, and additional encode information to generate a plurality of alternative video streams during the second pass through the received multimedia content using a plurality of parallel encoding processes using a source encoder, Encoding the source video data in parallel, the encoding of the source video comprising:
Reusing additional encoding information already determined for a portion of video by another parallel encoding process and stored in the shared memory using at least one of the plurality of parallel encoding processes, Reusing the additional encoding information, wherein the additional encoding information includes quantization parameters for the blocks of the encoding information;
Generating additional encoding information that has not yet been determined for a portion of the video by another of the plurality of parallel encoding processes, wherein the additional parallel encoding process further encodes the portion of the video, Wherein generating the information further comprises: determining a coding tree unit (CTU) size for encoding a portion of a frame of video in the source video data, the additional encoding information being generated; And
Further comprising: storing the generated additional encoding information in the shared memory using a parallel encoder process. ≪ Desc / Clms Page number 21 > 22. The method of claim 21,
Statistics for source video data include: statistics encoding, including statistics selected from the group consisting of average quantization parameters, size of header bits, size of texture bits, number of intra blocks, number of inter blocks, and number of skip blocks Lt; / RTI > 22. The method of claim 21,
Wherein determining the initial encoding information for the source video data further comprises calculating a frame complexity measure. delete 22. The method of claim 21,
Wherein determining a CTU size for encoding a portion of a frame of video in the source video data comprises:
Selecting a portion of a frame of video to encode as at least one output CTU in a first output stream;
Confirming that the size is determined for a similar CTU;
Selecting a CTU size if the size is not determined for the similar CTU;
Selecting a previously determined CTU size determined for the second output stream and comparing the resolution of the second output stream to the resolution of the second output stream if the magnitude is determined for a similar CTU;
Scaling the CTU size if the resolution of the first output stream is not the same resolution as the second output stream;
Determining if the selected CTU size is acceptable for the output CTU;
Selecting a smaller CTU size if the selected CTU size is unacceptable; And
Further comprising applying the selected CTU size to a portion of a frame of the video if the selected CTU size is acceptable for the output CTU. 22. The method of claim 21,
Wherein determining the initial encoding information for the source video data further comprises determining a mode distribution for at least one frame of video in at least one of the plurality of alternative video streams, Way. 27. The method of claim 26,
Encoding the source video data in parallel using collected statistics, initial encoding information, and additional encoding information to produce a plurality of alternative video streams includes:
Maintaining a count of blocks processed in a frame of video in an alternate video stream;
Determining a threshold number of blocks based on the mode distribution; And
Further comprising adjusting criteria for block type decisions if the count of blocks satisfies a threshold number of blocks. 22. The method of claim 21,
Reusing additional encoding information already determined for a portion of video by another parallel encoding process and stored in the shared memory includes:
Determining whether a motion vector exists for a second corresponding block in a second alternative stream when encoding a first block in a video frame in a first alternative stream;
Determining if the first alternative stream and the second alternative stream are of the same resolution;
Scaling the motion vector if the first alternative stream and the second alternative stream are not of the same resolution;
Refining the motion vector; And
Further comprising applying the motion vector when encoding the first block in the video frame in the first alternative stream. 22. The method of claim 21,
Wherein the initial encoding information further comprises a header size, a macroblock size, and a relative ratio of header size to macroblock size. 22. The method of claim 21,
Wherein the initial encoding information further comprises virtual reference decoder data. 22. The method of claim 21,
Wherein each of the parallel encoding processes encodes at different resolutions. 32. The method of claim 31,
Wherein each of the parallel encoding processes encodes one or more alternative video streams and wherein each of the alternative video streams encoded by the parallel encoding process is at a different bit rate. 22. The method of claim 21,
Wherein each of the parallel encoding processes sequentially encodes blocks from the source video data into respective streams in a subset of the plurality of alternative video streams. 22. The method of claim 21,
Wherein the additional encoding information comprises rate distortion information and quantization parameters. 22. The method of claim 21,
Wherein the blocks of pixels are coding tree units. 22. The method of claim 21,
Wherein the blocks of pixels are coding units. 22. The method of claim 21,
Wherein the blocks of pixels are transformation units. 22. The method of claim 21,
Wherein the quantization parameters for blocks of pixels are generated using complexity measures for blocks of pixels. 22. The method of claim 21,
Wherein the quantization parameters for blocks of pixels are generated using previously generated quantization parameters. 22. The method of claim 21,
Wherein the quantization parameters for blocks of pixels are generated using a distortion rate and a bit rate.

Images