CN106658000B - Method and system for the prediction selectively destroyed in Video coding - Google Patents

Abstract

This disclosure relates to the method and system of the prediction for selectively destroying in Video coding.Describe the technology in the prediction and/or intra-loop filtering for allowing selectively to destroy the segment boundary between the different sections across video image in Video coding and/or decoding.Regardless of whether can be with span outland using associated prediction or loop filtering tool, high level syntax element (such as parameter set or piece header) can include that one or more instructions of signal are sent to encoder and/or decoder.It is indicated in response to the one or more, then encoder and/or decoder can correspondingly control forecasting or loop filtering tools.

Description

Method and system for the prediction selectively destroyed in Video coding

It is on December 28th, 2011 that the application, which is application No. is the 201180062300.7, applying date, entitled " is used for The divisional application of the application for a patent for invention of the selectively method and system of the prediction in destruction Video coding ".

Cross reference to related applications

This application claims " PICTURE SEGMENTATION USING submitting, entitled on December 28th, 2010 The U.S. Provisional Patent Application No.61/427,569 of GENERALIZED SLICES " and on December 23rd, 2011 are submitting, mark Entitled " METHOD AND SYSTEM FOR SELECTIVELY BREAKING PREDICTION IN VIDEO CODING's " The full content of each of the above application is incorporated to this by reference by U.S. Patent application No.13/336,475 priority Wen Zhong.

Technical field

The embodiment of the present invention is related to video compress, and more specifically, is related to the image segment boundary in video image Place is selectively using prediction and in-loop filter mechanism.

Background technique

Digital video capabilities can be incorporated into the equipment of range broad sense, including DTV, Digital Direct Broadcast system System, wireless broadcast system, personal digital assistant (PDA), laptop computer or desktop PC, video camera, number note Recording apparatus, video game device, PlayStation 3 videogame console/PS3, honeycomb or satellite radio telephone etc..Digital-video equipment can be real Existing video compression technology, such as the video compression technology those of described in the standard of similar MPEG-2, MPEG-4, wherein It can be from positioned at the Geneva Switzerland CH-1211 20, P.O. Box 56, International Standards Organization of the First Avenue Voie-Creuse H.264/MPEG-4 Part X, advanced video coding (" AVC ") obtain by (" ISO ") or www.iso.org or ITU-T, from MPEG- is obtained positioned at the International Telecommunication Union (" ITU ") on 20 Palais des Nations square of the Geneva Switzerland CH-1211 or www.itu.int 2, the full content of each of the above standard is incorporated herein by MPEG-4 standard by reference, or according to other standards Or non-standard specification realizes video compression technology, thus effectively coding and/or decoding digital video information.Others compression skill Art still may be developed in future or be currently under in exploitation.For example, the new video of referred to as HEVC/H.265 compresses Standard is in the exploitation of the JCT-VC committee.By Wiegand et al. in March, 2011 written " WD3:Working The work grass of HEVC/H.265 is illustrated in Draft 3 of High-Efficiency Video Coding, JCTVC-E603 " Case is suggested, " 3 of High-Efficiency Video Coding, JCTVC-E603 " In of WD3:Working Draft Hereinafter referred to as " WD3 " and it is integrally incorporated by reference to herein.

Video encoder can receive uncoded video information, in order to be processed into any suitable format, the format Can be and meet (can from the International Telecommunication Union (" ITU ") for being located at 20 Palais des Nations square of the Geneva Switzerland CH-1211 or It is that www.itu.int is obtained and be by reference incorporated herein entire contents) number of ITU-R BT601 Format or some other number formats.Uncoded video can be spatially organized into and be arranged in one or more two-dimensional matrixes In pixel value and be organized into a series of uncoded images in time, each uncoded image includes one or more above-mentioned The two-dimensional pixel value matrix mentioned.In addition, each pixel may include for indicating a large amount of independent of color in a digital format Component.A general format for being input into the uncoded video of video encoder is directed to tool, and there are four each of pixels point There are four luminance sample (it includes the information of brightness/brightness or darkness in relation to pixel) and two chroma samples for tool for group (it includes colouring information (for example, YCrCb 4:2:0)).

One function of video encoder be by uncoded image translation (more generally " converting ") into bit stream, stream of packets, NAL unit stream or other suitable transformats (being all referred to as " bit stream " later), its object is to for example reduce coding The quantity of redundancy into bit stream increases the recovery of bit stream therefore to increase transmission rate, to inhibit during the transmission The bit-errors or packet erasure (being referred to as Error Resiliency) that may occur or other dedicated targets.The embodiment of the present invention The the removing or reducing of redundancy, the increase of Error Resiliency and video encoder in parallel processing architecture and/or associated are provided At least one of the realization of decoder.

One function of Video Decoder is to receive to be generated by the video encoder for meeting identical video compression standard Bit stream form encoded video as its input.Then video encoder translates the coded bit stream received (more Usually " convert ") it is the uncoded video information that can be shown, store or handle in other ways.

Hardware can be used and/or software configuration (including hardware and software combination) realizes video encoder and view Frequency decoder.Any one of video encoder and Video Decoder or two realizations may include programmable hardware group Part such as those of (for example, found in personal computer (PC)) general central processor (CPU), embeded processor, figure The use of Card processor, digital signal processor (DSP), field programmable gate array (FPGA) etc..In order to realize that video is compiled Code or decoded at least part, it may be necessary to instruct, and one or more non-transient computer-readable Jie can be used Matter stores and distributes those instructions.Computer-readable medium selection includes compact disk read-only memory (CD-ROM), number view Frequency disk read-only memory (DVD-ROM), memory stick, Embedded ROM etc..

Video compress and decompression refer to video encoder and/or the specific operation that decoder Shen executes.Video decoding Device can execute all or subset of the reverse operating of encoding operation.Unless otherwise indicated, Video coding skill described herein Art also aims to the reverse operating (i.e. relevant video decoding techniques) including described video coding technique.

Uncompressed, digital representation video can be regarded as sample flow, wherein scanning sequence can be pressed by video display Handle the sample.On the boundary that a type of boundary generally occurred in the sample flow is between image in sample flow.Perhaps More video compression standards identify the boundary, and general for example by being inserted into image header at the beginning of each uncoded image Or other metadata, in these boundary partition encoding bit streams.The other boundaries being likely to occur in sample flow include piece Boundary and tile boundaries can appear in uncoded image, as described below.

Prediction in Video coding can occur in multiple grades.

One grade is referred to as " entropy coding grade " later, and the prediction in the grade is referred to as " encoding pre- It surveys ".In the grade, the decoding of entropy coding symbol may need to be successfully decoded previous entropy coding symbol.It is all or almost All current compression standards all destroy the prediction of the coding at image and piece grade.That is, detecting bit stream (or equivalent) In image or piece header after, be reset to init state with entropy coding correlated condition used in entropy coding.Entropy coding One example of prediction is the reset of the CABAC state in ITU-T Rec.H.264.

Furthermore, it is possible to there are such a encoding mechanisms, wherein the decoding mechanism is not fallen within as described above and entropy coding In the general understanding of relevant prediction, but it is still related to reconfigurable control information associated with bit stream rather than pixel Value.As an example, allow some older standard (for example, ITU-T Rec.H.261 standards) relative to one Or the motion vector encoding motion vector of multiple previous codings.The detection of block group (GOB), piece or image header is by the pre- direction finding Amount is reset to (0,0).

There is also the forecasting mechanisms across multiple image.For example, motion compensation can will come from one or more with reference to figure (it is possible that by the motion compensation) pixel value of picture is for predicting.The pre- of the type is destroyed by macro block (mb) type (or equivalent) It surveys.For example, intra-frame macro block is often used without the prediction from reference picture, however inter macroblocks may use and come from reference picture Prediction.To the extent that frame inner sheet and interframe piece are only the accumulation for belonging to the macro block of those different macro block (mb) types.

There is also such a forecast ratings, and wherein the forecast ratings include based on the reconstruct in the image being encoded The prediction for the pixel value being reconstructed during process.One example is intra prediction mechanism, for example, ITU-T Rec.H.263 Attachment I described in that (similar mechanism is also available in other video encoding standards).

Other than forecasting mechanism, several video encoding standards specify the filter for executing intra-loop filtering.One Example is the in-loop filter specified in the attachment J of ITU-T Rec.H.263.

For some applications, the image segmentation being encoded may be advantageous at lesser data block, wherein should Segmentation can occur before the coding or during coding.The two kinds of service conditions that can benefit from image segmentation are described below.

The first service condition is related to parallel processing.Former, SD video (such as 720x480 or 720 x576 Pixel) it is maximal format in the business use of broad sense.Recently, occur and used (up in various application spaces 1920x1080 pixel) HD format and 4k (4096x2048 pixel), 8k (8192x4096 pixel) and bigger Format.Although affordable computing capability increases in recent years, due to these update or in bigger format one A little associated picture sizes of format are very big, so adjusting the efficiency of parallel processing to encode and decoding process is usually has Benefit.Parallel encoding and decoding can for example occur instruction level (such as using SIMD), can in different phase it is same When handle in the assembly line of multiple video encoding units or (such as more as independent entity by independent computing engines Core general processor) on the basis of large scale structure to handle the set of Video coding subelement.The last one lattice of parallel processing Formula may need image segmentation.

Second of service condition is related to image segmentation to create the ratio effectively transmitted being suitable for over a packet network Spy's stream.The codec that encoded video is transmitted based on IP or other packet network agreements may be by maximum transmission unit The limitation of (" MTU ") dimension constraint.For encoding chip size, sometimes the result grouping including coded slice connects as much as possible Nearly MTU size and be no more than the size and be advantageous, to keep high payload/packetization overhead ratio, avoid simultaneously By network segmentation (and higher loss probability caused by result).

MTU size changes very greatly from a network to another network of network.It can be for example by commonly used in internet The MTU size of many internet connections is arranged in the minimum MTU size of network infrastructure, wherein minimum MTU size is usually right It should be in the limit in Ethernet and can be with substantially 1500 bytes.

The quantity of bit in coded image dependent on many factors as the size of source images, desired quality, just prediction The code efficiency and other factors of content complexity, video encoding standard for suitability.However, even if being set in moderate It sets in quality and content complexity, for the sequence of HD resolution ratio and higher resolution, the size of average coded image is easily Ground is more than MTU size.Video conference encoder may for example need about 2Mbit/sec, to encode 720p60 video sequence.This Lead to the average coded image size of substantially 33000 bits or 4215 bytes, this 1500 word than the MTU size of internet It saves mostly quite a lot of.In higher resolution ratio, the average image size increases to the value for being significantly higher than the MTU size of internet.It is false If the compression ratio similar with the above 720p60,4096x2048 (4k) video (4kp60) on 60fps may be for each volume Code video image needs to be greater than the grouping of 300000 bits or 25 MUT sizes.

In many pervious video encoding standards (for example, until with include WD3), image sections (or at least one form Image sections) be referred to as " piece ".In the following description, it will can be used to destroy image interior prediction, the ring of at least one form Any kind of (such as based on Video coding) image segmentation of interior filtering or other encoding mechanisms can commonly referred to as " piece ". Similarly, such as in (above available for ITU H.264 from seeing) ITU.T Rec.H.261 or ITU Rec.H.263 Each of the structure of block group (" GOB "), piece H.264 or in MPEG family of standards etc structure may be constructed throughout this paper institute The term " piece " used.However, the segmenting unit of RFC3984 or data portion H.264 cannot be constituted throughout used herein Term " piece " because they do not destroy prediction in image, intra-loop filtering or another encoding mechanism.

The example 100 using the image segmentation of piece is shown with reference to Fig. 1, Fig. 1.Image 101 is divided into two scanning sequences Piece 102,103.Sheet border is shown as thick line 104.Second 103 first macro block 105 has address 11.For example, when making When being used for transmission the corresponding bit stream 106 of image 101 with the generation of H.264 standard, bit stream 106 may include one or more The piece header 108,110 and sheet data 109,111 of a parameter set 107 and then two pieces 102,103, the wherein one or more Parameter set 107 does not include the information in relation to sheet border, header.Enlargedly show second 103 piece header 110.For example, by solving Code device determines uncoded 103 size by the combination of at least two factors.Firstly, piece header 110 includes the of piece 103 The address of one macro block 105.Secondly, for example determining the ending of piece by the new piece header in detection bit stream, or described In example, the end of piece is determined by the end (that is, after macro block 24) of the coded image in bit stream 112.First is macro All macro blocks between block and the ending of the piece constitute the piece.It should be noted that scanning sequence modification (for example, H.264 Flexible macro block sequence) quantity of the macro block in piece can be changed by creating gap.

In the non-perceptual segmentation mechanism of medium for example by using one of piece on those of the IP offer of routing layer splicing mechanism Advantage is piece by the certain types of prediction of the boundary destroyed between piece so as at least to a certain extent independently Decoding piece (discusses) in greater detail below.Therefore a piece loss not necessarily cause other of coded image can not With or it is un-decodable.According to the realization of fragmentation scheme, the loss of segmentation may cause many other segmentations unavailable on the contrary, Because through terms used herein segmentation without destroying any type of prediction.

WD4 (can get from http://wftp3.itu.int/av-arch/jctvc-sit/2011_07_f_Forino/ , by B.Bross et al. written " 4 of High-Efficiency Video Coding of WD4:Working Draft ") Be to the draft specification relevant with digital video coding standard in exploitation, efficient video coding (HEVC) can be referred to as Or H.265.Other than piece, WD4 further includes the image segmentation mechanism referred to as " tile (title) ".It, can be according to WD4 Source images are divided into the rectangular element of referred to as tile, so that each pixel of source images is a part (other constraint items of tile Part can also be applicable in).Therefore, tile is the rectangle part of image.Tile is determined by available coordinate system in high-level syntax's structure Tile boundaries, high-level syntax's structure are referred to as parameter set in WD4.It is described in more detail below tile.

In addition to the prediction between image it is possible other than, it is every in the forecasting mechanism or encoding mechanism between above-mentioned image One may be destroyed by the decoding (or equivalent, such as the decoding with the frame number piece different from previous piece) of image header. It is to destroy those forecasting mechanisms depending on video compression standard and the sheet type used across sheet border or across tile boundaries.

In h .264, it can mark relative to motion vector prediction, intra prediction, CA-VLC and CABAC state and H.264 Quasi- other aspects, are decoded independently piece.Only allow the prediction between image (including by the picture except the sheet border of motion compensation The input of prime number evidence).Although the decoding independence increases Error Resiliency, does not allow the aforementioned prediction across sheet border to reduce and compile Code efficiency.

In H.263, video encoder is selection is by using piece or which GOB with non-empty GOB header destroys With more flexibility when forecasting mechanism.For example, in the presence of including the bit in image header, it is optional when using attachment R The bit is selected, which notifies that there is no any predictions on boundary across piece/GOB (having non-empty header) to decoder signals. Across the GOB with non-empty header and across sheet border destruction it is specific (for example, motion vector mechanism), without the state of pipe fittings R. Other forecasting mechanisms are controlled by attachment R.For example, motion vector can be directed to and currently join if the bit is not set The outside for the area of space that the piece in image/GOB with non-empty header is co-located at is examined, therefore will potentially be used to move benefit The sample value repaid is from the region " input " in the piece/GOB geometric areas being not in reference picture to current slice.In addition, removing Non- attachment R be it is movable, otherwise loop filtering may include the sample value outside piece/GOB.Similarly, exist in image header Another enables or disables intra-prediction image header.

However, in most standards, to be done at least one image granularity and in some cases in sequence granularity The decision of image prediction is destroyed out.In other words, use H.263 as an example, cannot be mixed in given image (difference) Enable or disable the piece of de-blocking filter, it is also not possible in piece grade be enabled/disabled intra prediction.

As has been described, image segmentation allows to divide the image into the area of space less than entire image.Although for scheming Most of common applications of picture segmentation seem to match and be parallelized with MTU size, but image segmentation can also be used In many other purposes, including those of the size and shape and the content-adaptive that make section purpose.The coding of area-of-interest is One in several examples.In the situation, when application different coding tool (including different forecasting mechanisms), the particular portion of image Dividing may be than other parts (in the sense that spending low amount bit to be encoded and generate similar visual experience) by more Effectively encode.For example, some contents may benefit from deblocking filtering, and intra prediction can not be responded well, however Other contents in identical image may be encoded preferably without deblocking filtering, but can benefit from intra prediction.Pass through Deblocking filtering and intra prediction are enabled, third content can be best encoded.When image is divided into tile, all these contents It can be located in identical image, person for example occurs in interview situation or video conference.

One of the existing mechanism that prediction at segment boundary destroys is the disadvantage is that the enabling and/or disabling generally quilt that prediction destroys Be hard coded into existing video encoding standard, therefore, it is difficult to or cannot for example be selected at segment boundary based on the characteristic of encoded content Property ground failure prediction mechanism.

Therefore, it is necessary to a kind of improvement that prediction and intra-loop filtering mechanism individually or are integrally enabled or disabled based on every Method and system.Therefore, it is desirable to a kind of scheme at least being partially solved above or other disadvantages.

Furthermore, it is necessary to individually or integrally be enabled or disabled based on each image (or image group, sequence group etc.) across nothing The forecasting mechanism and/or intra-loop filtering mechanism of header (or equivalent) image segment boundary (such as tile boundaries).Therefore, it is desirable to one Kind at least is partially solved the scheme of above or other disadvantages.

Summary of the invention

The embodiment of the present invention provides the method and system for encoding and/or decoding such a video image, wherein Multiple predictions and the intra-loop filtering tool of image sections are enabled or disabled to the property of can choose in the video image.

A scheme according to the present invention, encoder can indicate that the tool can take for one or more forecasting tools From currently just the outside of processed image sections information as be used for the image sections in processing reference information.It compiles Code device can provide for single forecasting tool (for example, entropy prediction, intra prediction, motion compensated prediction, motion vector prediction, Make forecasting tool later) and/or tool is individually filtered (for example, adaptive interpolation filtering, adaptive loop filter, deblocking, filter Wave, sample adaptively deviate, and make loop filtering tool later) and other instructions.Alternatively, encoder can mention For provide for multiple predefined tools or may include it is any of above prediction and loop filtering tool predefined tool group with And other instructions.The parallelization and certain application cases that can be used for supporting encoder and decoder are done so, such as Soft lasting presentation (coded image is stitched together in the compressed domain).

A scheme according to the present invention, when use is without header image section (for example, tile), encoder can indicate pre- Survey tool, loop filtering tool or multiple tool groups predetermined, but regardless of the tool may across horizontal, vertical or The information of both horizontal and vertical tile boundaries is used as reference information.

As an example, H.264 or in the specific condition of HEVC, encoder be can be set for predicting and filter in ring The value of " coding interrupts instruction " mark of wave tool, such as: the inter-frame predicated reference sample value outside piece/tile boundaries；Piece/watt The vector reference sample value of (that is, passing through motion compensation) outside sheet border；The use of CABAC state outside piece/tile boundaries； The use of CA-VLC state outside piece/tile boundaries；PIPE or similar V2V entropy coding state outside piece/tile boundaries Use (only HEVC)；And in-loop filter (such as adaptive interpolation filter, auto-adaptive loop filter, the filter of deblocking loop Wave device or sample adaptively deviate) piece/tile boundaries outside state and sample value use.

A scheme according to the present invention is encoded not in the form of mark but by the operation instruction of other data structures Tool uses or other enablings, for example, in some cases " coding interrupt instruction " integer can will it is multiple it is aforementioned indicate or those The preferred Permutations combination of mark is at single symbol.

A scheme according to the present invention, be directed toward sheet border outside motion vector maximum length can be encoded into it is whole The suitable entropy coding of type indicates, therefore not only indicates to move up to by used grade is permitted apart from upper do not use Compensation, and indicate permitted maximum value, this can for example help the resource allocation in decoder realization.

Afore-mentioned code can be interrupted Warning Mark or other data encodings interrupt instruction by a scheme according to the present invention At least one of structure is stored in piece header, image header, parameter set or equivalent.

A scheme according to the present invention, decoder can by across piece/tile boundaries rather than it is other potentially suitable Outland destroys indicated forecasting tool, to making a response for mark or other data structures.

In the scheme of a broad sense, a kind of method for decoding the encoded video image including multiple sections is provided. For at least one section without associated section of header of encoded video image, this method may include: from encoded video Image obtains at least one instruction that will be applied at least one prediction or the intra-loop filtering operation of encoded video image, and And in response to this, at least one is indicated, controls at least one prediction or intra-loop filtering operation.In some cases, encoded video figure Two sections as may include at least two no associated section of headers.

In another broad aspect, a kind of method for encoding the video image including multiple sections is provided.It should Method includes: at least one section without associated section of header for video image, and acquisition will be applied to not have There is at least one instruction that at least one of at least one section of associated section of header is predicted or intra-loop filtering operates；And it responds In at least one instruction, at least one prediction or intra-loop filtering operation are controlled during encoded video image.In some cases, Video image may include at least two without associated section of header section.

In another broad aspect again, a kind of non-transient computer-readable medium is provided, meter is stored thereon with The executable instruction of calculation machine, the instruction are used to plan one or more processors to execute the video image that coding includes multiple sections Method.This method may include: at least one section without associated section of header for encoded video image, from volume Code video image obtains at least one at least one prediction or intra-loop filtering operation that will be applied to encoded video image Instruction, and in response at least one instruction, control at least one prediction or intra-loop filtering operation.In some cases, coding view Frequency image may include at least two without associated section of header section.

In another broad aspect again, a kind of non-transient computer-readable medium is provided, meter is stored thereon with The executable instruction of calculation machine, the instruction are used to plan one or more processors to execute the video image that coding includes multiple sections Method.This method may include: at least one section without associated section of header for video image, and acquisition will It is applied at least one of at least one of at least one section prediction or intra-loop filtering operation without associated section of header Instruction, and in response at least one instruction, at least one prediction or intra-loop filtering operation are controlled during encoded video image.In In some cases, video image may include at least two without associated section of header section.

In the scheme of another broad sense again, data processing system is provided, which includes being configured as holding At least one of the processor of method of the row for encoding the video image including multiple sections and accelerator hardware.This method can To include: at least one section without associated section of header for encoded video image, obtained from encoded video image At least one instruction that at least one prediction or intra-loop filtering that will be applied to encoded video image operate, and in response to At least one instruction, to control at least one prediction or intra-loop filtering operation.In some cases, encoded video image can be with Including at least two without associated section of header section.

In the scheme of another broad sense again, data processing system is provided, which includes being configured as holding At least one of the processor of method of the row for encoding the video image including multiple sections and accelerator hardware.This method can To include: at least one section without associated section of header for video image, acquisition will be applied to not have At least one instruction of at least one of at least one section prediction or the intra-loop filtering operation of associated section of header, and in response to At least one instruction, to control at least one prediction or intra-loop filtering operation during encoded video image.In some cases In, video image may include at least two without associated section of header section.

In some embodiments, according to any one of above scheme scheme, at least one prediction or intra-loop filtering behaviour Work may include entropy prediction, intra prediction, motion vector prediction, motion compensated prediction, adaptive loop filter, adaptive interpolation Filtering, deblocking filtering or sample at least one of adaptively deviate.

In some embodiments, it according to any one of above scheme scheme, can be obtained from least one combination instruction Obtain at least one scheme in multiple instructions.

In some embodiments, according to any one of above scheme scheme, at least one instruction can be encoded as It is used to indicate the vector of the maximum length of motion vector.

In some embodiments, according to any one of above scheme scheme, at least one instruction can be encoded to In parameter set.

Further scheme according to the present invention, provide a kind of device (for example, data processing system), it is a kind of for changing The method and product of the device are (for example, record thereon and/or be stored with the program for executing any method described herein The non-transient computer-readable medium or product of instruction).

Detailed description of the invention

The various feature and advantage of the embodiment of the present invention will become from the detailed description below carried out in conjunction with attached drawing Obviously, in which:

Fig. 1 is to show the example images according to an embodiment of the invention with scanning sequence piece and indicate to compile The diagram of the bit stream of code image；

Fig. 2 is to show the diagram of tile and piece according to an embodiment of the invention；

Fig. 3 is to show the block diagram of coded bit stream according to an embodiment of the invention；

Fig. 4 is to show the block diagram of coded bit stream according to an embodiment of the invention；

Fig. 5 is to show the block diagram of coded bit stream according to an embodiment of the invention；

Fig. 6 is to show the flow chart of the operation of exemplary decoder according to an embodiment of the invention；

Fig. 7 is to show the process of the operation of exemplary decoder when decoding piece according to one embodiment of present invention Figure；

Fig. 8 be show embodiment according to the present invention based on data processing system (for example, personal computer (" PC ")) realize block diagram；

It should be noted that throughout the drawings, identifying identical feature by identical appended drawing reference.

Specific embodiment

Hereinafter, details is elaborated to provide understanding of the invention.In some instances, it does not describe in detail or shows Specific software, circuit, structures and methods, in order to avoid the fuzzy present invention.Term " data processing system " used herein refers to Alternative any machine in processing data, including computer system as described herein, wireless device and network configuration.It can use and appoint Computer programming language anticipate to realize embodiments of the present invention, as long as the operating system offer of data processing system can be supported The facility of the requirement of these embodiments.Embodiment party of the invention can also be realized with hardware or with the combination of hardware and software Formula.

At least some embodiments of the invention are related to combining with the image segmentation in video compress, selectively failure prediction Mechanism and/or selectively disabling intra-loop filtering mechanism.

Term such as " section " or " image sections " is used below to refer to any one or more macro blocks more smaller than whole image Or equivalent (for example, tree block in WD4), and in " section " or " image sections " boundary, destroy the prediction of at least one form And/or disable the intra-loop filtering of at least one form.The tile of class as described below piece H.264 and class WD4) (its Middle tile_boundary_independence_idc is equal to the non-limiting example for 1) being section.

Fig. 2 shows examples 200, wherein image 201 is depicted as the tile boundaries 204 of the vertical segmentation of black matrix solid line It is divided into two tiles 202,203.Tile can coexist in image 201 with piece.For example, image 201 is as by tile boundaries 204 It is divided into as tile 202,203, while two pieces is divided by sheet border 205.Tile (the tile_ as described in WD4 1) boundary_independence_idc is equal to can summarize another type of referred to as column in one or more schemes Image sections, wherein being " METHOD AND SYSTEM FOR submitting, entitled on December 23rd, 2011 The U.S. Patent application NO.13/336,675 of SELECTIVELY BREAKING PREDICTION IN VIDEO CODING " In further describe column, entire contents are incorporated herein by reference.

Bit stream 206 corresponding to the transmission in image 201 may include for example including tile boundaries 204 for identification The parameter set 207 of tile boundaries information 208 or other high level syntax elements.However, bit stream other than parameter set 207 Part does not include any information about tile boundaries.The inside shape for the macro block that decoder can be presently processing by association The information of state information and the tile dimensions known from parameter set 207 identifies coded macroblocks (also known as maximum coding unit (LCU) Or (Shen WD4) tree block) belonging to tile.

Tile and other rectangular image splicing mechanisms are (for example, the rectangular sheet (submodule of the attachment K of ITU-T Rec.H.263 Formula)) between a difference be that tile (different from rectangular sheet) does not need header.It, can be in the case where not including header It is changed to the physical size in parameter centralized definition tile.In some cases (tile_boundary_independence_idc etc. Under 1), the forecasting mechanism in all images is interrupted according to the tile boundaries of WD4, but allows to refer in reference picture in this way A kind of sample, wherein the sample is in different location with sample associated with the tile for being performed motion compensation.In addition, Tile boundaries do not interrupt to be filtered in the ring including de-blocking filter, sample self adaptation offset filter and auto-adaptive loop filter Wave.

However, it is also likely to be convenient or expectation that encoder or decoder, which destroy different forecasting mechanism set using tile, 's.For example, in the case where very high resolution, video image is divided into and is limited by the tile of such a requirement and is advantageous, Do not allow that motion vector is directed toward outside tile boundaries and/or encoder and decoder are by tile side in the requirement wherein Boundary is considered as image boundary (similar to attachment R H.263) or similarly, thus, for example, avoiding movement not only across tile boundaries It compensates and avoids intra-loop filtering.

In other cases, encoder or decoder are capable of handling the full resolution video other than the entropy coding of symbol Coding may be convenient or desired.The encoder or decoder may be for example including the special signal based on sample process at It manages hardware, but load can not be handled in monokaryon (the known calculating that CABAC entropy coding is especially in HEVC requires high) In the case of general multi-core CPU can be used for entropy coding and/or decoding.Therefore, in order to support the service condition, it may be necessary to Entropy coding is destroyed at tile boundaries, however the forecasting mechanism in other images or between image can be across piece and/or tile Boundary.

In other situations, encoder or decoder allow to coordinate the side of may be between the limited processor across tile boundaries Just or it is desired.It in this case, is impossible to the reference of pixel value, however communication channel between the processors can Obtain the reference to control information (such as information necessary to motion vector prediction).In this case, intra prediction be can not Can, but motion vector prediction can be used.

There may be such a encoding tools, wherein the encoding tool it is indirectly related to prediction but still can be with Advantageously the encoding tool is interrupted across piece or tile boundaries.For example, submitting, entitled on November 1st, 2011 The Co-pending U.S. Patent Application of " ADAPTIVE INTERPOLATION IN DIGITAL VIDEO CODING " No.13/286,828 disclose adaptive interpolation filter, wherein encoder may be selected the adaptive interpolation filter attribute and Coefficient, wherein being by reference incorporated herein all the contents of the application.Sample outside piece is used for interpolation by limitation Filtering is advantageous.Similarly, WD4 includes adaptive interpolation filter, wherein at least partly obtains this certainly from specific pixel Adapt to the control of interpolation filter.It is limited to the acquisition pixel only in piece or tile boundaries to be advantageous.(believe with FILTER TO CONTROL The acquisition of breath is opposite) pixel that filtering is only limitted in itself in piece or tile boundaries is also advantageous.WD4 further includes other loops Filter, such as auto-adaptive loop filter (and all samples of filtering are related), de-blocking filter (related with filtering block boundary) With the filtering mechanism for being referred to as adaptive shifts samples.All these filters can share performance identical with AIF.Example Such as, in the case where the auto-adaptive loop filter specified in such as WD4, (may be independently), disabling be to for across tile boundaries It obtains the access of the information of filter tap and disables the filtering across tile boundaries itself and be advantageous.

Can by image (or higher) grade syntactic structure (for example, parameter set when using WD4 tile), by section header Information (for example, the H.263 rectangular sheet of attachment K), by the arrangement of the section header in bit stream and the group of encoder/decoder state The combination of (for example, H.264 piece when the sequence of flexible macro block is not used) or two or more aforementioned mechanisms is closed (that is, FMO Piece group is defined, and passes through piece header arrangement (passing through the first macro block of its Address Recognition piece) and piece group in bit stream in piece group The combination of the implicit advantage of interior macroblock address defines image sections, until detecting piece by bit stream parsing or other means Until ending) segment boundary is defined.

The first mechanism for allowing forecasting tool to select tile boundaries will now be described, select piece followed by permission forecasting tool The mechanism on boundary.Finally, describing the interaction of two mechanism.

With reference to the example 300 in Fig. 3, Fig. 3 shows the encoding ratio including parameter set 302 and two coded slices 304,305 Spy's stream 301.Coded slice 304,305 may belong to one or two coded image.It, can be by with the address LCU 0 in WD4 Piece header identifies image boundary.Parameter set 302 may include tile control information 303 (such as tile boundaries), and in the reality Assume that the information in parameter set 302 is related to two coded slices (that is, it includes identical rope that the parameter set in piece header, which refers to, in example Draw).In many based on WD4 and H.264 in system, parameter set is related to tens of, hundreds of or more pieces.

According to one embodiment, parameter set 302 may include multiple forecasting tool Warning Marks (PTI).For example, when setting When (enabling) PTI, the prediction that can permit span boundary is associated with mark but regardless of which coding or decoding tool；Otherwise When PTI is not set and (disables), which may be prohibited.306, intra prediction for example can be predicted for entropy coding 307, motion vector prediction 308, motion compensated prediction 309, adaptive loop filter 310, adaptive interpolation filter 311, deblocking Filtering 312, sample adaptively deviate 313 and are possible to other predictions and intra-loop filtering work defined in video coding mechanism Tool, definition mark.

Individual prediction and intra-loop filtering mechanism packet for be related to referring to the parameter set all and image PTI is included to potentially contribute to that bit stream is made to be suitable for encoding and/or decoding the hardware structure of environment such as encoder or decoder.Due to Mark may be a part that can be applied to the parameter set of many or image, so the benefit with the offer of PTI in parameter set Benefit is compared, and the expense of PTI can be ignored in parameter set.

With reference to the example 400 described in Fig. 4, Fig. 4 shows coded bit stream 401, and coded bit stream 401 includes parameter Collection 402 and the coded image including two pieces 403,404.Each since header 405,406.Piece header 405 be amplified with The part of its information is shown.

According to embodiment, piece header 405 may include multiple forecasting tool Warning Marks (PTI).For example, when setting one Or when multiple PTI, can permit the prediction and/or intra-loop filtering on span boundary, but regardless of which coding or decoding tool and mark Will is associated；Otherwise when being set and (disabling) PTI, the prediction can be forbidden.Such as entropy can be predicted in 407, frame Prediction 408, motion vector prediction 409, motion compensated prediction 410, adaptive loop filter 411, adaptive interpolation filtering 412, Deblocking filtering 413, sample adaptively deviate 414 and are possible to filter in other predictions defined in video coding mechanism and ring Wave instrument definition mark.

For being related to the individual prediction of stator and intra-loop filtering mechanism including PTI, can contribute to keep bit stream suitable Content is answered, therefore improves code efficiency.

It will now be described how two above-mentioned mechanism can interact.

Example 500 with reference to shown in Fig. 5, Fig. 5 show the encoding ratio including parameter set 502 and two pieces 503,504 Spy's stream 501, each of two pieces 503,504 piece are since corresponding header 505,506.

Be amplified at 507 the parameter set 502 that shows include for example tile control information 508 or with no header segment boundary phase The other information of pass, for example, the boundary can indicate vertical tile boundaries 204 as shown in Figure 2.In addition, parameter set 502 may Including one or more PTI.There is illustrated three PTI, the prediction 509 of a PTI and entropy is associated, in a PTI and frame in advance Survey 510 is associated and a PTI is associated with motion compensation 511.These marks can control the solution at tile boundaries 204 Code device prediction.Tile boundaries 204 for example can be arranged by tile control information 508, so that image 201 is is vertically divided into two watts Piece 202,203.Mechanism described herein can be also used for the other configurations (group including vertically and horizontally boundary of tile boundaries It closes).

Coded image can also for example including two coded slices 503,504, each coded slice from corresponding header 505, 506 start.As shown in Fig. 2, (uncoded) piece for corresponding to coded slice 503,504 may for example respectively include macroblock address 1 and arrive 14 and 15 to 24 area of space.Piece header 506, which is amplified, to be shown at 512, and may include multiple PTI.Show two A PTI, a PTI is associated with intra prediction 513, another PTI is associated with adaptive loop filter (ALF) 514.So And, it is noted that there may be but the PTI of parameter set 502 or piece header 506 that does not require between overlapping.

According to embodiment, the PTI 509,510,511 of parameter set 502 control by tile control information 508 define across watt The prediction and intra-loop filtering of sheet border 204.

According to one embodiment, the PTI 513,514 of piece header 512 controls the prediction across the boundary between piece 503,504 And intra-loop filtering.For example, the sheet border of piece 504 has other than the image boundary marked by dotted line runic sheet border line 205 A boundary.

Therefore, in example 200, tile boundaries interrupt some predictions and intra-loop filtering mechanism (in order to allow image to compile Code is distributed between several processors), while other predictions are selectively interrupted at sheet border under the control of piece header 506 (therefore encoder is provided with intra-loop filtering mechanism by failure prediction and intra-loop filtering mechanism and is controlled entirely, in order to for compiling The content of code can select the combination of any specific prediction and intra-loop filtering mechanism, including being the phase to given application or purposes Combination hope or convenient).

If the PTI for being related to identical prediction or intra-loop filtering mechanism is appeared in parameter set 502 and piece header 506, and In the case where corresponding tile and sheet border are alignment, then at least two decoders reaction is possible.It can pass through and configure File/hierarchical selection so that the selection is statically designated in mark, or based on based in parameter set control information or Other high-level syntax's contents specify the selection dynamically in mark.

One selects to be the information contradicted in the PTI emulsion sheet header 506 in parameter set 502.The selection, which can have, to be made Decoder is determined to each processor or kernel dispensing section without realizing for allowing shared information between these sections Mechanism.

Another selection is that the PTI in piece header 508 covers the information contradicted in parameter set 502.The selection can be at it Tool selection in allow bigger encoder flexibility.Others reaction is also possible.

(if Optimized Coding Based mark is located in piece header 508 or parameter set 502) for Optimized Coding Based mark, one In a little situations, following any standard is specified to be advantageous:

(1) if indicating certain kinds and/or grade, some PTI may not be a part of parameter set or piece header, Because prediction or intra-loop filtering tool are disabled in such and/or grade.

(2) it is not necessarily to or is even not intended to for independently turning on/off if determined for example in particular profile The flexibility of these individual PTI, then by two or more PTI " binding " at single combination PTI.[93] (3) are in some cases In, PTI may not be encoded into Boolean type (that is, binary system) parameter preferably.For example, can be at least partly by such a fortune The length of moving vector determines the demand in the case where motion compensation between coordinating processor, wherein the motion vector be directed toward by piece or Outside the area of space of tile covering set altogether.Therefore, in one embodiment, PTI information be also possible to be encoded into integer or Other non-boolean's shape parameters thereby indicate that appropriate range of predicted value, such as are directed toward most greatly enhancing for motion vector outside segment boundary Degree.

(4) in some cases, PTI value may not be needed to occur in the bitstream in the physical sense, because can be from Other characteristics of bit stream obtain their value.For example, may not be needed in piece includes PTI relevant to motion compensation, because According to the design carried out by standard, motion compensation is unlikely to occur in piece.

The operation that the encoder of aforementioned any configuration of PTI information is likely to be suited for according to embodiment will now be described.

With reference to Fig. 6, in one embodiment, encoder may be operated according to flow chart 600.In encoded video sequence First before, encoder may determine (601) for PTI it is relevant to sequence setting and image sequence in video The tile of image is laid out.The hardware structure of encoder, the hardware structure of decoder can be considered, by hardware structure suggestion in the determination Or the possible tile of instruction is laid out, about knowledge of transmission network (if any), such as the size of MTU etc..One It, may admissible system-level standard ratifies PTI value in such a determination by encoder in a little situations.For example, following number Word TV standard, which can be envisaged, needs the specific tile layout and specific PTI that will control prediction and intra-loop filtering across tile boundaries to be arranged For specific (higher) resolution ratio, so that cost-effective multi-processor/multi-core is implemented as possibility.It may only need The subset of all PTI is determined on sequence grade.

Several selections of those settings have been described before.

After determination, encoder can (602) PTI relevant to sequence be encoded to suitable high-level syntax's structural example In sequence or picture parameter set, sequence, GOP or image header.Encoder can be (to pass through the grammer knot of video encoding standard Structure) it selects not define PTI during the coding.

PTI relevant to sequence can keep constant at least one intact video images (unless by based on piece header PTI it is capped, as will be described later), but in many cases can be at least one " sequence " (two in video flowing All images between a IDR image and leading IDR image) and may be kept constant in full Video coding ession for telecommunication. For example, PTI relevant to sequence at least partly can be driven by hardware limitation can not be changed in ession for telecommunication.Since then with The case where assuming the latter for convenience's sake afterwards.

Encoder is continued by being encoded to piece.For doing so, encoder can determine (603) piece grade PTI, It can be with PTI interaction relevant to sequence as has been described.Piece grade PTI can be encoded into the volume of (604) piece header A part of code.

Then can be according to applying which coding standard (for example, WD4 or H.264), while considering across such as being referred to by PTI The prediction of the piece and tile boundaries that show and/or the destruction of intra-loop filtering mechanism encode (605) piece.

Next continuation (606) is encoded.

The operation that the decoder of aforementioned any PTI information configuration is applicable to according to embodiment will now be described.

Fig. 7 is the flow chart 700 of the decoder of one embodiment for use in the present invention.Decoder may receive (701) NAL unit from bit stream, and determine the type of NAL unit.If NAL unit type is used to indicate parameter set (702), Then decoder may execute parameter set parsing and storage (703) (other high-level syntaxes according to used video encoding standard Structure (for example, sequence, GOP or image header) can be also used for the purpose).

If NAL unit type is used to indicate sheet data (704) (not describing other situations), decoder may parse piece It header (705) and is then responded according to the information such as PTI information that wherein encodes.For example, piece header may include parameter Collection reference, and the parameter may be " activated " (706), as described in video encoding standard, that is, indicate the ginseng of reference The value of manifold becomes effective.Since PTI may be a part of parameter set, by the activation (706), the value of PTI may also become It obtains effectively.

Piece header is also possible that the PTI of itself, and as has been described, which, which may be different from parameter set, includes PTI.Have been described in coding in piece header PTI information and coding between the PTI information in parameter set how into The selection of row ruling.For example, by the way that the PTI (if present) based on piece header and parameter set header PTI (if present) are closed Connection, and consider that in the other parts of video encoding standard, any restrictions condition that may be present (such as passes through configuration file The limitation of the PTI made with grade and/or default setting), decoder may determine that (707) will be used to decode theme piece Final PTI setting.It should be noted that depending on parameter set PTI setting and piece header PTI setting (including sheet border and Tile boundaries are the specific conditions of alignment), for the different edges of piece, PTI may be different.

Consider final PTI setting, decoder may be used across such as by the piece or tile of the information instruction being encoded into PTI The prediction on boundary and/or intra-loop filtering technology decode (708) piece.

(709) process is continued to next NAL unit.

The processing of the NAL unit other than piece or parameter set NAL unit is not shown in Fig. 7.

Fig. 8 is to show embodiment according to the present invention to be based on data processing system (such as personal computer (" PC ")) 800 realization block diagram.Until at this time, for convenience's sake, in detail will description clearly with encoder And/or the possible physics realization of decoder is related.Many different physics realizations of combination based on software and/or component are It is possible.It in some implementations, can example in many cases due to related to cost efficiency and/or power consumption efficiency Such as video encoder and/or decoder are realized using customization or gate array integration.

In addition, using the software-based realization of general procedure framework (its example is data processing system 800) It is possible.Such as by using personal computer or similar equipment (such as set-top box, kneetop computer, mobile device), such as The lower described implementation strategy can be possible.It, can be with including such one according to the embodiment as shown in the Shen Fig. 8 The form of the computer-readable medium 801 (such as CD-ROM, semiconductor ROM, memory stick) of kind instruction, which provides, is used for PC or similar Equipment encoder and/or decoder, wherein the instruction be configured as allow processor 802 it is individually or hard with accelerator Part (such as graphics processor) 803 is in combination in conjunction with degree of coupling processor 802 and/or the memory of accelerator hardware 803 804 Execute coding or decoding.Processor 802, memory 804 and accelerator hardware 803 can be coupled to can be used for to/from aforementioned The bus 805 of equipment transmitting bit stream and uncompressed video.Dependent on realization, bit stream and uncompressed video can will be used for The peripheral equipment of input/output is coupled to bus 805.It can be for example by suitable interface such as frame grabber 807 or USB link Camera 806 is attached to bus 805 so as to the real-time input of uncompressed video by 808.Similar interface can be used for uncompressed view Frequency storage equipment such as VTR.Uncompressed video can be exported by display apparatus such as computer monitor or TV screen 809.DVD R/W drive or equivalent (such as CD-ROM, CD-RW Blu-ray Disc, memory stick) 810 can be used for realizing bit stream input and/ Or output.Finally, network interface 811 can be used for dependent on the access to network 812 for the real-time Transmission on network 812 The capacity and network 812 itself of link transmits bit stream and/or uncompressed video.

According to various embodiments, the above method can be realized by respective software module.According to other implementations Mode can realize the above method by respective hardware module.It, can be with software module and hardware module according to other embodiments Combination realize the above method.

Although a kind of Primary Reference illustrative methods describe embodiment for convenience's sake, above with reference to data The device that processing system 800 is discussed can be programmed so that the method can be implemented according to the embodiment.In addition, Product for data processing system 800 such as includes the pre-recorded storage medium or other similar of the program instruction of record on it Computer-readable medium or product can help the realization of the method with guide data processing system 800.It will be appreciated that in addition to Except the method, the device and product are also entirely fallen among the range of the embodiment.

Specifically, according to embodiment of the present invention, will can cause upon being performed by data processing system The instruction sequence of 800 execution method described hereins is included in data carrier product.The data carrier product can be loaded It is run into data processing system 800 and by data processing system 800.In addition, according to embodiment of the present invention, it can This to be caused upon being performed to execute the instruction sequence of method described herein by data processing system 800 to include in computer In program or software product.The computer program or software product can be loaded into data processing system 800 and be counted It is run according to processing system 800.In addition, according to embodiment of the present invention, will can cause upon being performed by data Reason system 800 execute method described herein instruction sequence include may include the integrated electricity of coprocessor or memory In road product (such as hardware module or multiple modules).The IC products can be mounted in data processing system 800.

Above embodiment can contribute to a kind of filter for selectively destroying in the prediction in Video coding and/or ring The improved system and method for wave, and one or more advantages can be provided.For example, for individually predicting and intra-loop filtering Mechanism includes that such a PTI potentially contributes to that bit stream is made to be suitable for encoding and/or decoding environment such as encoder or decoder Hardware structure, wherein the PTI is related to all referred to the parameter set and image.In addition, for individually predicting It include being related to can contribute to make bit stream to adapt to content to the PTI of stator, therefore improve code efficiency with intra-loop filtering machine.

The embodiment of invention as described herein is only applicable to example.Therefore, these embodiments can be made pair During the various variations and/or modification of details, whole variations and/or modification are within the scope of the present invention.

Claims (5)

1. a kind of method for decoding the encoded video image including multiple and multiple tiles, which comprises

For at least one tile without associated tile header of the encoded video image, from the encoded video Image obtains the of the deblocking filtering operation on the boundary between two tiles will being applied in the encoded video image One instruction；

In response to first instruction, the deblocking filtering operation between two tiles is controlled；

For at least one piece of associated header, obtained from the encoded video image described in will being applied to Second instruction of the deblocking filtering operation on the boundary between two pieces in encoded video image；And

In response to second instruction, the deblocking filtering operation between two pieces is controlled,

Wherein, the encoded video image includes at least part including two tiles without associated tile header At least one piece.

2. a kind of method for encoding the video image including multiple and multiple tiles, which comprises

For at least one tile without associated tile header of the video image, acquisition will be applied to institute State the first instruction of the deblocking filtering operation on the boundary between two tiles in video image；

In response to first instruction, the deblocking filtering behaviour between two tiles is controlled during the coding of the video image Make；

For at least one piece of associated header, two pieces that will be applied in the video image are obtained Between boundary deblocking filtering operation second instruction；And

In response to second instruction, the deblocking filtering operation between two pieces is controlled during the coding of the video image,

Wherein, the video image include including two tiles without associated tile header it is at least part of extremely A few piece.

3. a kind of includes the equipment stored for storing the data of encoded video image data, the encoded video image data packet It includes:

Multiple of associated header in encoded video image；

Multiple tiles without associated tile header, wherein at least one piece include two watts in the multiple tile At least part of piece；

What the deblocking filtering for controlling the boundary between two tiles that will be applied at least one described piece operated First instruction；And

The second instruction that deblocking filtering for controlling the boundary that will be applied between two pieces operates,

Wherein, second instruction is comprised in the associated header.

4. it is a kind of store computer executable instructions computer readable storage medium, the computer executable instructions when by When managing device execution, configure processor to execute method described in any one of -2 according to claim 1.

5. a kind of for encoding or the equipment of decoding video images, comprising:

Store the computer readable storage medium of computer executable instructions；With

One or more processors, one or more of processors are couple to the computer readable storage medium and are configured To execute the computer executable instructions, so that the equipment executes side described in any one of -2 according to claim 1 Method.