CN105103552A - Method for encoding inter-layer video for compensating luminance difference and device therefor, and method for decoding video and device therefor - Google Patents

Method for encoding inter-layer video for compensating luminance difference and device therefor, and method for decoding video and device therefor Download PDF

Info

Publication number
CN105103552A
CN105103552A CN201480013688.5A CN201480013688A CN105103552A CN 105103552 A CN105103552 A CN 105103552A CN 201480013688 A CN201480013688 A CN 201480013688A CN 105103552 A CN105103552 A CN 105103552A
Authority
CN
China
Prior art keywords
layer
block
image
illuminance compensation
ground floor
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201480013688.5A
Other languages
Chinese (zh)
Other versions
CN105103552B (en
Inventor
朴慜佑
尹载元
崔秉斗
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Samsung Electronics Co Ltd
Original Assignee
Samsung Electronics Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Samsung Electronics Co Ltd filed Critical Samsung Electronics Co Ltd
Publication of CN105103552A publication Critical patent/CN105103552A/en
Application granted granted Critical
Publication of CN105103552B publication Critical patent/CN105103552B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/105—Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/117—Filters, e.g. for pre-processing or post-processing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/12—Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
    • H04N19/122—Selection of transform size, e.g. 8x8 or 2x4x8 DCT; Selection of sub-band transforms of varying structure or type
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
    • H04N19/159—Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/186—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/187—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a scalable video layer
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
    • H04N19/33—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability in the spatial domain
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/59—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/80—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
    • H04N19/82—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation involving filtering within a prediction loop
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
    • H04N19/86—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving reduction of coding artifacts, e.g. of blockiness

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Discrete Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

The invention provides a method for encoding an inter-layer video for performing luminance compensation according to each image characteristic of blocks of other layer images and a device therefor, and a method for decoding an inter-layer video and a device therefor. The provided method for decoding an inter-layer video comprises: restoring first layer images on the basis of encoding information obtained from a first layer bitstream; determining whether to carry out luminance compensation for a second layer restoration block which is determined by using a first layer reference block corresponding to a second layer block among the restored first layer images, so as to restore the second layer block determined by a prediction mode and a predetermined partition type; generating the second layer restoration block by using the first layer reference block and inter-layer prediction information obtained from a second layer bitstream; and generating a second layer image which includes the second layer restoration block of which the luminance is determined depending on whether to perform the luminance compensation.

Description

For the method for encoding to cross-layer video of compensate for brightness difference and device thereof and for the method for decoding to video and device thereof
Technical field
The disclosure relates to cross-layer video Code And Decode method.In more detail, the disclosure relates to a kind of method compensated illumination (illumination) difference between interlayer image.
Background technology
Along with for reproduce and store high-resolution or high-quality video content hardware exploitation and provide, for for effectively increasing the demand of the Video Codec that high-resolution or high-quality video content are encoded or decoded.According to the Video Codec of prior art, based on the macro block with preliminary dimension, according to limited coding method, video is encoded.
The view data of spatial domain is transformed to the coefficient of frequency domain via frequency translation.According to Video Codec, image is divided into the block of preliminary dimension, performs discrete cosine transform (DCT) to each piece, and in units of block, coefficient of frequency is encoded, to carry out the quick calculating of frequency translation.Compared with the view data of spatial domain, the coefficient of frequency domain is easily compressed.Particularly, due to according to the image pixel value coming representation space territory via the inter prediction of Video Codec or the predicated error of infra-frame prediction, therefore when performing frequency translation to predicated error, mass data can be transformed to 0.According to Video Codec, reproducing raw data by using small amount of data to replace laying equal stress on continuously, reducing data volume.
Multi-layer video codec carries out Code And Decode to ground floor video and one or more second layer video.By remove ground floor video and second layer video time/redundancy between spatial redundancy and layer, the data volume of ground floor video and second layer video can be reduced.
Summary of the invention
Technical problem
One or more exemplary embodiment of method for video coding and equipment and cross-layer video coding/decoding method and equipment thereof between disclosure providing layer, wherein, cross-layer video coding method is encoded to image sequence according to one or more layer, determine whether to perform illuminance compensation according to the characteristics of image of the block of other tomographic image, and illumination difference is compensated.
Solution
According to one side of the present disclosure, a kind of cross-layer video coding/decoding method is provided, comprises following operation: reconstruct ground floor image based on the coded message obtained from ground floor bit stream; Predetermined partition type is confirmed as and the second layer block being in predictive mode in order to reconstruct, second layer block for the reconstruct determined by using ground floor reference block determines whether to perform illuminance compensation, wherein, ground floor reference block is arranged in the ground floor image of reconstruct and corresponding to second layer block; By the second layer block using the inter-layer prediction information that obtains from second layer bit stream and ground floor reference block to produce reconstruct, and produce the second layer image comprising the second layer block of reconstruct, wherein, whether the illumination of the second layer block of reconstruct is performed according to illuminance compensation and determines.
Beneficial effect
According to one or more exemplary embodiment, only to more needing the block of illuminance compensation to determine whether to perform illuminance compensation, and illuminance compensation is not performed to other block.Therefore, the increase of calculated load can be reduced, and can code efficiency be improved due to illuminance compensation.
Accompanying drawing explanation
Fig. 1 a illustrates the block diagram of the cross-layer video encoding device according to exemplary embodiment.
Fig. 1 b illustrates the flow chart of the cross-layer video coding method according to exemplary embodiment.
Fig. 2 a illustrates the block diagram of the cross-layer video decoding device according to exemplary embodiment.
Fig. 2 b illustrates the flow chart of the cross-layer video coding/decoding method according to exemplary embodiment.
Fig. 3 illustrates the inter-layer prediction structure according to exemplary embodiment.
Fig. 4 a illustrates the flow chart of the illuminance compensation method performed by cross-layer video encoding device according to exemplary embodiment.
Fig. 4 b illustrates the flow chart of the illuminance compensation method performed by cross-layer video decoding device according to exemplary embodiment.
Fig. 5 illustrates the grammer for performing illuminance compensation according to the divisional type of current block and predictive mode according to exemplary embodiment.
Fig. 6 a illustrates the flow chart of the illuminance compensation method performed by cross-layer video encoding device according to another exemplary embodiment.
Fig. 6 b illustrates the flow chart of the illuminance compensation method performed by cross-layer video decoding device according to another exemplary embodiment.
Fig. 6 c illustrates the grammer for performing illuminance compensation according to color component according to another exemplary embodiment.
Fig. 7 a and Fig. 7 b illustrates the grammer for determining whether to perform each piece illuminance compensation and residual prediction.
Fig. 8 is the block diagram of the video encoder of the coding unit based on tree structure according to exemplary embodiment.
Fig. 9 is the block diagram of the video decoding apparatus of the coding unit based on tree structure according to exemplary embodiment.
Figure 10 is the diagram of the design for describing the coding unit according to exemplary embodiment.
Figure 11 is the block diagram of the image encoder based on coding unit according to exemplary embodiment.
Figure 12 is the block diagram of the image decoder based on coding unit according to exemplary embodiment.
Figure 13 illustrates the diagram compared with deep layer coding unit and subregion according to the degree of depth according to exemplary embodiment.
Figure 14 is the diagram for describing according to the relation between the coding unit of exemplary embodiment and converter unit.
Figure 15 illustrates many coded messages according to the degree of depth according to exemplary embodiment.
Figure 16 is the diagram compared with deep layer coding unit according to the degree of depth according to exemplary embodiment.
Figure 17, Figure 18 and Figure 19 are the diagrams for describing according to the coding unit of exemplary embodiment, the relation between predicting unit and converter unit.
Figure 20 is the diagram for describing the coding unit of the coding mode information according to table 1, the relation between predicting unit and converter unit.
Figure 21 is the diagram of the physical structure of stored program dish according to exemplary embodiment.
Figure 22 is for being coiled the diagram recorded with the disk drive of fetch program by use.
Figure 23 is the integrally-built diagram of the contents providing system for providing content distribution service.
Figure 24 and Figure 25 illustrates the external structure applying the mobile phone of method for video coding and video encoding/decoding method according to exemplary embodiment and internal structure.
Figure 26 is the diagram of the digit broadcasting system of application communication system according to exemplary embodiment.
Figure 27 is the diagram of the network configuration illustrated according to the use video encoder of exemplary embodiment and the cloud computing system of video decoding apparatus.
Preferred forms
According to one side of the present disclosure, a kind of cross-layer video coding/decoding method is provided, comprises following operation: reconstruct ground floor image based on the coded message obtained from ground floor bit stream; Predetermined partition type is confirmed as and the second layer block being confirmed as being in predictive mode in order to reconstruct, second layer block for the reconstruct determined by using ground floor reference block determines whether to perform illuminance compensation, wherein, ground floor reference block is arranged in the ground floor image of reconstruct and corresponding to second layer block; By the second layer block using the inter-layer prediction information that obtains from second layer bit stream and ground floor reference block to produce reconstruct, and produce the second layer image comprising the second layer block of reconstruct, wherein, whether the illumination of the second layer block of reconstruct is performed according to illuminance compensation and determines.
Determine whether that the operation performing illuminance compensation can comprise following operation: the divisional type information and the prediction mode information that obtain second layer block from second layer bit stream; If divisional type information indicates described predetermined partition type and prediction mode information does not indicate intra prediction mode, then obtain the illumination compensation information for second layer block from second layer bit stream; Determine whether to perform illuminance compensation to the second layer block of reconstruct based on the illumination compensation information for second layer block.
Determine whether that the operation performing illuminance compensation can comprise following operation: obtain the illumination compensation information being used for following piece, wherein, the divisional type information of described piece indicates 2N × 2N type and the prediction mode information of described piece does not indicate intra prediction mode.
Determine whether that the operation performing illuminance compensation can comprise following operation: obtain based on divisional type information and prediction mode information the illumination compensation information being used for being confirmed as the block being in skip mode or 2N × 2N fusion mode.
Determine whether that performing the operation of illuminance compensation also can comprise following operation: determine whether to perform illuminance compensation to brightness (luma) component of the second layer block of the reconstruct by carrying out illuminance compensation and colourity (chroma) component according to the size of the current bay of the second layer block of reconstruct.Such as, determination operation can comprise following operation: determine that the luminance component to partitions sizes is the block of 8 × 8 performs illuminance compensation, and determines not perform illuminance compensation to the chromatic component that partitions sizes is the block of 8 × 8.
Determine whether that performing the operation of illuminance compensation can comprise following operation: according to the size of the second layer block of reconstruct, for determining whether that performing illuminance compensation to the luminance component of the second layer block of reconstruct and chromatic component carries out different operations.Such as, determination operation can comprise following operation: when the size (luminance component about block) of subregion is equal to or greater than 8 × 8, determines to perform illuminance compensation to the luminance component of this block.Determination operation can comprise following operation: when the size of the subregion of the luminance component of block is greater than 8 × 8, determines to perform illuminance compensation to the chromatic component of this block.
Can for except being confirmed as described predetermined partition type and other second layer block except the second layer block being in described predictive mode skips the operation determining whether to perform illuminance compensation.In addition, illuminance compensation can not be performed to other second layer block described.
When illuminance compensation is confirmed as performing second layer block, can not to second layer block perform residual prediction by service time direction reference block and interlayer direction reference block in the residual information of residual information to second layer block of at least one reference block predict.
When residual prediction be confirmed as performing second layer block with by service time direction reference block and interlayer direction reference block in the residual information of residual information to second layer block of at least one reference block predict time, illuminance compensation can not be performed to second layer block.
According to another aspect of the present disclosure, a kind of cross-layer video coding method being provided, comprising following operation: producing the ground floor bit stream comprising the coded message produced by encoding to ground floor image; Predetermined partition type is confirmed as and the second layer block being confirmed as being in predictive mode in order to reconstruct, second layer block for the reconstruct determined by using ground floor reference block determines whether to perform illuminance compensation, wherein, ground floor reference block is arranged in the ground floor image of reconstruct and corresponding to second layer block; Produce second layer bit stream, wherein, second layer bit stream comprise by perform second layer block and corresponding to second layer block ground floor reference block between inter-layer prediction and the inter-layer prediction information that produces, and whether second layer bit stream comprises ground floor reference block and is performed according to illuminance compensation and determines the inter-layer prediction information between the second layer block of illumination.
Determine whether that the step performing illuminance compensation can comprise following operation: divisional type information and the prediction mode information of determining second layer block; If divisional type information indicates described predetermined partition type and prediction mode information does not indicate intra prediction mode, then determine to indicate whether that the second layer block to reconstruct performs the illumination compensation information of illuminance compensation.But the step producing second layer bit stream can comprise following operation: produce and comprise for not indicating frame mode being the second layer bit stream of the following information of the second layer block of described predetermined partition type: divisional type information, prediction mode information and illumination compensation information.
Determine whether that performing the operation of illuminance compensation also can comprise following operation: determine whether to perform illuminance compensation to the luminance component of the second layer block of the reconstruct by carrying out illuminance compensation and chromatic component according to the size of the current bay of the second layer block of reconstruct.Such as, determination operation can comprise following operation: when partitions sizes is 8 × 8, determines to perform illuminance compensation to luminance component; When partitions sizes is 8 × 8, determine not perform illuminance compensation to chromatic component.
Determine whether that performing the operation of illuminance compensation can comprise following operation: according to the size of the second layer block of reconstruct, for determining whether that performing illuminance compensation to the luminance component of the second layer block of reconstruct and chromatic component carries out different operations.Such as, determination operation can comprise following operation: if the size of subregion (luminance component about block) is equal to or greater than 8 × 8, then determine to perform illuminance compensation to the luminance component of this block.Determination operation can comprise following operation: the size of the luminance component of if block is greater than 8 × 8, then determine to perform illuminance compensation to the chromatic component of this block.
Can for except being confirmed as described predetermined partition type and other second layer block except the second layer block being in described predictive mode skips the operation determining whether to perform illuminance compensation, and, illuminance compensation is not performed to other second layer block described.When illuminance compensation is confirmed as performing second layer block, can not to second layer block perform residual prediction by service time direction reference block and interlayer direction reference block in the residual information of residual information to second layer block of at least one reference block predict.
When residual prediction be confirmed as performing second layer block with by service time direction reference block and interlayer direction reference block in the residual information of residual information to second layer block of at least one reference block predict time, illuminance compensation can not be performed to second layer block.
According to another aspect of the present disclosure, a kind of cross-layer video decoding device is provided, comprises: ground floor decoder, reconstruct ground floor image based on the coded message obtained from ground floor bit stream; Illuminance compensation determiner, predetermined partition type is confirmed as and the second layer block being confirmed as being in predictive mode in order to reconstruct, second layer block for the reconstruct determined by using ground floor reference block determines whether to perform illuminance compensation, wherein, ground floor reference block is arranged in the ground floor image of reconstruct and corresponding to second layer block; Second layer decoder, by the second layer block using the inter-layer prediction information that obtains from second layer bit stream and ground floor reference block to produce reconstruct, and produce the second layer image comprising the second layer block of reconstruct, wherein, whether the illumination of the second layer block of reconstruct is performed according to illuminance compensation and determines.
According to another aspect of the present disclosure, a kind of cross-layer video encoding device being provided, comprising: ground floor encoder, producing the first laminar flow comprising the coded message produced by encoding to ground floor image; Illuminance compensation determiner, predetermined partition type is confirmed as and the second layer block being confirmed as being in predictive mode in order to reconstruct, second layer block for the reconstruct determined by using ground floor reference block determines whether to perform illuminance compensation, wherein, ground floor reference block is arranged in the ground floor image of reconstruct and corresponding to second layer block; Second layer encoder, produce second layer bit stream, wherein, second layer bit stream comprise by perform second layer block and corresponding to second layer block ground floor reference block between inter-layer prediction and the inter-layer prediction information that produces, and whether second layer bit stream comprises ground floor reference block and is performed according to illuminance compensation and determines the inter-layer prediction information between the second layer block of illumination.
According to another aspect of the present disclosure, provide a kind of computer readable recording medium storing program for performing recording program for video encoding/decoding method between execution level.According to another aspect of the present disclosure, provide a kind of computer readable recording medium storing program for performing recording program for method for video coding between execution level.
Embodiment
Hereinafter, the cross-layer video coding techniques and the cross-layer video decoding technique that determine whether to perform illuminance compensation according to block feature according to exemplary embodiment is provided with reference to Fig. 1 a to Fig. 7 b.In addition, video coding technique according to the coding unit based on tree structure of exemplary embodiment and video decoding techniques is provided with reference to Fig. 8 to Figure 20, wherein, described video coding technique and video decoding techniques can be applied to cross-layer video coding techniques and cross-layer video decoding technique.In addition, the various embodiments of application video coding technique and video decoding techniques are provided with reference to Figure 21 to Figure 27.
Hereinafter, " image " can indicate the moving image of rest image or video, or video itself.
Hereinafter, " sampling point " represents the sampling location that is assigned to image and as the data of processing target.Such as, the pixel of the image in spatial domain can be sampling point.
First, with reference to Fig. 1 a to Fig. 7 b, provide according to the cross-layer video encoding device of exemplary embodiment and cross-layer video coding method and cross-layer video decoding device and cross-layer video coding/decoding method.
Fig. 1 a illustrates the block diagram of the cross-layer video encoding device 10 according to exemplary embodiment.Fig. 1 b illustrates the flow chart of the cross-layer video coding method according to exemplary embodiment.
Ground floor encoder 12, illuminance compensation determiner 14 and second layer encoder 16 is comprised according to the cross-layer video encoding device 10 of this exemplary embodiment.It is outside that illuminance compensation determiner 14 can be formed in second layer encoder 16.
Cross-layer video encoding device 10 according to this exemplary embodiment is encoded to each image sequence in multiple image sequence according to layer by using scalable video coding method, and the exportable independent stream comprising the coded data of each layer.Cross-layer video encoding device 10 can by layers extremely different with second layer coding image sequences for ground floor image sequence.
Ground floor encoder 12 can be encoded to ground floor image, and exportable the first laminar flow comprising the coded data of ground floor image.
Second layer encoder 16 can be encoded to second layer image, and exportable the second laminar flow comprising the coded data of second layer image.
Such as, according to the scalable video coding method based on spatial scalability, low-resolution image can be encoded as ground floor image, and high-definition picture can be encoded as second layer image.The coding result of ground floor image can be outputted as the first laminar flow, and the coding result of second layer image can be outputted as the second laminar flow.
As another example, can encode to multi-view point video according to scalable video coding method.Left visual point image can be encoded as ground floor image, and right visual point image can be encoded as second layer image.Alternatively, each image in central viewpoint picture, left visual point image and right visual point image can be encoded, and in these images, central viewpoint picture can be encoded as ground floor image, left visual point image can be encoded as the first second layer image, and right visual point image can be encoded as the second second layer image.
As another example, scalable video coding method can be performed according to the time hierarchical prediction based on time retractility.The first laminar flow comprising coded message can be output, and wherein, described coded message produces by encoding to the image with basic frame rate.Time rank can be classified according to frame per second, and can be coded separately to layer.Image by reference to basic frame rate is encoded to the image with high speed frame per second further, exportable the second laminar flow comprising coded message about high speed frame per second.
In addition, scalable video can be performed to ground floor and multiple second layer.When the quantity of the second layer is equal to or greater than 3, ground floor image, the first second layer image, the second second layer image ... can be encoded with K second layer image.Therefore, the coding result of ground floor image can be outputted as the first laminar flow, the coding result of the first second layer image, the second second layer image coding result ... with the coding result of K second layer image can be outputted as respectively the one the second laminar flows, the two the second laminar flow ... with K second laminar flow.
Cross-layer video encoding device 10 according to this exemplary embodiment can perform inter prediction, and wherein, present image is predicted via the image with reference to single layer by inter prediction.Via inter prediction, the motion vector of the movable information between instruction present image and reference picture can be produced, and the residual component between present image and reference picture.
In addition, cross-layer video encoding device 10 can inter-layer prediction, wherein, second layer image by inter-layer prediction via predicted with reference to ground floor image.
In addition, when allowing at least three layers in ground floor, the second layer, third layer etc. according to the cross-layer video encoding device 10 of this exemplary embodiment, cross-layer video encoding device 10 can perform the inter-layer prediction between ground floor image and third layer image according to multilayer predict, and can perform the inter-layer prediction between second layer image and third layer image.
Via inter-layer prediction, the residual component between the reference picture of alternate position spike component between the reference picture of another layer and present image and another layer described and present image can be produced.
Inter-layer prediction structure is described in detail with reference to Fig. 3.
Encode according to each image of video each piece of layer according to the cross-layer video encoding device 10 of this exemplary embodiment.The type of block can be square, rectangle or random geometry.Block is not limited to the data cell of fixed dimension.Block can be maximum coding unit, coding unit, predicting unit, converter unit etc. among the coding unit of tree structure.The maximum coding unit comprising the coding unit of tree structure can be variously referred to as encoding block tree, block tree, root block tree, code tree, coding root or trunk.Method for video coding and the video encoding/decoding method of the coding unit using tree structure are described with reference to Fig. 8 to Figure 20.
Inter prediction and inter-layer prediction is performed by using the data cell of coding unit, predicting unit or converter unit.
Ground floor encoder 12 according to this exemplary embodiment produces symbol data by performing the source code operation comprising inter prediction or infra-frame prediction to ground floor image.Symbol data indicates the sample value of each coding parameter and the sample value of residual component.
First, ground floor encoder 12 performs inter prediction or infra-frame prediction by the sampling point of the data cell to ground floor image, transform and quantization produces symbol data, and produces the first laminar flow by performing entropy code to symbol data.
Second layer encoder 16 can be encoded to second layer image based on the coding unit of tree structure.Second layer encoder 16 performs inter prediction or infra-frame prediction by the sampling point of the coding unit to second layer image, transform and quantization produces symbol data, and produces the second laminar flow by performing entropy code to symbol data.
Second layer encoder 16 according to this exemplary embodiment carrys out inter-layer prediction to predict second layer image by using the reconstruct sampling point of ground floor image.In order to encode to the second layer original image among second layer image sequence via inter-layer prediction structure, second layer encoder 16 produces second layer predicted picture by using the ground floor image of reconstruct, and can encode to the predicated error between second layer original image and second layer predicted picture.
Second layer encoder 16 can according to block (such as coding unit or predicting unit) to second layer image inter-layer prediction.Can determine ground floor image by by the block of the block reference of second layer image.Such as, by the reconstructed blocks of locating when can determine ground floor image corresponding to the position of the current block in second layer image.By using the ground floor block of the reconstruct corresponding to second layer block, second layer encoder 16 determines that the second layer predicts block.
The second layer can be predicted that block is used as to carry out the second layer original block reference picture of inter-layer prediction by second layer encoder 16, and wherein, second layer prediction block determines by using the ground floor block of reconstruct according to inter-layer prediction structure.Second layer encoder 16 carries out entropy code to residual component (that is, the second layer predicts the difference between the sample value of block and the sample value of second layer original block) according to inter-layer prediction by using the ground floor image of reconstruct.
As mentioned above, the ground floor image that second layer encoder 16 reconstructs by inter-layer prediction structural reference is encoded to current layer image sequence.But, can when not encoding to second layer image sequence according to individual layer predict with reference to when other layer of sampling point according to the second layer encoder 16 of this exemplary embodiment.Therefore, should narrowly not be interpreted as, second layer encoder 16 only inter-layer prediction to encode to second layer image sequence.
As mentioned above, when cross-layer video encoding device 10 pairs of multi-view point videos are encoded, ground floor encoder 12 can be encoded to the first viewpoint video, and second layer encoder 16 can be encoded to the second viewpoint video.By using different cameras to take the video with different points of view, or the video of different points of view can be obtained via different camera lenses.Because the characteristic of shooting angle, light or image-capturing apparatus (such as, camera, camera lens etc.) can change according to different points of view, therefore illumination can change in the video obtained via different points of view.This illumination mismatch can be relevant to the difference of sample value of the video with different points of view.
If illumination changes in the video with different points of view, then inter-layer prediction error increases, and code efficiency can be reduced.Therefore, consider the illumination mismatch in different points of view, the illuminance compensation determiner 14 of cross-layer video encoding device 10 is encoded to multi-view point video by compensating the illumination difference had in the video of different points of view.Such as, can the illumination difference between first visual point image of being encoded by ground floor encoder 12 and second visual point image of being encoded by second layer encoder 16 be encoded.Because the illumination difference between the first visual point image and the second visual point image is encoded, therefore when second layer encoder 16 is encoded to the second visual point image, illuminance compensation can be performed.
According to exemplary embodiment, in order to compensate illumination difference by ground floor block and second layer block being compared, the difference between the sampling point mean value of ground floor block and the sampling point mean value of second layer block can be defined as offset.
But, because residual error data is predicted between layers with inter-layer prediction structure, therefore can calculated load be increased to the encoding operation that the illumination difference between layer is predicted.Therefore, according to the illuminance compensation determiner 14 of this exemplary embodiment by determining whether to perform illuminance compensation with reference to the feature of predetermined unit of data (band of such as present image or block).
Hereinafter, the operation of the cross-layer video encoding device 10 considering illuminance compensation is described in detail with reference to figure 1b.
Fig. 1 b illustrates the flow chart of the cross-layer video coding method according to exemplary embodiment.
In operation 11, ground floor encoder 12 can be encoded to ground floor image, and can produce ground floor bit stream, and wherein, ground floor bit stream comprises the sample value of the coded message of generation.Illuminance compensation determiner 14 can be encoded to second layer image, and can produce second layer bit stream, and wherein, second layer bit stream comprises the sample value of the coded message of generation.If cross-layer video encoding device 10 pairs of multi-view point videos are encoded, then ground floor image can be corresponding to the first visual point image, and second layer image can be corresponding to the second visual point image.Image can be divided into block by each in ground floor encoder 12 and second layer encoder 16, and can encode to each piece.
In operation 13, illuminance compensation determiner 14 can determine whether each piece of execution illuminance compensation to second layer image based on the ground floor block reconstructed.
In operation 15, second layer encoder 16 can inter-layer prediction to encode to the error between ground floor image and second layer image, the residual component between the reference block (ground floor reference block) of the block of second layer image (second layer block) and its corresponding ground floor image can be encoded.Therefore, second layer bit stream can comprise various types of inter-layer prediction information and the inter-layer residue component of coding method between marker.
In more detail, for operation 13, in order to reconstruct second layer block, illuminance compensation determiner 14 can for by using corresponding to second layer block and belonging to the ground floor reference block of the ground floor image of reconstruct and the second layer block determined determines whether to perform illuminance compensation.Illuminance compensation determiner 14 can determine whether to be confirmed as predetermined partition type and be in predictive mode second layer block perform illuminance compensation.
Second layer encoder 16 can determine the divisional type information of the divisional type indicating second layer block, and can determine the prediction mode information of the indication predicting pattern of second layer block.In the indicating predetermined divisional type of divisional type information and prediction mode information is not intra prediction mode, illuminance compensation determiner 14 can determine to indicate whether illumination compensation information second layer block being performed to illuminance compensation.
Second layer encoder 16 can produce second layer bit stream, wherein, so second layer bit stream comprises by predetermined partition type and do not indicate the divisional type information of the second layer block of intra prediction mode, prediction mode information and illumination compensation information.
Such as, illuminance compensation determiner 14 can determine the illumination compensation information of the block of its divisional type information instruction 2N × 2N type.
Such as, illuminance compensation determiner 14 can determine the illumination compensation information of following piece, and wherein, the divisional type information of described piece indicates 2N × 2N type and the prediction mode information of described piece does not indicate intra prediction mode.
In addition, illuminance compensation determiner 14 can determine instruction skip mode based on divisional type information and prediction mode information or be confirmed as the illumination compensation information of the block being in 2N × 2N fusion mode.
In addition, illuminance compensation determiner 14 can determine whether to perform illuminance compensation to by by the luminance component of the block of illuminance compensation and chromatic component according to the size of current bay.Such as, can determine that the luminance component to partitions sizes is the block of 8 × 8 performs illuminance compensation, and can determine not perform illuminance compensation to the chromatic component that partitions sizes is the block of 8 × 8.
In addition, illuminance compensation determiner 14 can carry out different operation for determining whether to the luminance component of block and the process of chromatic component execution illuminance compensation according to the size of block.Such as, the partitions sizes of the luminance component of if block is equal to or greater than 8 × 8, then illuminance compensation determiner 14 can be determined to perform illuminance compensation to the luminance component of block.The partitions sizes of the luminance component of if block is greater than 8 × 8, then only can perform illuminance compensation to the chromatic component of block.The partitions sizes of the luminance component of if block is 8 × 8, then the partitions sizes of the chromatic component of block can be 4 × 4.If perform illuminance compensation to the block with 4 × 4 sizes, then calculated load increases, the partitions sizes of the luminance component of if block is made to be 8 × 8, then illuminance compensation is not performed to the chromatic component of block, and if the partitions sizes of the luminance component of block is greater than 8 × 8, then can perform illuminance compensation to the chromatic component of block.
Illuminance compensation determiner 14 can skip and determines whether except being confirmed as predetermined partition type and other second layer block except the block being in predictive mode performs the operation of illuminance compensation.
But according to interlayer coding method, the divisional type of second layer block or predictive mode can be confirmed as the divisional type and the predictive mode that are equal to the ground floor block corresponding to second layer block.In the case, even if illuminance compensation determiner 14 is formed on second layer encoder 16 outside, illuminance compensation determiner 14 still predicts divisional type and the predictive mode of second layer block by the divisional type information of use ground floor block and the sample value of prediction mode information.Therefore, illuminance compensation determiner 14 can determine whether to perform illuminance compensation to second layer block based on the predictive mode of the divisional type of the prediction of second layer block and prediction.
In addition, second layer encoder 16 can not perform residual prediction to the second layer block being confirmed as carrying out illuminance compensation, wherein, residual prediction by by service time direction reference block and interlayer direction reference block in the residual information of residual information to second layer block of at least one reference block predict.Therefore, only can determine for the block (band or picture) being confirmed as not carrying out illuminance compensation the information indicating whether execution residual prediction.
Second layer encoder 16 can not perform to the such second layer block operation being used for compensating the difference between the illumination of ground floor reference block and the illumination of second layer reference block, wherein, described second layer block is confirmed as carrying out residual prediction, so as by service time direction reference block and interlayer direction reference block in the residual information of at least one reference block predict the reference information of second layer block.Therefore, only illumination compensation information can be determined for the block (band or picture) being confirmed as not carrying out residual prediction.
In addition, according to the determination undertaken by illuminance compensation determiner 14, the illumination of second layer reference block can be determined via illuminance compensation, or, the illumination of second layer reference block can be determined when not carrying out illuminance compensation.Therefore, in operation 15, second layer encoder 16 can produce the second layer bit stream comprising the inter-layer prediction information relevant with the inter-layer prediction between ground floor reference block and second layer block, and wherein, whether the illumination of second layer block is performed according to illuminance compensation and determines.
When considering that ground floor image adjusts the illumination of second layer image, the difference between second layer image and ground floor image is reduced further, and the code efficiency of inter-layer prediction can be enhanced.In addition, according to the coding mode of block, illuminance compensation can be paid the utmost attention under specific coding pattern.
The central processing unit (not shown) of overall control ground floor encoder 12, illuminance compensation determiner 14 and second layer encoder 16 can be comprised according to the cross-layer video encoding device 10 of this exemplary embodiment.Alternatively, each processor (not shown) that can be had by it in ground floor encoder 12, illuminance compensation determiner 14 and second layer encoder 16 drives, and processor can operate mutually, cross-layer video encoding device 10 can be operated.Alternatively, according to the control of the ppu (not shown) of cross-layer video encoding device 10, ground floor encoder 12, illuminance compensation determiner 14 and second layer encoder 16 can be controlled.
Cross-layer video encoding device 10 can comprise one or more data storage cell (not shown), wherein, one or more data storage cell described is for storing the input and output data of ground floor encoder 12, illuminance compensation determiner 14 and second layer encoder 16.Cross-layer video encoding device 10 can comprise Memory Controller (not shown), and wherein, the input and output of Memory Controller to the data of one or more data storage cell described manage.
The interior video encode processor that cross-layer video encoding device 10 and inside embed or external video encode processor carry out interoperability with output video coding result, make cross-layer video encoding device 10 can perform the video encoding operations comprising map function.The interior video encode processor of cross-layer video encoding device 10 not only can be corresponding to independent processor, also can comprise Video coding processing module to the central processing unit of cross-layer video encoding device 10 (CPU) or graphical operation unit and therefore to perform the situation of elementary video encoding operation corresponding.
Fig. 2 a illustrates the block diagram of the cross-layer video decoding device 20 according to exemplary embodiment.
Ground floor decoder 22, illuminance compensation determiner 24 and second layer decoder 26 is comprised according to the cross-layer video decoding device 20 of this exemplary embodiment.Illuminance compensation determiner 24 can be included in second layer decoder 26.Second layer decoder 26 can be formed in outside according to the illuminance compensation determiner 24 of another exemplary embodiment.
Cross-layer video decoding device 20 according to this exemplary embodiment can based on flexible coding method reception according to the bit stream of layer.The quantity of the layer of the bit stream that cross-layer video decoding device 20 receives is not limited.But for convenience of description, hereinafter, in the exemplary embodiment, the ground floor decoder 22 of cross-layer video decoding device 20 receives the first laminar flow and decodes to ground floor stream, and second layer decoder 26 receives the second laminar flow and decodes to second layer stream.
Such as, the cross-layer video decoding device 20 based on spatial scalability can receive stream, and wherein, in described stream, the image sequence with different resolution is encoded into different layers.By carrying out decoding to reconstruct sequence of low resolution pictures to ground floor stream, and by carrying out decoding to second layer stream to reconstruct high-definition picture sequence.
As another example, by using scalable video coding method, multi-view point video is decoded.When the stereo video streaming with multiple layers is received, by carrying out decoding to ground floor stream to reconstruct left visual point image.By decoding further to the second layer stream except the first laminar flow, right visual point image can be recovered.
Alternatively, when the multiple vision point video stream with multiple layers is received, carry out reconstruction center visual point image by carrying out decoding to ground floor stream.By decoding further to the second layer stream except the first laminar flow, the left visual point image of restructural.By decoding further to the third layer stream except the first laminar flow, the right visual point image of restructural.
As another example, the scalable video coding method based on time retractility can be performed.By decoding to ground floor stream, restructural has the image of basic frame rate.By decoding further to the second layer stream except the first laminar flow, restructural has the image of high speed frame per second.
When the quantity of the second layer is equal to or greater than 3, from the first laminar flow reconstruct ground floor image, also second layer image can be reconstructed by carrying out decoding further with reference to the ground floor image of reconstruct to the second layer.If also by decoding to K laminar flow with reference to the second layer image of reconstruct, then go back restructural K tomographic image.
Cross-layer video decoding device 20 can obtain the coded data of ground floor image and second layer image from the first laminar flow and the second laminar flow, and can obtain the motion vector produced by inter prediction and the information of forecasting passing through inter-layer prediction and produce.
Such as, cross-layer video decoding device 20 can to the decoding data of the inter prediction of each layer, and can between multiple layer by the decoding data of inter-layer prediction.Based on coding unit or predicting unit, reconstruct can be performed by using motion compensation and room decipherer.
By reference pin, to present image, the image that motion compensation reconstructs each laminar flow is performed via the reconstructed image that inter prediction is predicted to same layer.Motion compensation represents the operation by carrying out synthesizing the reconstructed image reconstructing present image with reference to the residual component of image and present image, and wherein, reference picture is by using the motion vector of present image to determine.
In addition, cross-layer video decoding device 20 is by performing room decipherer, to reconstruct via the predicted second layer image of inter-layer prediction with reference to ground floor image.Room decipherer represents the operation by carrying out the residual component of the reference picture of another layer and present image to synthesize the reconstructed image reconstructing present image, and wherein, reference picture is determined predicting present image.
Room decipherer can be performed to reconstruct by reference to the predicted third layer image of second layer image according to the cross-layer video decoding device 20 of this exemplary embodiment.Inter-layer prediction structure is described in detail with reference to Fig. 3.
But, can when not decoding to second layer stream with reference to when ground floor image sequence according to the second layer decoder 26 of this exemplary embodiment.Therefore, should narrowly not be interpreted as: second layer decoder 26 only inter-layer prediction to decode to second layer image stream.
Decode for each piece of each image of cross-layer video decoding device 20 pairs of videos.Block can be maximum coding unit, coding unit, predicting unit, converter unit etc. among the coding unit of tree structure.
Ground floor decoder 22 is decoded to ground floor image by using the coded identification of the ground floor image of resolving.If cross-layer video decoding device 20 receives coding unit based on tree structure by the stream of encoding, then ground floor decoder 22 can perform decoding based on each maximum coding unit of the coding unit of tree structure to the first laminar flow.
Ground floor decoder 22 can perform entropy decoding to each maximum coding unit, and therefore can obtain coded message and coded data.Ground floor decoder 22 can perform inverse quantization and inverse transformation to the coded data that obtains from stream, and therefore restructural residual component.The bit stream of the conversion coefficient after quantification directly can be received according to the ground floor decoder 22 of another exemplary embodiment.As the inverse quantization performed the conversion coefficient after quantizing and the result of inverse transformation, the residual component of image can be reconstructed.
Ground floor decoder 22 is by carrying out combination to reconstruct ground floor image via the motion compensation between identical layer image by predicted picture and residual component.
According to inter-layer prediction structure, second layer decoder 26 produces second layer predicted picture by the sampling point of the ground floor image using reconstruct.Second layer decoder 26 can be decoded to second layer stream, and can obtain the predicated error produced due to inter-layer prediction.Second layer decoder 26 is by carrying out combining the second layer image producing reconstruct by second layer predicted picture and predicated error.
Second layer decoder 26 is by using the ground floor image of the reconstruct of being decoded by ground floor decoder 22 to determine second layer predicted picture.According to inter-layer prediction structure, second layer decoder 26 can determine ground floor image by by the block of the block of second layer image (such as coding unit or predicting unit) reference.Such as, can determine ground floor image while corresponding to the position of the current block in second layer image by the reconstructed blocks of locating.By using the ground floor block of the reconstruct corresponding to second layer block, second layer decoder 26 determines that the second layer predicts block.
The second layer can be predicted that block is used as to carry out the second layer original block reference picture of inter-layer prediction by second layer decoder 26, and wherein, second layer prediction block determines by using the ground floor block of reconstruct according to inter-layer prediction structure.In the case, second layer decoder 26 is by predicting that by the residual component obtained via inter-layer prediction and the second layer sample value of block carries out synthesis to reconstruct second layer block, and wherein, second layer prediction block is by using the ground floor image of reconstruct to determine.
According to spatial scalable video coding method, when ground floor decoder 22 reconstruct has the ground floor image of the resolution different from the resolution of second layer image, second layer decoder 26 can carry out interpolation to the ground floor image of reconstruct by the adjusted size of the ground floor image of reconstruct to be the resolution of second layer original image.The ground floor image of the reconstruct after interpolation can be confirmed as the second layer predicted picture for inter-layer prediction.
Therefore, the ground floor decoder 22 of cross-layer video decoding device 20 can be decoded to ground floor stream, and therefore restructural ground floor image sequence, and second layer decoder 26 can be decoded to second layer stream, and therefore restructural second layer image sequence.
In addition, consider the illumination mismatch in different points of view, the illuminance compensation determiner 24 of cross-layer video decoding device 20 can compensate the illumination difference between the video with different points of view, and restructural has the video of different points of view.Such as, the illumination that can obtain between first visual point image of being decoded by ground floor decoder 22 and second visual point image of being decoded by second layer decoder 26 from bit stream is poor.Because the second visual point image is obtained for the illumination difference of the first visual point image, therefore when second layer decoder 26 is decoded to the second viewpoint video, can determine whether to perform illuminance compensation operation.
Can consider that according to the illuminance compensation determiner 24 of this exemplary embodiment the feature of the predetermined unit of data (such as band or block) of current picture determines whether to perform illuminance compensation.
Hereinafter, the operation of the cross-layer video decoding device 20 considering illuminance compensation is described in detail with reference to Fig. 2 b.
Fig. 2 b illustrates the flow chart of the cross-layer video coding/decoding method according to exemplary embodiment.
In operation 21, ground floor decoder 22 can reconstruct ground floor image based on the coded message obtained from ground floor bit stream.
In operation 23, illuminance compensation determiner 24 can determine whether to be confirmed as predetermined partition type and be in predictive mode second layer block perform illuminance compensation.
In operation 25, second layer decoder 26 can determine by using the ground floor reference block corresponding to second layer block of the ground floor image reconstructed the second layer block reconstructed according to inter-layer prediction structure.By synthesizing to ground floor reference block and the residual component of the inter-layer prediction information obtained from the second layer bit stream second layer block determining to reconstruct.
Be defined as carrying out the second layer block of illuminance compensation by illuminance compensation determiner 24 in order to reconstruct, second layer decoder 26 can compensate the illumination difference between ground floor reference block and second layer block.
The information of the illumination difference between can obtaining about layer from bit stream.Alternatively, the brightness value of second layer block nearly can be preset brightness value by compensation.
According to this exemplary embodiment, illuminance compensation and interlayer compensation can be performed together.Can according to inter-layer prediction structure, according to the sampling point of second layer block additionally being performed to the mode of illuminance compensation to determine the sampling point of the reconstruct of second layer block, wherein, the sampling point of second layer block is performed by the sampling point of the reconstruct to ground floor block and determines about the compensation of residual component.
According to another exemplary embodiment, illuminance compensation and interlayer compensation can be performed during difference.Because illumination difference component can be included in residual component between layers according to inter-layer prediction structure, therefore illuminance compensation can not be performed according to inter-layer prediction structure to the sampling point of second layer block, wherein, the sampling point of second layer block is performed by the sampling point of the reconstruct to ground floor block and determines about the compensation of residual component.But, illuminance compensation can be performed to the second layer block do not compensated for the residual component between layer.
Therefore, second layer decoder 26 can produce the second layer image of the reconstruct be made up of the second layer block be reconstructed based at least one in inter-layer prediction and illuminance compensation.
Illuminance compensation determiner 24 can obtain divisional type information and the prediction mode information of second layer block from second layer bit stream.When the indicating predetermined divisional type of divisional type information, and when prediction mode information does not indicate intra prediction mode, illumination compensation information for second layer block can be obtained from second layer bit stream.Illuminance compensation determiner 24 can determine whether based on the illumination compensation information for second layer block to perform illuminance compensation to the second layer block of reconstruct.
Illuminance compensation determiner 24 can obtain the illumination compensation information for following piece, and wherein, the divisional type information of described piece indicates 2N × 2N type and the prediction mode information of described piece does not indicate intra prediction mode.
Illuminance compensation determiner 24 can obtain the illumination compensation information for following piece, and wherein, the divisional type information of described piece indicates 2N × 2N type and the prediction mode information of described piece does not indicate intra prediction mode.
As another example, illuminance compensation determiner 24 can obtain based on divisional type information and prediction mode information the illumination compensation information being used to indicate skip mode or being confirmed as the block being in 2N × 2N fusion mode.
Illuminance compensation determiner 24 can determine whether to perform illuminance compensation to by by the luminance component of the block of illuminance compensation and chromatic component according to the size of current bay.Such as, can determine that the luminance component to partitions sizes is the block of 8 × 8 performs illuminance compensation, and can determine not perform illuminance compensation to the chromatic component that partitions sizes is the block of 8 × 8.
In addition, illuminance compensation determiner 24 can according to the size of block for determining whether that process luminance component and chromatic component being performed to illuminance compensation carries out different operations.Such as, illuminance compensation determiner 24 can be determined to perform illuminance compensation to the luminance component of block, and wherein, the partitions sizes of luminance component is equal to or greater than 8 × 8.Only can perform illuminance compensation to the chromatic component of block, wherein, the partitions sizes of the luminance component of block is greater than 8 × 8.
Illuminance compensation determiner 24 can skip and determines whether except being confirmed as predetermined partition type and other second layer block except the block being in predictive mode performs the operation of illuminance compensation.
In addition, illuminance compensation determiner 24 according to this exemplary embodiment can not perform residual prediction to the second layer block being confirmed as carrying out illuminance compensation, wherein, residual prediction by be used for by service time direction reference block and interlayer direction reference block in the residual information of residual information to second layer block of at least one reference block predict.Therefore, only can determine for the block (band or picture) being confirmed as not carrying out illuminance compensation the information indicating whether execution residual prediction.
Can not perform for the illuminance compensation by the sample value of ground floor reference block compensation illumination difference being determined the sample value of second layer block according to the illuminance compensation determiner 24 of another exemplary embodiment, wherein, second layer block be confirmed as carrying out residual prediction in case by service time direction reference block and interlayer direction reference block in the residual information of residual information to second layer block of at least one reference block predict.Therefore, only can obtain for the block (band or picture) being confirmed as not carrying out residual prediction the information indicating whether to perform illuminance compensation.
But according to interlayer coding method, the divisional type of second layer block or predictive mode can be confirmed as the divisional type and the predictive mode that are equal to the ground floor block corresponding to second layer block.In the case, even if illuminance compensation determiner 24 is formed in second layer encoder 26 outside, illuminance compensation device 24 still predicts divisional type and the predictive mode of second layer block by the divisional type information of use ground floor block and the sample value of prediction mode information.Therefore, illuminance compensation determiner 24 can determine whether to perform illuminance compensation to second layer block based on the predictive mode of the divisional type of the prediction of second layer block and prediction.
The central processing unit (not shown) of overall control ground floor decoder 22, illuminance compensation determiner 24 and second layer decoder 26 can be comprised according to the cross-layer video decoding device 20 of this exemplary embodiment.Alternatively, each processor (not shown) that can be had by it in ground floor decoder 22, illuminance compensation determiner 24 and second layer decoder 26 drives, and processor can operate mutually, cross-layer video decoding device 20 can be operated.Alternatively, based on the control of the ppu (not shown) of the cross-layer video decoding device 20 according to this exemplary embodiment, ground floor decoder 22, illuminance compensation determiner 24 and second layer decoder 26 can be controlled.
One or more data storage cell (not shown) can be comprised according to the cross-layer video decoding device 20 of this exemplary embodiment, wherein, one or more data storage cell described is for storing the input and output data of ground floor decoder 22, illuminance compensation determiner 24 and second layer decoder 26.Cross-layer video decoding device 20 can comprise Memory Controller (not shown), and wherein, the input and output of Memory Controller to the data of one or more data storage cell described manage.
The interior video decoding processor embedded according to cross-layer video decoding device 20 and the inside of this exemplary embodiment or external video decoding processor carry out interoperability with via video decode reconstructing video, make cross-layer video decoding device 20 can perform the video decoding operation comprising inverse transformation.Interior video decoding processor according to the cross-layer video decoding device 20 of this exemplary embodiment not only can be corresponding to independent processor, also can comprise Video decoding module to the CPU of cross-layer video decoding device 20 or graphical operation unit and therefore to perform the situation of elementary video decode operation corresponding.
With reference to Fig. 2 a and Fig. 2 b, when cross-layer video decoding device 20 pairs of second layer images are decoded, the block of cross-layer video decoding device 20 pairs of particular types or band perform compensation about the illumination difference between different layers image or the compensation that performs about the illumination difference between viewpoint, make the illumination between the ground floor image that reconstructs and the second layer image of reconstruct can become consistent.With reference to Fig. 1 a and Fig. 1 b, the different layers image in the block of cross-layer video encoding device 10 pairs of particular types or band performs illuminance compensation, and the residual component between predicted picture and original image can be reduced.Therefore, decoding efficiency can be enhanced.
Fig. 3 illustrates the inter-layer prediction structure according to exemplary embodiment.
Cross-layer video encoding device 10 according to this exemplary embodiment can carry out predictive coding according to the reproduction order of the multi-view point video predict 50 shown in Fig. 3 to basic visual point image, left visual point image and right visual point image.
According to the reproduction order of multi-view point video predict 50, the image of same viewpoint arranges in the horizontal direction.Therefore, the left visual point image being labeled " left side " is horizontally arranged in a line, is labeled that " the basic visual point image of " center " is horizontally arranged in a line, and the right visual point image being labeled " right side " is horizontally arranged in a line.Compared with left visual point image/right visual point image, basic visual point image can be central viewpoint picture.
In addition, the image with same frame sequential counting (POC) order vertically arranges.The POC order instruction of image forms the reproduction order of the image of video." POCX " instruction be labeled in multi-view point video predict 50 is arranged in the relative reproduction order of the image of respective column, and along with the number of X reduces, the reproduction order of image is shifted to an earlier date, and along with the number of X increases, the reproduction order of image is postponed.
Therefore, according to the reproduction order of multi-view point video predict 50, the left visual point image being labeled " left side " is arranged in the horizontal direction according to POC order (reproduction order), be labeled that " the basic visual point image of " center " is arranged in the horizontal direction according to POC order (reproduction order), and the right visual point image being labeled " right side " is arranged in the horizontal direction according to POC order (reproduction order).In addition, the left visual point image and the right visual point image that are positioned at same column with basic visual point image are the images having different points of view but have identical POC order (reproduction order).
For each viewpoint, four consecutive images form a picture group (GOP).Each GOP comprises picture between continuous print anchor (anchor) picture and an anchor picture (key picture).
Anchor picture instruction random access point, and when video is reproduced, if reproducing positions is optional in the image be arranged from the reproduction order (that is, POC order) according to image, then reproduces its POC sequentially close to the anchor picture of reproducing positions.Basic visual point image comprises basic viewpoint anchor picture 51,52,53,54 and 55, and left visual point image comprises left viewpoint anchor picture 151,152,153,154 and 155, and right visual point image comprises right viewpoint anchor picture 251,252,253,254 and 255.
Multi-view image is reproduced and prediction (reconstruct) according to GOP order.First, according to the reproduction order of multi-view point video predict 50, according to viewpoint, the image that GOP0 comprises can be reproduced, and the image that then GOP1 comprises can be reproduced.That is, the image that each GOP comprises can be reproduced according to the order of GOP0, GOP1, GOP2 and GOP3.In addition, according to the coded sequence of multi-view point video predict, according to viewpoint, the image that GOP0 comprises can predicted (reconstruct), and then the image that comprises of GOP1 can predicted (reconstruct).That is, according to the order of GOP0, GOP1, GOP2 and GOP3, the image that each GOP comprises can predicted (reconstruct).
According to the reproduction order of multi-view point video predict 50, interview prediction (inter-layer prediction) and inter prediction are performed to image.In multi-view point video predict, the image that arrow starts is reference picture, and the image that arrow terminates is predicted image by use reference picture.
Predicting the outcome of basic visual point image also can be outputted as basic visual point image stream by coding, and predicting the outcome of additional viewpoint image also can be outputted as a layer bit stream by coding.In addition, predicting the outcome of left visual point image can be outputted as ground floor bit stream, and predicting the outcome of right visual point image can be outputted as second layer bit stream.
Only inter prediction is performed to basic visual point image.That is, as the anchor picture 51,52,53,54 and 55 of I picture type not with reference to other image, but predicted by reference to other basic visual point image as other image of B picture type and b picture type.By reference to the I picture type anchor picture be positioned at by POC order above and by POC sequential bits I picture type anchor picture later to predict B picture type image.By reference to the I picture picture type be positioned at by POC order above and by POC sequential bits B picture type image later to predict b picture type image, or by reference to the B picture type image be positioned at by POC order above and by POC sequential bits I picture type anchor picture later to predict b picture type image.
For left visual point image and right visual point image, perform the inter prediction of the interview prediction (inter-layer prediction) with reference to the image of different points of view and the image with reference to same viewpoint.
Come to perform interview prediction (inter-layer prediction) to left viewpoint anchor picture 151,152,153,154 and 155 by the basic viewpoint anchor picture 51,52,53,54 and 55 identical respectively with reference to POC order.By with reference to laying respectively at the basic anchor picture 51,52,53,54 and 55 of identical POC order or left viewpoint anchor picture 151,152,153,154 and 155, interview prediction is performed to right viewpoint anchor picture 251,252,253,254 and 255.In addition, can to not being that other image of anchor picture 151,152,153,154,155,251,252,253,254 and 255 performs with reference to having different points of view but having the interview prediction (inter-layer prediction) of image of identical POC order in left visual point image and right visual point image.
By reference to the image of same viewpoint to not being that other image of anchor picture 151,152,153,154,155,251,252,253,254 and 255 is predicted in left visual point image and right visual point image.
But possibly cannot by reference to having predicting each image in left visual point image and right visual point image by reproduction order preceding anchor picture among the additional viewpoint image of same viewpoint.That is, for the inter prediction when front left visual point image, can with reference to the reproduction order except left viewpoint anchor picture at the left visual point image before front left visual point image.Similarly, for the inter prediction when front right visual point image, can with reference to the reproduction order except right viewpoint anchor picture at the right visual point image before front right visual point image.
In addition, for the inter prediction when front left visual point image, preferably: predicted operation is with reference to comprising left visual point image included in the previous GOP before the current GOP of front left visual point image, and described predicted operation is by reference to included performing at the left visual point image be reconstructed before front left visual point image in current GOP.This operation is also applied to right visual point image.
Cross-layer video decoding device 20 according to this exemplary embodiment can reconstruct basic visual point image, left visual point image and right visual point image according to the reproduction order of the multi-view point video predict 50 shown in Fig. 3.
Can via parallax compensation between the viewpoint with reference to basic visual point image and with reference to left visual point image image between motion compensation reconstruct left visual point image.Can via parallax compensation between the viewpoint with reference to basic visual point image and left visual point image and with reference to right visual point image image between motion compensation reconstruct right visual point image.For parallax compensation and the motion compensation of left visual point image and right visual point image, must first reconstructed reference image.
For left visual point image image between motion compensation, can via with reference to reconstruct left viewpoint reference picture image between motion compensation reconstruct left visual point image.For right visual point image image between motion compensation, can via with reference to reconstruct right viewpoint reference picture image between motion compensation reconstruct right visual point image.
For motion compensation when between the image of front left visual point image, preferably: not with reference to comprising left visual point image included in the previous GOP before the current GOP of front left visual point image, and only with reference to the left visual point image will be reconstructed before working as front left visual point image included in current GOP.This operation is also applied to right visual point image.
As mentioned above, cross-layer video encoding device 10 and cross-layer video decoding device 20 can determine whether to perform illuminance compensation according to the feature of image.In addition, cross-layer video encoding device 10 and cross-layer video decoding device 20 can determine whether to perform illuminance compensation to each band or each piece.
Such as, can only determine whether to perform illuminance compensation to skip mode block.Due in non-skip mode block, the residual component produced by being used as the result of inter-layer prediction or inter prediction compensates the predicated error between original block and prediction block, therefore can not perform illuminance compensation to non-skip mode block.On the other hand, because the residual error data of skip mode block is not encoded, therefore can determine whether to perform illuminance compensation to skip mode block.
For predictive coding, the coding unit of 2N × 2N size can be divided into the predicting unit of 2N × 2N size, N × 2N size, 2N × N size or N × N size.Here, the sample value of the divisional type information of the shape of indication predicting unit can be confirmed as 2N × 2N, N × 2N, 2N × N and N × N.Such as, can only determine whether to perform illuminance compensation to the 2N × 2N block (merging block hereinafter referred to as 2N × 2N) under fusion mode.
In addition, can determine whether to perform illuminance compensation to each in luminance block and chrominance block.Such as, when the block (skipping block) under luminance block is to skip pattern or the 2N × 2N block under fusion mode (2N × 2N merges block), can determine whether to perform illuminance compensation.But, only have when chrominance block is to skip pattern, just can determine whether to perform illuminance compensation.
As another example, can be that all luminance block of 2N × 2N and chrominance block determine whether to perform illuminance compensation to partitions sizes.
As another example, even if determine that the block to partitions sizes is 2N × 2N performs illuminance compensation, still differently can determine whether to perform illuminance compensation to luminance component and chromatic component according to the size of current bay.Such as, can determine that the luminance component to partitions sizes is the block of 8 × 8 performs illuminance compensation, and can determine not perform illuminance compensation to the chromatic component that partitions sizes is the block of 8 × 8.
In addition, the size of carrying out the block of illuminance compensation can be changed according to luminance block and chrominance block.Such as, can determine that luminance component partitions sizes being equal to or greater than to the block of 8 × 8 performs illuminance compensation.Can for determining whether that the process chromatic component of same block being performed to illuminance compensation carries out different operations.Be in the block of 8 × 8 in the partitions sizes of luminance component, the partitions sizes of the chromatic component of this block can be 4 × 4.Therefore, the partitions sizes of the luminance component of if block is 8 × 8, then do not perform illuminance compensation to the chromatic component of this block, the partitions sizes of the luminance component of if block is greater than 8 × 8, then can perform illuminance compensation to the chromatic component of this block.
In addition, when current picture is not random access point, illuminance compensation can not be performed to the block in current picture.That is, can only determine whether to perform illuminance compensation to the block of random access point.
As another example, illuminance compensation can not be performed to the chrominance block of the picture not being random access point.Therefore, and not can that the luminance block of picture of random access point determines whether to perform illuminance compensation to the luminance block/chrominance block of the picture as random access point.
As another example, can determine whether to perform illuminance compensation to current block based on coding characteristic (such as comprising the picture type of the picture of block, time rank, network abstract layer (NAL) cell type or the present image during time prediction and the distance between reference picture).The high coding mode of the possibility of carrying out illuminance compensation can be pre-set according to coding characteristic.Can determine whether the block with pre-arranged code pattern to perform illuminance compensation.
As another example, can determine whether to perform illuminance compensation according to the coding mode determined for each piece.Such as, for each band, illumination compensation information can be sent as strip data or slice header, or can be used as strip data or slice header and be received, wherein, whether illumination compensation information instruction performs illuminance compensation when the indicating predetermined coding mode of the block of respective strap.Alternatively, for each picture, illumination compensation information can be sent as PPS or picture related data, or can be used as PPS or picture related data and be received, wherein, whether illumination compensation information instruction performs illuminance compensation when the indicating predetermined coding mode of the block of respective picture.
Such as, the coding mode of block, the prediction direction of block and the type of coding that can determine according to optimizing based on rate distortion (RD) determine whether to perform illuminance compensation to block.
Hereinafter, the method determining whether to perform illuminance compensation according to exemplary embodiment is provided with reference to Fig. 4 a to Fig. 7 b.
Describe with reference to Fig. 4 a, Fig. 4 b and Fig. 5 and determine whether to merge to skip mode or 2N × 2N the exemplary embodiment that block performs illuminance compensation, describe the exemplary embodiment determining whether each in luminance block and chrominance block to be performed to illuminance compensation with reference to Fig. 6 a, Fig. 6 b and Fig. 6 c, and describe the exemplary embodiment optionally performing illuminance compensation and residual prediction with reference to Fig. 7 a and Fig. 7 b.
Fig. 4 a illustrates the flow chart of the illuminance compensation method performed by cross-layer video encoding device 10 according to exemplary embodiment.
In operation 41, if the block type of current block neither merge block by 2N × 2N neither skip block, then whether the illuminance compensation determiner 14 of cross-layer video encoding device 10 uncertainly can perform illuminance compensation, and can end process (40).
But in operation 41, if the block type of current block is to skip block or 2N × 2N merges block, then in operation 42, illuminance compensation determiner 14 can determine whether to perform illuminance compensation to current block.Determine according to described, illumination compensation information " ic_flag " can be set to current block by illuminance compensation determiner 14.The illumination compensation information " ic_flag " being set to current block can be encoded and can be included in the transport stream.
In operation 43, determine whether illumination compensation information " ic_flag " indicates 1, in operation 44, illuminance compensation can be performed to the block being confirmed as carrying out illuminance compensation.
Fig. 4 b illustrates the flow chart of the illuminance compensation method performed by cross-layer video decoding device 20 according to exemplary embodiment.
In operation 46, the illuminance compensation determiner 24 of cross-layer video decoding device 20 can obtain the block type information of current block, and can determine whether current block is to skip block or 2N × 2N merges block.If current block neither merge block by 2N × 2N neither skip block, then whether illuminance compensation determiner 24 uncertainly can perform illuminance compensation and can end process (45).
But, in operation 46, if determine that the block type of current block is to skip block or 2N × 2N merges block, then in operation 47, illuminance compensation determiner 24 can obtain the illumination compensation information " ic_flag " of current block, and determines whether to perform illuminance compensation by reference illumination compensated information " ic_flag ".The illumination compensation information " ic_flag " of current block can be obtained from reception stream, or the illumination compensation information " ic_flag " of current block can be determined according to the type of coding of the coding mode as current block or prediction direction.
In operation 48, determine whether illumination compensation information " ic_flag " indicates 1, in operation 49, illuminance compensation can be performed to the block being confirmed as carrying out illuminance compensation.
Fig. 5 illustrates the grammer for performing illuminance compensation according to the divisional type of current block and predictive mode according to exemplary embodiment.
Grammer coding_unit () 55 for current block can comprise the condition 56 for determining whether to perform current block illuminance compensation.
That is, when illuminance compensation is activated (icEnableFlag=1) for the block in band, and when the type of prediction of block is 2N × 2N fusion block (PartMode==PART_2Nx2N & & merge_flag [x0] [y0]) or skip mode (skip_flag [x0] [y0]), illumination compensation information " ic_flag " can be used, to indicate whether to perform illuminance compensation to current block.
Fig. 6 a to Fig. 6 c illustrates the example determining whether each in luminance block and chrominance block to be performed to illuminance compensation.
Fig. 6 a illustrates the flow chart of the illuminance compensation method performed by cross-layer video encoding device 10 according to another exemplary embodiment.
First, in operation 61, the illuminance compensation determiner 14 of cross-layer video encoding device 10 can determine whether can perform illuminance compensation to the chrominance block in current picture or band, and can arrange the illumination compensation information " chroma_ic_flag " for chrominance block.
As mentioned above, the illumination compensation information by using coding mode (such as comprising the picture type of current block), NAL unit type, distance etc. between original image and temporal reference picture to determine chrominance block.Illumination compensation information " chroma_ic_flag " for chrominance block can be included in the transport stream.
In operation 61, determine whether to perform illuminance compensation at picture or band rank, in operation 62 to 66, can determine whether to perform illuminance compensation in block rank.
If determine whether perform illuminance compensation to the chrominance block in current picture or band at prior operation, then in operation 62, illuminance compensation determiner 14 can determine whether to perform illuminance compensation to each in luminance block and chrominance block.The brightness illumination compensation information " ic_flag " indicating whether to perform illuminance compensation to luminance block can be set, and the colourity illumination compensation information " chroma_ic_flag " indicating whether to perform illuminance compensation to chrominance block can be set.
In operation 63, illuminance compensation determiner 14 can be determined whether brightness illumination compensation information " ic_flag " indicates and perform illuminance compensation to present intensity block.If brightness illumination compensation information " ic_flag " indicates 0 value, then whether illuminance compensation determiner 14 uncertainly can perform illuminance compensation and can end process (60).
But in operation 63, if brightness illumination compensation information " ic_flag " indicates 1, then in operation 64, illuminance compensation determiner 14 can perform illuminance compensation to present intensity block.
In operation 65, if determine that colourity illumination compensation information " chroma_ic_flag " indicates 1, then in operation 66, illuminance compensation determiner 14 can perform illuminance compensation to current chroma block.
Fig. 6 b illustrates the flow chart of the illuminance compensation method performed by cross-layer video decoding device 20 according to another exemplary embodiment.
First, in operation 611, the illuminance compensation determiner 24 of cross-layer video decoding device 20 can obtain the illumination compensation information " chroma_ic_flag " for chrominance block.Illumination compensation information " chroma_ic_flag " for chrominance block can be obtained from transport stream.As mentioned above, the illumination compensation information by using coding mode (such as comprising the picture type of current block), NAL unit type, distance etc. between original image and temporal reference picture to determine chrominance block.
In operation 611, determine whether to perform illuminance compensation at picture or band rank, in operation 621 to 671, can determine whether to perform illuminance compensation in block rank.
If determine whether to perform illuminance compensation to the chrominance block in current picture or band at prior operation, then in operation 621, illuminance compensation determiner 24 can determine whether to perform illuminance compensation to each in luminance block and chrominance block.The brightness illumination compensation information " ic_flag " indicating whether to perform illuminance compensation to luminance block can be obtained, and the colourity illumination compensation information " chroma_ic_flag " indicating whether to perform illuminance compensation to chrominance block can be obtained.
In operation 631, illuminance compensation determiner 24 can be determined whether brightness illumination compensation information " ic_flag " indicates and perform illuminance compensation to present intensity block.If brightness illumination compensation information " ic_flag " indicated value 0, then illuminance compensation determiner 24 can not perform the operation determining whether to perform illuminance compensation, and can end process (601).
But in operation 631, if brightness illumination compensation information " ic_flag " indicates 1, then in operation 641, illuminance compensation determiner 24 can perform illuminance compensation to present intensity block.
In operation 651, if determine that colourity illumination compensation information " chroma_ic_flag " indicates 1, then in operation 661, illuminance compensation determiner 24 can perform illuminance compensation to current chroma block.
Fig. 6 c illustrates the grammer for performing illuminance compensation according to color component according to another exemplary embodiment.
The head slice_header68 of current band can comprise " slice_ic_enable_flag " that indicate illuminance compensation whether to be activated in band.When the band comprising current block is non-separate panels (dependent_slice_flag), the head slice_header68 of band can comprise " slice_ic_enable_flag " that indicate illuminance compensation whether to be activated in current band.
In addition, according to " slice_ic_enable_flag " that whether instruction illuminance compensation is activated in current band, can determine whether to perform illuminance compensation to chrominance block.That is, if illuminance compensation is activated in current band, then colourity illumination compensation information " slice_chroma_ic_flag " can be used, to indicate whether to perform illuminance compensation to the chrominance block in current band.
If the head slice_header68 of current band does not comprise colourity illumination compensation information " slice_chroma_ic_flag ", then colourity illumination compensation information can be regarded as 0.
Although Fig. 6 c provides this exemplary embodiment only determining whether to perform illuminance compensation to chrominance block, in a further exemplary embodiment, optionally determine whether to perform illuminance compensation to luminance block.
Hereinafter, the exemplary embodiment optionally performing illuminance compensation and residual prediction is described in detail with reference to Fig. 7 a and Fig. 7 b.
Fig. 7 a and Fig. 7 b illustrates the grammer for determining whether to perform each piece illuminance compensation and residual prediction.
Residual prediction can be performed, to perform prediction between the residual component produced in ground floor via inter prediction and the residual component produced in the second layer via inter prediction according to the cross-layer video encoding device 10 of exemplary embodiment and cross-layer video decoding device 20.That is, according to residual prediction, inter-layer prediction between the residual error produced in layer via inter prediction, and therefore can perform two predicted operation altogether.Although code efficiency is enhanced due to residual prediction, owing to must perform two predicted operation, therefore calculated load is also very high.
When according to the cross-layer video encoding device 10 of exemplary embodiment and cross-layer video coding/decoding apparatus 20 time of implementation direction prediction (inter prediction) and viewpoint direction prediction (inter-layer prediction), illuminance compensation and residual prediction can be performed to current block simultaneously.
But, in order to reduce calculating, when illuminance compensation and residual prediction can be performed to current block, the operation carrying out in illuminance compensation and residual prediction may not be allowed.
Residual prediction is performed according to the block that cross-layer video encoding device 10 and the cross-layer video decoding device 20 of exemplary embodiment can not be activated illuminance compensation.
Illuminance compensation is performed according to the block that cross-layer video encoding device 10 and the cross-layer video decoding device 20 of other exemplary embodiment can not be activated residual prediction.
Comprise according to the block grammer coding_unit (70) of this exemplary embodiment, for determining whether, the condition 71 of illuminance compensation and the condition 72 for determining whether to perform residual prediction are performed to current block.
Therefore, when determining that illuminance compensation is activated in current band based on " icEnableFlag ", can determine whether to perform illuminance compensation to current block based on " ic_flag " (condition 71).When determining that residual prediction is activated in current band based on " resPredEnableFlag ", and based on "! Ic_flag " when determining that illuminance compensation is not performed, can determine whether to perform residual prediction to current block based on " res_pred_flag " (condition 72).
Comprise according to the block grammer coding_unit (75) of this exemplary embodiment, for determining whether, the condition 76 of residual prediction and the condition 77 for determining whether to perform current block illuminance compensation are performed to current block.
Therefore, when determining that residual prediction is activated in current band based on " resPredEnableFlag ", can determine to perform residual prediction to current block based on " res_pred_flag " (condition 76).In addition, when determining that illuminance compensation is activated in current band based on " icEnableFlag ", and based on "! Res_pred_flag " when determining that residual prediction is not performed, can determine whether to perform illuminance compensation to current block based on " ic_flag " (condition 77).
With reference to Fig. 1 a to Fig. 7 b, describe cross-layer video encoding device 10 and the cross-layer video decoding device 20 for determining whether to perform block illuminance compensation according to the feature of image.If perform illuminance compensation to all pieces, then calculated load can increase, thus only determines whether to perform illuminance compensation to the block meeting predetermined condition, and, do not determine whether to perform illuminance compensation to the block not meeting predetermined condition, and illuminance compensation is not performed to this block.
Therefore, can be determined whether to perform illuminance compensation by the block of encoding for by using pre-arranged code pattern.In addition, can indicate whether that the illumination compensation information performing illuminance compensation comprises in the transport stream by for the block with described coding mode is determined, or described illumination compensation information can be obtained from reception stream.
Therefore, based on according to the cross-layer video encoding device 10 of exemplary embodiment and cross-layer video decoding device 20, only to more needing the block of illuminance compensation to determine whether to perform illuminance compensation, and illuminance compensation is not performed to other block, make the increase that can reduce the calculated load caused due to illuminance compensation, and the code efficiency caused due to illuminance compensation can be improved.
As mentioned above, according to the cross-layer video encoding device 10 of exemplary embodiment and cross-layer video decoding device 20, the block of the video data after dividing is divided into the coding unit of tree structure, and coding unit, predicting unit and converter unit are used to inter-layer prediction or the inter prediction of coding unit.Hereinafter, with reference to Fig. 8 to Figure 20, describe based on the coding unit of tree structure and the method for video coding of converter unit and equipment thereof and video encoding/decoding method and equipment thereof.
Generally speaking, in the coding/decoding process of multi-layer video, perform individually and be used for the coding/decoding process of ground floor image and the coding/decoding process for second layer image.That is, when inter-layer prediction occurs in multi-layer video, reciprocally with reference to the coding/decoding result of single-layer video, but coding/decoding process can be performed for each single-layer video in multiple single-layer video.
Therefore, for convenience of description, the video coding process of the coding unit based on tree structure described with reference to Fig. 8 to Figure 20 and video decoding process are video coding process for single-layer video and video decoding process, and therefore, inter prediction and motion compensation are described in detail.But described by above reference Fig. 1 a to Fig. 7 b, in order to carry out coding/decoding to video flowing, the inter-layer prediction between basic visual point image and second layer image and compensation are performed.
Therefore, multi-layer video is encoded based on the coding unit of tree structure to make the ground floor encoder 12 of cross-layer video encoding device 10, ground floor encoder 12 can comprise the video encoder of Fig. 8 corresponding to the number of plies of multi-layer video to perform Video coding to each single-layer video in multiple single-layer video, and can control video encoder 100 and encode to multiple single-layer video respectively.In addition, cross-layer video encoding device 10 is by using about the coding result of the single view of the separation obtained by video encoder 100 to perform interview prediction.Therefore, the ground floor encoder 12 of cross-layer video encoding device 10 can produce basic viewpoint video stream and the second layer video flowing of the coding result comprising each layer.
Similarly, multi-layer video is decoded based on the coding unit of tree structure to make the second layer decoder 26 of cross-layer video decoding device 20, second layer decoder 26 can comprise the video decoding apparatus 200 of Fig. 9 corresponding to the number of plies of multi-layer video to perform video decode to each layer in the ground floor video flowing received and the layer of second layer video flowing received, and can control video decoding apparatus 200 and decode to multiple single-layer video respectively.Then, cross-layer video decoding device 20 comes to compensate between execution level by using the decoded result about the single layer of the separation obtained by video decoding apparatus 200.Therefore, the second layer decoder 26 of cross-layer video decoding device 20 can produce the ground floor image and second layer image that reconstruct for each layer.
Fig. 8 is the block diagram of the video encoder 100 of the coding unit based on tree structure according to exemplary embodiment.
The video encoder 100 relating to the video estimation of the coding unit based on tree structure comprises coding unit determiner 120 and output unit 130.Below, for ease of describing, the video encoder 100 relating to the video estimation of the coding unit based on tree structure is called as " video encoder 100 ".
Coding unit determiner 120 can divide current picture based on the maximum coding unit of the current picture of image, and wherein, maximum coding unit has maximum sized coding unit.If current picture is greater than maximum coding unit, then the view data of current picture can be divided at least one maximum coding unit.According to the maximum coding unit of exemplary embodiment can be of a size of 32 × 32,64 × 64,128 × 128,256 × 256 etc. data cell, wherein, the square of the shape of data cell to be width and length be some powers of 2.
Coding unit according to embodiment can be characterized by full-size and the degree of depth.Depth representing coding unit by the number of times of spatial division, and along with depth down, can be divided into minimum code unit from maximum coding unit according to the degree of depth compared with deep layer coding unit from maximum coding unit.The degree of depth of maximum coding unit is most high depth, and the degree of depth of minimum code unit is lowest depth.Due to the depth down along with maximum coding unit, the size of the coding unit corresponding to each degree of depth reduces, and therefore corresponding at greater depths coding unit can comprise multiple coding unit corresponding with more low depth.
As mentioned above, the view data of current picture is divided into maximum coding unit according to the full-size of coding unit, and each maximum coding unit can comprise the comparatively deep layer coding unit be divided according to the degree of depth.Owing to dividing the maximum coding unit according to exemplary embodiment according to the degree of depth, therefore can classify to the view data of the spatial domain be included in maximum coding unit according to Depth Stratification.
Can pre-determine depth capacity and the full-size of coding unit, wherein, described depth capacity and full-size limit the total degree that the height of maximum coding unit and width are layered division.
Coding unit determiner 120 is encoded at least one zoning obtained by dividing according to the region of the degree of depth to maximum coding unit, and determines the degree of depth of the view data exporting final coding according at least one zoning described.In other words, coding unit determiner 120 is encoded to view data with the comparatively deep layer coding unit according to the degree of depth by the maximum coding unit according to current picture, and selects the degree of depth with minimum code error, determines coding depth.Output unit 130 is outputted to by the coding depth determined with according to the view data of the coding of the coding depth determined.
Based on the comparatively deep layer coding unit corresponding at least one degree of depth being equal to or less than depth capacity, the view data in maximum coding unit is encoded, and based on each compared with deep layer coding unit relatively to the result that view data is encoded.After the encoding error compared with deep layer coding unit is compared, the degree of depth with minimum code error can be selected.At least one coding depth can be selected for each maximum coding unit.
Dividing with being layered according to the degree of depth along with coding unit and increase along with the quantity of coding unit, the size of maximum coding unit is divided.In addition, even if coding unit is corresponding to the same degree of depth in a maximum coding unit, the encoding error still by measuring the view data of each coding unit respectively determines whether each coding unit corresponding to the same degree of depth to be divided into more low depth.Therefore, even if view data is included in a maximum coding unit, encoding error can be different according to the region in a described maximum coding unit, and therefore coding depth can be different according to the region in view data.Therefore, one or more coding depth can be determined in a maximum coding unit, and can divide according to the view data of the coding unit of at least one coding depth to maximum coding unit.
Therefore, coding unit determiner 120 can determine the coding unit with tree structure that is included in maximum coding unit.According to exemplary embodiment " there is the coding unit of tree structure " comprise that maximum coding unit comprises all compared with the coding unit corresponding to the degree of depth being defined as coding depth in deep layer coding unit.Hierarchically can determine the coding unit of coding depth according to the degree of depth in the same area of maximum coding unit, and the coding unit of coding depth can be determined in the different areas independently.Similarly, the coding depth current region can be determined independently from the coding depth in another region.
According to the depth capacity of exemplary embodiment be to from maximum coding unit to the relevant index of the division number of times of minimum code unit.The total division number of times from maximum coding unit to minimum code unit can be represented according to the first depth capacity of exemplary embodiment.The sum from maximum coding unit to the depth levels of minimum code unit can be represented according to the second depth capacity of exemplary embodiment.Such as, when the degree of depth of maximum coding unit is 0, can be set to 1 to the degree of depth of the coding unit that maximum coding unit divides once, the degree of depth maximum coding unit being divided to the coding unit of twice can be set to 2.Here, if minimum code unit is the coding unit that maximum coding unit is divided after four times, then there are 5 depth levels of the degree of depth 0,1,2,3 and 4, and therefore the first depth capacity can be set to 4, the second depth capacity can be set to 5.
Predictive coding and conversion can be performed according to maximum coding unit.Also according to maximum coding unit, perform predictive coding and conversion based on according to the comparatively deep layer coding unit of the degree of depth being equal to or less than depth capacity.
Due to whenever dividing maximum coding unit according to the degree of depth, increase compared with the quantity of deep layer coding unit, therefore comprise the codings of predictive coding and conversion by all to perform compared with deep layer coding unit produced along with depth down.For the ease of explaining, in maximum coding unit, now predictive coding and conversion are described the coding unit based on current depth.
Video encoder 100 differently can select size or the shape of the data cell for encoding to view data.In order to encode to view data, perform the operation of such as predictive coding, conversion and entropy code, now, identical data cell can be used for all operations, or different data cells can be used for each operation.
Such as, video encoder 100 not only can select the coding unit for encoding to view data, also can select the data cell being different from coding unit, to perform predictive coding to the view data in coding unit.
In order to perform predictive coding in maximum coding unit, predictive coding can be performed based on the coding unit (that is, based on the coding unit being no longer divided into the coding unit corresponding with more low depth) corresponding to coding depth.Below, to be no longer divided and the coding unit become for the elementary cell of predictive coding will be called as " predicting unit " now.The data cell that the subregion obtained by dividing predicting unit can comprise predicting unit or obtain by dividing at least one selection from the height and width of predicting unit.Subregion is the data cell that the predicting unit of coding unit is divided, and predicting unit can be the subregion with the size identical with coding unit.
Such as, when the coding unit of 2N × 2N (wherein, N is positive integer) is no longer divided, when the coding unit of 2N × 2N becomes the predicting unit being of a size of 2N × 2N, and the size of subregion can be 2N × 2N, 2N × N, N × 2N or N × N.The example of divisional type comprises symmetrical subregion by dividing symmetrically height or the width of predicting unit and obtain, by asymmetricly dividing (such as, 1:n or n:1) and the subregion obtained, the subregion obtained by geometrically dividing predicting unit to the height of predicting unit or width and having the subregion of arbitrary shape.
The predictive mode of predicting unit can be at least one that select from frame mode, inter-frame mode and skip mode.Such as, frame mode or inter-frame mode can be performed to the subregion of 2N × 2N, 2N × N, N × 2N or N × N.In addition, only skip mode can be performed to the subregion of 2N × 2N.Coding can be performed independently to the predicting unit of in coding unit, thus select the predictive mode with minimum code error.
Video encoder 100 not only also can, based on the data cell different from coding unit, can come to perform conversion to the view data in coding unit based on the coding unit for encoding to view data.In order to perform conversion in coding unit, conversion can be performed based on the data cell with the size being less than or equal to coding unit.Such as, the data cell for converting can comprise the data cell of frame mode and the data cell of inter-frame mode.
With with the mode similar according to the coding unit of tree structure, the converter unit in coding unit can be recursively divided into the region of smaller szie.Therefore, based on the converter unit with tree structure according to transformed depth, the residual error data in coding unit can be divided.
Also can arrange transformed depth in converter unit, wherein, transformed depth instruction is by dividing the height of coding unit and width and reach the division number of times of converter unit.Such as, in the current coded unit of 2N × 2N, when the size of converter unit is 2N × 2N, transformed depth can be 0, and when the size of converter unit is N × N, transformed depth can be 1, when the size of converter unit is N/2 × N/2, transformed depth can be 2.In other words, also the converter unit with tree structure can be set according to transformed depth.
Not only require the information about coding depth according to the coded message of the coding unit corresponding to coding depth, also require about to predictive coding and the information converting relevant information.Therefore, coding unit determiner 120 not only determines the coding depth with minimum code error, also determines the divisional type in predicting unit, according to the predictive mode of predicting unit and the size of converter unit for converting.
Describe in detail according to the coding unit according to tree structure in the maximum coding unit of exemplary embodiment and the method determining predicting unit/subregion and converter unit with reference to Figure 10 to Figure 20 subsequently.
Coding unit determiner 120 measures the encoding error compared with deep layer coding unit according to the degree of depth by using based on the rate-distortion optimization of Lagrange's multiplier.
Output unit 130 exports the view data of maximum coding unit and the information about the coding mode according to coding depth in the bitstream, wherein, the view data of described maximum coding unit is encoded based at least one coding depth determined by coding unit determiner 120.
By carrying out coding to obtain coded image data to the residual error data of image.
Information about the coding mode according to coding depth can comprise the information about the information of coding depth, the information about the divisional type in predicting unit, the information about predictive mode and the size about converter unit.
Defining information about coding depth by using according to the division information of the degree of depth, wherein, indicating whether to perform coding to the coding unit of more low depth instead of current depth according to the division information of the degree of depth.If the current depth of current coded unit is coding depth, then the view data in current coded unit is encoded and export, therefore division information can be defined as and current coded unit be divided into more low depth.Alternatively, if the current depth of current coded unit is not coding depth, then coding is performed to the coding unit of more low depth, and therefore division information can be defined as the coding unit obtaining more low depth is divided to current coded unit.
If current depth is not coding depth, then coding is performed to the coding unit of the coding unit being divided into more low depth.Because at least one coding unit of more low depth is present in a coding unit of current depth, therefore coding is repeated to each coding unit of more low depth, and therefore recursively can perform coding to the coding unit with same depth.
Owing to determining the coding unit with tree structure for a maximum coding unit, and the coding unit for coding depth determines the information about at least one coding mode, so the information can determining about at least one coding mode for a maximum coding unit.In addition, owing to carrying out layering division according to the degree of depth to view data, therefore the coding depth of the view data of maximum coding unit can be different according to position, therefore can arrange information about coding depth and coding mode for view data.
Therefore, the coded message about corresponding coding depth and coding mode can be distributed at least one in coding unit, predicting unit and the minimum unit be included in maximum coding unit by output unit 130.
By being 4 parts of square data cells obtained by forming the minimum code dividing elements of lowest depth according to the minimum unit of exemplary embodiment.Selectively, can be to be included in maximum coding unit the largest square data cell in included all coding units, predicting unit, zoning unit and converter unit according to the minimum unit of embodiment.
Such as, the coded message exported by output unit 130 can be classified as according to the coded message compared with deep layer coding unit and the coded message according to predicting unit.The information about predictive mode and the information about partitions sizes can be comprised according to the coded message compared with deep layer coding unit.The information of the information in the estimation direction about inter-frame mode, the information about the reference picture index of inter-frame mode, the information about motion vector, the information about the chromatic component of frame mode and the interpolation method about frame mode can be comprised according to the coded message of predicting unit.
The head of bit stream, sequence parameter set or frame parameter can be inserted into according to the maximum sized information about coding unit of picture, band or GOP definition and the information about depth capacity to concentrate.
Also export the information about the maximum sized information of the converter unit allowed for current video and the minimum dimension about converter unit by the head of bit stream, sequence parameter set or parameter sets.Output unit 130 can be encoded to reference information, information of forecasting and the type of strip information relevant to prediction and export.
According to the simplest embodiment for video encoder 100, comparatively deep layer coding unit can be divided into two parts and the coding unit that obtains by the height of the coding unit by greater depths (higher level) or width.In other words, when the size of the coding unit of current depth is 2N × 2N, more the size of the coding unit of low depth is N × N.In addition, the coding unit with current depth being of a size of 2N × 2N can comprise maximum 4 have described in the coding unit of more low depth.
Therefore, for each maximum coding unit, the size of the maximum coding unit that video encoder 100 can be determined based on the feature of consideration current picture and depth capacity, by determining that the coding unit with optimum shape and optimal size forms the coding unit with tree structure.In addition, due to by using various predictive mode to perform coding with any one in conversion to each maximum coding unit, therefore can consider that the feature of the coding unit of various picture size is to determine optimum code pattern.
Therefore, if encoded to the image with high resolution or big data quantity with conventional macro block, then the quantity of the macro block of each picture extremely increases.Therefore, for the number increase of the compressed information that each macro block produces, be therefore difficult to the information sending compression, and efficiency of data compression reduces.But, by use video encoder 100, owing to increasing the full-size of coding unit while the size considering image, and while the feature considering image, adjust coding unit simultaneously, therefore can improve picture compression efficiency.
The cross-layer video encoding device 10 described with reference to Fig. 1 a above can comprise the video encoder 100 corresponding to the number of plies, to encode to the single layer image in each layer of multi-layer video.Such as, ground floor encoder 12 can comprise a video encoder 100, and second layer encoder 16 can comprise the video encoder 100 corresponding to the quantity of the second layer.
When video encoder 100 pairs of ground floor images are encoded, coding unit determiner 120 can determine the predicting unit of each coding unit in the coding unit of tree structure being carried out to inter picture prediction according to each maximum coding unit, and can predict between each predicting unit carries out image.
When video encoder 100 pairs of second layer images are encoded, coding unit determiner 120 according to the predicting unit of each maximum coding unit determination tree structure and coding unit, and can perform inter prediction to each predicting unit.
Video encoder 100 can be encoded to illumination difference, to compensate the illumination difference between ground floor image and second layer image.But, can determine whether to perform illuminance compensation according to the coding mode of coding unit.Such as only can perform illuminance compensation to the predicting unit being of a size of 2N × 2N.
Fig. 9 is the block diagram of the video decoding apparatus 200 of the coding unit based on tree structure according to exemplary embodiment.
The video decoding apparatus 200 relating to the video estimation of the coding unit based on tree structure comprises receiver 210, view data and coded message extractor 220 and image data decoding device 230.Hereinafter, for convenience of description, the video decoding apparatus 200 relating to the video estimation of the coding unit based on tree structure is called as " video decoding apparatus 200 ".
Definition for the various terms (such as coding unit, the degree of depth, predicting unit, converter unit and the information about various coding mode) of the decode operation of video decoding apparatus 200 is identical with the definition that video encoder 100 describes with reference Fig. 8.
Receiver 210 receives and resolves the bit stream of encoded video.View data and coded message extractor 220 bit stream analytically, coded image data is extracted for each coding unit, and the view data of extraction is outputted to image data decoding device 230, wherein, coding unit has the tree structure according to each maximum coding unit.View data and coded message extractor 220 can from the maximum sized information extracted about the head of current picture, sequence parameter set or parameter sets about the coding unit of current picture.
In addition, view data and coded message extractor 220 bit stream analytically, according to each maximum coding unit, extract the information of coding depth about the coding unit with tree structure and coding mode.The information about coding depth and coding mode extracted is output to image data decoding device 230.In other words, the view data in bit stream is divided into maximum coding unit, and image data decoding device 230 is decoded to view data for each maximum coding unit.
Can arrange about according to the coding depth of maximum coding unit and the information of coding mode for the information about at least one coding unit corresponding to coding depth, the information about coding mode can comprise the information of the information of the divisional type about the corresponding encoded unit corresponding to coding depth, the information about predictive mode and the size about converter unit.In addition, the information about coding depth can be extracted as according to the division information of the degree of depth.
By view data and coded message extractor 220 extract about the information according to each coding depth of maximum coding unit and the information of coding mode being coding depth about such and coding mode: this coding depth and coding mode are determined to be in encoder (such as, video encoder 100) and produce minimum code error according to each maximum coding unit to when repeatedly performing coding according to each comparatively deep layer coding unit of the degree of depth.Therefore, video decoding apparatus 200 carrys out reconstructed image by carrying out decoding according to the coding depth and coding mode that produce minimum code error to view data.
Owing to can be assigned to the predetermined unit of data in corresponding coding unit, predicting unit and minimum unit about the coded message of coding depth and coding mode, therefore view data and coded message extractor 220 according to predetermined unit of data, can extract the information about coding depth and coding mode.If be recorded according to predetermined unit of data about the corresponding coding depth of maximum coding unit and the information of coding mode, then the predetermined unit of data being assigned with the identical information about coding depth and coding mode can be inferred as is be included in the data cell in same maximum coding unit.
Image data decoding device 230, based on about according to the coding depth of maximum coding unit and the information of coding mode, by decoding to the view data in each maximum coding unit, reconstructs current picture.In other words, image data decoding device 230 based on the information of the divisional type about each coding unit among the coding unit with tree structure be included in each maximum coding unit extracted, predictive mode and converter unit, can be decoded to the view data of coding.Decoding process can comprise prediction (comprising infra-frame prediction and motion compensation) and inverse transformation.
Image data decoding device 230 based on the information of the divisional type of the predicting unit about the coding unit according to coding depth and predictive mode, according to subregion and the predictive mode of each coding unit, can perform infra-frame prediction or motion compensation.
In addition, for the inverse transformation of each maximum coding unit, image data decoding device 230 can read information about the converter unit according to tree structure, to perform inverse transformation based on the converter unit of each coding unit for each coding unit.Via inverse transformation, the pixel value of the spatial domain of restructural coding unit.
The coding depth of current maximum coding unit determined by image data decoding device 230 according to the division information of the degree of depth by using.If division information indicating image data are no longer divided in current depth, then current depth is coding depth.Therefore, image data decoding device 230, by using the information of the size about the divisional type of the predicting unit for each coding unit corresponding to coding depth, predictive mode and converter unit, is decoded to the coded data in current maximum coding unit.
In other words, collect by observing the coded message collection being assigned to predetermined unit of data in coding unit, predicting unit and minimum unit the data cell comprising the coded message comprising identical division information, and the data cell of collecting can be considered to the data cell of will be carried out with same-code pattern decoding by image data decoding device 230.Therefore, current coded unit is decoded about the information of the coding mode of each coding unit by obtaining.
The cross-layer video decoding device 20 described with reference to Fig. 2 a above can comprise the video decoding apparatus 200 corresponding to viewpoint number, to decode to the ground floor image stream received and the second layer image stream received and to reconstruct ground floor image and second layer image.
When ground floor image stream is received, the sampling point of the ground floor image extracted from ground floor image stream by extractor 220 can be divided into the coding unit of the tree structure according to maximum coding unit by the image data decoding device 230 of video decoding apparatus 200.Image data decoding device 230 can perform motion compensation based on the predicting unit for inter picture prediction to each coding unit in the coding unit of the tree structure of the sampling point according to ground floor image, and restructural ground floor image.
When second layer image stream is received, the sampling point of the second layer image extracted from second layer image stream by extractor 220 can be divided into the coding unit of the tree structure according to maximum coding unit by the image data decoding device 230 of video decoding apparatus 200.Image data decoding device 230 can perform motion compensation based on the predicting unit for inter picture prediction to each coding unit in the coding unit of the sampling point of second layer image, and restructural second layer image.
Extractor 220 can obtain the information with illumination difference correlation from bit stream, to compensate the illumination difference between ground floor image and second layer image.But, can determine whether to perform illuminance compensation according to the coding mode of coding unit.Such as, only illuminance compensation can be performed to the predicting unit being of a size of 2N × 2N.
Therefore, video decoding apparatus 200 can obtain the information about at least one coding unit producing minimum code error when recursively performing coding to each maximum coding unit, and described information can be used to decode to current picture.In other words, can the coding unit with tree structure being confirmed as optimum code unit in each maximum coding unit be decoded.
Therefore, even if image has high-resolution or has big data quantity, also by using size and the coding mode of coding unit, effectively view data decoded and rebuild, wherein, the size of described coding unit and coding mode be by use from encoder accepts to the information about optimum code pattern, determined adaptively according to the feature of image.
Figure 10 is the diagram of the design for describing the coding unit according to exemplary embodiment.
The size of coding unit can be represented as width × highly, and can be 64 × 64,32 × 32,16 × 16 and 8 × 8.The coding unit of 64 × 64 can be divided into the subregion of 64 × 64,64 × 32,32 × 64 or 32 × 32, the coding unit of 32 × 32 can be divided into the subregion of 32 × 32,32 × 16,16 × 32 or 16 × 16, the coding unit of 16 × 16 can be divided into the subregion of 16 × 16,16 × 8,8 × 16 or 8 × 8, and the coding unit of 8 × 8 can be divided into the subregion of 8 × 8,8 × 4,4 × 8 or 4 × 4.
In video data 310, resolution is 1920 × 1080, and the full-size of coding unit is 64, and depth capacity is 2.In video data 320, resolution is 1920 × 1080, and the full-size of coding unit is 64, and depth capacity is 3.In video data 330, resolution is 352 × 288, and the full-size of coding unit is 16, and depth capacity is 1.Depth capacity shown in Figure 10 represents the division total degree from maximum coding unit to minimum decoding unit.
If resolution is high or data volume large, then the full-size of coding unit may be comparatively large, thus not only improve code efficiency, and reflect the feature of image exactly.Therefore, the full-size with the coding unit of video data 310 and 320 more high-resolution than video data 330 can be 64.
Depth capacity due to video data 310 is 2, therefore owing to passing through to divide twice maximum coding unit, depth down to two-layer, therefore the coding unit 315 of video data 310 can comprise major axis dimension be 64 maximum coding unit and major axis dimension be the coding unit of 32 and 16.On the other hand, depth capacity due to video data 330 is 1, therefore due to by dividing once maximum coding unit, depth down to one deck, therefore the coding unit 335 of video data 330 can comprise major axis dimension be 16 maximum coding unit and major axis dimension be the coding unit of 8.
Depth capacity due to video data 320 is 3, therefore due to by dividing three times to maximum coding unit, depth down to 3 layer, therefore the coding unit 325 of video data 320 can comprise major axis dimension be 64 maximum coding unit and major axis dimension be 32,16 and 8 coding unit.Along with depth down, details can accurately be represented.
Figure 11 is the block diagram of the image encoder 400 based on coding unit according to exemplary embodiment.
The operation that image encoder 400 performs the coding unit determiner 120 of video encoder 100 is encoded to view data.In other words, coding unit under frame mode in intra predictor generator 410 pairs of present frames 405 performs infra-frame prediction, exercise estimator 420 and motion compensator 425, by using present frame 405 and reference frame 495, perform interframe respectively to the coding unit under the inter-frame mode in present frame 405 and estimate and motion compensation.
The data exported from intra predictor generator 410, exercise estimator 420 and motion compensator 425 are outputted as the conversion coefficient after quantification by converter 430 and quantizer 440.Conversion coefficient after quantification is reconfigured as the data in spatial domain by inverse DCT 460 and inverse converter 470, the data in the spatial domain of reconstruct being outputted as reference frame 495 after going module unit 480 and offset compensating unit 490 reprocessing.Conversion coefficient after quantification is outputted as bit stream 455 by entropy coder 450.
In order to application image encoder 400 in video encoder 100, all elements of image encoder 400 (namely, intra predictor generator 410, exercise estimator 420, motion compensator 425, converter 430, quantizer 440, entropy coder 450, inverse DCT 460, inverse converter 470, go module unit 480 and offset compensating unit 490) while the depth capacity considering each maximum coding unit, based on each coding unit executable operations had in the coding unit of tree structure.
Particularly, intra predictor generator 410, exercise estimator 420 and motion compensator 425 determine subregion and the predictive mode of each coding unit had in the coding unit of tree structure while the full-size considering current maximum coding unit and depth capacity, and the size of the converter unit had in each coding unit in the coding unit of tree structure determined by converter 430.
Figure 12 is the block diagram of the image decoder 500 based on coding unit according to exemplary embodiment.
Resolver 510 is resolved the information about coding needed for decoded coded image data and decoding from bit stream 505.Coded image data is outputted as the data of inverse quantization by entropy decoder 520 and inverse DCT 530, and the data of inverse quantization are resorted to the view data in spatial domain by inverse converter 540.
For the view data in spatial domain, the coding unit under intra predictor generator 550 pairs of frame modes performs infra-frame prediction, and motion compensator 560 performs motion compensation by using the coding unit under reference frame 585 pairs of inter-frame modes.
Can being outputted as reconstructed frame 595 after going module unit 570 and offset compensating unit 580 reprocessing by the view data in the spatial domain of intra predictor generator 550 and motion compensator 560.In addition, by going the view data of module unit 570 and offset compensating unit 580 reprocessing can be outputted as reference frame 585.
In order to decode to view data in the image data decoding device 230 of video decoding apparatus 200, image decoder 500 can perform the operation performed after resolver 510.
In order to application image decoder 500 in video decoding apparatus 200, all elements of image decoder 500 (that is, resolver 510, entropy decoder 520, inverse DCT 530, inverse converter 540, intra predictor generator 550, motion compensator 560, go module unit 570 and offset compensating unit 580) for each maximum coding unit based on the coding unit executable operations with tree structure.
Particularly, intra predictor generator 550 and motion compensator 560 must for each coding unit determination subregion and the predictive modes with tree structure, and inverse converter 540 must for the size of each coding unit determination converter unit.
The encoding operation of Figure 11 and the decode operation of Figure 12 are described to video flowing encoding operation in individual layer and decoding video stream operation respectively.Therefore, if the ground floor encoder 12 of Fig. 1 a is encoded to the video flowing had to least two layers, then ground floor encoder 12 can comprise the image encoder 400 for each layer.Similarly, if the second layer decoder 26 of Fig. 2 a is decoded to the video flowing had to least two layers, then second layer decoder 26 can comprise the image decoder 500 for each layer.
Figure 13 illustrates the diagram compared with deep layer coding unit and subregion according to the degree of depth according to exemplary embodiment.
Video encoder 100 and video decoding apparatus 200 use hierarchical coding unit to consider the feature of image.The maximum height of coding unit, Breadth Maximum and depth capacity can be determined adaptively according to the feature of image, or the maximum height of coding unit, Breadth Maximum and depth capacity can be differently set by user.The size compared with deep layer coding unit can determining according to the degree of depth according to the predetermined full-size of coding unit.
According to exemplary embodiment, in the hierarchy 600 of coding unit, the maximum height of coding unit and Breadth Maximum are all 64, and depth capacity is 3.In the case, depth capacity refers to the total degree that coding unit is divided from maximum coding unit to minimum code unit.Because the vertical axis of the degree of depth along hierarchy 600 is deepened, be therefore all divided compared with the height of deep layer coding unit and width.In addition, predicting unit and subregion are illustrated along the trunnion axis of hierarchy 600, and wherein, described predicting unit and subregion are to each basis of carrying out predictive coding compared with deep layer coding unit.
In other words, in hierarchy 600, coding unit 610 is maximum coding units, and wherein, the degree of depth is 0, and size (that is, highly taking advantage of width) is 64 × 64.The degree of depth is deepened along vertical axis, exist be of a size of 32 × 32 and the degree of depth be 1 coding unit 620, be of a size of 16 × 16 and the degree of depth be 2 coding unit 630, be of a size of 8 × 8 and the degree of depth be 3 coding unit 640.Be of a size of 8 × 8 and the degree of depth be 3 coding unit 640 are minimum code unit.
The predicting unit of coding unit and subregion are arranged along trunnion axis according to each degree of depth.In other words, if be of a size of 64 × 64 and the degree of depth be 0 coding unit 610 are predicting unit, then predicting unit can be divided into the subregion be included in coding unit 610, that is, the subregion 610 being of a size of 64 × 64, the subregion 612 being of a size of 64 × 32, be of a size of the subregion 614 of 32 × 64 or be of a size of the subregion 616 of 32 × 32.
Similarly, can by be of a size of 32 × 32 and the degree of depth be that the predicting unit of the coding unit 620 of 1 is divided into the subregion be included in coding unit 620, that is, the subregion 620 being of a size of 32 × 32, the subregion 622 being of a size of 32 × 16, be of a size of the subregion 624 of 16 × 32 and be of a size of the subregion 626 of 16 × 16.
Similarly, can by be of a size of 16 × 16 and the degree of depth be that the predicting unit of the coding unit 630 of 2 is divided into the subregion be included in coding unit 630, that is, be included in coding degree unit 630 subregion being of a size of 16 × 16, the subregion 632 being of a size of 16 × 8, be of a size of the subregion 634 of 8 × 16 and be of a size of the subregion 636 of 8 × 8.
Similarly, can by be of a size of 8 × 8 and the degree of depth be that the predicting unit of the coding unit 640 of 3 is divided into the subregion be included in coding unit 640, that is, be included in coding unit 640 subregion being of a size of 8 × 8, the subregion 642 being of a size of 8 × 4, be of a size of the subregion 644 of 4 × 8 and be of a size of the subregion 646 of 4 × 4.
In order to determine at least one coding depth of the coding unit forming maximum coding unit 610, the coding unit determiner 120 of video encoder 100 performs coding to the coding unit corresponding to each degree of depth be included in maximum coding unit 610.
Along with depth down, the quantity compared with deep layer coding unit according to the degree of depth comprising the data with same range and same size increases.Such as, the coding unit that needs four are corresponding to the degree of depth 2 covers the data be included in a coding unit corresponding with the degree of depth 1.Therefore, in order to according to depth ratio comparatively to the result that identical data is encoded, the coding unit corresponding to the degree of depth 1 coding unit corresponding with the degree of depth 2 with four is all encoded.
In order to perform coding for the current depth among multiple degree of depth, can along the trunnion axis of hierarchy 600, by performing coding to each predicting unit in the coding unit corresponding to current depth, coming for current depth, selecting the minimum code error of representatively property encoding error.Alternatively, along with the vertical axis of the degree of depth along hierarchy 600 is deepened, by performing the representative encoding error of encoding and comparing according to the degree of depth for each degree of depth, to search for minimum code error.The degree of depth with minimum code error in coding unit 610 and subregion can be chosen as coding depth and the divisional type of coding unit 610.
Figure 14 is the diagram for describing according to the relation between the coding unit 710 of exemplary embodiment and converter unit 720.
Video encoder 100 or video decoding apparatus 200, for each maximum coding unit, according to the coding unit with the size being less than or equal to maximum coding unit, are encoded to image or decode.Based on the data cell being not more than corresponding encoded unit, the size for carrying out the converter unit converted during encoding can be selected.
Such as, in video encoder 100 or video decoding apparatus 200, if the size of coding unit 710 is 64 × 64, then by using the converter unit 720 being of a size of 32 × 32 to perform conversion.
In addition, by performing conversion to being less than 64 × 64 each converter units being of a size of 32 × 32,16 × 16,8 × 8 and 4 × 4, the data of the coding unit 710 being of a size of 64 × 64 are encoded, then can select the converter unit with minimum code error.
Figure 15 illustrates many coded messages according to the degree of depth according to exemplary embodiment.
The output unit 130 of video encoder 100 can be encoded to the information 800 about divisional type of each coding unit corresponding to coding depth, the information 810 about predictive mode and the information 820 about the size of converter unit, and information 800, information 810 and information 820 is sent as the information about coding mode.
Information 800 indicates the information of the shape of the subregion obtained about the predicting unit by dividing current coded unit, and wherein, described subregion is the data cell for carrying out predictive coding to current coded unit.Such as, the current coded unit CU_0 being of a size of 2N × 2N can be divided into any one in following subregion: the subregion 802 being of a size of 2N × 2N, the subregion 804 being of a size of 2N × N, be of a size of the subregion 806 of N × 2N and be of a size of the subregion 808 of N × N.Here, be provided to instruction about the information 800 of divisional type and be of a size of the subregion 802 of 2N × 2N, the subregion 804 being of a size of 2N × N, the subregion 806 being of a size of N × 2N and of being of a size of in the subregion 808 of N × N.
Information 810 indicates the predictive mode of each subregion.Such as, information 810 can indicate the pattern to the predictive coding that the subregion indicated by information 800 performs, that is, frame mode 812, inter-frame mode 814 or skip mode 816.
Information 820 indicate when to current coded unit perform convert time based on converter unit.Such as, converter unit can be the first frame inner conversion unit 822, second frame inner conversion unit 824, first Inter-frame Transformation unit 826 or the second Inter-frame Transformation unit 828.
The view data of video decoding apparatus 200 and coded message extractor 220 according to each comparatively deep layer coding unit, can extract and use the information 800,810 and 820 for decoding.
Figure 16 is the diagram compared with deep layer coding unit according to the degree of depth according to exemplary embodiment.
Division information can be used to the change of indicated depth.Whether the coding unit of division information instruction current depth is divided into the coding unit of more low depth.
For to the degree of depth be 0 and the coding unit 900 that the is of a size of 2N_0 × 2N_0 predicting unit 910 of carrying out predictive coding can comprise the subregion of following divisional type: the divisional type 912 being of a size of 2N_0 × 2N_0, the divisional type 914 being of a size of 2N_0 × N_0, be of a size of the divisional type 916 of N_0 × 2N_0 and be of a size of the divisional type 918 of N_0 × N_0.Diagram illustrate only the divisional type 912 to 918 obtained by dividing predicting unit 910 symmetrically, but divisional type is not limited thereto, and the subregion of predicting unit 910 can comprise asymmetric subregion, has the subregion of reservation shape and have the subregion of geometry.
According to often kind of divisional type, to being of a size of a subregion of 2N_0 × 2N_0, two subregions being of a size of 2N_0 × N_0, two subregions being of a size of N_0 × 2N_0 and four subregions being of a size of N_0 × N_0 repeatedly perform predictive coding.The predictive coding under frame mode and inter-frame mode can be performed to the subregion being of a size of 2N_0 × 2N_0, N_0 × 2N_0,2N_0 × N_0 and N_0 × N_0.Only the subregion being of a size of 2N_0 × 2N_0 is performed to the predictive coding under skip mode.
If encoding error is minimum in a divisional type in the divisional type 912 to 916 being of a size of 2N_0 × 2N_0,2N_0 × N_0 and N_0 × 2N_0, then predicting unit 910 can not be divided into more low depth.
If encoding error is minimum in the divisional type 918 being of a size of N_0 × N_0, then the degree of depth changes to 1 to divide divisional type 918 operation 920 from 0, and to the degree of depth be 2 and the coding unit 930 that is of a size of N_0 × N_0 repeatedly perform coding to search for minimum code error.
For to the degree of depth be 1 and the coding unit 930 that the is of a size of 2N_1 × 2N_1 (=N_0 × N_0) predicting unit 940 of carrying out predictive coding can comprise the subregion of following divisional type: the divisional type 942 being of a size of 2N_1 × 2N_1, the divisional type 944 being of a size of 2N_1 × N_1, be of a size of the divisional type 946 of N_1 × 2N_1 and be of a size of the divisional type 948 of N_1 × N_1.
If encoding error is minimum in the divisional type 948 being of a size of N_1 × N_1, then the degree of depth changes to 2 to divide divisional type 948 operation 950 from 1, and to the degree of depth be 2 and the coding unit 960 that is of a size of N_2 × N_2 repeat coding to search for minimum code error.
When depth capacity is d, the division operation according to each degree of depth can be performed until the degree of depth becomes d-1, and division information can be encoded until the degree of depth 0 arrives one of d-2.In other words, be performed when encoding until when to be divided the degree of depth in operation 970 be d-1 to the coding unit corresponding to the degree of depth of d-2, for to the degree of depth being the subregion that predicting unit 990 that d-1 and the coding unit 980 that is of a size of 2N_ (d-1) × 2N_ (d-1) carry out predictive coding can comprise following divisional type: the divisional type 992 being of a size of 2N_ (d-1) × 2N_ (d-1), be of a size of the divisional type 994 of 2N_ (d-1) × N_ (d-1), be of a size of the divisional type 996 of N_ (d-1) × 2N_ (d-1) and be of a size of the divisional type 998 of N_ (d-1) × N_ (d-1).
Can to being of a size of a subregion of 2N_ (d-1) × 2N_ (d-1) in divisional type 992 to 998, two subregions being of a size of 2N_ (d-1) × N_ (d-1), two subregions being of a size of N_ (d-1) × 2N_ (d-1), four subregions being of a size of N_ (d-1) × N_ (d-1) repeatedly perform predictive coding, to search for the divisional type with minimum code error.
Even if when the divisional type 998 being of a size of N_ (d-1) × N_ (d-1) has minimum code error, because depth capacity is d, therefore the degree of depth is that the coding unit CU_ (d-1) of d-1 is also no longer divided into more low depth, coding depth for the coding unit forming current maximum coding unit 900 is confirmed as d-1, and the divisional type of current maximum coding unit 900 can be confirmed as N_ (d-1) × N_ (d-1).In addition, because depth capacity is d, therefore the division information of minimum code unit 980 is not set.
Data cell 999 can for current maximum coding unit " minimum unit ".According to the square data cell that the minimum unit of the present embodiment can be by the minimum code unit 980 with minimum coding depth being divided into 4 parts and acquisition.By repeatedly performing coding, video encoder 100 according to the present embodiment selects to have the degree of depth of minimum code error to determine coding depth by comparing according to the encoding error of the degree of depth of coding unit 900, and respective partition type and predictive mode is set to the coding mode of coding depth.
Like this, all degree of depth 0,1 ... in d-1, the minimum code error according to the degree of depth is compared, and the degree of depth with minimum code error can be confirmed as coding depth.The information that the divisional type of coding depth, predicting unit and predictive mode can be used as about coding mode is encoded and is sent.In addition, because the degree of depth of coding unit from 0 is divided into coding depth, therefore only the division information of coding depth is set to 0, and the division information of the degree of depth except coding depth is set to 1.
Can extract according to the view data of the video decoding apparatus 200 of the present embodiment and coded message extractor 220 and use the information of coding depth about coding unit 900 and predicting unit, subregion 912 is decoded.According to the video decoding apparatus 200 of the present embodiment by using the division information according to the degree of depth, be that the degree of depth of 0 is defined as coding depth by division information, and use and decode about the information of the coding mode of respective depth.
Figure 17, Figure 18 and Figure 19 are the diagrams in coding unit 1010, relation between predicting unit 1060 and converter unit 1070 for describing according to exemplary embodiment.
Coding unit 1010 is the comparatively deep layer coding units according to the degree of depth determined by video encoder 100 in maximum coding unit.Predicting unit 1060 is subregions of the predicting unit in each coding unit 1010, and converter unit 1070 is converter units of each coding unit 1010.
When the degree of depth of coding unit maximum in coding unit 1010 is 0, the degree of depth of coding unit 1012 and 1054 is 1, the degree of depth of coding unit 1014,1016,1018,1028,1050 and 1052 is 2, the degree of depth of coding unit 1020,1022,1024,1026,1030,1032 and 1048 is 3, and the degree of depth of coding unit 1040,1042,1044 and 1046 is 4.
In predicting unit 1060, obtain some coding units 1014,1016,1022,1032,1048,1050,1052 and 1054 by the coding unit divided in coding unit 1010.In other words, the size of the divisional type in coding unit 1014,1022,1050 and 1054 is 2N × N, and the size of the divisional type in coding unit 1016,1048 and 1052 is N × 2N, and the divisional type of coding unit 1032 is of a size of N × N.The predicting unit of coding unit 1010 and subregion are less than or equal to each coding unit.
In converter unit 1070 in the data cell being less than coding unit 1052, conversion or inverse transformation are performed to the view data of coding unit 1052.In addition, in size and dimension, the coding unit 1014,1016,1022,1032,1048,1050 and 1052 in converter unit 1070 is different from the coding unit 1014,1016,1022,1032,1048,1050 and 1052 in predicting unit 1060.In other words, according to the video encoder 100 of embodiment with infra-frame prediction/motion estimation/motion compensation/conversion/inverse transformation can be performed independently to the data cell in same coding unit according to the video decoding apparatus 200 of embodiment.
Therefore, coding is recursively performed to determine optimum code unit to each coding unit with hierarchy in each region of maximum coding unit, thus the coding unit with recursive tree structure can be obtained.Coded message can comprise the information of the division information about coding unit, the information about divisional type, the information about predictive mode and the size about converter unit.Table 1 illustrates the coded message that can be arranged by the video encoder 100 according to embodiment and the video decoding apparatus 200 according to embodiment.
[table 1]
According to the exportable coded message of coding unit about having tree structure of the output unit 130 of the video encoder 100 of embodiment, can from the bitstream extraction received about the coded message of coding unit with tree structure according to the view data of the video decoding apparatus 200 of embodiment and coded message extractor 220.
Division information indicates whether coding unit current coded unit being divided into more low depth.If the division information of current depth d is 0, then current coded unit is no longer divided into the degree of depth of more low depth is coding depth, thus can define the information of the size about divisional type, predictive mode and converter unit for described coding depth.If current coded unit by Further Division according to division information, then divides coding units to four of more low depth and performs coding independently.
Predictive mode can be the one in frame mode, inter-frame mode and skip mode.Can in all divisional types definition frame internal schema and inter-frame mode, only in the divisional type being of a size of 2N × 2N, define skip mode.
Information about divisional type can indicate the height by dividing predicting unit symmetrically or width and obtain the symmetrical divisional type being of a size of 2N × 2N, 2N × N, N × 2N and N × N, and obtains by the height that asymmetricly divides predicting unit or width the asymmetric divisional type being of a size of 2N × nU, 2N × nD, nL × 2N and nR × 2N.Height by dividing predicting unit by 1:3 and 3:1 obtains the asymmetric divisional type being of a size of 2N × nU and 2N × nD respectively, and the width by dividing predicting unit by 1:3 and 3:1 obtains the asymmetric divisional type being of a size of nL × 2N and nR × 2N respectively.
Converter unit can be sized to two types under frame mode and two types under inter-frame mode.In other words, if the division information of converter unit is 0, then the size of converter unit can be 2N × 2N, i.e. the size of current coded unit.If the division information of converter unit is 1, then by carrying out division to obtain converter unit to current coded unit.In addition, if when the divisional type being of a size of the current coded unit of 2N × 2N is symmetrical divisional type, then the size of converter unit can be N × N, if the divisional type of current coded unit is asymmetric divisional type, then the size of converter unit can be N/2 × N/2.
At least one in the coding unit corresponding to coding depth, predicting unit and minimum unit can be assigned to according to the coded message about the coding unit with tree structure of embodiment.The coding unit corresponding to coding depth can comprise at least one in the predicting unit and minimum unit comprising same-code information.
Therefore, the coded message by comparing proximity data unit determines whether proximity data unit is included in the same coding unit corresponding to coding depth.In addition, determine the corresponding encoded unit corresponding to coding depth by the coded message of usage data unit, and therefore can determine the distribution of the coding depth in maximum coding unit.
Therefore, if predicted current coded unit based on the coded message of proximity data unit, then can directly with reference to also using the coded message compared with data cell in deep layer coding unit contiguous with current coded unit.
Alternatively, if predicted current coded unit based on the coded message of proximity data unit, then the coded message of usage data unit searches for the data cell contiguous with current coded unit, and can with reference to the contiguous coding unit searched to predict current coded unit.
Figure 20 is the diagram for describing the coding unit of the coding mode information according to table 1, the relation between predicting unit and converter unit.
Maximum coding unit 1300 comprises the coding unit 1302,1304,1306,1312,1314,1316 and 1318 of multiple coding depth.Here, because coding unit 1318 is coding units of a coding depth, therefore division information can be configured to 0.The information of divisional type about the coding unit 1318 being of a size of 2N × 2N can be arranged to the one comprised in the divisional type of 2N × 2N1322,2N × N1324, N × 2N1326, N × N1328,2N × nU1332,2N × nD1334, nL × 2N1336 and nR × 2N1338.
Converter unit division information (TU dimension mark) is the manipulative indexing of a type.The size of the converter unit corresponding to manipulative indexing can change according to the predicting unit type of coding unit or divisional type.
Such as, when to be configured in symmetrical divisional type 2N × 2N1322,2N × N1324, N × 2N1326 and N × N1328 a kind of for the information about divisional type, if converter unit division information is 0, the converter unit 1342 being of a size of 2N × 2N is then set, if converter unit division information is 1, then the converter unit 1344 being of a size of N × N is set.
When to be configured in asymmetric divisional type 2N × nU1332,2N × nD1334, nL × 2N1336 and nR × 2N1338 a kind of for the information about divisional type, if converter unit division information is 0, the converter unit 1352 being of a size of 2N × 2N is then set, if converter unit division information is 1, then the converter unit 1354 being of a size of N/2 × N/2 is set.
Described by above reference Figure 20, converter unit division information (TU dimension mark) is the mark with value 0 or 1, but converter unit division information is not limited to the mark of 1 bit, and converter unit can be layered division when converter unit division information increases from 0.Converter unit division information can be the example of manipulative indexing.
In this case, according to embodiment, by the size using converter unit division information and the full-size of converter unit and the minimum dimension of converter unit to represent the converter unit in fact used.Video encoder 100 according to embodiment can be encoded to maximum converter unit dimension information, minimum converter unit dimension information and maximum converter unit division information.SPS can be inserted into the result that maximum converter unit dimension information, minimum converter unit dimension information and maximum converter unit division information are encoded.Video decoding apparatus 200 according to embodiment is decoded to video by using maximum converter unit dimension information, minimum converter unit dimension information and maximum converter unit division information.
Such as, if a the size of () current coded unit is 64 × 64 and maximum converter unit size is 32 × 32, then (a-1) is when TU dimension mark is 0, the size of converter unit can be 32 × 32, (a-2) when TU dimension mark is 1, the size of converter unit can be 16 × 16, and (a-3), when TU dimension mark is 2, the size of converter unit can be 8 × 8.
As another example, if the size of (b) current coded unit is 32 × 32 and minimum converter unit size is 32 × 32, then (b-1) is when TU dimension mark is 0, and the size of converter unit can be 32 × 32.Here, because the size of converter unit can not be less than 32 × 32, therefore TU dimension mark can not be set to the value except 0.
As another example, if the size of (c) current coded unit is 64 × 64 and maximum TU dimension mark is 1, then TU dimension mark can be 0 or 1.Here, TU dimension mark can not be set to the value except 0 or 1.
Therefore, when TU dimension mark is 0, if define maximum TU dimension mark for " MaxTransformSizeIndex ", minimum converter unit is of a size of " MinTransformSize ", converter unit is of a size of " RootTuSize ", then define the current minimum converter unit size " CurrMinTuSize " can determined in current coded unit by equation (1):
Equation (1):
CurrMinTuSize＝max(MinTransformSize,RootTuSize/(2^MaxTransformSizeIndex))…(1)
Compared with the current minimum converter unit size " CurrMinTuSize " can determined in current coded unit, the converter unit size " RootTuSize " being 0 when TU dimension mark can indicate the maximum converter unit size that can select in systems in which.In equation (1), " RootTuSize/ (2^MaxTransformSizeIndex) " instruction is when TU dimension mark is 0, converter unit size " RootTuSize " has been divided the converter unit size during number of times corresponding to maximum TU dimension mark, and " MinTransformSize " indicates minimum transform size.Therefore, in " RootTuSize/ (2^MaxTransformSizeIndex) " and " MinTransformSize ", less value can be the current minimum converter unit size " CurrMinTuSize " can determined in current coded unit.
According to exemplary embodiment, maximum converter unit size RootTuSize can change according to the type of predictive mode.
Such as, if current prediction mode is inter-frame mode, then by using following equation (2) to determine " RootTuSize ".In equation (2), " MaxTransformSize " indicates maximum converter unit size, and " PUSize " indicates current prediction unit size:
RootTuSize＝min(MaxTransformSize,PUSize)……(2)
That is, if current prediction mode is inter-frame mode, then the converter unit size " RootTuSize " when TU dimension mark is 0 can be value less in maximum converter unit size and current prediction unit size.
If the predictive mode of current bay unit is frame mode, then by using following equation (3) to determine " RootTuSize ".In equation (3), " PartitionSize " indicates the size of current bay unit:
RootTuSize＝min(MaxTransformSize,PartitionSize)……(3)
That is, if current prediction mode is frame mode, then the converter unit size " RootTuSize " when TU dimension mark is 0 can be value less among the size of maximum converter unit size and current bay unit.
But according to the type of the predictive mode in zoning unit and the current maximum converter unit size " RootTuSize " changed is only exemplary embodiment, and embodiment is not limited thereto.
According to the method for video coding of the above coding unit based on tree structure with reference to Fig. 8 to Figure 20 description, can encode to the view data in spatial domain in each coding unit in the coding unit of tree structure, and according to the view data come according to the mode of video encoding/decoding method to each maximum coding unit execution decoding based on the coding unit of tree structure in reconstruction attractor territory, the video be made up of picture and image sequence can be reconstructed.The video of reconstruct can be reproduced by reproducer, can be stored in storage medium, or send by network.
One or more exemplary embodiment can be written as computer program, and can to use the general purpose digital computer of computer readable recording medium storing program for performing executive program to realize.The example of computer readable recording medium storing program for performing comprises magnetic storage medium (such as, ROM, floppy disk, hard disk etc.) and optical record medium (such as, CD-ROM or DVD) etc.
For convenience of description, the above cross-layer video coding method with reference to Fig. 1 a to Figure 20 description and/or method for video coding will be collectively referred to as " method for video coding ".In addition, the above cross-layer video coding/decoding method with reference to Fig. 1 a to Figure 20 description and/or video encoding/decoding method will be collectively referred to as " video encoding/decoding method ".
In addition, the above video encoder comprising cross-layer video encoding device 10, video encoder 100 or image encoder 400 described with reference to Fig. 1 a to Figure 20 will be collectively referred to as " video encoder ".In addition, the above video decoding apparatus comprising cross-layer video decoding device 20, video decoding apparatus 200 or image decoder 500 described with reference to Fig. 1 a to Figure 20 will be collectively referred to as " video decoding apparatus ".
Now by the stored program computer readable recording medium storing program for performing described in detail according to exemplary embodiment (such as, coiling 26000).
Figure 21 is the diagram of the physical structure of stored program dish 26000 according to exemplary embodiment.Dish 26000 as storage medium can be hard disk drive, compact disk read-only memory (CD-ROM) dish, Blu-ray disc or digital versatile disc (DVD).Dish 26000 comprises multiple concentric magnetic track Tr, and each concentric magnetic track Tr is divided into the sector Se of specific quantity along the circumferencial direction coiling 26000.In the specific region of dish 26000, can distribute and store the program performing quantization parameter defining method, method for video coding and video encoding/decoding method described above.
The computer system using and store and realize for the storage medium of the program performing method for video coding as above and video encoding/decoding method is described now with reference to Figure 22.
Figure 22 is recorded by use dish 26000 and the diagram of the disk drive 26800 of fetch program.Execution can be stored in dish 26000 according to the program of at least one in the method for video coding of exemplary embodiment and video encoding/decoding method via disk drive 26800 by computer system 26700.In order to run the program be stored in dish 26000 in computer system 26700, by using disk drive 26800 from dish 26000 fetch program and program being sent to computer system 26700.
Execution not only can be stored in the dish 26000 shown in Figure 21 and Figure 22 according to the program of at least one in the method for video coding of exemplary embodiment and video encoding/decoding method, also can be stored in storage card, ROM cassette tape or solid-state drive (SSD).
The system of application method for video coding described above and video encoding/decoding method will be described below.
Figure 23 is the integrally-built diagram of the contents providing system 11000 for providing distribution of content to serve.The coverage of communication system is divided into the community of preliminary dimension, and wireless base station 11700,11800,11900 and 12000 is arranged in these communities respectively.
Contents providing system 11000 comprises multiple self-contained unit.Such as, multiple self-contained units of such as computer 12100, personal digital assistant (PDA) 12200, video camera 12300 and mobile phone 12500 are connected to the Internet 11100 via ISP 11200, communication network 11400 and wireless base station 11700,11800,11900 and 12000.
But contents providing system 11000 is not limited to as shown in Figure 23, and device is optionally connected to contents providing system 11000.Multiple self-contained unit can not be directly connected to communication network 11400 via wireless base station 11700,11800,11900 and 12000.
Video camera 12300 is can the imaging device of captured video image, such as, and digital video camera.Mobile phone 12500 can utilize at least one communication means in various agreement (such as, individual digital communication (PDC), code division multiple access (CDMA), Wideband Code Division Multiple Access (WCDMA) (W-CDMA), global system for mobile communications (GSM) and personal handyphone system (PHS)).
Video camera 12300 can be connected to streaming server 11300 via wireless base station 11900 and communication network 11400.Streaming server 11300 allows the content received from user via video camera 12300 to be streamed via real-time broadcast.Video camera 12300 or streaming server 11300 can be used to encode to the content received from video camera 12300.The video data captured by video camera 12300 can be sent to streaming server 11300 via computer 12100.
The video data captured by camera 12600 also can be sent to streaming server 11300 via computer 12100.Similar with digital camera, camera 12600 is can the imaging device of catching static images and video image.Camera 12600 or computer 12100 can be used the coding video data captured by camera 12600.Software video being performed to Code And Decode can be stored in the computer readable recording medium storing program for performing (such as, CD-ROM dish, floppy disk, hard disk drive, SSD or storage card) can accessed by computer 12100.
If video data is caught in by the camera be built in mobile phone 12500, then can from mobile phone 12500 receiving video data.
Also come coding video data by large scale integrated circuit (LSI) system be arranged in video camera 12300, mobile phone 12500 or camera 12600.
Contents providing system 1100 can to the content-data being used video camera 12300, camera 12600, mobile phone 12500 or another imaging device to record by user (such as, the content recorded during concert) encode, and the content-data after coding is sent to streaming server 11300.Content-data after coding can be sent to other client of request content data by streaming server 11300 with the type of streaming content.
Client is the device can decoded to the content-data after coding, such as, and computer 12100, PDA12200, video camera 12300 or mobile phone 12500.Therefore, contents providing system 11000 allows client to receive and reproduces the content-data after encoding.In addition, the content-data after contents providing system 11000 allows client real-time reception to encode also is decoded to the content-data after coding and reproduces, thus can carry out personal broadcaster.
The Code And Decode operation being included in the multiple self-contained units in contents providing system 11000 can be similar to and operate according to the video encoder of exemplary embodiment and the Code And Decode of video decoding apparatus.
The mobile phone 12500 be included in according in the contents providing system 11000 of exemplary embodiment is described in further detail now with reference to Figure 24 and Figure 25.
Figure 24 illustrates the external structure according to the application method for video coding of exemplary embodiment and the mobile phone 12500 of video encoding/decoding method.Mobile phone 12500 can be smart phone, and the function of described smart phone is not limited, and most of functions of described smart phone can be changed or expand.
Mobile phone 12500 comprises and can exchange the inside antenna 12510 of radio frequency (RF) signal with wireless base station 12000, and comprise for show the image that captured by camera 12530 or via antenna 12510 receive and the display screen 12520 (such as, liquid crystal display (LCD) or Organic Light Emitting Diode (OLED) screen) of decoded image.Mobile phone 12500 comprises the guidance panel 12540 including control button and touch panel.If display screen 12520 is touch-screens, then guidance panel 12540 also comprises the touch-sensing panel of display screen 12520.Mobile phone 12500 comprises loud speaker 12580 for exporting voice and sound or another type voice output unit and for the microphone 12550 that inputs voice and sound or another type Speech input unit.Mobile phone 12500 also comprises the camera 12530 for catching video and rest image, such as charge coupled device (CCD) camera.Mobile phone 12500 also can comprise: storage medium 12570, for store captured by camera 12530, receive via e-mail or the encoding/decoding data (such as, video or rest image) that obtains according to various mode; Slot 12560, storage medium 12570 is loaded in mobile phone 12500 via slot 12560.Storage medium 12570 can be flash memory, such as, is included in secure digital (SD) card in plastic housing or electric erasable and programmable read only memory (EEPROM).
Figure 25 illustrates the internal structure of mobile phone 12500.In order to systematically control the parts comprising the mobile phone 12500 of display screen 12520 and guidance panel 12540, power supply circuits 12700, operation input control device 12640, image encoder 12720, camera interface 12630, lcd controller 12620, image decoder 12690, multiplexer/demultiplexer 12680, recording/reading unit 12670, modulation/demodulation unit 12660 and Sound Processor Unit 12650 are connected to central controller 12710 via synchronous bus 12730.
If user operation power knob, and be set to " electric power starting " state from " power-off " state, then power supply circuits 12700 are powered from battery pack to all parts of mobile phone 12500, thus during operator scheme, arrange mobile phone 12500.
Central controller 12710 comprises central processing unit (CPU), ROM and RAM.
While communication data is sent to outside by mobile phone 12500, under the control of central controller 12710, produce digital signal by mobile phone 12500.Such as, Sound Processor Unit 12650 can produce digital audio signal, and image encoder 12720 can produce data image signal, and the text data of message can be produced via guidance panel 12540 and operation input control device 12640.When under the control at central controller 12710, digital signal is sent to modulation/demodulation unit 12660, the frequency band of modulation/demodulation unit 12660 pairs of digital signals is modulated, and digital audio signal actual figure mode convertion (DAC) after telecommunication circuit 12610 pairs of band modulation and frequency inverted.The transmission signal exported from telecommunication circuit 12610 can be sent to voice communication base station or wireless base station 12000 via antenna 12510.
Such as, when mobile phone 12500 is in call mode, under the control of central controller 12710, the voice signal obtained via microphone 12550 is transformed into digital audio signal by Sound Processor Unit 12650.Digital audio signal can be transformed into figure signal via modulation/demodulation unit 12660 and telecommunication circuit 12610, and can be sent out via antenna 12510.
When text message (such as, Email) is sent out during data communication mode, the text data of text message is transfused to via guidance panel 12540, and is sent to central controller 12610 via operation input control device 12640.Under the control of central controller 12610, text data is transformed into transmission signal via modulation/demodulation unit 12660 and telecommunication circuit 12610, and is sent to wireless base station 12000 via antenna 12510.
In order to send view data during data communication mode, the view data captured by camera 12530 is provided to image encoder 12720 via camera interface 12630.The view data captured can be displayed directly on display screen 12520 via camera interface 12630 and lcd controller 12620.
The structure of image encoder 12720 can be corresponding to the structure of video encoder 100 described above.Image encoder 12720 can based on the above-mentioned method for video coding according to the present embodiment, be compression and the view data after encoding by the image data transformation received from camera 12530, and then the view data after coding outputted to multiplexer/demultiplexer 12680.During the record operation of camera 12530, the voice signal obtained by the microphone 12550 of mobile phone 12500 can be transformed into digital audio data via Sound Processor Unit 12650, and digital audio data can be sent to multiplexer/demultiplexer 12680.
Multiplexer/demultiplexer 12680 carries out multiplexing to the view data after the coding received from image encoder 12720 together with the voice data received from Sound Processor Unit 12650.Multiplexing result is carried out to data and can be transformed into transmission signal via modulation/demodulation unit 12660 and telecommunication circuit 12610, then can be sent out via antenna 12510.
When mobile phone 12500 is from external reception communication data, frequency retrieval and ADC can be performed so that signal is transformed into digital signal to the signal that receives via antenna 12510.The frequency band of modulation/demodulation unit 12660 pairs of digital signals is modulated.Described digital signal is sent to Video Decoder 12690, Sound Processor Unit 12650 or lcd controller 12620 by the type according to the digital signal after band modulation.
During call mode, mobile phone 12500 amplifies the signal received via antenna 12510, and by obtaining digital audio signal to the signal execution frequency inverted after amplification and ADC.Under the control of central controller 12710, the digital audio signal received is transformed into analoging sound signal via modulation/demodulation unit 12660 and Sound Processor Unit 12650, and analoging sound signal is output via loud speaker 12580.
When during data communication mode, be received in the data of the video file that internet site is accessed, via modulation/demodulation unit 12660, the signal received from wireless base station 12000 via antenna 12510 is exported as multiplex data, and multiplex data is sent to multiplexer/demultiplexer 12680.
In order to decode to the multiplex data received via antenna 12510, multiplex data is demultiplexed into the audio data stream after the video data stream after coding and coding by multiplexer/demultiplexer 12680.Via synchronous bus 12730, the video data stream after coding and the audio data stream after encoding are provided to Video Decoder 12690 and Sound Processor Unit 12650 respectively.
The structure of image decoder 12690 can be corresponding to the structure of video decoding apparatus described above.Image decoder 12690 is by using the above-mentioned video encoding/decoding method according to the present embodiment, video data after coding is decoded and obtains the video data of reconstruct, and via lcd controller 12620, the video data of reconstruct is supplied to display screen 12520.
Therefore, the data of the video file that internet site is accessed can be presented on display screen 12520.Meanwhile, voice data can be transformed into analoging sound signal by Sound Processor Unit 12650, and analoging sound signal is supplied to loud speaker 12580.Therefore, also can be reproduced in via loud speaker 12580 voice data comprised in the video file that internet site is accessed.
The communication terminal of mobile phone 12500 or another type can be the transceiver terminal comprising video encoder according to exemplary embodiment and video decoding apparatus, can be the transceiver terminal only comprising video encoder, or can be the transceiver terminal only comprising video decoding apparatus.
The above communication system described with reference to Figure 24 is not limited to according to the communication system of exemplary embodiment.Such as, Figure 26 illustrates the digit broadcasting system of the employing communication system according to exemplary embodiment.The digit broadcasting system of Figure 26 is by using according to the video encoder of exemplary embodiment and video decoding apparatus receives via satellite or ground network sends digital broadcasting.
In more detail, broadcasting station 12890 is by using radio wave by video data stream to communication satellite or broadcasting satellite 12900.Broadcasting satellite 12900 sends broadcast singal, and broadcast singal is sent to satellite broadcast reception converter via family expenses antenna 12860.In each house, by TV receiver 12810, Set Top Box 12870 or another device, the video flowing after coding is decoded and reproduced.
When being implemented in reproducer 12830 according to the video decoding apparatus of exemplary embodiment, reproducer 12830 can be resolved and decode the video flowing after the coding be recorded on storage medium 12820 (such as dish or storage card), with reconstructed number signal.Therefore, the vision signal of reconstruct can be reproduced on such as monitor 12840.
Be connected to for satellite/terrestrial broadcast antenna 12860 or for receive the cable antenna 12850 that cable TV (TV) is broadcasted Set Top Box 12870 in, the video decoding apparatus according to exemplary embodiment can be installed.The data exported from Set Top Box 12870 also can be reproduced in TV Monitor 12880.
As another example, can be arranged on according to the video decoding apparatus of exemplary embodiment in TV receiver 12810, instead of in Set Top Box 12870.
The automobile 12920 with suitable antenna 12910 can receive the signal sent from satellite 12900 or the wireless base station 11700 of Figure 21.Decoded video can be reproduced on the display screen of the auto-navigation system 12930 be arranged in automobile 12920.
Vision signal can be encoded by the video encoder according to exemplary embodiment, then can be stored in storage medium.Particularly, by DVD register, picture signal can be stored in DVD dish 12960, or by hdd recorder 12950, picture signal can be stored in a hard disk.As another example, vision signal can be stored in SD card 12970.If hdd recorder 12950 comprises the video decoding apparatus according to exemplary embodiment, then the vision signal be recorded on DVD dish 12960, SD card 12970 or another storage medium can be reproduced on TV Monitor 12880.
Auto-navigation system 12930 can not comprise the camera 12530 of Figure 26, camera interface 12630 and image encoder 12720.Such as, computer 12100 and TV receiver 12810 can not included in the camera 12530 of Figure 26, camera interface 12630 and image encoders 12720.
Figure 27 is the diagram of the network configuration illustrated according to the use video encoder of exemplary embodiment and the cloud computing system of video decoding apparatus.
Cloud computing system can comprise cloud computing server 14000, customer data base (DB) 14100, multiple computational resource 14200 and user terminal.
In response to the request carrying out user terminal, cloud computing system provides the program request outsourcing service of multiple computational resource 14200 via data communication network (such as, the Internet).Under cloud computing environment, the computational resource of service provider by using virtual technology combination to be positioned at the place of data center of different physical locations, for user provides the service wanted.Service-user not will computational resource (such as, application, memory, operating system (OS) and fail-safe software) be arranged on to use them in the terminal that he/her has, but can in the time point the wanted service that choice and operation is wanted from service in the Virtual Space produced by virtual technology.
The user terminal of appointed service-user is connected to cloud computing server 14000 via the data communication network comprising the Internet and mobile communications network.Cloud computing service can be provided to user terminal, particularly rabbit service from cloud computing server 14100.User terminal can be various types of electronic installations that can be connected to the Internet, such as, desktop PC14300, intelligent TV14400, smart phone 14500, notebook 14600, portable media player (PMP) 14700, dull and stereotyped PC14800 etc.
The cloud computing server 14000 multiple computational resources 14200 be distributed in cloud network capable of being combined, and the result of combination is provided to user terminal.Described multiple computational resource 14200 can comprise various data, services, and can comprise the data uploaded from user terminal.As described above, cloud computing server 14000 is by providing according to virtual technology combination distribution video database in the different areas the service wanted to user terminal.
User profile about the user ordering cloud computing service is stored in user DB14100.User profile can comprise the log-on message of user, address, name and personal credit information.User profile also can comprise the index of video.Here, described index can comprise reproduced video list, just reproduced video list, before just in the breakpoint etc. of reproduced video.
The information about video be stored in user DB14100 can be shared between the user device.Such as, when Video service being supplied to notebook 14600 in response to the request from notebook 14600, the representation of the historical of Video service is stored in user DB14100.When receiving the request for reproducing this Video service from smart phone 14500, cloud computing server 14000 is searched for based on user DB14100 and is reproduced this Video service.When smart phone 14500 receives video data stream from cloud computing server 14000, the class of operation of the mobile phone 12500 described by process and above reference Figure 24 of rendered video of decoding to video data stream seemingly.
Cloud computing server 14000 can with reference to the representation of the historical of the Video service wanted be stored in user DB14100.Such as, cloud computing server 14000 is from the request of user terminal reception for reproducing the video be stored in user DB14100.If the reproduced mistake of this video, then by cloud computing server 14000 perform the method for flow transmission is carried out to this video can according to carrying out the request (that is, according to be by rendered video from the starting point of video or the breakpoint of video) of user terminal and different.Such as, if user terminal requests rendered video from the starting point of video, then the flow data of the video from the first frame of video is sent to user terminal by cloud computing server 14000.If user terminal requests is rendered video from the breakpoint of video, then the flow data of the video from the frame corresponding to breakpoint is sent to user terminal by cloud computing server 14000.
In the case, user terminal can comprise as the above video decoding apparatus described with reference to Fig. 1 a to Figure 20.In another example, user terminal can comprise as the above video encoder described with reference to Fig. 1 a to Figure 20.Alternatively, user terminal can comprise as the above video decoding apparatus with reference to Fig. 1 a to Figure 20 description and video encoder.
The various application of the above method for video coding according to embodiment with reference to Fig. 1 a to Figure 20 description, video encoding/decoding method, video encoder and video decoding apparatus are described above with reference to Figure 21 to Figure 27.But, method for video coding and video encoding/decoding method are stored in the method in storage medium or video encoder and the video decoding apparatus method realized in the device described with reference to Fig. 1 a to Figure 20 are above not limited to the above embodiment described with reference to Figure 21 to Figure 27 according to various embodiment.
Although specifically illustrate with reference to exemplary embodiment of the present disclosure and describe the disclosure, but those of ordinary skill in the art will understand, when not departing from the spirit and scope of claim, the various changes in form and details can be made in the exemplary embodiment.Exemplary embodiment only should be considered describing significance, and is not used in the object of restriction.Therefore, the scope of the present disclosure be can't help detailed description of the present disclosure and is limited, and is limited by claim, and all differences within the scope of this all will be believed to comprise in the disclosure.

Claims (15)

1. a cross-layer video coding/decoding method, comprising:
Ground floor image is reconstructed based on the coded message obtained from ground floor bit stream;
Predetermined partition type is confirmed as and the second layer block being confirmed as being in predictive mode in order to reconstruct, determine whether the illuminance compensation of the second layer block performed for the reconstruct determined by using ground floor reference block, wherein, ground floor reference block is arranged in the ground floor image of reconstruct and corresponding to second layer block;
By use ground floor reference block and the second layer block producing reconstruct from the inter-layer prediction information that second layer bit stream obtains, and produce the second layer image comprising the second layer block of reconstruct, wherein, whether the illumination of the second layer block of reconstruct is performed according to illuminance compensation and determines.
2. cross-layer video coding/decoding method as claimed in claim 1, wherein, determines whether that the step performing illuminance compensation comprises:
Divisional type information and the prediction mode information of second layer block is obtained from second layer bit stream;
If divisional type information indicates described predetermined partition type and prediction mode information does not indicate intra prediction mode, then obtain the illumination compensation information for second layer block from second layer bit stream;
Determine whether to perform illuminance compensation to the second layer block of reconstruct based on the illumination compensation information for second layer block.
3. cross-layer video coding/decoding method as claimed in claim 2, wherein, determine whether that the step performing illuminance compensation comprises the illumination compensation information obtained for following piece: the divisional type information of described piece indicates 2N × 2N type and the prediction mode information of described piece does not indicate intra prediction mode.
4. cross-layer video coding/decoding method as claimed in claim 2, wherein, determine whether that the step performing illuminance compensation comprises: obtain based on divisional type information and prediction mode information the illumination compensation information being used for being confirmed as the block being in skip mode or 2N × 2N fusion mode.
5. cross-layer video coding/decoding method as claimed in claim 1, wherein, determine whether that the step performing illuminance compensation comprises: determine whether to perform illuminance compensation to by by the luminance component of the second layer block of the reconstruct of illuminance compensation and chromatic component according to the size of the current bay of the second layer block of reconstruct.
6. cross-layer video coding/decoding method as claimed in claim 5, wherein, determine whether to comprise by the step being performed illuminance compensation by the luminance component of the second layer block of the reconstruct of illuminance compensation and chromatic component: determine that the luminance component to partitions sizes is the block of 8 × 8 performs illuminance compensation, and determine not perform illuminance compensation to the chromatic component that partitions sizes is the block of 8 × 8.
7. cross-layer video coding/decoding method as claimed in claim 1, wherein, determining whether that the step performing illuminance compensation comprises: according to the size of the second layer block of reconstruct, for determining whether, different operations being carried out to the luminance component of the second layer block of reconstruct and the process of chromatic component execution illuminance compensation.
8. cross-layer video coding/decoding method as claimed in claim 7, wherein, determine whether to comprise the luminance component of the second layer block of reconstruct and the step of chromatic component execution illuminance compensation: when the size of the brightness subregion of block is equal to or greater than 8 × 8, determine to perform illuminance compensation to the luminance component of this block, the size of the brightness subregion of if block is greater than 8 × 8, then determine to perform illuminance compensation to the chromatic component of this block.
9. cross-layer video coding/decoding method as claimed in claim 1, wherein, for except being confirmed as described predetermined partition type and other second layer block except the second layer block being confirmed as being in described predictive mode skips the step determining whether to perform illuminance compensation, further, illuminance compensation is not performed to other second layer block described.
10. cross-layer video coding/decoding method as claimed in claim 1, wherein, when determine illuminance compensation is performed to the second layer block time, not to second layer block perform be used for by service time direction reference block and interlayer direction reference block in the residual prediction predicted of the residual information of residual information to second layer block of at least one reference block.
11. cross-layer video coding/decoding methods as claimed in claim 1, wherein, when determine second layer block performed residual prediction with by service time direction reference block and interlayer direction reference block in the residual information of residual information to second layer block of at least one reference block predict time, illuminance compensation is not performed to second layer block.
12. 1 kinds of cross-layer video coding methods, comprising:
Produce the ground floor bit stream comprising the coded message produced by encoding to ground floor image;
Predetermined partition type is confirmed as and the second layer block being confirmed as being in predictive mode in order to reconstruct, determine whether the illuminance compensation of the second layer block performed for the reconstruct determined by using ground floor reference block, wherein, ground floor reference block is arranged in the ground floor image of reconstruct and corresponding to second layer block;
Produce second layer bit stream, wherein, second layer bit stream comprise by second layer block and corresponding to second layer block ground floor reference block between inter-layer prediction and the inter-layer prediction information produced, and whether second layer bit stream comprises ground floor reference block and is performed according to illuminance compensation and determines the inter-layer prediction information between the second layer block of illumination.
13. 1 kinds of cross-layer video decoding devices, comprising:
Ground floor decoder, reconstructs ground floor image based on the coded message obtained from ground floor bit stream;
Illuminance compensation determiner, predetermined partition type is confirmed as and the second layer block being confirmed as being in predictive mode in order to reconstruct, determine whether the illuminance compensation of the second layer block performed for the reconstruct determined by using ground floor reference block, wherein, ground floor reference block is arranged in the ground floor image of reconstruct and corresponding to second layer block;
Second layer decoder, by use ground floor reference block and the second layer block producing reconstruct from the inter-layer prediction information that second layer bit stream obtains, and produce the second layer image comprising the second layer block of reconstruct, wherein, whether the illumination of the second layer block of reconstruct is performed according to illuminance compensation and determines.
14. 1 kinds of cross-layer video encoding devices, comprising:
Ground floor encoder, produces the ground floor bit stream comprising the coded message produced by encoding to ground floor image;
Illuminance compensation determiner, predetermined partition type is confirmed as and the second layer block being confirmed as being in predictive mode in order to reconstruct, determine whether the illuminance compensation of the second layer block performed for the reconstruct determined by using ground floor reference block, wherein, ground floor reference block is arranged in the ground floor image of reconstruct and corresponding to second layer block;
Second layer encoder, produce second layer bit stream, wherein, second layer bit stream comprise by second layer block and corresponding to second layer block ground floor reference block between inter-layer prediction and the inter-layer prediction information produced, and whether second layer bit stream comprises ground floor reference block and is performed according to illuminance compensation and determines the inter-layer prediction information between the second layer block of illumination.
15. 1 kinds of computer readable recording medium storing program for performing recorded for the program of any one in the cross-layer video coding/decoding method of enforcement of rights requirement 1 and the cross-layer video coding method of claim 12.
CN201480013688.5A 2013-01-10 2014-01-10 Method and device thereof for the method and device thereof of compensation brightness difference encoded to cross-layer video and for being decoded to video Active CN105103552B (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US201361750966P 2013-01-10 2013-01-10
US61/750,966 2013-01-10
PCT/KR2014/000314 WO2014109594A1 (en) 2013-01-10 2014-01-10 Method for encoding inter-layer video for compensating luminance difference and device therefor, and method for decoding video and device therefor

Publications (2)

Publication Number Publication Date
CN105103552A true CN105103552A (en) 2015-11-25
CN105103552B CN105103552B (en) 2019-04-05

Family

ID=51167175

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201480013688.5A Active CN105103552B (en) 2013-01-10 2014-01-10 Method and device thereof for the method and device thereof of compensation brightness difference encoded to cross-layer video and for being decoded to video

Country Status (4)

Country Link
US (1) US9877020B2 (en)
KR (1) KR102270787B1 (en)
CN (1) CN105103552B (en)
WO (1) WO2014109594A1 (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109417640A (en) * 2016-07-05 2019-03-01 株式会社Kt Method and apparatus for handling vision signal
CN110166783A (en) * 2019-06-03 2019-08-23 深圳市华星光电技术有限公司 Compensate table compression method, display manufacturing equipment and the device with store function
CN112203085A (en) * 2020-09-30 2021-01-08 字节跳动(香港)有限公司 Image processing method, device, terminal and storage medium
CN112292863A (en) * 2018-06-12 2021-01-29 光云大学校产学协力团 Video signal processing method and apparatus based on reference between components

Families Citing this family (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9948951B2 (en) * 2012-12-26 2018-04-17 Sharp Kabushiki Kaisha Image decoding device which generates a predicted image of a target prediction unit
KR20140121315A (en) * 2013-04-04 2014-10-15 한국전자통신연구원 Method and apparatus for image encoding and decoding based on multi-layer using reference picture list
WO2014166104A1 (en) * 2013-04-12 2014-10-16 Mediatek Singapore Pte. Ltd. Removal of parsing dependency for illumination compensation
US10554967B2 (en) * 2014-03-21 2020-02-04 Futurewei Technologies, Inc. Illumination compensation (IC) refinement based on positional pairings among pixels
US10631002B2 (en) * 2016-09-30 2020-04-21 Qualcomm Incorporated Frame rate up-conversion coding mode
US10880570B2 (en) 2016-10-05 2020-12-29 Qualcomm Incorporated Systems and methods of adaptively determining template size for illumination compensation
KR20180074000A (en) * 2016-12-23 2018-07-03 삼성전자주식회사 Method of decoding video data, video decoder performing the same, method of encoding video data, and video encoder performing the same
KR20240023240A (en) * 2017-07-06 2024-02-20 삼성전자주식회사 Image encoding method and apparatus, and image decoding method and apparatus
WO2019021803A1 (en) * 2017-07-24 2019-01-31 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ Encoding device, decoding device, encoding method, and decoding method
US10652571B2 (en) * 2018-01-25 2020-05-12 Qualcomm Incorporated Advanced motion vector prediction speedups for video coding
WO2020085235A1 (en) * 2018-10-22 2020-04-30 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ Encoding device, decoding device, encoding method and decoding method
WO2020149630A1 (en) * 2019-01-15 2020-07-23 엘지전자 주식회사 Method and device for decoding image on basis of cclm prediction in image coding system
US11206396B2 (en) * 2019-01-16 2021-12-21 Qualcomm Incorporated Local illumination compensation in video coding

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101176348A (en) * 2005-04-01 2008-05-07 Lg电子株式会社 Method for scalably encoding and decoding video signal
CN101836457A (en) * 2007-10-25 2010-09-15 日本电信电话株式会社 Video scalable encoding method, video scalable decoding method, devices therefor, programs therefor, and recording medium where program is recorded
US20110122944A1 (en) * 2009-11-24 2011-05-26 Stmicroelectronics Pvt. Ltd. Parallel decoding for scalable video coding
US20130182779A1 (en) * 2010-09-29 2013-07-18 Industry-Academic Cooperation Foundation Hanbat National Univ Method and apparatus for video-encoding/decoding using filter information prediction

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH1032824A (en) * 1996-07-16 1998-02-03 Nippon Telegr & Teleph Corp <Ntt> Luminance change detecting, luminance change compensating and position detecting methods for dynamic image and dynamic image encoding and decoding device
US7289154B2 (en) * 2000-05-10 2007-10-30 Eastman Kodak Company Digital image processing method and apparatus for brightness adjustment of digital images
US8761252B2 (en) 2003-03-27 2014-06-24 Lg Electronics Inc. Method and apparatus for scalably encoding and decoding video signal
KR20060105407A (en) * 2005-04-01 2006-10-11 엘지전자 주식회사 Method for scalably encoding and decoding video signal
KR20070012201A (en) 2005-07-21 2007-01-25 엘지전자 주식회사 Method for encoding and decoding video signal
KR20070014956A (en) * 2005-07-29 2007-02-01 엘지전자 주식회사 Method for encoding and decoding video signal
JP2007329678A (en) * 2006-06-07 2007-12-20 Matsushita Electric Ind Co Ltd Super resolution processing method and apparatus using luminance masking and luminance compensation
KR101244917B1 (en) 2007-06-11 2013-03-18 삼성전자주식회사 Method and apparatus for compensating illumination compensation and method and apparatus for encoding and decoding video based on illumination compensation
USRE49308E1 (en) * 2010-09-29 2022-11-22 Electronics And Telecommunications Research Instit Method and apparatus for video-encoding/decoding using filter information prediction
GB2486733A (en) * 2010-12-24 2012-06-27 Canon Kk Video encoding using multiple inverse quantizations of the same reference image with different quantization offsets
US9282343B2 (en) * 2011-08-31 2016-03-08 Dolby Laboratories Licensing Corporation Multiview and bitdepth scalable video delivery

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101176348A (en) * 2005-04-01 2008-05-07 Lg电子株式会社 Method for scalably encoding and decoding video signal
CN101836457A (en) * 2007-10-25 2010-09-15 日本电信电话株式会社 Video scalable encoding method, video scalable decoding method, devices therefor, programs therefor, and recording medium where program is recorded
US20110122944A1 (en) * 2009-11-24 2011-05-26 Stmicroelectronics Pvt. Ltd. Parallel decoding for scalable video coding
US20130182779A1 (en) * 2010-09-29 2013-07-18 Industry-Academic Cooperation Foundation Hanbat National Univ Method and apparatus for video-encoding/decoding using filter information prediction

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109417640A (en) * 2016-07-05 2019-03-01 株式会社Kt Method and apparatus for handling vision signal
US10986358B2 (en) 2016-07-05 2021-04-20 Kt Corporation Method and apparatus for processing video signal
CN109417640B (en) * 2016-07-05 2021-08-17 株式会社Kt Method and apparatus for processing video signal
US11394988B2 (en) 2016-07-05 2022-07-19 Kt Corporation Method and apparatus for processing video signal
US11743481B2 (en) 2016-07-05 2023-08-29 Kt Corporation Method and apparatus for processing video signal
CN112292863A (en) * 2018-06-12 2021-01-29 光云大学校产学协力团 Video signal processing method and apparatus based on reference between components
US11765348B2 (en) 2018-06-12 2023-09-19 Kwangwoon University Industry-Academic Collaboration Foundation Method and an apparatus for processing a video signal based on inter-component reference
CN110166783A (en) * 2019-06-03 2019-08-23 深圳市华星光电技术有限公司 Compensate table compression method, display manufacturing equipment and the device with store function
WO2020244022A1 (en) * 2019-06-03 2020-12-10 Tcl华星光电技术有限公司 Compensation table compression method, apparatus for manufacturing display, and memory
CN112203085A (en) * 2020-09-30 2021-01-08 字节跳动(香港)有限公司 Image processing method, device, terminal and storage medium
CN112203085B (en) * 2020-09-30 2023-10-17 字节跳动(香港)有限公司 Image processing method, device, terminal and storage medium

Also Published As

Publication number Publication date
US20150350642A1 (en) 2015-12-03
WO2014109594A1 (en) 2014-07-17
US9877020B2 (en) 2018-01-23
KR20140091493A (en) 2014-07-21
CN105103552B (en) 2019-04-05
KR102270787B1 (en) 2021-06-29

Similar Documents

Publication Publication Date Title
CN105103552B (en) Method and device thereof for the method and device thereof of compensation brightness difference encoded to cross-layer video and for being decoded to video
CN106063271B (en) For executing cross-layer video coding/decoding method and its equipment and the cross-layer video coding method and its equipment for executing the prediction based on sub-block of the prediction based on sub-block
CN105144713B (en) For the method and device thereof of decoder setting encoded to video and based on decoder, the method and device thereof being decoded to video are set
CN105308966A (en) Video encoding method and apparatus thereof, and a video decoding method and apparatus thereof
CN104396252A (en) Multiview video encoding method using reference picture set for multiview video prediction and device thereof, and multiview video decoding method using reference picture set for multiview video prediction and device thereof
CN104620578A (en) Method and apparatus for multilayer video encoding for random access, and method and apparatus for multilayer video decoding for random access
CN105532001A (en) Method and apparatus for inter-layer decoding video using depth-based disparity vector, and method and apparatus for inter-layer encoding video using depth-based disparity vector
CN105472396A (en) Method for entropy-decoding slice segment and apparatus therefor
CN105532004B (en) Cross-layer video coding/decoding method and equipment
CN104488272A (en) Method and apparatus for predicting motion vector for coding video or decoding video
CN104796723A (en) Method and device for determining motion vector for video coding or video decoding
CN104813665A (en) Method for sao compensation for encoding inter-layer prediction error and apparatus therefor
CN104365101A (en) Method and apparatus for determining reference images for inter prediction
CN105340271A (en) Multi-layer video coding method for random access and device therefor, and multi-layer video decoding method for random access and device therefor
CN106031175B (en) Use the cross-layer video coding method of luminance compensation and its device and video encoding/decoding method and its device
CN105264894A (en) Method for determining inter-prediction candidate for interlayer decoding and encoding method and apparatus
CN104365104A (en) Method for multi-view video encoding based on tree structure encoding unit and apparatus for same, and method for multi-view video decoding based on tree structure encoding unit and apparatus for same
CN105556972A (en) Intra scene prediction method of depth image for interlayer video decoding and encoding apparatus and method
CN105165011A (en) Device and method for scalable video encoding considering memory bandwidth and computational quantity, and device and method for scalable video decoding
CN104541510A (en) Inter prediction method in which reference picture lists can be changed and apparatus for same
CN104396251A (en) Method for encoding multiview video using reference list for multiview video prediction and device therefor, and method for decoding multiview video using refernece list for multiview video prediction and device therefor
CN105325002A (en) Multi-viewpoint video encoding method using viewpoint synthesis prediction and apparatus for same, and multi-viewpoint video decoding method and apparatus for same
CN105340275A (en) Multi-view video encoding method using view synthesis prediction and apparatus therefor, and multi-view video decoding method and apparatus therefor
CN103875249A (en) Method for multiview video prediction encoding and device for same, and method for multiview video prediction decoding and device for same
CN105264890B (en) Cross-layer video coding method and equipment and cross-layer video coding/decoding method and equipment for compensation brightness difference

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant