CN104813663A

CN104813663A - Encoding device, encoding method, decoding device, and decoding method

Info

Publication number: CN104813663A
Application number: CN201380061119.3A
Authority: CN
Inventors: 佐藤数史; 陆硕
Original assignee: Sony Corp
Current assignee: Sony Corp
Priority date: 2012-09-28
Filing date: 2013-09-20
Publication date: 2015-07-29
Anticipated expiration: 2033-09-20
Also published as: CN108521574A; JP6432809B2; MX345239B; EP2899983A4; EP2899983B1; JP2018085757A; RU2015110063A; US20180270503A1; KR20150063379A; US10085038B2; US10499079B2; EP2899983A1; BR112015006313A2; RU2666305C1; CN108337524A; WO2014050748A1; US20150222921A1; RU2646340C2; JPWO2014050748A1; JP6274108B2

Abstract

The present technology pertains to an encoding device, encoding method, decoding device, and decoding method that enable the sharing or prediction of information pertaining to a reference image of an image having a multi-layered structure. An enhancement encoding unit sets reference-image-specification generation information used in generating reference-image-specifying information of an enhancement image using reference-image-specifying information as information pertaining to the reference image of a base image and reference-image-specifying information as information pertaining to the reference image used when encoding the enhancement image. The enhancement encoding unit encodes the enhancement image using the reference image, generating encoded data. A transmission unit transmits the encoded data and the reference-image-specification generation information. The present technology can, for example, be applied to an encoding device.

Description

Encoding device, coding method, decoding device and coding/decoding method

Technical field

This technology relates to encoding device, coding method, decoding device and coding/decoding method, more specifically, relates to and can share or predict and the encoding device of the information relevant with reference to image of the image with hierarchy, coding method, decoding device and coding/decoding method.

Background technology

In recent years, in order to digital image information, and at that time efficiently transmit and cumulative information, for broadcasting station etc. information transmit and average family in receives information, observe the peculiar redundancy such as wherein utilizing image information, by orthogonal transform and the motion compensation of such as discrete cosine transform (DCT) and so on, the equipment carrying out the Motion Picture Experts Group (MPEG) compressed and so on mode is all universal.

Especially, MPEG2 (ISO/IEC 13818-2) mode is defined as general image coded system, the current standard as covering horizontally interlaced image, progressive scanning picture, standard-resolution image and high-definition image, is widely used in the various application of professional purpose and consumer applications.Utilize MPEG2 mode, such as, by in the horizontally interlaced image situation of the standard resolution in 720 × 480 pixels, distribute the encoding amount (bit rate) of 4-8Mbps, with in the horizontally interlaced image situation of the high definition in 1920 × 1088 pixels, distribute the encoding amount of 18-22Mbps, high compression rate and good picture quality can be realized.

MPEG2 is mainly encoded to target with the high-quality being suitable for broadcasting, but the coded system not supporting encoding amount (bit rate) lower than the encoding amount (bit rate) of MPEG1, that is, the coded system of high compression rate.Along with popularizing of mobile terminal, think in not far future, can increase the demand of this coded system, thus MPEG4 coded system is standardized.As the international standard of MPEG4 Image Coding mode, have approved ISO/IEC 14496-2 in December, 1998.

In addition, in recent years, the standardization being such as initially the H.26L class standard of (ITU-T Q6/16 VCEG) designed for the Image Coding of video conference has been carried out.Known compared with such as MEPG2 or MPEG4 and so on coded system, H.26L need larger Code And Decode amount of calculation, but can high coding efficiency be realized.

In addition, in recent years, as one of MPEG 4 activity, to strengthen the form of compressed video coding united mode, carried out, based on H.26L, comprising the function H.26L do not supported, and realizing the standardization of high coding efficiency.H.264 or MPEG-4Part 10 (advanced video coding (AVC)) this standardization, in March, 2003, is approved for international standard.

In addition, as its expansion, in February, 2005, made to comprise such as RGB or YUV422 or YUV444 and so on the coding tools needed for business, and the fidelity range extension of the 8 × 8DCT specified in MPEG-2 and quantization matrix (FRExt) standardization.As a result, AVC mode has become the coded system that also can show the film noise be included in film, for the various application of such as BD (blue light (registered trade mark) CD) and so on.

But, in recent years, to the image that can compress about 4000 × 2000 pixels (4 times for high-definition image), or the demand of the high compression rate coding transmitting high-definition image in the limited environment of the transmission capacity of such as internet and so on constantly increases.For this reason, the Video Coding Experts group (VCEG) under ITU-T leader constantly examines the improvement of code efficiency.

At present, in order to improve code efficiency further, to realize the code efficiency higher than AVC, as the integration and cooperation team-Video coding (JCTVC) of the combination with standard tissue of ITU-T and ISO/IEC, carry out the standardization of the coded system being called efficient video coding (HEVC).In August, 2012, issue non-patent literature 1 as draft.

Meanwhile, the Image Coding mode of such as MPEG-2 and AVC and so on has the scalable function of layering and coded image.According to scalable function, the coded data corresponding to the handling property of side of decoding can be transmitted, and do not carry out transcoding process.

Particularly, such as, to the low terminal of such as mobile phone and so on disposal ability, the encoding stream of the image of Primary layer (serving as the layer on basis) can only be transmitted.Meanwhile, to the high terminal of such as television receiver or personal computer and so on disposal ability, the image of Primary layer can be transmitted, and the encoding stream of the image of enhancement layer (layer except Primary layer).

HEVC mode also has scalable function, in HEVC mode, except such as in non-patent literature 1 illustrate sequence parameter set (SPS) and picture parameter set (PPS) except, also define the video parameter collection (VPS) comprising the parameter relevant to scalable function.

Fig. 1 is the diagram of the illustration grammer of VPS in graphic extension HEVC version 1.

In HEVC version 1, owing to only providing according to frame rate, the scalable function (hereinafter referred to " time scalability ") of layering and coded image, therefore as shown in diagram in Fig. 1, in VPS, the parameter that only definition is relevant to time scalability.

In HEVC version 2, make a reservation for the standardization of the scalable function also supported except time scalability.

Quoted passage list

Non-patent literature

Non-patent literature 1:Benjamin Bross, Woo-Jin Han, Jens-Rainer Ohm, Gary J.Sullivan, Thomas Wiegand, " High efficiency video coding (HEVC) text specification draft 8 ", JCTVC-I1003_d7,2012.7.11-7.20.

Summary of the invention

When utilizing scalable function to encode, if support the image of Primary layer and the image of enhancement layer, be so considered to have high correlation to the every terms of information relevant with reference to image of two kinds of images.

But in the HEVC mode of prior art, owing to being every one deck, set the information relevant to reference image, therefore code efficiency is poor.

Described in above, create this technology, it is desirable to share or to predict the information relevant with reference to image to the image with hierarchy.

A kind of encoding device according to the encoding device of the first aspect of this technology, comprise setup unit, described setup unit utilizes the first reference image information of serving as the information relevant to the first reference image used when encoding the first scalable image of the image with hierarchy, second reference image information of the information relevant with the second reference image serving as image scalable to second, setting is used for the reference image generation information of the generation of the first reference image information, coding unit, described coding unit utilizes first with reference to image, to encode the first scalable image, thus generation coded data, and transmission unit, described transmission unit transmits the coded data generated by coding unit, with the reference image generation information set by setup unit.

The encoding device of the first aspect of this technology is corresponded to according to the coding method of the first aspect of this technology.

In the first aspect of this technology, utilize the first reference image information of serving as the information relevant to the first reference image used when encoding the first scalable image of the image with hierarchy, second reference image information of the information relevant with the second reference image serving as image scalable to second, setting is used for the reference image generation information of the generation of the first reference image information, utilize first with reference to image, to encode the first scalable image, thus generation coded data, and transmit described coded data and described reference image generation information.

A kind of decoding device according to the decoding device of the second aspect of this technology, comprise receiving element, described receiving element receives the first reference image information utilizing and serve as the information relevant to the first reference image used when encoding the first scalable image of the image with hierarchy, second of the information relevant with the second reference image serving as image scalable to second generates with reference to image information, for the reference image generation information of the generation of the first reference image information, with the coded data of the first scalable image, generation unit, described generation unit utilizes with reference to image generation information, generate first with reference to image information, and decoding unit, the first reference image information that described decoding unit generates according to generation unit, utilize first with reference to image, to decode the coded data of the first scalable image.

The decoding device according to the second aspect of this technology is corresponded to according to the coding/decoding method of the second aspect of this technology.

In the second aspect of this technology, receive the first reference image information utilizing and serve as the information relevant to the first reference image used when encoding the first scalable image of the image with hierarchy, second of the information relevant with the second reference image serving as image scalable to second generates with reference to image information, for the reference image generation information of the generation of the first reference image information, with the coded data of the first scalable image, utilize with reference to image generation information, generate first with reference to image information, with utilize first with reference to image, according to generate first with reference to image information, to decode the coded data of the first scalable image.

In addition, by making executive program, the encoding device of first aspect and the decoding device of second aspect can be realized.

In addition, for the decoding device of the encoding device and second aspect that realize first aspect and the program performed by computer by transmission medium transmission, or to be recorded in the recording medium.

The encoding device of first aspect and the decoding device of second aspect can be independently equipment, or can be the internal parts forming individual equipment.

According to this technology, can share or predict the information relevant with reference to image to the image with hierarchy.

Accompanying drawing explanation

Fig. 2 is the diagram that spatial scalability is described.

Fig. 3 is the diagram of description time scalability.

Fig. 4 is the diagram that SNR scalability is described.

Fig. 5 is the diagram of the overview of the first embodiment that this technology is described.

Fig. 6 is the block diagram of graphic extension according to the Exemplary structures of the encoding device of first embodiment of this technology.

Fig. 7 is the diagram of the illustration grammer of the SPS that graphic extension is flowed substantially.

Fig. 8 is the diagram of the illustration grammer of the SPS that graphic extension is flowed substantially.

Fig. 9 is the diagram of the illustration grammer of the section head that graphic extension is flowed substantially.

Figure 10 is the diagram of the illustration grammer of the section head that graphic extension is flowed substantially.

Figure 11 is the diagram of the illustration grammer of the section head that graphic extension is flowed substantially.

Figure 12 is the diagram of the illustration grammer of the RPS that graphic extension is flowed substantially.

Figure 13 is the block diagram of the Exemplary structures of the enhancing coding unit of graphic extension Fig. 6.

Figure 14 is the block diagram of the Exemplary structures of the coding unit of graphic extension Figure 13.

Figure 15 is the diagram that CU is described.

Figure 16 is the block diagram of the Exemplary structures with reference to image setting unit of graphic extension Figure 14.

Figure 17 is the diagram of the illustration grammer of the SPS that graphic extension is set by the setup unit of Figure 13.

Figure 18 is the diagram of the illustration grammer of the SPS that graphic extension is set by the setup unit of Figure 13.

Figure 19 is the diagram of the Exemplary structures of the grammer of the section head of graphic extension enhanced flow.

Figure 20 is the diagram of the Exemplary structures of the grammer of the section head of graphic extension enhanced flow.

Figure 21 is the diagram of the Exemplary structures of the grammer of the section head of graphic extension enhanced flow.

Figure 22 is the diagram of the illustration grammer of the RPS of graphic extension enhanced flow.

Figure 23 is the diagram of the illustration grammer of graphic extension VPS.

Figure 24 is the flow chart of the ges forschung process of the encoding device of key diagram 6.

Figure 25 is the flow chart of the details of the enhanced flow generating process that Figure 24 is described.

Figure 26 is the flow chart of the details of the coded treatment that Figure 25 is described.

Figure 27 is the flow chart of the details of the coded treatment that Figure 25 is described.

Figure 28 is the flow chart of the details of the generating process that Figure 26 is described.

Figure 29 is the block diagram of graphic extension according to the Exemplary structures of the decoding device of first embodiment of this technology.

Figure 30 is the block diagram of the Exemplary structures of the enhancing decoding unit of graphic extension Figure 29.

Figure 31 is the block diagram of the Exemplary structures of the decoding unit of graphic extension Figure 30.

Figure 32 is the block diagram of the Exemplary structures with reference to image setting unit of graphic extension Figure 31.

Figure 33 is the flow chart of the scalable decoding process of the decoding device that Figure 29 is described.

Figure 34 is the flow chart of the enhancing Computer image genration process that Figure 30 is described.

Figure 35 is the flow chart of the details of the decoding process that Figure 34 is described.

Figure 36 is the flow chart of the details of the generating process that Figure 35 is described.

Figure 37 is the diagram of the overview of the second embodiment that this technology is described.

Figure 38 is the diagram that weight estimation is described.

Figure 39 is the block diagram of graphic extension according to the Exemplary structures of the encoding device of second embodiment of this technology.

Figure 40 is the diagram of the illustration grammer of PPS in graphic extension AVC mode.

Figure 41 is the diagram of the illustration grammer of PPS in graphic extension AVC mode.

Figure 42 is the diagram of the illustration grammer of the PPS that graphic extension is flowed substantially.

Figure 43 is the diagram of the illustration grammer of the PPS that graphic extension is flowed substantially.

Figure 44 is the diagram of the illustration grammer of section head in graphic extension AVC mode.

Figure 45 is the diagram of the illustration grammer of section head in graphic extension AVC mode.

Figure 46 is the diagram of the illustration grammer of weighted information in graphic extension AVC mode.

Figure 47 is the diagram of the illustration grammer of the weighted information that graphic extension is flowed substantially.

Figure 48 is the block diagram of the Exemplary structures of the enhancing coding unit of graphic extension Figure 39.

Figure 49 is the block diagram of the Exemplary structures of the coding unit of graphic extension Figure 48.

Figure 50 is the block diagram of the weight buffer of graphic extension Figure 49 and the Exemplary structures of weight setting unit.

Figure 51 is the diagram of the illustration grammer of the weighted information of graphic extension enhanced flow.

Figure 52 is the diagram of the illustration grammer of the weighted information of graphic extension enhanced flow.

Figure 53 is the flow chart of the enhanced flow generating process that Figure 48 is described.

Figure 54 is the flow chart of the details of the coded treatment that Figure 53 is described.

Figure 55 is the flow chart of the details of the coded treatment that Figure 53 is described.

Figure 56 is the flow chart of the details of the generating process that Figure 54 is described.

Figure 57 is the block diagram of graphic extension according to the Exemplary structures of the decoding device of second embodiment of this technology.

Figure 58 is the block diagram of the Exemplary structures of the enhancing decoding unit of graphic extension Figure 57.

Figure 59 is the block diagram of the Exemplary structures of the decoding unit of graphic extension Figure 58.

Figure 60 is the block diagram of the weight buffer of graphic extension Figure 59 and the Exemplary structures of weight setting unit.

Figure 61 is the flow chart of the scalable decoding process of the decoding device that Figure 57 is described.

Figure 62 is the flow chart of the details of the decoding process of the decoding unit that Figure 59 is described.

Figure 63 is the flow chart of the details of the generating process that Figure 62 is described.

Figure 64 is the diagram of the overview of the 3rd embodiment that this technology is described.

Figure 65 is the block diagram of graphic extension according to the Exemplary structures of the encoding device of the 3rd embodiment of this technology.

Figure 66 is the block diagram of the Exemplary structures of the enhancing coding unit of graphic extension Figure 65.

Figure 67 is the block diagram of the Exemplary structures of the coding unit of graphic extension Figure 66.

Figure 68 is the diagram of the illustration grammer of the SPS that graphic extension is set by the setup unit of Figure 66.

Figure 69 is the diagram of the illustration grammer of the section head of enhanced flow.

Figure 70 is the diagram of the effect illustrated in encoding device.

Figure 71 is the flow chart of the ges forschung process of the encoding device that Figure 65 is described.

Figure 72 is the flow chart of the details of the SPS setting process illustrated in the ges forschung process of Figure 71.

Figure 73 is the flow chart of the details of the never mark setting process illustrated in the enhancing coded treatment of Figure 71.

Figure 74 is the block diagram of graphic extension according to the Exemplary structures of the decoding device of third embodiment of the present disclosure.

Figure 75 is the block diagram of the Exemplary structures of the enhancing decoding unit of graphic extension Figure 74.

Figure 76 is the block diagram of the Exemplary structures of the decoding unit of graphic extension Figure 75.

Figure 77 is the flow chart of the enhancing Computer image genration process of the enhancing decoding unit that Figure 74 is described.

Figure 78 is the flow chart of the details of the SPS extraction process that Figure 77 is described.

Figure 79 is the flow chart of the details of the generating process of the enhancing decoding process that Figure 77 is described.

Figure 80 is illustrated in the diagram of the illustration grammer of the SPS when the setting of another never mark.

Figure 81 is illustrated in the diagram of the illustration grammer of the section head of the enhanced flow when the setting of another never mark.

Figure 82 is the flow chart of the SPS setting process illustrated when the setting of another never mark.

Figure 83 is the flow chart of the never mark setting process illustrated when the setting of another never mark.

Figure 84 is the flow chart of the SPS extraction process illustrated when the setting of another never mark.

Figure 85 is the flow chart of the generating process illustrated when the setting of another never mark.

Figure 86 is the diagram of the illustration grammer of the expansion of VPS in graphic extension the 3rd embodiment.

Figure 87 is illustrated in the diagram of the illustration grammer of the SPS when set model utilizes.

Figure 88 is illustrated in the diagram of the illustration grammer of the section head of the enhanced flow when set model utilizes.

Figure 89 is the flow chart of the SPS setting process illustrated when set model utilizes.

Figure 90 is the flow chart of the never mark setting process illustrated when set model utilizes.

Figure 91 is the flow chart of the SPS extraction process illustrated when set model utilizes.

Figure 92 is the flow chart of the generating process illustrated when set model utilizes.

The diagram of the illustration grammer of the SPS when Figure 93 is illustrated in based on the public never mark setting of coded system.

Figure 94 is the block diagram of graphic extension according to the Exemplary structures of the encoding device of the 4th embodiment of this technology.

Figure 95 is the block diagram of the Exemplary structures of the enhancing coding unit of graphic extension Figure 94.

Figure 96 is the block diagram of the Exemplary structures of the coding unit of graphic extension Figure 95.

Figure 97 is the diagram of the illustration grammer of the SPS that graphic extension is set by the setup unit of Figure 95.

Figure 98 is the diagram of the illustration grammer of the section head of graphic extension enhanced flow.

Figure 99 is the diagram of graphic extension for the illustration grammer of the RPS of part RPS never mark.

Figure 100 is the flow chart of the SPS setting process of the encoding device that Figure 94 is described.

Figure 101 is the flow chart of the details of the never mark setting process of the encoding device that Figure 94 is described.

Figure 102 is the block diagram of graphic extension according to the Exemplary structures of the decoding device of fourth embodiment of the present disclosure.

Figure 103 is the block diagram of the Exemplary structures of the enhancing decoding unit of graphic extension Figure 102.

Figure 104 is the block diagram of the Exemplary structures of the decoding unit of graphic extension Figure 103.

Figure 105 is the flow chart of the details of the SPS extraction process of the decoding device that Figure 102 is described.

Figure 106 is the flow chart of the details of the generating process of the decoding device that Figure 102 is described.

Figure 107 is illustrated in the diagram of the illustration grammer of the RPS when public RPS sets.

Figure 108 is illustrated in the flow chart of the SPS setting process when public RPS sets.

Figure 109 is the flow chart of the details of the never mark setting process illustrated when public RPS setting.

Figure 110 is the flow chart of the SPS extraction process illustrated when public RPS setting.

Figure 111 is the diagram of the multi-view image coded system of graphic extension illustration.

Figure 112 graphic extension utilizes the another kind of illustration coding of scalable function.

Figure 113 is the block diagram of the illustration hardware configuration of graphic extension computer.

Figure 114 is the diagram of graphic extension according to the illustration schematic construction of the television set of this technology.

Figure 115 is the diagram of graphic extension according to the illustration schematic construction of the mobile phone of this technology.

Figure 116 is the diagram of graphic extension according to the illustration schematic construction of the recording/reproducing apparatus of this technology.

Figure 117 is the diagram of graphic extension according to the illustration schematic construction of the imaging device of this technology.

Figure 118 is the block diagram utilizing example of graphic extension ges forschung.

Figure 119 is the block diagram that another of graphic extension ges forschung utilizes example.

Figure 120 is the block diagram that another of graphic extension ges forschung utilizes example.

Embodiment

The explanation > of the scalable function of <

(explanation of spatial scalability)

Fig. 2 is the diagram that spatial scalability is described.

As illustrated in Figure 2 shows, spatial scalability is according to spatial resolution, image is carried out to the scalable function of layering and coding.Particularly, in spatial scalability, low-resolution image is encoded into Primary layer image, and the difference image between high-definition picture and low-resolution image is encoded into enhancement layer image.

Thus because encoding device only sends the low decoding device of disposal ability to the coded data of Primary layer image, therefore this decoding device can generate low-resolution image.In addition, because encoding device sends the high decoding device of disposal ability to the coded data of Primary layer image and enhancement layer image, therefore by decoding and synthesizing Primary layer image and enhancement layer image, this decoding device can generate high-definition picture.

(explanation of time scalability)

Fig. 3 is the diagram of description time scalability.

As mentioned above, time scalability is according to frame rate, image is carried out to the scalable function of layering and coding.Particularly, as shown in diagram in figure 3, in time scalability, the image of low frame rate (in the example in figure 3,7.5fps) is encoded into Primary layer image.Difference image between the image of intermediate frame speed (in the example in figure 3,15fps) and the image of low frame rate is encoded into enhancement layer image.Difference image between the image of high frame rate (in the example in figure 3,30fps) and the image of intermediate frame speed is encoded into enhancement layer image.

Thus because encoding device only sends the low decoding device of disposal ability to the coded data of Primary layer image, therefore this decoding device can generate the image of low frame rate.In addition, because encoding device sends the high decoding device of disposal ability to the coded data of Primary layer image and enhancement layer image, therefore by decoding and synthesizing Primary layer image and enhancement layer image, this decoding device can generate the image of high frame rate or intermediate frame speed.

(explanation of SNR scalability)

Fig. 4 is the diagram that SNR scalability is described.

As shown in diagram in Fig. 4, SNR scalability is according to signal to noise ratio (SNR), image is carried out to the scalable function of layering and coding.Particularly, in SNR scalability, the image of low SNR is encoded into Primary layer image, and the difference image between the image of the image of high SNR and low SNR is encoded into enhancement layer image.

Thus because encoding device only sends the low decoding device of disposal ability to the coded data of Primary layer image, therefore this decoding device can generate the image of low SNR, that is, low-quality image.In addition, because encoding device sends the high decoding device of disposal ability to the coded data of Primary layer image and enhancement layer image, therefore by decoding and synthesizing Primary layer image and enhancement layer image, this decoding device can generate the image of high SNR, that is, high quality graphic.

In addition, although not shown, but except spatial scalability, time scalability and SNR scalability, also there is other scalable function.

Such as, as scalable function, also exist according to bit number, image is carried out to the bit-depth scalability of layering and coding.In this case, such as, 8 bit video images are encoded into Primary layer image, and the difference between 10 bit video images and 8 bit video images is encoded into enhancement layer image.

In addition, as scalable function, also there is the form according to carrier chrominance signal, image is carried out to the colourity scalability of layering and coding.In this case, such as, YUV 420 image is encoded into Primary layer image, and the difference image between YUV 422 image and YUV 420 image is encoded into enhancement layer image.

For convenience of explanation, the number going on to say wherein enhancement layer is below the example of 1.

< first embodiment >

(explanation of the overview of the first embodiment)

As shown in diagram in Fig. 5, in a first embodiment, except between identical layer, share or predict the information (hereinafter referred to " with reference to image appointed information ") of the appointment reference image serving as the information relevant to reference image between the different layers.

(Exemplary structures of the first embodiment of encoding device)

The encoding device 10 of Fig. 6 comprises basic coding unit 11, strengthens coding unit 12, synthesis unit 13 and transmission unit 14, utilizes scalable function, according to the mode observing HEVC mode, and coded image.

Primary layer image (hereinafter referred to " primary image ") is transfused to the basic coding unit 11 of encoding device 10 from outside.Basic coding unit 11 is that the encoding device of the HEVC mode being similar to prior art is formed, according to HEVC mode, and encoded base image.Here, basic coding unit 11 specify use when encoded base image be supplied to enhancing coding unit 12 with reference to image with reference to image appointed information.Basic coding unit 11 is the encoding stream comprising the coded data obtained by coding, and SPS, PPS etc., be supplied to synthesis unit 13 as basic stream.

Enhancement layer image (hereinafter referred to " enhancing image ") is transfused to enhancing coding unit 12 from outside.Strengthen coding unit 12 according to the mode observing HEVC mode, coding strengthens image.In addition, strengthen the reference image appointed information that coding unit 12 utilizes primary image, with the reference image appointed information with reference to image used when encoding and strengthening image, what generate the generation with reference to image appointed information for strengthening image specifies information generated (with reference to image generation information) with reference to image.

Strengthen coding unit 12 and specify information generated etc. and the coded data strengthening image by being added with reference to image, generate encoding stream, and the encoding stream generated, be supplied to synthesis unit 13 as enhanced flow.

Synthesis unit 13 synthesizes the basic stream supplied from basic coding unit 11 and the enhanced flow supplied from enhancing coding unit 12, adds VPS etc., thus generates the encoding stream of all layers.Synthesis unit 13 is supplied to transmission unit 14 the encoding stream of all layers.

Transmission unit 14 sends the encoding stream of all layers supplied from synthesis unit 13 decoding device illustrated to below.

Here, encoding device 10 transmits the encoding stream of all layers, but if desired, can only transmit basic stream.

(the illustration grammer of the SPS of basic stream)

Fig. 7 and 8 is diagrams that graphic extension is included in the illustration grammer of the SPS in basic stream.

If the 9th row at Fig. 8 is to shown in diagram in the 11st row, the SPS of basic stream comprises the information relevant with reference to image set (RPS) with reference to image appointed information to the short-term of the image sets (GOP) of serving as corresponding to SPS.Particularly, be included in the number (num_short_term_ref_pic_sets) of the RPS in SPS, and RPS (short_term_ref_pic_set) is included in SPS.The index provided from 0 order is assigned to RPS.

In addition, as shown in diagram in the 12nd row to the 17th row, in SPS, record and the long-term information relevant with reference to image appointed information.Particularly, indicate long-term with reference to image whether can long-term mark (long_term_ref_pics_present_flag) be included in SPS.

In addition, when being labeled as 1 for a long time, when indicating long-term reference image available, the long-term number (num_long_term_ref_pics_sps) with reference to image appointed information be included in SPS is included in SPS.In addition, comprise and serve as long-term with reference to image appointed information, indicate the figure sequence with reference to image to count the information (lt_ref_pic_poc_lsb_sps) of the least significant bit of (POC).In addition, whether record instruction utilizes the reference image of specifying with reference to image appointed information by the reference marker of self reference (used_by_curr_pic_lt_sps_flag).Long-term reference image appointed information is by the index sequentially given from 0.

Below, when distinguishing RPS and long-term reference image appointed information especially, they are referred to simply as " with reference to image appointed information ".

(the illustration grammer of the section head of basic stream)

Fig. 9-11 is that graphic extension is served as in units of cutting into slices, and adds the diagram of the illustration grammer of the section head of the head in the coded data be included in basic stream.

As shown in diagram in the 18th row of Fig. 9, the section head of basic stream comprises RPS mark (short_term_ref_pic_set_sps_flag) that the corresponding RPS cut into slices of instruction is included in the RPS in SPS.

In addition, as shown in diagram in the 19th row and the 20th row, when RPS is labeled as 0, when the RPS of the corresponding section of instruction is not included in the RPS in SPS, the RPS of corresponding section is comprised in section head as the RPS (short_term_ref_pic_set (num_short_term_ref_pic_sets)) that index is num_short_term_ref_pic_sets.

As shown in diagram in the 21st row and the 22nd row, when RPS is labeled as 1, when the RPS of the corresponding section of instruction is included in the RPS in SPS, the index (short_term_ref_pic_set_idx) of the RPS of corresponding section is included in section head.

In addition, as as shown in the 23rd row to the 26th row, when be included in corresponding SPS be labeled as 1 for a long time time, section head comprises the inner number (num_long_term_sps) of SPS of serving as the long-term number with reference to image appointed information be included in SPS, and serves as the SPS external number (num_long_term_pics) of the long-term number with reference to image appointed information be included in section head.

In addition, as shown in the 27th row to the 29th row, section head comprises among the long-term reference image appointed information in corresponding section, is included in the long-term index (lt_idx_sps) with reference to image appointed information in SPS.In addition, as as shown in the 30th row to the 32nd row, section head comprises as among the long-term reference image appointed information in correspondence section, be included in long-term in SPS with reference to image appointed information, instruction is with reference to the information (poc_lsb_sps) of the least significant bit of the POC of image and reference marker (used_by_curr_pic_lt_flag).

(the illustration grammer of the RPS of basic stream)

As shown in diagram in the 2nd row of Figure 12, RPS comprises inter_ref_pic_set_prediction_flag.Inter_ref_pic_set_prediction_flag indicates according to coded sequence, the reference information with reference to image appointed information whether being used as current image to be encoded with reference to image appointed information at front image before the current encoded image in the GOP of current image to be encoded.

When being used as the reference image appointed information of current image to be encoded in the reference image appointed information of front image, described is 1 with reference to information, and when being not used as the reference image appointed information of current image to be encoded in the reference image appointed information of front image, described is 0 with reference to information.

If the 3rd row at Figure 12 is to shown in diagram in the 5th row, when inter_ref_pic_set_prediction_flag is 1, RPS comprise specify in front image front image appointed information (delta_idx_minus1).Particularly, delta_idx_minus1 is from by numbering (coded sequence) from the coding of current encoded image, deducts in the value obtained in the coding numbering of front image, deducts the value of 1 and acquisition.Here, coding numbering is as coded sequence, gives the numbering of each image in GOP from less value.

In addition, as shown in diagram in the 6th row and the 7th row of Figure 12, RPS is included in the symbol (delta_rps_sign) with reference to the difference between image appointed information (POC) and reference image appointed information (POC) of current image to be encoded of front image, and the absolute value of described difference (abs_delta_rps_minus1).

In addition, as shown in diagram in the eighth row and the 9th row of Figure 12, RPS comprises the mark (used_by_curr_pic_lt_flag) with reference to image indicating whether to use and utilize and specify with reference to image appointed information.In addition, as shown in diagram in the 10th row and the 11st row, when mark (used_by_curr_pic_lt_flag) is 0, instruction do not use utilize with reference to image appointed information specify with reference to image time, in RPS, comprise instruction whether be comprised in mark (use_delta_flag) in RPS with reference to image.

In addition, if the 14th row at Figure 12 is to shown in diagram in the 23rd row, when inter_ref_pic_set_prediction_flag is 0, the information etc. of the number with reference to image, instruction POC in RPS, is comprised.

(strengthening the Exemplary structures of coding unit)

Figure 13 is the block diagram of the Exemplary structures of the enhancing coding unit 12 of graphic extension Fig. 6.

The enhancing coding unit 12 of Figure 13 comprises coding unit 21 and setup unit 22.

The coding unit 21 strengthening coding unit 12 receives as input signal, from the enhancing image of the frame unit of outside input.Coding unit 21 is with reference to the reference image appointed information provided from basic coding unit 11, and what provide from setup unit 22 specifies information generated etc. with reference to image, according to the mode observing HEVC mode, and coded input signal.Coding unit 21 is supplied to setup unit 22 the coded data obtained as a result.

Setup unit 22 sets specifies information generated with reference to image.The index that order provides from 0 is assigned to the reference image appointed information of specifying information generated with reference to image, or the difference of reference image appointed information.Setup unit 22 specifies information generated to be supplied to coding unit 21 with reference to image.In addition, setup unit 22 setting comprises SPS, PPS etc. of specifying information generated with reference to image.

Setup unit 22, according to SPS, PPS of setting and the coded data from coding unit 21 supply, generates encoding stream, and using the encoding stream generated as enhanced flow, is supplied to synthesis unit 13.

(Exemplary structures of coding unit)

Figure 14 is the block diagram of the Exemplary structures of the coding unit 21 of graphic extension Figure 13.

The coding unit 21 of Figure 14 comprises A/D converter 31, screen reorder buffer 32, arithmetic element 33, orthogonal transform unit 34, quantifying unit 35, lossless encoding unit 36, accumulation buffer 37, inverse quantization unit 38, inverse orthogonal transformation unit 39, adder unit 40, deblocking filter 41, self adaptation offset filter 42, auto-adaptive loop filter 43, frame memory 44, switch 45, intraprediction unit 46, motion prediction/compensating unit 47, predicted picture selected cell 48, with reference to buffer 49, with reference to image setting unit 50 and Rate control unit 51.

Particularly, the A/D converter 31 of coding unit 21 carries out the A/D conversion of the image of the frame unit as input signal input, and exports the image after conversion, to be kept in screen reorder buffer 32.Screen reorder buffer 32, according to gop structure, resets the image pressing the frame unit of DISPLAY ORDER preserved by coded sequence, and the image reset is exported to arithmetic element 33, intraprediction unit 46 and motion prediction/compensating unit 47.

Arithmetic element 33 plays coding unit, by calculate from predicted picture selected cell 48 supply predicted picture, and from screen reorder buffer 32 export current encoded image between difference, encode.Particularly, arithmetic element 33, by from the current encoded image output from screen reorder buffer 32, deducts the predicted picture supplied from predicted picture selected cell 48, encodes.Arithmetic element 33 as residual information, exports to orthogonal transform unit 34 using the image obtained as a result.In addition, when not from predicted picture selected cell 48 availability forecast image, arithmetic element 33 as residual information, exports to orthogonal transform unit 34 using the image read from screen reorder buffer 32 steadily.

Orthogonal transform unit 34 carries out orthogonal transform to the residual information provided from arithmetic element 33, and the orthogonal transform coefficient generated is supplied to quantifying unit 35.

Quantifying unit 35 quantizes the orthogonal transform coefficient supplied from orthogonal transform unit 34, and the coefficient obtained as a result is supplied to lossless encoding unit 36.

Lossless encoding unit 36, from intraprediction unit 46, obtains the information (hereinafter referred to " intraprediction mode information ") of instruction optimum frame inner estimation mode.In addition, lossless encoding unit 36, from motion prediction/compensating unit 47, obtains the information (hereinafter referred to " inter-frame forecast mode information "), motion vector etc. of the best inter-frame forecast mode of the instruction supplied from motion prediction/compensating unit 47.In addition, lossless encoding unit 36, from reference to image setting unit 50, obtains and specifies information generated, RPS mark etc. with reference to image.

In addition, lossless encoding unit 36 obtains the skew filtering information relevant to offset filter from self adaptation offset filter 42, and from auto-adaptive loop filter 43, obtains filter factor.

Lossless encoding unit 36 is to the quantization parameter supplied from quantifying unit 35, carry out lossless coding, such as variable-length encoding (such as, context-adaptive transition coding (CAVLC)), or arithmetic coding (such as, context adaptive binary arithmetic coding (CABAC).

In addition, lossless encoding unit 36 pairs of intraprediction mode information or inter-frame forecast mode information, motion vector, reference image appointment information generated, RPS mark, skew filtering information and filter factor carry out lossless coding, as the coded message relevant to coding.Lossless encoding unit 36 is set as the coded message of lossless coding head of cutting into slices, and the coefficient of lossless coding is set as coded data, and section head is added in coded data.Lossless encoding unit 36 supplies the coded data with section head, to be accumulated in accumulation buffer 37.

The coded data supplied from lossless encoding unit 36 preserved by accumulation buffer 37 temporarily.In addition, accumulation buffer 37 is supplied to the coded data of preserving the setup unit 22 of Figure 13.

In addition, the quantization parameter exported from quantifying unit 35 is also provided to inverse quantization unit 38.The coefficient that inverse quantization unit 38 pairs of quantifying unit 35 quantize, carries out re-quantization, and the orthogonal transform coefficient obtained as a result is supplied to inverse orthogonal transformation unit 39.

Inverse orthogonal transformation unit 39, to the orthogonal transform coefficient supplied from inverse quantization unit 38, carries out inverse orthogonal transformation, and the residual information obtained as a result is supplied to adder unit 40.

Adder unit 40 is added the residual information supplied from inverse orthogonal transformation unit 39, and from the predicted picture that predicted picture selected cell 48 supplies, thus obtain local decoder image.In addition, when not from predicted picture selected cell 48 availability forecast image, adder unit 40 is considered as local decoder image the residual information supplied from inverse orthogonal transformation unit 39.Adder unit 40 is supplied to deblocking filter 41 local decoder image, and provides local decoder image, to be accumulated in frame memory 44.

Deblocking filter 41, to the local decoder image supplied from adder unit 40, carries out the adaptive de-blocking filtering process eliminating block distortion, and the image obtained as a result is supplied to self adaptation offset filter 42.

Self adaptation offset filter 42, to the image of adaptive de-blocking filtering process experiencing deblocking filter 41, mainly eliminates self adaptation skew filtering (sampled point self adaptation skew (the SAO)) process of ring.

Particularly, self adaptation offset filter 42 determines the kind of the self adaptation skew filtering process for each the maximum coding unit (LCU) as maximum coding unit, and obtains the side-play amount used in self adaptation skew filtering process.Self adaptation offset filter 42 utilizes the side-play amount obtained, and to the image through adaptive de-blocking filtering process, carries out the self adaptation skew filtering process determining kind.Subsequently, self adaptation offset filter 42 is supplied to auto-adaptive loop filter 43 the image through self adaptation skew filtering process.

In addition, self adaptation offset filter 42 comprises the buffer preserving side-play amount.For each LCU, self adaptation offset filter 42 judges whether the side-play amount being used for adaptive de-blocking filtering process has been stored in described buffer.

When the side-play amount judging to be used for adaptive de-blocking filtering process has been stored in described buffer, self adaptation offset filter 42 is saved storage mark in a buffer instruction side-play amount, be set to that the described side-play amount of instruction is saved value (, 1) in a buffer here.

Subsequently, in units of LCU, self adaptation offset filter 42, the storage mark being set to 1, indicates the index of side-play amount memory location in a buffer, indicate the kind of information of the kind of the self adaptation skew filtering process carried out, be supplied to lossless encoding unit 36 as skew filtering information.

Meanwhile, when the side-play amount for adaptive de-blocking filtering process is not stored in described buffer, self adaptation offset filter 42 is preserved side-play amount in a buffer.In addition, storage mark, self adaptation offset filter 42 is set to that instruction side-play amount is not saved value (, 0) in a buffer here.Subsequently, in units of LCU, self adaptation offset filter 42 is the storage mark being set to 0, and side-play amount and described kind of information, be supplied to lossless encoding unit 36 as skew filtering information.

Such as, auto-adaptive loop filter 43 is made up of TWO DIMENSIONAL WIENER FILTER.In units of LCU, auto-adaptive loop filter 43, to the image through self adaptation skew filtering process supplied from self adaptation offset filter 42, carries out adaptive loop filter (ALF) process.

Particularly, in units of LCU, auto-adaptive loop filter 43 calculates the filter factor used in adaptive loop filter process, and to cause the original image as the image exported from screen reorder buffer 32, and the residual error between the image of adaptive loop filter process is down to minimum.Subsequently, in units of LCU, auto-adaptive loop filter 43, by utilizing the filter factor calculated, to the image through self adaptation skew filtering process, carries out adaptive loop filter process.

Auto-adaptive loop filter 43 is supplied to frame memory 44 the image through adaptive loop filter process.In addition, auto-adaptive loop filter 43 is supplied to lossless encoding unit 36 filter factor.

Here, although supposition is in units of LCU, carry out adaptive loop filter process, but, the process unit of adaptive loop filter process is not limited to LCU.Here, mated by the process unit of the process unit and auto-adaptive loop filter 43 that make self adaptation offset filter 42, can process efficiently.

Frame memory 44 accumulates the image supplied from auto-adaptive loop filter 43, and from the image that adder unit 40 supplies.Be accumulated in image in frame memory 44 by switch 45, be exported to intraprediction unit 46 or motion prediction/compensating unit 47, as with reference to image.

Intraprediction unit 46, by utilizing through switch 45, from the reference image that frame memory 44 reads, carries out the intra-prediction process of all intra prediction modes serving as candidate.

In addition, intraprediction unit 46 according to the image read from screen reorder buffer 32, and by the predicted picture that intra-prediction process generates, calculates the cost function value (details is illustrating) of all intra prediction modes of alternatively person below.Subsequently, intraprediction unit 46, intra prediction mode minimum for its cost function value, is defined as optimum frame inner estimation mode.

Intraprediction unit 46 is the predicted picture generated according to optimum frame inner estimation mode, and the cost function value of correspondence is supplied to predicted picture selected cell 48.When sending the notice of selection of the predicted picture generated by optimum frame inner estimation mode from predicted picture selected cell 48, intraprediction unit 46 is supplied to lossless encoding unit 36 intraprediction mode information.

In addition, cost function value is also referred to as rate distortion (RD) cost, and is according to the technique computes utilizing one of the high complexity pattern and low-complexity mode determined as the conjunctive model (JM) of the reference software in H.264/AVC mode.In addition, the reference software H.264/AVC in mode at http://iphome.hhi.de/suehring/tml/index.htm to public.

Particularly, when adopting high complexity pattern as cost function value computing technique, assuming that for all predictive modes of alternatively person, be performed until decoding, and for various predictive mode, calculate the cost function value represented with following formula (1).

[mathematical expression 1]

Cost (pattern)=D+ λ R... (1)

D represents the difference (distortion) between original image and decoded picture, and R represents and comprises until the generation encoding amount of orthogonal transform coefficient, and λ represents the Lagrangian undertermined multiplier that the function as quantization parameter QP provides.

Meanwhile, when adopting low-complexity mode as cost function value computing technique, to all predictive modes of alternatively person, carry out the generation of predicted picture, with the calculating of the encoding amount of coded message, and to various predictive mode, calculate the cost function represented with following formula (2).

Cost (pattern)=D+QPtoQuant (QP) Header_Bit... (2)

[mathematical expression 2]

Cost (pattern)=D+QPtoQuant (QP) Header_Bit... (2)

D indicates the difference (distortion) between original image and decoded picture, and Header_Bit indicates the encoding amount of coded message, the function that QPtoQuant indicates the function as quantization parameter QP to provide.

Under low-complexity mode, due to for all predictive modes, need only generation forecast image, and decoded picture need not be generated, thus amount of calculation is less.

Motion prediction/compensating unit 47 carries out the motion prediction/compensation deals of all inter-frame forecast modes of alternatively person.Particularly, motion prediction/compensating unit 47 according to the image supplied from screen reorder buffer 32, and by switch 45, from the reference image that frame memory 44 reads, detects the motion vector of all inter-frame forecast modes of alternatively person.Be set by the user for example, referring to image.Motion prediction/compensating unit 47, according to the motion vector detected, compensates process to reference to image, thus generation forecast image.

Now, motion prediction/compensating unit 47 is according to the image supplied from screen reorder buffer 32, and predicted picture, calculate the cost function value of all inter-frame forecast modes of alternatively person, inter-frame forecast mode minimum for its cost function value is defined as best inter-frame forecast mode.Subsequently, motion prediction/compensating unit 47 is the cost function value of best inter-frame forecast mode, and the predetermined image of correspondence is supplied to predicted picture selected cell 48.In addition, when sending the notice of selection of the predicted picture that fortune generates by best inter-frame forecast mode from predicted picture selected cell 48, motion prediction/compensating unit 47 is inter-frame forecast mode information, corresponding motion vector etc. exports to lossless encoding unit 36, and exports to reference to image setting unit 50 with reference to image appointed information.

Predicted picture selected cell 48, according to the cost function value supplied from intraprediction unit 46 and motion prediction/compensating unit 47, less for corresponding cost function value one of optimum frame inner estimation mode and best inter-frame forecast mode, is defined as optimum prediction mode.Subsequently, predicted picture selected cell 48 is supplied to arithmetic element 33 and adder unit 40 the predicted picture of optimum prediction mode.In addition, predicted picture selected cell 48, the selection of the predicted picture of optimum prediction mode, informs intraprediction unit 46 or motion prediction/compensating unit 47.

Preserve with reference to buffer 49 and supply from the basic coding unit 11 of Fig. 6, the reference image appointed information with reference to image used when encoded base image.

Reference image setting unit 50 compares the reference image appointed information supplied from motion prediction/compensating unit 47 and the reference image appointed information be kept at reference buffer 49, and determines the predictive mode with reference to image appointed information of enhancing image.Here, assuming that the predictive mode with reference to image appointed information comprises replication mode, difference prediction pattern and nonanticipating pattern.

Replication mode refers to the predictive mode with reference to image appointed information being used as to strengthen image with reference to image appointed information of the reference layer of wherein serving as another layer (, Primary layer) treating reference here.Difference prediction pattern refers to wherein by being added the difference between reference image appointed information and the reference image appointed information of reference layer strengthening image, with the reference image appointed information of reference layer, prediction strengthens the predictive mode with reference to image appointed information of image.Nonanticipating pattern refers to and wherein independently sets with the reference image appointed information of reference layer the predictive mode with reference to image appointed information strengthening image.

When the predictive mode with reference to image appointed information is replication mode, if the predictive mode with reference to image appointed information served as with reference to image appointment information generated supplied from the setup unit 22 of Figure 13 is replication mode, be so supplied to lossless encoding unit 36 with reference to image setting unit 50 serve as RPS mark 1.Simultaneously, if the predictive mode with reference to image appointed information served as with reference to image appointment information generated supplied from setup unit 22 is not replication mode, so serve as RPS mark 0 is provided to lossless encoding unit 36, replication mode is configured to specify information generated with reference to image, and is provided to lossless encoding unit 36.

In addition, when the predictive mode of reference image appointed information is difference prediction pattern, the difference between reference image appointed information and the reference image appointed information of primary image strengthening image is calculated with reference to image setting unit 50.Subsequently, compare the calculating difference with reference to image appointed information with reference to image setting unit 50, with the difference of serving as the reference image appointed information of specifying information generated with reference to image supplied from setup unit 22.

Subsequently, when two difference are mutually the same, identify corresponding index with reference to image setting unit 50, be supplied to lossless encoding unit 36 serve as RPS mark 1, index is set as specifying information generated with reference to image, then specifies information generated to be supplied to lossless encoding unit 36 with reference to image.

Simultaneously, when two difference are differing from each other, lossless encoding unit 36 is supplied to serve as RPS mark 0 with reference to image setting unit 50, calculating difference with reference to image appointed information is set as specifying information generated with reference to image, then specifies information generated to be supplied to lossless encoding unit 36 with reference to image.

In addition, when the predictive mode with reference to image appointed information is non-predictive mode, reference image setting unit 50 compares the reference image appointed information of enhancing image, and from the reference image appointed information of serving as with reference to image appointment information generated that setup unit 22 supplies.Subsequently, when these two identical with reference to image appointed information, corresponding index is identified with reference to image setting unit 50, lossless encoding unit 36 is supplied to serve as RPS mark 1, index is set as specifying information generated with reference to image, then specifies information generated to be supplied to lossless encoding unit 36 with reference to image.

Simultaneously, when these two are not identical with reference to image appointed information, lossless encoding unit 36 is supplied to when 0 of RPS mark with reference to image setting unit 50, strengthening being set to reference to image appointment information generated with reference to image appointed information of image, then information generated is specified to be supplied to lossless encoding unit 36 with reference to image.

Rate control unit 51, according to the coded data be accumulated in accumulation buffer 37, controls the speed of the quantization operation of quantifying unit 35, so that overflow or underflow can not occur.

(explanation of coding processing unit)

Figure 15 is the diagram of the coding unit (CU) that the coding unit served as in HEVC mode is described.

In HEVC mode, because the image of the large picture frame of the ultrahigh resolution (UHD) and so on of such as 4000 × 2000 pixels is also object, therefore the size of coding unit is fixed as 16 × 16 pixels and non-optimal.Thus in HEVC mode, CU is defined as coding unit.

CU is also referred to as code tree block (CTB), bears the task identical with the macro block in AVC mode.Particularly, CU is divided into the predicting unit (PU) of the unit serving as infra-frame prediction or inter prediction, or serves as the converter unit (TU) of unit of orthogonal transform.Here, the size of CU is the square for each sequence variation, represents by the pixel of the power for 2.In addition, at present, in HEVC mode, as the size of TU, not only can use 4 × 4 pixels and 8 × 8 pixels, and 16 × 16 pixels and 32 × 32 pixels can be used.

In the example of Figure 15, the size of serving as the maximum coding unit (LCU) of the maximum CU of size is 128, and the size of serving as the minimum code unit (SCU) of the minimum CU of size is 8.Thus the depth of seam division of wherein carrying out the CU of 2N × 2N size of layering in units of N is 0-4, and depth of seam division number is 5.In addition, when the value of split_flag is 1, the CU of 2N × 2N size is divided into the CU of N × N size of the layer serving as a low level.

The information of the size of LCU and the size of SCU is specified to be included in SPS.The details of CU is described in non-patent literature 1.

(Exemplary structures with reference to image setting unit)

Figure 16 is the block diagram of the Exemplary structures with reference to image setting unit 50 of graphic extension Figure 14.

Figure 16 comprises acquiring unit 71, identifying unit 72 and generation unit 73 with reference to image setting unit 50.

Acquiring unit 71 with reference to image setting unit 50 obtains with reference to image appointed information from the motion prediction/compensating unit 47 of Figure 14, is then supplied to identifying unit 72 and generation unit 73 with reference to image appointed information.

Identifying unit 72, from reference to buffer 49, reads the reference image appointed information of primary image.The reference image appointed information that identifying unit 72 compares the primary image of reading and the reference image appointed information supplied from acquiring unit 71.Subsequently, when the reference image appointed information of the primary image read is identical with the reference image appointed information supplied from acquiring unit 71, identifying unit 72 is judged to be the predictive mode with reference to image appointed information replication mode.Identifying unit 72 is supplied to generation unit 73 replication mode.

In addition, identifying unit 72 judges to specify information generated as with reference to image, and whether the predictive mode with reference to image appointed information supplied from the setup unit 22 of Figure 13 is replication mode.When judgement specifies information generated as with reference to image, when the predictive mode with reference to image appointed information of supply is replication mode, identifying unit 72 is supplied to the lossless encoding unit 36 of Figure 14 serve as RPS mark 1.But, when judgement specifies information generated as with reference to image, when the predictive mode with reference to image appointed information of supply is not replication mode, identifying unit 72 is supplied to lossless encoding unit 36 serve as RPS mark 0, and specifies information generated to be supplied to lossless encoding unit 36 as with reference to image replication mode.

When not supplying replication mode from identifying unit 72, generation unit 73, from reference to buffer 49, reads the reference image appointed information of primary image.Generation unit 73, according to the input from user, is the predictive mode with reference to image appointed information difference prediction pattern or nonanticipating mode decision, then the predictive mode judged is specified information generated as with reference to image, be supplied to lossless encoding unit 36.

When difference predictive mode is judged as the predictive mode with reference to image appointed information, generation unit 73 calculate the primary image read with reference to the difference between image appointed information and reference image appointed information supply from acquiring unit 71.Subsequently, generation unit 73 compares the calculating difference with reference to image appointed information and the difference of serving as the reference image appointed information of specifying information generated with reference to image from setup unit 22 supply.

When two difference are mutually the same, generation unit 73 identifies corresponding index, is supplied to lossless encoding unit 36, then index is specified information generated as with reference to image, be supplied to lossless encoding unit 36 serve as RPS mark 1.

But when two difference are differing from each other, generation unit 73 is supplied to lossless encoding unit 36 serve as RPS mark 0, then the difference calculated is specified information generated as with reference to image, is supplied to lossless encoding unit 36.

In addition, when nonanticipating pattern is judged as the predictive mode with reference to image appointed information, generation unit 73 compares the reference image appointed information from acquiring unit 71 supply, and from the reference image appointed information of serving as with reference to image appointment information generated that setup unit 22 supplies.Subsequently, when these two mutually the same with reference to image appointed information, generation unit 73 identifies corresponding index, is supplied to lossless encoding unit 36, then index is specified information generated as with reference to image, be supplied to lossless encoding unit 36 serve as RPS mark 1.

But, when these two differing from each other with reference to image appointed information, generation unit 73 is supplied to lossless encoding unit 36 serve as RPS mark 0, then specifies information generated with reference to image appointed information as with reference to image using what supply from acquiring unit 71, is supplied to lossless encoding unit 36.

(the illustration grammer of the SPS of enhanced flow)

Figure 17 and Figure 18 is the diagram of the illustration grammer of the SPS that graphic extension is set by the setup unit 22 of Figure 13.

As shown in diagram in the 9th row of Figure 18, SPS comprises the RPS prediction mode information (short_term_ref_pic_pred_mode) of the predictive mode as the instruction RPS specifying information generated with reference to image.RPS prediction mode information is 0 when its instruction replication mode, is 1 when its instruction difference prediction pattern, is 2 when its instruction nonanticipating pattern.

As shown in diagram in the 10th row to the 13rd row, when RPS prediction mode information is not 0, SPS comprises the difference of RPS or RPS of often kind of predictive mode of RPS.The index that order provides from 0 is assigned to the difference of RPS or RPS.

As shown in the 15th row, be similar to the SPS of basic stream, this SPS comprises long-term mark (long_term_ref_pics_present_flag).As shown in diagram in the 16th row and the 17th row, when being labeled as 1 for a long time, SPS comprises the long-term forecast pattern information (long_term_ref_pic_pred_mode) as the predictive mode with reference to image appointed information of specifying the instruction of information generated long-term with reference to image.Long-term forecast pattern information (long_term_ref_pic_pred_mode) is 0 when its instruction replication mode, is 1 when its instruction difference prediction pattern, is 2 when its instruction nonanticipating pattern.

As shown in diagram in the 18th row to the 22nd row, when long-term forecast pattern information is 2, be similar to the SPS of basic stream, this SPS comprises the long-term item number with reference to image appointed information, long-term reference image appointed information, and reference marker, specify information generated as with reference to image.The index that order provides from 0 is assigned to long-term reference image appointed information.

But, when long-term forecast pattern information is 1, as shown in diagram in the 24th row to the 27th row, SPS comprises the difference (diff_num_long_term_ref_pics_sps) between the item number of the long-term item number with reference to image appointed information be included in SPS and the long-term reference image appointed information be included in the SPS of reference layer, and be endowed same index long-term with reference to image appointed information and to be included in the SPS of reference layer long-term with reference to the difference (diff_lt_ref_pic_poc_lsb_sps) between image appointed information, information generated is specified as with reference to image.

In addition, when long-term forecast pattern information is 1, the SPS of enhanced flow does not comprise reference marker, and reference marker is regarded as the reference marker of reference layer.

(the illustration grammer of the section head of enhanced flow)

Figure 19-21 is diagrams of the Exemplary structures of the grammer of the section head of graphic extension enhanced flow.

As shown in diagram in the 18th row of Figure 19, be similar to the section head of basic stream, the section head of enhanced flow comprises RPS mark.In addition, as shown in diagram in the 19th row and the 20th row, when RPS is labeled as 0, section head comprises the RPS pattern information as the correspondence section of specifying information generated with reference to image.

As shown in diagram in the 21st row and the 22nd row of Figure 19, when RPS prediction mode information is not 0, section head comprises the difference of RPS or RPS of the correspondence section of often kind of predictive mode of RPS, is the short_term_ref_pic_set of num_short_term_ref_pic_sets as wherein index.

In addition, as shown in diagram in the 23rd row and the 24th row of Figure 19, when RPS prediction mode information is not 0, and when RPS is labeled as 1, section head comprises the index (short_term_ref_pic_set_idx) of the difference of RPS or RPS of corresponding section.

In addition, as shown in diagram in the 25th row and the 26th row of Figure 19, when be included in SPS be labeled as 1 for a long time time, section head comprises specifies the long-term forecast pattern information of information generated as with reference to image.

If the 27th row at Figure 19 is to shown in diagram in the 30th row, when long-term forecast pattern information is 2, be similar to basic stream, section head comprises as specifying the inner number of the SPS of information generated and SPS external number with reference to image.

In addition, if the 31st row at Figure 19 is to shown in diagram in the 36th row, be similar to basic stream, section head comprises index (lt_idx_sps), information (poc_lsb_sps) and reference marker (used_by_curr_pic_lt_flag) as specifying information generated with reference to image.

In addition, as shown in diagram in the 38th row of Figure 19, section head comprises highest significant position mark (delta_poc_msb_present_flag) of the highest significant position information (delta_poc_msb_cycle_lt) indicating whether the highest significant position that there is the long-term POC with reference to image of instruction.In addition, as shown in diagram in the 39th row and the 40th row, when highest significant position is labeled as 1, when instruction exists highest significant position information, section head comprises highest significant position information.

Simultaneously, as the 42nd row at Figure 19 to the 45th in shown in diagram, when long-term forecast pattern information is 1, section head comprises the difference (diff_num_long_term_sps) between the inner number of SPS of corresponding section and the inner number of SPS of reference layer, and the difference (diff_num_long_term_pics) between the SPS external number of correspondence section and the SPS external number of reference layer, specify information generated as with reference to image.

In addition, as shown in diagram in the 46th row of Figure 19 and the 1st row of Figure 20 to the 4th row, section head comprises index (lt_idx_sps), and the difference (diff_poc_lsb_lt) between the information (poc_lsb_lt) of information (poc_lsb_lt) and reference layer, specify information generated as with reference to image.

In addition, when long-term forecast pattern information is 1, the section head of enhanced flow does not comprise reference marker and highest significant position mark, and reference marker and highest significant position mark are regarded as reference marker and the highest significant position mark of reference layer.

In addition, as shown in diagram in the 6th row and the 7th row of Figure 20, when highest significant position is labeled as 1, section head comprises the difference (diff_delta_poc_msb_cycle_lt) between the highest significant position information of corresponding section and the highest significant position information of reference layer.

(the illustration grammer of the RPS of enhanced flow)

As shown in diagram in the 2nd row of Figure 22, be similar to the RPS of basic stream, the RPS of enhanced flow comprises with reference to information.As shown in diagram in the 3rd row to the 13rd row, when reference information is 1, and when RPS predictive mode is 2, be similar to the RPS of basic stream, the RPS of enhanced flow is included in front image appointed information, symbol (delta_rps_sign), absolute value (abs_delta_rps_minus1), mark (used_by_curr_pic_lt_flag), and mark (use_delta_flag), specify information generated as with reference to image.

In addition, as as shown in the 14th row to the 17th row, when reference information is 1, and when RPS predictive mode is 1, RPS comprises the difference (diff_delta_idx_minus1) between front image appointed information in front image appointed information and reference layer of corresponding section, and the difference (diff_abs_delta_rps_minus1) between the absolute value (abs_delta_rps_minus1) of correspondence section and the absolute value (abs_delta_rps_minus1) of reference layer.

In addition, when long-term forecast pattern information is 1, the section head of enhanced flow does not comprise symbol (delta_rps_sign), mark (used_by_curr_pic_lt_flag) and mark (use_delta_flag), symbol (delta_rps_sign), mark (used_by_curr_pic_lt_flag) and mark (use_delta_flag) are regarded as the symbol (delta_rps_sign) of reference layer respectively, mark (used_by_curr_pic_lt_flag) and mark (use_delta_flag).

Meanwhile, as shown in diagram in the 21st row to 32 row, when reference information is 0, and when RPS predictive mode is 2, be similar to the RPS of basic stream, RPS comprises the information of the number with reference to image or the POC with reference to image and so on.In addition, as shown in diagram in the 33rd row to the 40th row, when reference information is 0, and when RPS predictive mode is 1, RPS comprises the information of the number with reference to image of such as corresponding section or the POC with reference to image and so on, and the difference between the information of the number with reference to image of such as reference layer or the POC with reference to image and so on.

(the illustration grammer of VPS)

Figure 23 is the diagram of the illustration grammer of graphic extension VPS.

As shown in the 6th row of Figure 23, VPS comprises the information (vps_max_layer_minus1) of the number of plies of instruction scalability.In addition, as shown in the 7th row, be similar to prior art, VPS comprises the information (vps_max_sub_layer_minus1) of the number of plies of scalability instruction time.

In addition, as shown in the 15th row, VPS comprises 0, specifies its index to be difference (diff_ref_layer [0]) between the Primary layer of the information of the Primary layer of 0 and reference layer as serving as.In addition, as shown in diagram in the 16th row and the 17th row, VPS comprises the difference (diff_ref_layer) of enhancement layer.

Here, when current layer curr_layer indicates, and when reference layer ref_layer indicates, utilize difference diff_ref_layer, with following formula (3) statement reference layer ref_layer.

[mathematical expression 3]

ref_layer＝curr_layer-diff_ref_layer...(3)

Thus, when the difference (diff_ref_layer) between enhancement layer is 0, be similar to basic stream, generate enhanced flow not with reference to another layer with reference to image appointed information and other places.

(explanation of the process of encoding device)

Figure 24 is the flow chart of the ges forschung process of the encoding device 10 of key diagram 6.When from outside input primary image with when strengthening image, start ges forschung process.

At the step S1 of Figure 24, the basic coding unit 11 of encoding device 10, according to HEVC mode, is encoded from the primary image of outside input.Basic coding unit 11 use when encoded base image with reference to image with reference to image appointed information, be supplied to and strengthen coding unit 12.Basic coding unit 11 is comprising the coded data obtained by coding, and the basic stream of SPS, PPS etc., is supplied to synthesis unit 13 as basic stream.

In step S2, strengthen coding unit 12 and carry out, according to the enhancing image from outside input, generating the enhanced flow generating process of enhanced flow.The details of enhanced flow generating process is with reference to Figure 25 explanation illustrated below.

In step S3, synthesis unit 13 synthesizes the basic stream supplied from basic coding unit 11, and from strengthening the enhanced flow of coding unit 12 supply, adding VPS etc., thus generating the encoding stream of all layers.Synthesis unit 13 is supplied to transmission unit 14 the encoding stream of all layers.

In step S4, transmission unit 14 sends the encoding stream of all layers supplied from synthesis unit 13 decoding device illustrated to below.

Figure 25 is the flow chart of the details of the enhanced flow generating process of the step S2 that Figure 24 is described.

At the step S10 of Figure 25, the setup unit 22 strengthening coding unit 12 sets specifies information generated with reference to image, then specifies information generated to be supplied to coding unit 21 with reference to image.In step S11, coding unit 21, according to the mode observing HEVC mode, carries out encoding as input signal, from the coded treatment of the enhancing image of the frame unit of outside input.The details of coded treatment is with reference to the Figure 26 illustrated below and 27 explanations.

In step S12, setup unit 22 setting is included in the SPS specifying information generated with reference to image of step S10 setting.In step S13, setup unit 22 sets PPS.In step S14, setup unit 22 according to SPS and PPS of setting, and from the coded data that coding unit 21 supplies, generates enhanced flow.

In step S15, setup unit 22 is supplied to synthesis unit 13 enhanced flow, then ends process.

Figure 26 and 27 is flow charts of the details of the coded treatment of the step S11 that Figure 25 is described.

At the step S31 of Figure 26, the A/D converter 31 of coding unit 21, to the image of the frame unit inputted as input signal, carries out A/D conversion, then exports image as a result, to be kept in screen reorder buffer 32.

In step S32, screen reorder buffer 32, according to gop structure, by coded sequence, resets the two field picture of the DISPLAY ORDER preserved.Screen reorder buffer 32 is supplied to arithmetic element 33, intraprediction unit 46 and motion prediction/compensating unit 47 the image of the frame unit after rearrangement.

In step S33, intraprediction unit 46 carries out the intra-prediction process of all intra prediction modes of alternatively person.In addition, intraprediction unit 46 according to the image read from screen reorder buffer 32, and by the predicted picture that intra-prediction process generates, calculates the cost function value of all intra prediction modes of alternatively person.Subsequently, intraprediction unit 46 is defined as optimum frame inner estimation mode intra prediction mode minimum for cost function value.Intraprediction unit 46 is the predicted picture generated according to optimum frame inner estimation mode, and the cost function value of correspondence is supplied to predicted picture selected cell 48.

In addition, motion prediction/compensating unit 47 carries out the motion prediction/compensation deals of all inter-frame forecast modes of alternatively person.In addition, motion prediction/compensating unit 47 is according to the image supplied from screen reorder buffer 32, and predicted picture, calculate the cost function value of all inter-frame forecast modes of alternatively person, and inter-frame forecast mode minimum for cost function value is defined as best inter-frame forecast mode.Subsequently, motion prediction/compensating unit 47 is the cost function value of best inter-frame forecast mode, and the predicted picture of correspondence is supplied to predicted picture selected cell 48.

In step S34, predicted picture selected cell 48 is according to the process utilizing step S33, from the cost function value that intraprediction unit 46 and motion prediction/compensating unit 47 supply, less for cost function value one of optimum frame inner estimation mode and best inter-frame forecast mode, be defined as optimum prediction mode.Subsequently, predicted picture selected cell 48 is supplied to arithmetic element 33 and adder unit 40 the predicted picture of optimum prediction mode.

In step S35, predicted picture selected cell 48 judges whether optimum prediction mode is best inter-frame forecast mode.When in step S35, when judging that optimum prediction mode is best inter-frame forecast mode, predicted picture selected cell 48, the selection of the predicted picture generated by best inter-frame forecast mode, informs motion prediction/compensating unit 47.

Subsequently, in step S36, motion prediction/compensating unit 47, inter-frame forecast mode information and motion vector, is supplied to lossless encoding unit 36.In step S37, preserve the reference image appointed information of the primary image supplied from basic coding unit 11 with reference to buffer 49.In step S38, carry out with reference to image setting unit 50 being created on the generating process of specifying information generated with reference to image with reference to image used in motion prediction/compensation deals.The details of generating process is with reference to Figure 28 explanation illustrated below.

Meanwhile, when in step S35, when judging that optimum prediction mode is not best inter-frame forecast mode, namely, when optimum prediction mode is optimum frame inner estimation mode, predicted picture selected cell 48, the selection of pressing the predicted picture that optimum frame inner estimation mode generates, informs intraprediction unit 46.Subsequently, in step S39, intraprediction unit 46 is supplied to lossless encoding unit 36 intraprediction mode information, and then process enters step S40.

In step S40, arithmetic element 33, by from supplying from the image of screen reorder buffer 32, deducting the predicted picture supplied from predicted picture selected cell 48, encoding.Arithmetic element 33, the image obtained as a result, exports to orthogonal transform unit 34 as residual information.

In step S41, orthogonal transform unit 34, to the residual information supplied from arithmetic element 33, carries out orthogonal transform, and the orthogonal transform system obtained as a result is supplied to quantifying unit 35.

In step S42, quantifying unit 35 quantizes the coefficient supplied from orthogonal transform unit 34, and the coefficient obtained as a result is supplied to lossless encoding unit 36 and inverse quantization unit 38.

At the step S43 of Figure 27, inverse quantization unit 38, to the quantization parameter supplied from quantifying unit 35, carries out re-quantization, and the orthogonal transform coefficient obtained as a result is supplied to inverse orthogonal transformation unit 39.

In step S44, inverse orthogonal transformation unit 39, to the orthogonal transform coefficient supplied from inverse quantization unit 38, carries out inverse orthogonal transformation, and the residual information obtained as a result is supplied to adder unit 40.

In step S45, adder unit 40 is added the residual information supplied from inverse orthogonal transformation unit 39, and from the predicted picture that predicted picture selected cell 48 supplies, thus obtain local decoder image.Adder unit 40 is supplied to deblocking filter 41 and frame memory 44 the image obtained.

In step S46, deblocking filter 41 carries out de-blocking filter process to the local decoder image supplied from adder unit 40.Deblocking filter 41 is supplied to self adaptation offset filter 42 the image obtained as a result.

In step S47, in units of LCU, self adaptation offset filter 42, to the image supplied from deblocking filter 41, carries out self adaptation skew filtering process.Self adaptation offset filter 42 is supplied to auto-adaptive loop filter 43 the image obtained as a result.In addition, in units of LCU, self adaptation offset filter 42, storage mark, index or side-play amount and kind of information, is supplied to lossless encoding unit 36 as skew filtering information.

In step S48, in units of LCU, auto-adaptive loop filter 43, to the image supplied from self adaptation offset filter 42, carries out adaptive loop filter process.Auto-adaptive loop filter 43 is supplied to frame memory 44 the image obtained as a result.In addition, auto-adaptive loop filter 43 is supplied to lossless encoding unit 36 the filter factor used in adaptive loop filter process.

In step S49, frame memory 44 accumulates the image supplied from auto-adaptive loop filter 43, and from the image that adder unit 40 supplies.Be accumulated in image in frame memory 44 by switch 45, as with reference to being exported to intraprediction unit 46 or motion prediction/compensating unit 47.

In step S50, lossless encoding unit 36 pairs of intraprediction mode information and inter-frame forecast mode information is one of any, motion vector, specify information generated, RPS mark, skew filtering information and filter factor with reference to image, carry out lossless coding, as coded message.

In step S51, lossless encoding unit 36, to the quantization parameter supplied from quantifying unit 35, carries out lossless coding.Subsequently, lossless encoding unit 36 is according to the process utilizing step S50, and the coded message of lossless coding, and lossless coding coefficient, generate coded data, and the coded data generated is supplied to accumulation buffer 37.

In step S52, the coded data that accumulation buffer 37 temporary cumulative supplies from lossless encoding unit 36.

In step S53, Rate control unit 50, according to the coded data be accumulated in accumulation buffer 37, controls the speed of the quantization operation of quantifying unit 35, so that overflow or underflow does not occur.

In step S54, accumulation buffer 37 exports to the setup unit 22 of Figure 13 the coded data of preserving.Subsequently, process returns the step S11 of Figure 25, then enters step S12.

In addition, in the coded treatment of Figure 26 and 27, for the purpose of simplifying the description, carry out intra-prediction process and motion prediction/compensation deals consistently, but in fact, exist according to picture kind etc., only carry out the situation of one of intra-prediction process and motion prediction/compensation deals.

Figure 28 is the flow chart of the details of the generating process of the step S38 that Figure 26 is described.

At the step S70 of Figure 28, the acquiring unit 71 with reference to image setting unit 50 obtains the reference image appointed information strengthening image from motion prediction/compensating unit 47, and the reference image appointed information strengthening image is supplied to identifying unit 72 and generation unit 73.In step S71, identifying unit 72 and generation unit 73, from reference to buffer 49, read the reference image appointed information of primary image.

In step S72, identifying unit 72 judges to strengthen the whether identical with the reference image appointed information of primary image with reference to image appointed information of image.When in step S72, judge to strengthen image with reference to image appointed information and primary image identical with reference to image appointed information time, in step S73, identifying unit 72 is judged to be the predictive mode with reference to image appointed information replication mode.

In step S74, identifying unit 72 determines whether to supply to give replication mode from the setup unit 22 of Figure 13, specifies information generated as with reference to image.When in step S74, judge for giving replication mode, as during with reference to image appointment information generated, in step S75, identifying unit 72 is 1 RPS flag settings, and RPS mark is supplied to the lossless encoding unit 36 of Figure 14.Subsequently, process returns the step S38 of Figure 26, then enters step S40.

Meanwhile, when in step S74, judge not supply replication mode, as during with reference to image appointment information generated, process enters step S76.In step S76, identifying unit 72 is 0 RPS flag settings, RPS mark is supplied to lossless encoding unit 36, and replication mode is specified information generated as with reference to image, be supplied to lossless encoding unit 36.Subsequently, process returns the step S38 of Figure 26, then enters step S40.

Meanwhile, when in step S72, judge to strengthen image with reference to image appointed information be different from primary image with reference to image appointed information time, process enters step S77.In step S77, generation unit 73, according to the input from user, judges whether difference prediction pattern is set to the predictive mode with reference to image appointed information.

When in step S77, when judging the predictive mode that difference prediction pattern is set to reference to image appointed information, process enters step S78.In step S78, generation unit 73 is defined as the predictive mode with reference to image appointed information difference prediction pattern.

In step S79, generation unit 73 obtains primary image and strengthens the difference with reference to image appointed information between image.In step S80, generation unit 73 judges that whether the difference calculated in step S79 is identical with the difference with reference to image appointed information that the index supplied from setup unit 22 is assigned to.

In step S80, when the difference with reference to image appointed information that the difference judging to calculate in step S79 and described index are assigned to is identical, process enters step S81.In step S81, generation unit 73 is 1 RPS flag settings, and RPS mark is supplied to lossless encoding unit 36, then and the differential pair identical with the difference of the reference image appointed information calculated in step 79 index of answering be supplied to lossless encoding unit 36.Subsequently, process returns the step S38 of Figure 26, then enters step S40.

Meanwhile, when in step S80, when judging that the difference calculated in step S79 is different from the difference with reference to image appointed information that described index is assigned to, process enters step S82.In step S82, generation unit 73 is 0 RPS flag settings, and RPS mark is supplied to lossless encoding unit 36, is then supplied to lossless encoding unit 36 difference prediction pattern with in the difference with reference to image appointed information that step S79 calculates.Subsequently, process returns the step S38 of Figure 26, then enters step S40.

In addition, when in step S77, when judging the predictive mode that difference prediction pattern is not set to reference to image appointed information, generation unit 73 is defined as the predictive mode with reference to image appointed information nonanticipating pattern, and then process enters step S83.

In step S83, generation unit 73 judges to strengthen the whether identical with the reference image appointed information that the index supplied from setup unit 22 is assigned to reference to image appointed information of image.When in step S83, judge to strengthen image with reference to image appointed information and described index be assigned to identical with reference to image appointed information time, process enters step S84.

In step S84, generation unit 73 is 1 RPS flag settings, and RPS mark is supplied to lossless encoding unit 36, then and be supplied to lossless encoding unit 36 with the index corresponding with reference to image appointed information identical with reference to image appointed information strengthening image.Subsequently, process returns the step S38 of Figure 26, then enters step S40.

Meanwhile, when in step S83, judge to strengthen image with reference to image appointed information be different from that described index is assigned to reference to image appointed information time, process enters step S85.In step S85, generation unit 73 is 0 RPS flag settings, and RPS mark is supplied to lossless encoding unit 36, is then supplied to lossless encoding unit 36 nonanticipating pattern with reference to image appointed information.Subsequently, process returns the step S38 of Figure 26, then enters step S40.

As mentioned above, because setting specifies information generated with reference to image, therefore encoding device 10 can be shared or predict with reference to image appointed information between Primary layer and enhancement layer.Thus, the amount of information of enhanced flow can be reduced, thus improve code efficiency.

(Exemplary structures according to the decoding device of the first embodiment)

Figure 29 is the block diagram of the Exemplary structures of the decoding device of the encoding stream of all each layer that graphic extension transmits from the encoding device 10 of Fig. 6 according to the decoding of first embodiment of this technology.

The decoding device 90 of Figure 29 comprises receiving element 91, separative element 92, basic decoding unit 93 and strengthens decoding unit 94.

Receiving element 91 receives the encoding stream of all each layer transmitted from the encoding device 10 of Fig. 6, and the encoding stream of all each layer received is supplied to separative element 92.

Separative element 92 from the encoding stream of all each layer of receiving element 91, extracts VPS from supply, according to the difference be included in VPS (diff_ref_layer), identifies the presence or absence of the reference layer of enhanced flow.Here, because encoding device 10 utilizes Primary layer as the reference layer of enhanced flow, therefore separative element 72 identifies the existence of reference layer.

When identifying the existing of reference layer, the basic decoding unit 93 of the encoding stream of separative element 92 instruction decoding reference layer is the enhancing decoding unit 94 being supplied to decoding enhanced flow with reference to image appointed information.

In addition, separative element 92, from the encoding stream of all each layers, is separated and substantially flows, then basic stream is supplied to basic decoding unit 93, and is separated enhanced flow, then enhanced flow is supplied to and strengthens decoding unit 94.

Basic decoding unit 93 has the structure similar with the decoding device of HEVC mode conventionally, according to HEVC mode, and the basic stream supplied from separative element 92 of decoding, thus generate primary image.Now, basic decoding unit 93 when decode primary image time use strengthen decoding unit 94 with reference to being supplied to reference to image appointed information of image.Basic decoding unit 93 exports the primary image generated.

Strengthen decoding unit 94 according to the mode observing HEVC mode, the enhanced flow supplied from separative element 92 of decoding, thus generating enhanced images.Now, decoding unit 94 is strengthened with reference to the reference image appointed information supplied from basic decoding unit 93, decoding enhanced flow.Strengthen decoding unit 94 and export the enhancing image generated.

(strengthening the Exemplary structures of decoding unit)

Figure 30 is the block diagram of the Exemplary structures of the enhancing decoding unit 94 of graphic extension Figure 29.

The enhancing decoding unit 94 of Figure 30 comprises extraction unit 111 and decoding unit 112.

The extraction unit 111 strengthening decoding unit 94, from supplying from the enhanced flow of the separative element 92 of Figure 29, extracts SPS, PPS, coded data etc., then SPS, PPS, coded data etc. is supplied to decoding unit 112.

The reference image appointed information of the primary image that decoding unit 112 supplies with reference to the basic decoding unit 93 from Figure 29, according to the mode observing HEVC mode, the coded data supplied from extraction unit 111 of decoding.Now, decoding unit 112 also takes the circumstances into consideration SPS, the PPS etc. with reference to supplying from extraction unit 111.Decoding unit 112 exports the image obtained by decoding, as enhancing image.

(Exemplary structures of decoding unit)

Figure 31 is the block diagram of the Exemplary structures of the decoding unit 112 of graphic extension Figure 30.

The decoding unit 112 of Figure 31 comprises accumulation buffer 131, lossless decoding unit 132, inverse quantization unit 133, inverse orthogonal transformation unit 134, adder unit 135, deblocking filter 136, self adaptation offset filter 137, auto-adaptive loop filter 138, screen reorder buffer 139, D/A converter 140, frame memory 141, switch 142, intraprediction unit 143, with reference to buffer 144, with reference to image setting unit 145, motion compensation units 146 and switch 147.

The accumulation buffer 131 of decoding unit 112 from the extraction unit 111 of Figure 30, received code data, and the coded data of accumulative reception.Accumulation buffer 131 is supplied to lossless decoding unit 132 the decoded data of accumulation.

Lossless decoding unit 132, to the coded data supplied from accumulation buffer 131, carries out the losslessly encoding of such as variable length decoding or arithmetic decoding and so on, thus obtains quantization parameter and coded message.Lossless decoding unit 132 is supplied to inverse quantization unit 133 quantization parameter.In addition, lossless decoding unit 132, using the intraprediction mode information etc. as coded message, is supplied to intraprediction unit 143, and motion vector, inter-frame forecast mode information etc. are supplied to motion compensation units 146.

In addition, lossless decoding unit 132 specifies information generated using the reference image as coded message, and RPS mark etc. is supplied to reference to setup unit 145.In addition, lossless decoding unit 132, using as the intraprediction mode information of coded message or inter-frame forecast mode information, is supplied to switch 147.Lossless decoding unit 132, using the skew filtering information as coded message, is supplied to self adaptation offset filter 137, and filter factor is supplied to auto-adaptive loop filter 138.

Inverse quantization unit 133, inverse orthogonal transformation unit 134, adder unit 135, deblocking filter 136, self adaptation offset filter 137, auto-adaptive loop filter 138, frame memory 141, switch 142, intraprediction unit 143 and motion compensation units 146 are carried out and the inverse quantization unit 38 of Figure 14, inverse orthogonal transformation unit 39, adder unit 40, deblocking filter 41, self adaptation offset filter 42, auto-adaptive loop filter 43, frame memory 44, switch 45, process that intraprediction unit 46 is identical with motion prediction/compensating unit 47, thus decoded picture.

Particularly, inverse quantization unit 133 to the re-quantization of the quantization parameter supplied from lossless decoding unit 132, and is supplied to inverse orthogonal transformation unit 134 the orthogonal transform coefficient obtained as a result.

Inverse orthogonal transformation unit 134, to the orthogonal transform coefficient supplied from inverse quantization unit 133, carries out inverse orthogonal transformation.Inverse orthogonal transformation unit 134 is supplied to adder unit 135 the residual information obtained by inverse orthogonal transformation.

Adder unit 135 plays decoding unit, by being added the residual information as current image to be decoded supplied from inverse orthogonal transformation unit 134, and from the predicted picture that switch 147 supplies, decodes.Adder unit 135 is supplied to deblocking filter 136 and frame memory 141 the image obtained by decoding.In addition, when not from switch 147 availability forecast image, adder unit 135 is supplied to deblocking filter 136 using the image as the residual information supplied from inverse orthogonal transformation unit 134, as the image obtained by decoding, and supply this image, to be accumulated in frame memory 141.

Deblocking filter 136, to the image supplied from adder unit 135, carries out adaptive de-blocking filtering process, and the image obtained as a result, is supplied to self adaptation offset filter 137.

Self adaptation offset filter 137 comprises the buffer that order preserves the side-play amount supplied from lossless decoding unit 132.In addition, self adaptation offset filter 137, according to the skew filtering information supplied from lossless decoding unit 132, in units of LCU, to the image of the adaptive de-blocking filtering process through deblocking filter 136, carries out self adaptation skew filtering process.

Particularly, when being included in the storage mark offset in filtering information and being 0, self adaptation offset filter 137 utilizes the side-play amount be included in skew filtering information, to in units of LCU, through the image of de-blocking filter process, carry out the self adaptation skew filtering process of the kind indicated by kind of information.

Meanwhile, when being included in the storage mark offset in filtering information and being 1, self adaptation offset filter 137 is in units of LCU, and through the image of de-blocking filter process, reading and saving is in the side-play amount of the position indicated by the index be included in skew filtering information.Subsequently, self adaptation offset filter 137 utilizes the side-play amount read, and carries out the self adaptation skew filtering process of the kind indicated by kind of information.Self adaptation offset filter 137 is supplied to auto-adaptive loop filter 138 the image through self adaptation skew filtering process.

Auto-adaptive loop filter 138 utilizes the filter factor supplied from lossless decoding unit 132, in units of LCU, carries out adaptive loop filter process to the image supplied from self adaptation offset filter 137.Auto-adaptive loop filter 138 is supplied to frame memory 141 and screen reorder buffer 139 the image obtained as a result.

Screen reorder buffer 139, in units of frame, preserves the image supplied from auto-adaptive loop filter 138.Screen reorder buffer 139, according to original display order, resets the memory image in units of frame of coded sequence, and the image reset is supplied to D/A converter 140.

D/A converter 140, to the image in units of frame supplied from screen reorder buffer 139, carries out D/A conversion, and exports the image after conversion, as enhancing image.Frame memory 141 accumulates the image supplied from auto-adaptive loop filter 138, and from the image that adder unit 135 supplies.The image be accumulated in frame memory 141 is read, and as reference image, and is provided to intraprediction unit 143 or motion compensation units 146 by switch 142.

Intraprediction unit 143 utilizes through switch 142, from the reference image that frame memory 141 reads, carries out the intra-prediction process of the intra prediction mode indicated by the intraprediction mode information supplied from lossless decoding unit 132.Intraprediction unit 143 is supplied to switch 147 the predicted picture generated as a result.

With reference to the reference image appointed information with reference to image that buffer 144 preservation uses when the coded data of the primary image that decoding supplies from the basic decoding unit 93 of Figure 29.

With reference to image setting unit 145 according to the RPS mark supplied from lossless decoding unit 132, specify information generated with reference to image being included in the SPS that supplies from extraction unit 111, or from lossless decoding unit 132 supply specify information generated to be set as current coded data to be decoded with reference to image specify information generated with reference to image.Take the circumstances into consideration, from reference to buffer 144, to read the reference image appointed information of primary image with reference to image setting unit 145.Information generated is specified according to the reference image of current coded data to be decoded with reference to image setting unit 145, with the reference image appointed information of the primary image read, generate with reference to image appointed information, and this reference image appointed information is supplied to motion compensation units 146.

Motion compensation units 146, through switch 142, reads by the reference image of specifying from the reference image appointed information supplied with reference to image setting unit 145 from frame memory 141.Motion compensation units 146 utilizes the motion vector supplied from lossless decoding unit 132, and with reference to image, carries out the motion compensation process of the best inter-frame forecast mode indicated by the inter-frame forecast mode information supplied from lossless decoding unit 132.Motion compensation units 146 is supplied to switch 147 the predicted picture generated as a result.

When supplying intraprediction mode information from lossless decoding unit 132, switch 147 is supplied to adder unit 135 the predicted picture supplied from intraprediction unit 143.Meanwhile, when supplying inter-frame forecast mode information from lossless decoding unit 132, switch 147 is supplied to adder unit 135 the predicted picture supplied from motion compensation units 146.

(Exemplary structures with reference to image setting unit)

Figure 32 is the block diagram of the Exemplary structures with reference to image setting unit 145 of graphic extension Figure 31.

Figure 32 comprises information buffer 161, pattern buffer 162 and generation unit 163 with reference to image setting unit 145.

Information buffer 161 is preserved and is specified information generated except as with reference to image, from the predictive mode with reference to image appointed information that the lossless decoding unit 132 of Figure 31 supplies, and the information outside RPS mark.Pattern buffer 162 is preserved and is specified information generated as with reference to image, from the predictive mode with reference to image appointed information that lossless decoding unit 132 supplies.

Generation unit 163 reads RPS mark from information buffer 161.Generation unit 163 marks according to RPS, the reference image that extraction unit 111 from Figure 30 supplies is specified information generated, or specifies information generated from the reference image that the reference image that lossless decoding unit 132 supplies specifies information generated to be defined as current coded data to be decoded.Is when specifying information generated from the reference image of lossless decoding unit 132 supply when current coded data to be decoded with reference to image appointment information generated, generation unit 163 is from information buffer 161, read the information except the predictive mode with reference to image appointed information, and from pattern buffer 162, read the predictive mode with reference to image appointed information.

When as current coded data to be decoded with reference to image specify the predictive mode with reference to image appointed information of information generated be replication mode or difference prediction pattern time, generation unit 163, from reference to buffer 144, reads the reference image appointed information of primary image.When the predictive mode with reference to image appointed information is replication mode, generation unit 163 generates the reference image appointed information of the reading of primary image, as the reference image appointed information strengthening image, and the motion compensation units 146 being supplied to Figure 31 with reference to image appointed information of the reading of the primary image generated.

Simultaneously, when the predictive mode with reference to image appointed information is difference prediction pattern, and when the information except the predictive mode with reference to image appointed information is index, generation unit 163 identifies specifies information generated, the difference of the reference image appointed information that described index supply from the extraction unit 111 of Figure 30 is assigned to as with reference to image.Generation unit 163 is added the reference image appointed information of the difference of reference image appointed information and the reading of primary image identified, generate the addition value obtained as a result, as the reference image appointed information strengthening image, and the addition value generated is supplied to motion compensation units 146.

In addition, when the predictive mode with reference to image appointed information is difference prediction pattern, and when the information except the predictive mode with reference to image appointed information is the difference with reference to image appointed information, generation unit 163 is added the reference image appointed information of the reading of described difference and primary image.Generation unit 163 generates the addition value obtained as a result, as the reference image appointed information strengthening image, and the addition value generated is supplied to motion compensation units 146.

In addition, when the predictive mode with reference to image appointed information is non-predictive mode, and when the information except the predictive mode with reference to image appointed information is index, being identified as with reference to image appointed information that the described index that the extraction unit 111 from Figure 30 supplies is assigned to by generation unit 163 specifies information generated with reference to image.Generation unit 163 generates the reference image appointed information identified, as the reference image appointed information strengthening image, and is supplied to motion compensation units 146 what generate with reference to image appointed information.

In addition, when the predictive mode with reference to image appointed information is non-predictive mode, and when the information except the predictive mode with reference to image appointed information is with reference to image appointed information, generation unit 163 generates with reference to image appointed information, as the reference image appointed information strengthening image, and be supplied to motion compensation units 146 what generate with reference to image appointed information.

(explanation of the process of decoding device)

Figure 33 is the flow chart of the scalable decoding process of the decoding device 90 that Figure 29 is described.

At the step S100 of Figure 33, the receiving element 91 of decoding device 90 receives the encoding stream of all each layer transmitted from the encoding device 10 of Fig. 6, then the encoding stream of all each layer received is supplied to separative element 92.In step S101, separative element 92 from the encoding stream of receiving element 91, extracts VPS from supply.

In step S102, separative element 92, according to the difference be included in VPS (diff_ref_layer), identifies the existence of the reference layer of enhanced flow.In step S103, the basic decoding unit 93 of the encoding stream of separative element 92 instruction decoding reference layer is the enhancing decoding unit 94 being supplied to decoding enhanced flow with reference to image appointed information.

In step S104, separative element 92, from the encoding stream of all each layers, is separated basic stream and enhanced flow.Separative element 92 is supplied to basic decoding unit 93 basic stream, enhanced flow is supplied to and strengthens decoding unit 94.

In step S105, basic decoding unit 93, according to HEVC mode, is decoded from the basic stream of separative element 92 supply, thus generates primary image.Now, basic decoding unit 93 when decode primary image time use strengthen decoding unit 94 with reference to being supplied to reference to image appointed information of image.Basic decoding unit 93 exports the primary image generated.

In step S106, strengthening decoding unit 94 with reference to the reference image appointed information supplied from basic decoding unit 93, carrying out the enhanced flow according to supplying from separative element 92, the enhancing Computer image genration process of generating enhanced images.Strengthen the details of Computer image genration process with reference to Figure 34 explanation illustrated below.

Figure 34 is the flow chart of the enhancing Computer image genration process of the enhancing decoding unit 94 that Figure 30 is described.

At the step S111 of Figure 34, the extraction unit 111 strengthening decoding unit 94, from the enhanced flow of supply self-separation unit 92, extracts SPS, PPS, coded data etc., then SPS, PPS, coded data etc. is supplied to decoding unit 112.

In step S112, decoding unit 112 carries out taking the circumstances into consideration SPS and PPS with reference to supplying from extraction unit 111, from the reference image appointed information etc. that basic decoding unit 93 supplies, according to the mode observing HEVC mode, decode from the decoding process of the coded data of extraction unit 111 supply.The details of decoding process is with reference to Figure 35 explanation illustrated below.Subsequently, process terminates.

At the step S131 of Figure 35, the accumulation buffer 131 strengthening decoding unit 112, from the extraction unit 111 of Figure 30, receives the coded data in units of frame, and the coded data of accumulative reception.Accumulation buffer 131 is supplied to lossless decoding unit 132 the coded data of accumulation.

In step S132, lossless decoding unit 132 carries out losslessly encoding to the coded data supplied from accumulation buffer 131, thus obtains quantization parameter and coded message.Lossless decoding unit 132 is supplied to inverse quantization unit 133 quantization parameter.In addition, lossless decoding unit 132, using the intraprediction mode information etc. as coded message, is supplied to intraprediction unit 143, and motion vector, inter-frame forecast mode information etc., is supplied to motion compensation units 146.

In addition, lossless decoding unit 132 specifies information generated using the reference image as coded message, and RPS mark etc. is supplied to reference to image setting unit 145.In addition, lossless decoding unit 132, using as the intraprediction mode information of coded message or inter-frame forecast mode information, is supplied to switch 147.Lossless decoding unit 132 is supplied to self adaptation offset filter 137 using the skew filtering information as coded message, and filter factor is supplied to auto-adaptive loop filter 138.

In step S133, inverse quantization unit 133 carries out re-quantization to the quantization parameter supplied from lossless decoding unit 132, and the orthogonal transform coefficient obtained as a result is supplied to inverse orthogonal transformation unit 134.

In step S134, motion compensation units 146 determines whether from lossless decoding unit 132 for giving inter-frame forecast mode information.When in step S134, judge that process enters step S135 for when giving inter-frame forecast mode information.

In step S135, with reference to the reference image appointed information with reference to image that buffer 144 preservation uses when the coded data of the primary image that decoding supplies from the basic decoding unit 93 of Figure 29.

In step S136, carry out specifying information generated and RPS mark according to what supply from lossless decoding unit 132 with reference to image with reference to image setting unit 145, be kept at the reference image appointed information etc. with reference to the primary image in buffer 144, the generating process with reference to image appointed information of generating enhanced images.The details of generating process is with reference to Figure 36 explanation illustrated below.

In step S137, motion compensation units 146 is according to the reference image appointed information supplied from reference image setting unit 145, read with reference to image, and utilize motion vector and with reference to image, carry out the motion compensation process of the best inter-frame forecast mode indicated by inter-frame forecast mode information.Motion compensation units 146, the predicted picture generated as a result, is supplied to adder unit 135 by switch 147, and then process enters step S139.

Meanwhile, when in step S134, when judging not supply inter-frame forecast mode information, that is, when supplying intraprediction mode information to intraprediction unit 143, process enters step S138.

In step S138, intraprediction unit 143 utilizes through switch 142, from the reference image that frame memory 141 reads, carries out the intra-prediction process of the intra prediction mode represented by intraprediction mode information.Intraprediction unit 143, by switch 147, is supplied to adder unit 135 the predicted picture generated by intra-prediction process, enters step S139 with reprocessing.

In step S139, inverse orthogonal transformation unit 134, to the orthogonal transform coefficient supplied from inverse quantization unit 133, carries out inverse orthogonal transformation, and the residual information obtained as a result is supplied to adder unit 135.

In step S140, adder unit 135 is added the residual information supplied from inverse orthogonal transformation unit 134, and from the predicted picture that switch 147 supplies.Adder unit 135 is supplied to deblocking filter 136 and frame memory 141 the image obtained as a result.

In step S141, deblocking filter 136, to the image supplied from adder unit 135, carries out de-blocking filter process, thus eliminates block distortion.Deblocking filter 136 is supplied to self adaptation offset filter 137 the image obtained as a result.

In step S142, self adaptation offset filter 137, according to the skew filtering information supplied from lossless decoding unit 132, in units of LUC, to the image of the de-blocking filter process through deblocking filter 136, carries out self adaptation skew filtering process.Self adaptation offset filter 137 is supplied to auto-adaptive loop filter 138 the image through self adaptation skew filtering process.

In step S143, auto-adaptive loop filter 138 utilizes the filter factor supplied from lossless decoding unit 132, in units of LCU, to the image supplied from self adaptation offset filter 137, carries out adaptive loop filter process.Auto-adaptive loop filter 138 is supplied to frame memory 141 and screen reorder buffer 139 the image obtained as a result.

In step S144, frame memory 141 accumulates the image supplied from adder unit 135, and from the image that auto-adaptive loop filter 138 supplies.Be accumulated in image in frame memory 141 by switch 142, be provided to intraprediction unit 143 or motion compensation units 146 as with reference to image.

In step S145, screen reorder buffer 139, in units of frame, preserves the image supplied from auto-adaptive loop filter 138, and according to original display order, the coded sequence that rearrangement is preserved, the image in units of frame, is then supplied to D/A converter 140 the image reset.

In step S146, D/A converter 140, to the image in units of frame supplied from screen reorder buffer 139, carries out D/A conversion, and exports the image after conversion, as enhancing image.Subsequently, process returns the step S112 of Figure 34, then terminates.

Figure 36 is the flow chart of the details of the generating process of the step S136 that Figure 35 is described.

At the step S161 of Figure 36, information buffer 161 (Figure 32) with reference to image setting unit 145 is preserved and is specified information generated as with reference to image, from the RPS mark that the lossless decoding unit 132 of Figure 31 supplies, and the information except the predictive mode with reference to image appointed information.In step S162, pattern buffer 162 is preserved and is specified information generated as with reference to image, from the predictive mode with reference to image appointed information that lossless decoding unit 132 supplies.

In step S163, generation unit 163 reads RPS mark from information buffer 161.In step S164, generation unit 163 judges that RPS marks whether as 1.

When in step S164, when judging that RPS is labeled as 1, in step S165, what generation unit 163 supplied the extraction unit 111 from Figure 30 specifies information generated to be defined as the reference image appointment information generated of current coded data to be decoded with reference to image.

In step S166, generation unit 163 reads the information except the predictive mode with reference to image appointed information from information buffer 161, reads the predictive mode with reference to image appointed information from pattern buffer 162.Subsequently, process enters step S168.

Simultaneously, when in step S164, judge that RPS mark is not as 1, namely judge RPS as 0 time, in step S167, generation unit 163 specifies information generated to be defined as the reference image appointment information generated of current coded data to be decoded what supply from lossless decoding unit 132 with reference to image.Subsequently, process enters step S168.

In step S168, generation unit 163 judges whether the predictive mode with reference to image appointed information with reference to image appointment information generated serving as current coded data to be decoded is non-predictive mode.When in step S168, when the predictive mode of judgement reference image appointed information is non-predictive mode, in step S169, generation unit 163, from reference to buffer 144, reads the reference image appointed information of primary image.

In step S170, generation unit 163 judges whether the predictive mode with reference to image appointed information is replication mode.

When in step S170, when the predictive mode of judgement reference image appointed information is replication mode, in step S171, generation unit 163 is the reference image appointed information being defined as enhancing image with reference to image appointed information of the reading of primary image.Generation unit 163 is the motion compensation units 146 being supplied to Figure 31 with reference to image appointed information of the enhancing image determined.Subsequently, process returns the step S136 of Figure 35, then enters step S137.

Simultaneously, when in step S170, judge that the predictive mode with reference to image appointed information is not replication mode, namely, when the predictive mode of judgement reference image appointed information is difference prediction pattern, in step S172, generation unit 163 judges whether the information except the predictive mode with reference to image appointed information is index.

When in step S172, when the information judging except the predictive mode with reference to image appointed information is index, process enters step S173.In step S173, generation unit 163 identifies that from the difference with reference to image appointed information that the described index of extraction unit 111 supply is assigned to, then process enters step S174 as reference image appointment information generated.

Meanwhile, when in step S172, judge that the information except the predictive mode with reference to image appointed information is not index, when namely judging to be the difference with reference to image appointed information, process enters step S174.

In step S174, generation unit 163 is added the reference image appointed information with reference to the difference of image appointed information and the primary image of reading, and the addition value obtained as a result is defined as the reference image appointed information strengthening image.Generation unit 163 is supplied to motion compensation units 146 the reference image appointed information of the enhancing image determined.Subsequently, process returns the step S136 of Figure 35, then enters step S137.

In addition, when in step S168, when the predictive mode judging with reference to image appointed information is non-predictive mode, in step S175, generation unit 163 judges whether the information except the predictive mode with reference to image appointed information is index.

When in step S175, when the information judging except the predictive mode with reference to image appointed information is index, process enters step S176.In step S176, generation unit 163 identification specifies information generated as with reference to image, and from the reference image appointed information that the described index of extraction unit 111 supply is assigned to, then process enters step S177.

Meanwhile, when in step S175, when the information judging except the predictive mode with reference to image appointed information is not index, that is, when judging that the information except the predictive mode with reference to image appointed information is with reference to image appointed information, process enters step S177.

In step S177, generation unit 163 is the reference image appointed information identified in step S175, or that serves as the information except the predictive mode with reference to image appointed information is supplied to motion compensation units 146 with reference to image appointed information, as the reference image appointed information strengthening image.Subsequently, process returns the step S136 of Figure 35, then enters step S137.

As mentioned above, decoding device 90 utilizes reference image to specify information generated, the reference image appointed information of generating enhanced images, thus between Primary layer and enhancement layer, can share or prediction reference image appointed information.Thus, the amount of information of enhanced flow can be reduced, thus improve code efficiency.

In addition, in a first embodiment, according to the kind of scalable function, the presence or absence of reference layer can be determined.In this case, such as, when scalable function exists wherein between the layers, carry out the multiple service condition of the coupling (alignment) of picture kind, and between the layers, the SNR scalability that the reference relation of picture may be identical, spatial scalability, chroma format scalability, when bit scalability or depth scalability, setting reference layer.

< second embodiment >

(explanation of the overview of the second embodiment)

As shown in diagram in Figure 37, in a second embodiment, to share between the different layers or prediction is served as and the weighted information with reference to the relevant information of image, described weighted information comprises weight coefficient in weight estimation and deviant.

Figure 38 is the diagram that weight estimation is described.

As shown in diagram in Figure 38, in AVC mode and HEVC mode, when inter prediction, be weighted prediction.Weight estimation is to the weighting of reference image, thus the process of generation forecast image.Such as, particularly, as the two frame Ys of coded sequence before current frame X to be encoded ₁and Y ₀decoded picture when being used as with reference to image, in weight estimation, utilize following formula (4), obtain the predicted picture X' of frame X:

[mathematical expression 4]

X'＝w ₀×Y ₀+w ₀×Y ₁+d...(4)

In formula (4), w ₀and w ₁be weight coefficient, d is deviant.Weight coefficient calculates according to the POC in the implicit weighted prediction in such as AVC mode and so on weight estimation.Meanwhile, in weight estimation in explicit weighting in such as AVC mode prediction or HEVC mode and so on weight estimation, weight coefficient and deviant are included in and transmit in encoding stream.

Owing to being weighted prediction, though when owing to fading in, fade out, cross compound turbine etc., when occurring with reference to brightness change between image and current image to be encoded, also can reduce the difference between predicted picture and current image to be encoded.As a result, code efficiency can be improved.

On the other hand, when not being weighted prediction, owing to fading in, fade out, cross compound turbine etc., becoming the difference between predicted picture and current image to be encoded with reference to the brightness change former state occurred between image and current image to be encoded, thus code efficiency worsens.

Here, in the coding utilizing scalable function, in each scalable image, the differences such as the form of SNR, spatial resolution, frame rate, bit number (bit-depth), carrier chrominance signal, but content is considered to identical.

Thus, when fading in certain scalable image, fade out, cross compound turbine etc. time, think fading in other scalable image, fade out, cross compound turbine etc.Thus, think that the every weighted information being suitable for scalable image has correlation.In this, in a second embodiment, by between the different layers, share or prediction weighted information, improve code efficiency.

(Exemplary structures of the second embodiment of encoding device)

Among assembly in Figure 39 shown in diagram, the identical Reference numeral of the assembly identical with the assembly in Fig. 6 represents.By suitably the repetitive description thereof will be omitted.

The structure of encoding device 180 of Figure 39 and the difference of the structure of Fig. 6 are to replace basic coding unit 11, arrange basic coding unit 181, replace strengthening coding unit 12, arrange and strengthen coding unit 182, and VPS are the VPS in Fig. 1 shown in diagram.Encoding device 180 generates the weighting information generated (with reference to image generation information) of the generation of the weighted information for strengthening image, and weighting information generated is included in enhanced flow.

Particularly, primary image is transfused to the basic coding unit 181 of encoding device 180 from outside.Basic coding unit 181 has the structure similar with the encoding device of HEVC mode conventionally, according to HEVC mode encoded base image.Here, basic coding unit 181 is supplied to the weighted information etc. used when encoded base image and strengthens coding unit 182.Basic coding unit 181 as basic stream, is supplied to synthesis unit 13 using the encoding stream comprising coded data, SPS, PPS etc. obtained by coding.

Strengthen image and be transfused to enhancing coding unit 182 from outside.Strengthen coding unit 182 according to the mode observing HEVC mode, coding strengthens image.In addition, the weighted information that coding unit 182 utilizes primary image is strengthened, and the weighted information used when encoding and strengthening image, generate weighted information information generated.

Strengthening coding unit 182 by being added weighted information etc. and the coded data strengthening image, generating encoding stream, then using the encoding stream generated as enhanced flow, be supplied to synthesis unit 13.

(the illustration grammer of the PPS of basic stream)

Figure 40 and 41 is diagrams of the illustration grammer of PPS in graphic extension AVC mode, and Figure 42 and 43 is diagrams of the illustration grammer of the PPS that graphic extension is flowed substantially.

As shown in diagram in the 28th row of Figure 40, the PPS of AVC mode comprises the mark (weighted_pred_flag) indicating whether P section to be weighted to prediction.In addition, as shown in diagram in the 29th row, the PPS of AVC mode comprises the information (weighted_bipred_idc) indicating whether B section to be carried out to bidirectional weighting prediction.In addition, bidirectional weighting prediction is that wherein the image of coded sequence before and after current image to be encoded is used as the weight estimation with reference to image.

In addition, as shown in diagram in the 17th row of Figure 42, the PPS of basic stream comprises mark (weighted_pred_flag).In addition, as shown in diagram in the 18th row, the PPS of basic stream comprises the mark (weighted_bipred_flag) indicating whether B section to be weighted to prediction.

(the illustration grammer of the section head of basic stream)

Figure 44 and 45 is diagrams of the illustration grammer of section head in graphic extension AVC mode.The grammer of the section head of basic stream is the grammer identical with the grammer of Fig. 9-11.

As shown in diagram in the 42nd row of Figure 44, the section head of AVC mode comprises the information (pred_weight_table) relevant to weighted information.

In addition, as shown in diagram in the 19th row of Figure 10, the section head of basic stream also comprises the information (pred_weight_table) relevant to weighted information.

(the illustration grammer of the weighted information of basic stream)

Figure 46 is the diagram of the illustration grammer of weighted information in graphic extension AVC mode, and Figure 47 is the diagram of the illustration grammer of the weighted information that graphic extension is flowed substantially.

As shown in diagram in the 2nd row of Figure 46, the weighted information in AVC mode comprises the denominator (luma_log2_weight_denom) of the weight coefficient of luminance signal.In addition, as shown in diagram in the 4th row, the weighted information in AVC mode comprises the denominator (chroma_log2_weight_denom) of the weight coefficient of carrier chrominance signal.

On the other hand, as shown in the 2nd row of Figure 47, the weighted information of basic stream also comprises the denominator (luma_log2_weight_denom) of the weight coefficient of luminance signal.In addition, as shown in the 4th row, the basic weighted information flowed comprises the difference (delta_chroma_log2_weight_denom) between the denominator of the denominator of the weight coefficient of luminance signal and the weight coefficient of carrier chrominance signal.Utilize described difference (delta_chroma_log2_weight_denom), calculate the weight coefficient (chroma_log2_weight_denom) of carrier chrominance signal with following formula (5).

[mathematical expression 5]

chroma_log2_weight_denom＝luma_log2_weight_denom+delta_chroma_log2_weight_denom...(5)

In addition, as shown in diagram in the 6th row and the 22nd row of Figure 46, the weight coefficient in AVC mode comprises and indicates whether to exist for the mark (luma_weight_l0_flag, luma_weight_l1_flag) of the weighted information of luminance signal.

As shown in diagram in the 7th row, eighth row, the 23rd row and the 24th row, as mark (luma_weight_l0_flag, luma_weight_l1_flag) be 1, when indicating whether to there is the weighted information for luminance signal, weighted information in AVC mode comprises the weight coefficient (luma_weight_l0, luma_weight_l1) of luminance signal.In addition, as shown in diagram in the 9th row and the 25th row, the skew (luma_offset_l0, luma_offset_l1) of luminance signal is comprised.

On the other hand, as shown in diagram in the 6th row and the 23rd row of Figure 47, the weighted information of basic stream also comprises mark (luma_weight_l0_flag, luma_weight_l1_flag).As shown in diagram in the 11st row, the 12nd row, the 28th row and the 29th row, as mark (luma_weight_l0_flag, when luma_weight_l1_flag) being 1, the weighted information of basic stream comprises the information (delta_luma_weight_l0, delta_luma_weight_l1) of the weight coefficient of luminance signal.

Utilize the information (delta_luma_weight_l0 of the weight coefficient of luminance signal, delta_luma_weight_l1), the weight coefficient (luma_weight_l0, luma_weight_l1) of luminance signal is calculated with following formula (6).

[mathematical expression 6]

luma_weight_l0[i]＝(1<<luma_log2_weight_denom)+delta_luma_weight_l0[i]

luma_weight_l1[i]＝(1<<luma_log2_weight_denom)+delta_luma_weight_l1[i]...(6)

In addition, as shown in diagram in the 13rd row and the 30th row of Figure 47, the weighted information of basic stream also comprises the skew (luma_offset_l0, luma_offset_l1) of luminance signal.

In addition, as shown in diagram in the 12nd row and the 28th row of Figure 46, the weighted information in AVC mode comprises and indicates whether to exist for the mark (chroma_weight_l0_flag, chroma_weight_l1_flag) of the weighted information of carrier chrominance signal.

As mark (chroma_weight_l0_flag, chroma_weight_l1_flag) be 1, when there is the weighted information for carrier chrominance signal in instruction, as shown in diagram in the 13rd row to the 15th row and the 29th row to the 31st row, weighted information in AVC mode comprises the weight coefficient (chroma_weight_l0, chroma_weight_l1) of carrier chrominance signal.In addition, as shown in the 16th row and the 32nd row, the skew (chroma_offset_l0, chroma_offset_l1) of carrier chrominance signal is comprised.

On the other hand, as shown in diagram in the 9th row and the 26th row of Figure 47, the weighted information of basic stream also comprises mark (chroma_weight_l0_flag, chroma_weight_l1_flag).As shown in diagram in the 15th row to the 17th row and the 32nd row to the 34th row, as mark (chroma_weight_l0_flag, when chroma_weight_l1_flag) being 1, the weighted information of basic stream comprises the information (delta_chroma_weight_l0, delta_chroma_weight_l1) of the weight coefficient of carrier chrominance signal.

Utilize the information (delta_chroma_weight_l0 of the weight coefficient of carrier chrominance signal, delta_chroma_weight_l1), the weight coefficient (chroma_weight_l0, chroma_weight_l1) of carrier chrominance signal is calculated with following formula (7).

[mathematical expression 7]

chroma_weight_l0[i][j]＝(1<<chroma_log2_weight_denom)+delta_chroma_weight_l0[i][j]

chroma_weight_l1[i][j]＝(1<<chroma_log2_weight_denom)+delta_chroma_weight_l1[i][j]...(7)

In addition, as shown in diagram in the 18th row and the 35th row of Figure 47, the weighted information of basic stream comprises the information (delta_chroma_offset_l0, delta_chroma_offset_l1) of the skew of carrier chrominance signal.Utilize the information (delta_chroma_offset_l0 of the skew of carrier chrominance signal, delta_chroma_offset_l1), the skew (chroma_offset_l0, chroma_offset_l1) of carrier chrominance signal is calculated with following formula (8).

[mathematical expression 8]

chroma_offset_l0[i][j]＝

Clip3(-128,127,(delta_chroma_offset_l0[i][j]

-((shift*chroma_weight_l0[i][j])>>chroma_log2_weight_denom)-shift))

chroma_offset_l1[i][j]

＝Clip3(-128,127,(delta_chroma_offset_l1[i][j]

-((shift*chroma_weight_l1[i][j])>>chroma_log2_weight_denom)-shift))

Here, shift=1<< (BitDepthC-1) ... (8)

(strengthening the Exemplary structures of coding unit)

Figure 48 is the block diagram of the Exemplary structures of the enhancing coding unit 182 of graphic extension Figure 39.

The enhancing coding unit 182 of Figure 48 comprises coding unit 201 and setup unit 202.

The coding unit 201 strengthening coding unit 182 receives the enhancing image of the frame unit from outside input, as input signal.Coding unit 201 with reference to the weighted information etc. supplied from basic coding unit 11, according to observing the mode of HEVC mode, coded input signal.Coding unit 201 is supplied to setup unit 202 the coded data obtained as a result.

Setup unit 202 sets SPS, PPS etc.In addition, PPS does not comprise mark (weighted_pred_flag) and mark (weighted_bipred_flag), is included in the mark (weighted_pred_flag) in the PPS of the basic stream of the encoding stream serving as reference layer and marks mark (weighted_pred_flag) and the mark (weighted_bipred_flag) that (weighted_bipred_flag) is used as to strengthen image.

Thus, when being weighted the reference image of primary image, to strengthening being weighted with reference to image of image, and when not being weighted the reference image of primary image, the reference image strengthening image is not weighted.

Setup unit 202, according to SPS, PPS of setting and the coded data from coding unit 201 supply, generates encoding stream, then the encoding stream generated is supplied to synthesis unit 13 as enhanced flow.

(Exemplary structures of coding unit)

Figure 49 is the block diagram of the Exemplary structures of the coding unit 201 of graphic extension Figure 48.

Among assembly in Figure 49 shown in diagram, the identical Reference numeral of the assembly identical with the assembly of Figure 14 represents.By suitably the repetitive description thereof will be omitted.

The structure of coding unit 201 of Figure 49 and the difference of the structure of Figure 14 are to replace motion prediction/compensating unit 47, with reference to buffer 49, with reference to image setting unit 50 and lossless encoding unit 36, be provided with motion prediction/compensating unit 221, weight buffer 222, weight setting unit 223 and lossless encoding unit 224.

Motion prediction/compensating unit 221 plays weighting processing unit, according to the weight estimation instruction sent from weight setting unit 223, utilizes the weight estimation of all inter-frame forecast modes of alternatively person, carries out motion prediction/compensation deals.Particularly, motion prediction/compensating unit 221 according to the image supplied from screen reorder buffer 32, and by switch 45, from the reference image that frame memory 44 reads, detects the motion vector of all inter-frame forecast modes of alternatively person.Be set by the user for example, referring to image.

Motion prediction/compensating unit 221, according to the motion vector detected, compensates process to reference to image.Motion prediction/compensating unit 221 calculates the weighted information in weight estimation.Motion prediction/compensating unit 221 utilizes the weighted information calculated, and is weighted prediction to the reference image through compensation deals, thus generation forecast image.

Now, motion prediction/compensating unit 221 is according to the image supplied from screen reorder buffer 32 and predicted picture, calculate the cost function value of all inter-frame forecast modes of alternatively person, inter-frame forecast mode minimum for cost function value is defined as best inter-frame forecast mode.Subsequently, motion prediction/compensating unit 221 is supplied to predicted picture selected cell 48 the cost function value of best inter-frame forecast mode and corresponding predicted picture.

In addition, when sending the notice of selection of the predicted picture generated by best inter-frame forecast mode from predicted picture selected cell 48, motion prediction/compensating unit 221 exports to lossless encoding unit 224 inter-frame forecast mode information, corresponding motion vector, reference image appointed information etc.In addition, motion prediction/compensating unit 221 is supplied to weight setting unit 223 the weighted information in weight estimation.

Weight buffer 222 preserves the mark (weighted_bipred_flag) and mark (weighted_pred_flag) that are included in the PPS of the basic stream supplied from the basic coding unit 181 of Figure 39.In addition, the weighted information the section head being included in the basic stream supplied from basic coding unit 181 preserved by weight buffer 222.

Weight setting unit 223, from weight buffer 222, reads mark (weighted_pred_flag) and mark (weighted_bipred_flag).Weight setting unit 223 is according to the kind of the section of mark (weighted_pred_flag), mark (weighted_bipred_flag) and current image to be encoded, and instruction motion prediction/compensating unit 221 is weighted prediction.

Weight setting unit 223 compares the weighted information supplied from motion prediction/compensating unit 221, and is kept at the weighted information in weight buffer 222, determines the predictive mode of the weighted information strengthening image.Here, assuming that the predictive mode of weighted information comprises replication mode, difference prediction pattern and nonanticipating pattern.

Replication mode is the predictive mode of weighted information as the weighted information of enhancing image of wherein reference layer (here, Primary layer).Difference prediction pattern is the difference being added the weighted information strengthened between image and reference layer, and the weighted information of reference layer, thus the predictive mode of the weighted information of generating enhanced images.Nonanticipating pattern is the predictive mode independently setting the weighted information strengthening image with the weighted information of reference layer.

When the predictive mode of weighted information is replication mode, weight setting unit 223 is set as weighting information generated replication mode, and replication mode is supplied to lossless encoding unit 224.In addition, when the predictive mode of weighted information is difference prediction pattern, weight setting unit 223 calculates the difference between weighted information and the weighted information of primary image strengthening image.Subsequently, weight setting unit 223 is set as weighting information generated the difference of the weighted information calculated and difference prediction pattern, then the difference of the weighted information calculated and difference prediction pattern is supplied to lossless encoding unit 224.

In addition, when the predictive mode of weighted information is non-predictive mode, weight setting unit 223 is set as weighting information generated the weighted information strengthening image, then the weighted information strengthening image is supplied to lossless encoding unit 224.

Be similar to the lossless encoding unit 36 of Figure 14, lossless encoding unit 224, from intraprediction unit 46, obtains intraprediction mode information.In addition, lossless encoding unit 224, from motion prediction/compensating unit 221, obtains the inter-frame forecast mode information, motion vector, reference image appointed information etc. that supply from motion prediction/compensating unit 221.In addition, lossless encoding unit 224 obtains weighting information generated from weight setting unit 223.

Be similar to lossless encoding unit 36, lossless encoding unit 224, from self adaptation offset filter 42, obtains skew filtering information, and from auto-adaptive loop filter 43, obtains filter factor.

Be similar to lossless encoding unit 36, lossless encoding unit 224, to the quantization parameter supplied from quantifying unit 35, carries out lossless coding.In addition, lossless encoding unit 224 by intraprediction mode information and inter-frame forecast mode information one of any, motion vector, with reference to image appointed information, weighting information generated, skew filtering information is used as the coded message relevant to coding with filter factor, carries out lossless coding.

Be similar to lossless encoding unit 36, lossless encoding unit 224 is set as the coded message of lossless coding head of cutting into slices, and the coefficient of lossless coding is set as coded data, and section head is added in coded data.Be similar to lossless encoding unit 36, lossless encoding unit 224 provides the coded data of wherein adding section head, to be accumulated in accumulation buffer 37.

(Exemplary structures of weight buffer and weight setting unit)

Figure 50 is the block diagram of the weight buffer 222 of graphic extension Figure 49 and the Exemplary structures of weight setting unit 223.

The weight buffer 222 of Figure 50 comprises information buffer 241 and mark buffer 242.

The information buffer 241 of weight buffer 222 preserves the weighted information be included in the section head of the basic stream supplied from the basic coding unit 181 of Figure 39.Mark buffer 242 is preserved the mark (weighted_bipred_flag) the PPS being included in the basic stream supplied from basic coding unit 181 and is marked (weighted_pred_flag).

The weight setting unit 223 of Figure 50 comprises control unit 261, information buffer 262, identifying unit 263 and setup unit 264.

The control unit 261 of weight setting unit 223, from mark buffer 242, reads mark (weighted_pred_flag) and mark (weighted_bipred_flag).Weight setting unit 223 is according to the kind of the section of mark (weighted_pred_flag), mark (weighted_bipred_flag) and current image to be encoded, and the motion prediction/compensating unit 221 of instruction Figure 49 is weighted prediction.

Information buffer 262 obtains weighted information from motion prediction/compensating unit 221, and preserves described weighted information.Identifying unit 263 reads the weighted information of primary image from information buffer 241, and reads the weighted information strengthening image from information buffer 262.Identifying unit 263 compares the weighted information of the primary image of reading and strengthens the weighted information of image.

Subsequently, when the weighted information of primary image is identical with the weighted information strengthening image, identifying unit 263 is defined as the predictive mode of weighted information replication mode.Identifying unit 263 is supplied to setup unit 264 replication mode.In addition, identifying unit 263 is set as weighting information generated replication mode, then replication mode is supplied to the lossless encoding unit 224 of Figure 49.

When not supplying replication mode from identifying unit 263, setup unit 264, from information buffer 241, reads the weighted information of primary image, and reads the weighted information strengthening image from information buffer 262.According to the input from user, setup unit 264 is defined as difference prediction pattern or nonanticipating pattern the predictive mode of weighted information.

When difference predictive mode is confirmed as the predictive mode of weighted information, setup unit 264 calculates the difference between the weighted information of the primary image read and the weighted information strengthening image.Subsequently, the difference of the weighted information of calculating and difference prediction pattern are set as weighting information generated by weight setting unit 223, then the weighting information generated of setting are supplied to lossless encoding unit 224.

Meanwhile, when nonanticipating pattern is confirmed as the predictive mode of weighted information, setup unit 264 is set as weighting information generated the weighted information and nonanticipating pattern strengthening image, then the weighting information generated of setting is supplied to lossless encoding unit 36.

(grammer of the section head of enhanced flow)

Due to except weighted information, the grammer of the section head of enhanced flow is similar to the grammer of the section head substantially flowed, and therefore the grammer of the information (pred_weight_table) relevant to weighted information is only described.

Figure 51 and 52 is diagrams of the illustration grammer of the weighted information of graphic extension enhanced flow.

As shown in diagram in the 2nd row of Figure 51, the information relevant to weighted information comprises the predictive mode of weighted information (pred_mode).The predictive mode of weighted information is 0 when its instruction replication mode, is 1, and is 2 when indicating nonanticipating pattern when indicating difference prediction pattern.

As shown in the 3rd row and the 4th row, when the predictive mode of weighted information is not 2, that is, when the predictive mode of weighted information be replication mode or difference prediction pattern time, comprise the difference (diff_ref_layer_minus1) between Primary layer and reference layer.Here, when indicating current layer with curr_layer, during by ref_layer instruction reference layer, utilize difference diff_ref_layer_minus1, with following formula (9) statement reference layer ref_layer.

[mathematical expression 9]

ref_layer＝curr_layer-diff_ref_layer_minus1...(9)

In addition, as as shown in the 5th row and the 6th row, when the predictive mode of weighted information is 1, the information relevant to weighted information comprises the difference (diff_luma_log2_weight_denom) of the denominator (luma_log2_weight_denom) between corresponding section and reference layer.In addition, as shown in eighth row, the difference (diff_delta_chroma_log2_weight_denom) of the difference (delta_chroma_log2_weight_denom) between corresponding section and reference layer is comprised.

In addition, relevant to weighted information information does not comprise mark (luma_weight_l0_flag, luma_weight_l1_flag) and mark (chroma_weight_l0_flag, chroma_weight_l1_flag).

In other words, as mentioned above, in the coding utilizing scalable function, when fading in certain scalable image, fade out, cross compound turbine etc. time, think also will fade in other scalable image, fade out, cross compound turbine etc.Thus, when being weighted prediction to the primary image of reference layer, with regard to code efficiency, preferably same to enhancing image be weighted prediction.

Thus, do not transmit the mark (luma_weight_l0_flag strengthening image, luma_weight_l1_flag) and mark (chroma_weight_l0_flag, chroma_weight_l1_flag), but use the mark (luma_weight_l0_flag of reference layer, luma_weight_l1_flag) and mark (chroma_weight_l0_flag, chroma_weight_l1_flag).

In addition, as shown in diagram in the 10th row to the 12nd row and the 22nd row to the 24th row, as the mark (luma_weight_l0_flag of reference layer, when luma_weight_l1_flag) being 1, the information relevant to weighted information comprises the information (delta_luma_weight_l0 of the weight coefficient of the luminance signal between corresponding section and reference layer, delta_luma_weight_l1) difference (diff_delta_luma_weight_l0, diff_delta_luma_weight_l1).In addition, the difference (diff_luma_offset_l0, diff_luma_offset_l1) of the skew (luma_offset_l0, luma_offset_l1) of the luminance signal between corresponding section and reference layer is comprised.

In addition, as shown in diagram in the 14th row to the 17th row and the 26th row to the 29th row, as the mark (chroma_weight_l0_flag of reference layer, when chroma_weight_l1_flag) being 1, the information relevant to weighted information comprises the information (delta_chroma_weight_l0 of the weight coefficient of the carrier chrominance signal between corresponding section and reference layer, delta_chroma_weight_l1) difference (diff_delta_chroma_weight_l0, diff_delta_chroma_weight_l1).In addition, the difference (diff_chroma_offset_l0, diff_chroma_offset_l1) of the skew (chroma_offset_l0, chroma_offset_l1) of the carrier chrominance signal between corresponding section and reference layer is comprised.

Meanwhile, when the predictive mode of weighted information is 2, if the 33rd row at Figure 51 and Figure 52 is to shown in diagram in the 37th row, the information of being correlated with weighted information comprises the information identical with the weighted information substantially flowed.

In addition, the information (pred_weight_table) relevant to weighted information can layer be unit, is included in the layer higher than such as VPS and SPS and so on section head.

(explanation of the process of encoding device)

Weighted information is comprised except supplying from basic coding unit 181 information strengthening coding unit 182, mark (weighted_pred_flag) and mark (weighted_bipred_flag), and outside the enhanced flow generating process of step S2 of carrying out Figure 24, the ges forschung process of the encoding device 180 of Figure 39 is similar to the ges forschung process of Figure 24, thus will proceed the explanation of enhanced flow generating process.

Figure 53 is the flow chart of the enhanced flow generating process of the enhancing coding unit 182 that Figure 48 is described.

At the step S191 of Figure 53, the coding unit 201 strengthening coding unit 182, according to the mode observing HEVC mode, carries out encoding as input signal, from the coded treatment of the enhancing image of the frame unit of outside input.The details of this coded treatment is with reference to the Figure 54 illustrated below and 55 explanations.

In step S192, setup unit 202 sets SPS.In step S193, setup unit 202 sets the PPS not comprising mark (weighted_pred_flag) and mark (weighted_bipred_flag).The process of step S194 and S195 is similar to the step S14 of Figure 25 and the process of S15, thus the description thereof will be omitted.

Figure 54 and 55 is flow charts of the details of the coded treatment of the step S191 that Figure 53 is described.

The step S211 of Figure 54 and the process of S212 are similar to the step S31 of Figure 26 and the process of S32, thus the description thereof will be omitted.

In step S213, the information buffer 241 of weight buffer 222 preserves the weighted information supplied from the basic coding unit 181 of Figure 39, and mark buffer 242 preserves mark (weighted_bipred_flag) and mark (weighted_pred_flag).

In step S214, control unit 261 is according to the mark (weighted_pred_flag) be kept in mark buffer 242 and mark (weighted_bipred_flag), kind with the section of current image to be encoded, determines whether to be weighted prediction.

When in step S214, when judgement will be weighted prediction, control unit 261 instruction motion prediction/compensating unit 221 is weighted prediction, and then process enters step S215.

In step S215, intraprediction unit 46 carries out the intra-prediction process of all intra prediction modes of alternatively person.In addition, intraprediction unit 46 according to the image read from screen reorder buffer 32, and by the predicted picture that intra-prediction process generates, calculates the cost function value of all intra prediction modes of alternatively person.Subsequently, intraprediction unit 46 is defined as optimum frame inner estimation mode intra prediction mode minimum for cost function value.Intraprediction unit 46 is supplied to predicted picture selected cell 48 the predicted picture generated by optimum frame inner estimation mode and corresponding cost function value.

In addition, motion prediction/compensating unit 221 utilizes the weight estimation of all inter-frame forecast modes of alternatively person, carries out motion prediction/compensation deals.In addition, motion prediction/compensating unit 221 is according to the image supplied from screen reorder buffer 32 and predicted picture, calculate the cost function value of all inter-frame forecast modes of alternatively person, inter-frame forecast mode minimum for cost function value is defined as best inter-frame forecast mode.Subsequently, motion prediction/compensating unit 221 is supplied to predicted picture selected cell 48 the cost function value of best inter-frame forecast mode and corresponding predicted picture.

Meanwhile, when in step S214, when determining not to be weighted prediction, control unit 261 makes process enter step S216.

In step S216, be similar to the process of step S215, intraprediction unit 46 carries out the intra-prediction process of all intra prediction modes of alternatively person, and the functional value that assesses the cost.Subsequently, intraprediction unit 46 is defined as optimum frame inner estimation mode intra prediction mode minimum for cost function value, and the predicted picture generated by optimum frame inner estimation mode and corresponding cost function value are supplied to predicted picture selected cell 48.

In addition, motion prediction/compensating unit 221 carries out the motion prediction/compensation deals of all inter-frame forecast modes of alternatively person.In addition, motion prediction/compensating unit 221 is according to the image supplied from screen reorder buffer 32 and predicted picture, calculate the cost function value of all inter-frame forecast modes of alternatively person, inter-frame forecast mode minimum for cost function value is defined as best inter-frame forecast mode.Subsequently, motion prediction/compensating unit 221 is supplied to predicted picture selected cell 48 the cost function value of best inter-frame forecast mode and corresponding predicted picture.

After the process of step S215 and S216, process enters step S217.The process of step S217 and S218 is similar to the step S34 of Figure 26 and the process of S35, thus the description thereof will be omitted.

In step S219, motion prediction/compensating unit 221 is inter-frame forecast mode information, motion vector and be supplied to lossless encoding unit 36 with reference to image appointed information.In step S220, motion prediction/compensating unit 221 determines whether to have carried out weight estimation.When in step S220, when judging to have carried out weight estimation, in step S221, motion prediction/compensating unit 221 is supplied to weight setting unit 223 the weighted information in weight estimation.

In step S222, weight setting unit 223 carries out the generating process generating weighting information generated.The details of generating process is with reference to Figure 56 explanation illustrated below.After the process of step S222, process enters step S224.

Meanwhile, when in step S220, when judging not to be weighted prediction, skip the process of step S221 and S222, then process enters step S224.

Except coded message comprise intraprediction mode information and inter-frame forecast mode information one of any, motion vector, with reference to image appointed information, weighting information generated, offset outside filtering information and filter factor, the process of step S223-S238 is similar to the process of the step S39-S54 of Figure 27, thus the description thereof will be omitted.

Figure 56 is the flow chart of the details of the generating process of the step S222 that Figure 54 is described.

At the step S251 of Figure 56, the information buffer 262 of weight setting unit 223 obtains the weighted information strengthening image from the motion prediction/compensating unit 47 of Figure 49, preserves the weighted information of the enhancing image obtained.In step S252, identifying unit 263 judges that whether the weighted information of the enhancing image be kept in information buffer 262 is identical with the weighted information of the primary image be kept in information buffer 241.

When in step S252, judge to strengthen the weighted information of image identical with the weighted information of primary image time, in step S253, identifying unit 263 is defined as the predictive mode of weighted information replication mode.In step S254, identifying unit 263 is supplied to setup unit 264 replication mode.In addition, identifying unit 263 is set as weighting information generated replication mode, and the replication mode of setting is supplied to the lossless encoding unit 224 of Figure 49.Subsequently, process returns the step S222 of Figure 54, then enters step S224.

Meanwhile, when in step S252, when the weighted information judging to strengthen image is different from the weighted information of primary image, process enters step S255.In step S255, setup unit 264, according to the input from user, judges whether difference prediction pattern is set to the predictive mode of weighted information.

When in step S255, when judging that difference prediction pattern is set to the predictive mode of weighted information, in step S256, setup unit 264 is defined as the predictive mode of weighted information difference prediction pattern.

In step S257, setup unit 264 obtains the difference between weighted information and the weighted information of primary image strengthening image.In step S258, weight setting unit 223 is set as weighting information generated the difference of the weighted information obtained in step S257 and difference prediction pattern, then the weighting information generated of setting is supplied to lossless encoding unit 224.Subsequently, process returns the step S222 of Figure 54, and enters step S224.

Meanwhile, when in step S255, when judging that difference prediction pattern is not set to the predictive mode of weighted information, in step S259, setup unit 264 is defined as the predictive mode of weighted information nonanticipating pattern.

In step S260, setup unit 264 is set as weighting information generated the weighted information and nonanticipating pattern strengthening image, then the weighting information generated of setting is supplied to lossless encoding unit 36.Subsequently, process returns the step S222 of Figure 54, and enters step S224.

As mentioned above, encoding device 180 sets weighting information generated, thus between Primary layer and enhancement layer, can share or predict weighted information.Thus, the amount of information of enhanced flow can be reduced and improve code efficiency.

(Exemplary structures according to the decoding device of the second embodiment)

Figure 57 is the block diagram of the Exemplary structures of the decoding device of the encoding stream of all each layer that graphic extension transmits from the encoding device 180 of Figure 39 according to the decoding of second embodiment of this technology.

Among assembly in Figure 57 shown in diagram, the identical Reference numeral of the assembly identical with Figure 29 represents.By suitably the repetitive description thereof will be omitted.

The structure of decoding device 280 of Figure 57 and the difference of the structure of Figure 29 are replace separative element 92, basic decoding unit 93 and strengthen decoding unit 94, are provided with separative element 281, basic decoding unit 282 and strengthen decoding unit 283.

The separative element 281 of decoding device 280 from supply, is separated and substantially flows, basic stream is supplied to basic decoding unit 282 from the encoding stream of all each layer of receiving element 91, is separated enhanced flow, and enhanced flow is supplied to enhancing decoding unit 283.

Basic decoding unit 282 has the structure identical with the decoding device of HEVC mode conventionally, according to HEVC mode, and the basic stream supplied from separative element 281 of decoding, thus generate primary image.Here, basic decoding unit 282 when decode primary image time the weighted information that uses, and the mark (weighted_bipred_flag) be included in the PPS of basic stream and mark (weighted_pred_flag) are supplied to enhancing decoding unit 283.Basic decoding unit 282 exports the primary image generated.

Strengthen decoding unit 283 according to the mode observing HEVC mode, the enhanced flow supplied from separative element 281 of decoding, thus generating enhanced images.Now, strengthen decoding unit 283 with reference to the weighted information supplied from basic decoding unit 282, mark (weighted_bipred_flag) and mark (weighted_pred_flag), decoding enhanced flow.Strengthen decoding unit 283 and export the enhancing image generated.

(strengthening the Exemplary structures of decoding unit)

Figure 58 is the block diagram of the Exemplary structures of the enhancing decoding unit 283 of graphic extension Figure 57.

The enhancing decoding unit 283 of Figure 58 comprises extraction unit 301 and decoding unit 302.

The extraction unit 301 strengthening decoding unit 283, from supplying from the enhanced flow of the separative element 281 of Figure 57, extracts SPS, PPS, coded data etc., and SPS, PPS, coded data etc. is supplied to decoding unit 302.

The weighted information of the primary image that decoding unit 302 supplies with reference to the basic decoding unit 282 from Figure 57, mark (weighted_bipred_flag) and mark (weighted_pred_flag), according to the mode observing HEVC mode, the coded data supplied from extraction unit 301 of decoding.Now, decoding unit 302 also takes the circumstances into consideration SPS, the PPS etc. with reference to supplying from extraction unit 301.Decoding unit 302 exports the image obtained by decoding, as enhancing image.

(Exemplary structures of decoding unit)

Figure 59 is the block diagram of the Exemplary structures of the decoding unit 302 of graphic extension Figure 58.

In assembly in Figure 59 shown in diagram, the identical Reference numeral of the assembly identical with Figure 31 represents.By suitably the repetitive description thereof will be omitted.

The structure of decoding unit 302 of Figure 59 and the difference of the structure of Figure 31 are to replace lossless decoding unit 132, with reference to buffer 144, with reference to image setting unit 145 and motion compensation units 146, be provided with lossless decoding unit 320, weight buffer 321, weight setting unit 322 and motion compensation units 323.

The lossless decoding unit 320 of decoding unit 302, to the coded data supplied from accumulation buffer 131, carries out the losslessly encoding of such as variable length decoding or arithmetic decoding and so on, obtains quantization parameter and coded message.Lossless decoding unit 320 is supplied to inverse quantization unit 133 quantization parameter.In addition, lossless decoding unit 320 is supplied to intraprediction unit 143 using the intraprediction mode information etc. as coded message, and motion vector, inter-frame forecast mode information, reference image appointed information etc. are supplied to motion compensation units 146.

In addition, lossless decoding unit 320 is supplied to weight setting unit 322 using the weighting information generated as coded message.In addition, lossless decoding unit 320 is supplied to switch 147 using as the intraprediction mode information of coded message or inter-frame forecast mode information.Lossless decoding unit 320 is supplied to self adaptation offset filter 137 using the skew filtering information as coded message, and filter factor is supplied to auto-adaptive loop filter 138.

The weighted information of the primary image supplied from the basic decoding unit 282 of Figure 57 preserved by weight buffer 321, mark (weighted_bipred_flag) and mark (weighted_pred_flag).

Weight setting unit 322 is according to the mark be kept in weight buffer 321 (weighted_bipred_flag) and mark (weighted_pred_flag), and the kind of the section of current image to be decoded, instruction motion compensation units 323 is weighted prediction.

In addition, weight setting unit 322 according to the weighted information of the primary image be kept in weight buffer 321, and from lossless decoding unit 320 supply weighting information generated, the weighted information of generating enhanced images.Weight setting unit 322 is supplied to motion compensation units 323 the weighted information generated.

Motion compensation units 323, by switch 142, reads by the reference image of specifying with reference to image appointed information supplied from lossless decoding unit 320 from frame memory 141.Motion compensation units 323 is according to the weight estimation instruction supplied from weight setting unit 322, utilize motion vector, with reference to image and the weighted information from weight setting unit 322 supply, carry out the motion compensation process of the weight estimation of the best inter-frame forecast mode utilizing inter prediction information to indicate.

Particularly, motion compensation units 323 plays weighting processing unit, according to motion vector, compensates process, utilize weighted information to reference to image, is weighted prediction to the reference image through compensation deals, thus generation forecast image.Motion compensation units 323 is supplied to switch 147 predicted picture.

(Exemplary structures of weight buffer and weight setting unit)

Figure 60 is the block diagram of the weight buffer 321 of graphic extension Figure 59 and the Exemplary structures of weight setting unit 322.

The weight buffer 321 of Figure 60 comprises information buffer 341 and mark buffer 342.

The information buffer 341 of weight buffer 321 preserves the weighted information be included in the section head of the basic stream supplied from the basic decoding unit 282 of Figure 57.Mark buffer 342 is preserved the mark (weighted_bipred_flag) the PPS being included in the basic stream supplied from basic decoding unit 282 and is marked (weighted_pred_flag).

The weight setting unit 322 of Figure 60 comprises control unit 351, pattern buffer 352, information buffer 353 and generation unit 354.

The control unit 351 of weight setting unit 322, from mark buffer 342, reads mark (weighted_pred_flag) and mark (weighted_bipred_flag).Weight setting unit 322 is according to the kind of the section of mark (weighted_pred_flag), mark (weighted_bipred_flag) and current image to be decoded, and instruction motion compensation units 323 is weighted prediction.

Pattern buffer 352 obtains the predictive mode of the weighted information among the weighting information generated that supplies from the lossless decoding unit 320 of Figure 59, and preserves the predictive mode of the weighted information obtained.Information buffer 353 obtains the difference from the weighted information among the weighting information generated of lossless decoding unit 320 supply or weighted information, and preserves the difference of the weighted information of weighted information or the acquisition obtained.

Generation unit 354, from pattern buffer 352, reads the predictive mode of weighted information.When the predictive mode of weighted information be replication mode or difference prediction pattern time, generation unit 354 reads the weighted information of primary image from information buffer 341.When the predictive mode of weighted information is replication mode, generation unit 354 generates the weighted information of primary image, as the weighted information strengthening image.

Meanwhile, when the predictive mode of weighted information is difference prediction pattern, generation unit 354, from information buffer 353, reads the difference of weighted information.Generation unit 354 is added the difference of weighted information and the weighted information of primary image, thus generates the addition value obtained as a result, as the weighted information strengthening image.

In addition, when the predictive mode of weighted information is non-predictive mode, generation unit 354 reads weighted information from information buffer 353, generates weighted information, as the weighted information strengthening image.Generation unit 354 is supplied to motion compensation units 323 the weighted information of the enhancing image generated.

(explanation of the process of decoding device)

Figure 61 is the flow chart of the scalable decoding process of the decoding device 280 that Figure 57 is described.

At the step S280 of Figure 61, the receiving element 91 of decoding device 280 receives the encoding stream of all each layer transmitted from the encoding device 180 of Figure 39, then the encoding stream received is supplied to separative element 281.

In step S281, separative element 281, from the encoding stream of all each layers, is separated basic stream and enhanced flow.Separative element 281 is supplied to basic decoding unit 281 basic stream, enhanced flow is supplied to and strengthens decoding unit 283.

In step S282, basic decoding unit 282, according to HEVC mode, is decoded from the basic stream of separative element 281 supply, thus generates primary image.Now, basic decoding unit 282 is the weighted information that uses during decoding primary image, and the mark (weighted_bipred_flag) be included in the PPS of basic stream and mark (weighted_pred_flag) are supplied to enhancing decoding unit 283.Basic decoding unit 282 exports the primary image generated.

In step S283, strengthen decoding unit 283 with reference to the weighted information supplied from basic decoding unit 282, mark (weighted_bipred_flag), and mark (weighted_pred_flag), carry out the process of enhancing Computer image genration.Except the decoding process of the step S112 of Figure 34, this enhancing Computer image genration process is similar to the enhancing Computer image genration process of Figure 34, thus goes on to say this decoding process below.

Figure 62 is the flow chart of the details of the decoding process of the decoding unit 302 that Figure 59 is described.

At the step S301 of Figure 62, strengthen the extraction unit 301 of accumulation buffer 131 from Figure 58 of decoding unit 283, the coded data of received frame unit, and accumulation coded data.Accumulation buffer 131 is supplied to lossless decoding unit 320 the coded data of accumulation.

In step S302, lossless decoding unit 320, to the coded data supplied from accumulation buffer 131, carries out losslessly encoding, obtains quantization parameter and coded message.Lossless decoding unit 320 is supplied to inverse quantization unit 133 quantization parameter.In addition, lossless decoding unit 320 is supplied to intraprediction unit 143 using the intraprediction mode information etc. as coded message, motion vector, inter-frame forecast mode information, is supplied to motion compensation units 323 with reference to image appointed information etc.

The process of step S303 and S304 is similar to the step S133 of Figure 35 and the process of S134, thus the description thereof will be omitted.

In step S305, the information buffer 341 (Figure 60) of weight buffer 321 preserves the weighted information of the primary image supplied from the basic decoding unit 282 of Figure 57, and mark buffer 342 preserves mark (weighted_bipred_flag) and mark (weighted_pred_flag).

In step S306, control unit 351 is according to the mark (weighted_bipred_flag) be kept in mark buffer 342 and mark (weighted_pred_flag), and the kind of the section of current image to be decoded, determine whether to be weighted prediction.

When in step S306, when judgement will be weighted prediction, control unit 351 instruction motion compensation units 323 is weighted prediction, and process enters step S307.In step S307, weight setting unit 322, according to the weighting information generated supplied from lossless decoding unit 320, is kept at the weighted information etc. of the primary image in weight buffer 321, carries out the generating process of the weighted information of generating enhanced images.The details of generating process is with reference to Figure 63 explanation illustrated below.

In step S308, motion compensation units 323 is according to the reference image appointed information supplied from lossless decoding unit 320, read with reference to image, utilize motion vector, with reference to image and the weighted information from weight setting unit 322 supply, carry out the motion compensation process of the weight estimation of the best inter-frame forecast mode utilizing inter prediction information to indicate.Motion compensation units 146, through switch 147, is supplied to adder unit 135 the predicted picture generated as a result, and then process enters step S311.

Simultaneously, when in step S306, when judging not to be weighted prediction, in step S309, motion compensation units 323 is according to the reference image appointed information supplied from lossless decoding unit 320, read with reference to image, and utilize motion vector and with reference to image, carry out the motion compensation process of the best inter-frame forecast mode utilizing inter-frame forecast mode information to indicate.Motion compensation units 146, through switch 147, provides adder unit 135 the predicted picture generated as a result, and then process enters step S311.

The process of step S310-S318 is similar to the process of the step S138-S146 of Figure 35, thus the description thereof will be omitted.

Figure 63 is the flow chart of the details of the generating process of the step S307 that Figure 62 is described.

At the step S330 of Figure 63, the pattern buffer 352 of weight setting unit 322 obtains the predictive mode of the weighted information among the weighting information generated that supplies from the lossless decoding unit 320 of Figure 59, and preserves the predictive mode of the weighted information obtained.

In step S331, generation unit 354 judges whether the predictive mode of the weighted information be kept in pattern buffer 352 is non-predictive mode.

When in step S331, when the predictive mode judging weighted information is not non-predictive mode, that is, when the predictive mode of weighted information be replication mode or difference prediction pattern time, process enters step S332.In step S332, read the weighted information of primary image from information buffer 341.

In step S333, generation unit 354 judges whether the predictive mode of weighted information is replication mode.When in step S333, when the predictive mode of judgement weighted information is replication mode, in step S334, generation unit 354 generates the weighted information of primary image, as the weighted information strengthening image, and the weighted information of the primary image generated is supplied to motion compensation units 323.Subsequently, generating process terminates.

Meanwhile, when in step S333, when the predictive mode judging weighted information is not replication mode, that is, when the predictive mode of weighted information is difference prediction pattern, process enters step S335.In step S335, information buffer 353 obtains the difference from the weighted information among the weighting information generated of lossless decoding unit 320 supply, and preserves the difference of weighted information.In step S336, generation unit 354, from information buffer 353, reads the difference of weighted information.

In step S337, generation unit 354 is added the difference of weighted information and the weighted information of primary image, thus generates the addition value obtained as a result, as the weighted information strengthening image.Generation unit 354 is supplied to motion compensation units 323 the weighted information of the enhancing image generated, and then, generating process terminates.

In addition, when in step S331, when the predictive mode of judgement weighted information is non-predictive mode, in step S338, information buffer 353 obtains from the weighted information among the weighting information generated of lossless decoding unit 320 supply, and preserves the weighted information obtained.In step S339, generation unit 354, from information buffer 353, reads weighted information.

In step S340, generation unit 354 is created on the weighted information read in step S339, as the weighted information strengthening image, and the weighted information generated is supplied to motion compensation units 323.Subsequently, generating process terminates.

As mentioned above, decoding device 280 utilizes weighting information generated, the weighted information of generating enhanced images, thus between Primary layer and enhancement layer, can share or predict weighted information.Thus, the amount of information of enhanced flow can be reduced and improve code efficiency.

In addition, in a second embodiment, be similar to the first embodiment, VPS can be the VPS in Figure 23 shown in diagram.

In addition, in the first and second embodiment, the number of plies is 2, but the number of plies can be 2 or larger.Can picture be unit, or set reference layer in units of GOP.So same in 3rd embodiment described below and the 4th embodiment.

In addition, in the first and second embodiment, primary image is encoded according to HEVC mode, but can encode according to AVC mode.In this case, in a second embodiment, through type (5)-(8), the information (pred_weight_table) relevant to the weighted information in AVC mode of primary image is converted to the information (pred_weight_table) relevant with the weighted information in HEVC mode, uses the information after conversion subsequently.

< the 3rd embodiment >

(explanation of the overview of the 3rd embodiment)

Figure 64 is the diagram of the overview of the 3rd embodiment illustrated according to this technology.

As shown in diagram in Figure 64, there are many situations that GOP (image sets) structure of wherein primary image (BL) is identical with the gop structure strengthening image (EL).In other words, in the identical time, primary image is identical all the time in section kind with the reference relation of time orientation with enhancing image.Thus in the third embodiment, between the different layers, share with reference to image appointed information.Particularly, as reference image appointed information of certain section of enhancing image, use the reference image appointed information with the section of the juxtaposed primary image of this section.

In addition, " juxtaposition " position of referring on screen is mutually corresponding.Such as, be comprise the position with regard on screen with the section of the juxtaposed primary image of section strengthening image, with the section of the LCU at primary image corresponding to the LCU of head of the section strengthening image.

(Exemplary structures according to the encoding device of the 3rd embodiment)

In each assembly in Figure 65 shown in diagram, the identical Reference numeral of the assembly identical with Fig. 6 represents.By suitably the repetitive description thereof will be omitted.

The structure of encoding device 400 of Figure 65 and the difference of the structure of Fig. 6 are to replace strengthening coding unit 12, are provided with and strengthen coding unit 411.In encoding device 400, between primary image and enhancing image, share all reference image appointed information.

Particularly, image is transfused to encoding device 400 enhancing coding unit 411 from outside is strengthened.Strengthen coding unit 411 according to the mode observing HEVC mode, coding strengthens image.

In addition, strengthen the reference image appointed information that coding unit 411 utilizes the primary image supplied from basic coding unit 11, never mark is set as specifying information generated (with reference to image generation information) with reference to image.Subsequently, strengthen coding unit 411 and never mark etc. is added in coding result, thus generate coded data.In addition, never mark is all marks with reference to image appointed information whether being used as strengthening image with reference to image appointed information of instruction primary image.

Strengthening coding unit 411 using comprising the encoding stream of coded data, SPS, PPS etc. as enhanced flow, being supplied to synthesis unit 13.

Here, assuming that encoding device 400 transmits the encoding stream of all each layers, but where necessary, can only transmit basic stream.

(strengthening the Exemplary structures of coding unit)

Figure 66 is the block diagram of the Exemplary structures of the enhancing coding unit 411 of graphic extension Figure 65.

The enhancing coding unit 411 of Figure 66 comprises setup unit 431 and coding unit 432.

The setup unit 431 strengthening coding unit 411 takes the circumstances into consideration to set the parameter set comprising never mark, such as SPS and PPS.Setup unit 431 is supplied to coding unit 432 the parameter set of setting.

Coding unit 432 receives as input signal, and from the enhancing image of the frame unit of outside input, according to the mode observing HEVC mode, coding strengthens image.In addition, coding unit 432 according to the reference image appointed information used when encoding, and from the reference image appointed information that basic coding unit 11 supplies, sets never mark.

Coding unit 432 according to the never mark of setting, and is included in from the never mark the SPS of setup unit 431 supply, never mark etc. is added in coding result, thus generates coded data.Subsequently, coding unit 432, according to coded data and the parameter set from setup unit 431 supply, generates enhanced flow, and enhanced flow is supplied to the synthesis unit 13 of Figure 65.

(Exemplary structures of coding unit)

Figure 67 is the block diagram of the Exemplary structures of the coding unit 432 of graphic extension Figure 66.

In each assembly in Figure 67 shown in diagram, the identical Reference numeral of the assembly identical with Figure 14 represents.By suitably the repetitive description thereof will be omitted.

The structure of coding unit 432 of Figure 67 and the difference of the structure of Figure 14 are to replace lossless encoding unit 36, accumulation buffer 37 and reference image setting unit 50, are provided with lossless encoding unit 451, accumulation buffer 452 and reference image setting unit 453.

Lossless encoding unit 451, from intraprediction unit 46, obtains intraprediction mode information.In addition, lossless encoding unit 451, from motion prediction/compensating unit 47, obtains the inter-frame forecast mode information, motion vector etc. that supply from motion prediction/compensating unit 47.In addition, lossless encoding unit 451, from reference to image setting unit 453, obtains RPS mark, never mark etc.In addition, lossless encoding unit 451, from self adaptation offset filter 42, obtains information filter information, from auto-adaptive loop filter 43, obtains filter factor.

Lossless encoding unit 451, to the quantization parameter supplied from quantifying unit 35, carries out lossless coding.In addition, lossless encoding unit 451 pairs of intraprediction mode information or inter-frame forecast mode information one of any, motion vector, RPS mark, never mark, skew filtering information and filter factor carry out lossless coding, as coded message.Lossless encoding unit 451 adds the coded message of lossless coding in the coefficient of lossless coding, thus generates coded data.Lossless encoding unit 451 supplies coded data, to be accumulated in accumulation buffer 452.

The coded data supplied from lossless encoding unit 451 preserved by accumulation buffer 452 temporarily.In addition, accumulation buffer 452 supplies the coded data of preserving parameter set together with the setup unit 431 from Figure 66, is supplied to synthesis unit 13 (Figure 65) as enhanced flow.

Compare the reference image appointed information supplied from motion prediction/compensating unit 47 with reference to image setting unit 453, and be kept at reference to the reference image appointed information in buffer 49, and set never mark.Subsequently, when the never mark set is different from the never mark the SPS being included in and supplying from setup unit 431, with reference to image setting unit 453, the never mark of setting is supplied to lossless encoding unit 451.

In addition, with reference to the never mark of image setting unit 453 according to setting, compare the reference image appointed information supplied from motion prediction/compensating unit 47, and be included in the reference image appointed information in SPS, setting RPS mark, and RPS mark is supplied to lossless encoding unit 451.In addition, mark according to RPS with reference to image setting unit 453, lossless encoding unit 451 is supplied to reference to image appointed information what supply from motion prediction/compensating unit 47, and specifying the index with reference to image appointed information identical with this reference image appointed information be included in SPS, be supplied to lossless encoding unit 451.

(first example of the grammer of the SPS of enhanced flow)

Figure 68 is the diagram of the illustration grammer of the SPS that graphic extension is set by the setup unit 431 of Figure 66.

As shown in diagram in the 3rd row of Figure 68, never mark (inter_layer_copy_flag) is set in the SPS of enhanced flow.When the reference image appointed information of primary image is used as the reference image appointed information strengthening image, never mark is 1, and when the reference image appointed information of primary image is not used as the reference image appointed information strengthening image, never mark is 0.

In addition, if the 4th row at Figure 68 is to shown in diagram in the 14th row, when never mark is 0, is similar to the SPS of basic stream, sets the information relevant to RPS, and the information relevant to long-term reference image appointed information.

(first example of the grammer of the section head of enhanced flow)

Figure 69 is the diagram of the illustration grammer of the section head of graphic extension enhanced flow.

As shown in diagram in the 5th row of Figure 69, when the never mark of correspondence section is different from the never mark be included in SPS, never mark is set to the section head of enhanced flow.In addition, as shown in diagram in the 6th row to the 15th row, when the never mark that correspondence is cut into slices is 0, be similar to the section head of basic stream, the index of setting RPS mark and RPS or RPS, and according to long-term mark, set long-term reference image appointed information.

(explanation of effect)

Figure 70 is the diagram of the effect illustrated in encoding device 400.

As mentioned above, in encoding device 400, when never mark is set to SPS, when the never mark of each section is different from the never mark of SPS, never mark is set to the section head of cutting into slices.Thus only have when never mark is included in section head, the decoding device illustrated below just must upgrade never mark, thus can easily carry out utilizing never mark, identifies the process with reference to image appointed information.

On the other hand, at Do-Kyoung Kwon, Madhukar Budagavi, " Inter-layer slice header syntax element prediction in SHVC " (JCTVC-L0231 of Minhua Zhou, 2013.1.14-1.23) in, propose never mark not to be set to SPS, as shown in the 5th row of Figure 70, never mark (inter_layer_rps_prediction_flag) is set to each section head all the time.In this case, in decoding device, in units of section, never mark must be upgraded, thus make to utilize never mark, identify that the process with reference to image appointed information is complicated.

(explanation of first example of the process of encoding device)

Figure 71 is the flow chart of the ges forschung process of the encoding device 400 that Figure 65 is described.

At the step S361 of Figure 71, the basic coding unit 11 of encoding device 400, according to HEVC mode, is encoded from the primary image of outside input, is added parameter set, thus generates basic stream.Subsequently, basic coding unit 11 is supplied to synthesis unit 13 basic stream.

In step S362, basic coding unit 11 strengthens coding unit 411 exporting to reference to image appointed information of primary image.

In step S363, setup unit 431 (Figure 66) setting strengthening coding unit 411 strengthens the parameter set of image.In step S364, coding unit 432 carries out the enhancing coded treatment of encoding from the enhancing image of outside input.

Except replacing the generating process of step S38 of Figure 26, carry out the never mark setting process setting never mark etc., and outside the process of step S54 not carrying out Figure 27, strengthen the coded treatment that coded treatment is similar to Figure 26 and 27.Thus never mark setting process describes in detail with reference to the Figure 73 illustrated below.

In step S365, the accumulation buffer 452 (Figure 67) of coding unit 432 according to the coded data generated in step S364, and from the parameter set that setup unit 431 supplies, generates enhanced flow, and enhanced flow is exported to synthesis unit 13.

In step S366, synthesis unit 13 synthesizes the basic stream supplied from basic coding unit 11 and the enhanced flow supplied from enhancing coding unit 411, adds VPS etc., thus generates the encoding stream of all each layers.Synthesis unit 13 is supplied to transmission unit 14 the encoding stream of all each layers.

In step S367, transmission unit 14 sends the encoding stream of all each layer supplied from synthesis unit 13 decoding device illustrated to below, and then process terminates.

Figure 72 is the flow chart of the details of the SPS setting process of setting SPS in the process of the step S363 that Figure 71 is described.

In the step S381 of Figure 72, setup unit 431 is set to SPS never mark.In step S382, setup unit 431 judges that whether the never mark being set to SPS is as 1.

When in step S381, when judging that never mark is not 1, that is, when never mark is 0, in step S383, setup unit 431 is set to SPS RPS.

In step S384, setup unit 431 long-term flag settings to SPS.In step S385, setup unit 431 judges to be set to marking whether for a long time as 1 of SPS.When in step S385, when judging to be labeled as 1 for a long time, in step S386, setup unit 431 sets long-term reference image appointed information, and then process terminates.

Meanwhile, when in step S382, judge never mark as 1 time, or when in step S385, judge long-term mark not as 1 time, process terminates.

Figure 73 is the flow chart of the details of the never mark setting process of the enhancing coded treatment of the step S364 that Figure 71 is described.Such as, in units of cutting into slices, carry out never mark setting process.

The step S390 of Figure 73, Figure 67 with reference to image setting unit 453 from motion prediction/compensating unit 47, obtain strengthen image with reference to image appointed information.In step S391, with reference to image setting unit 453 from reference to buffer 49, read the reference image appointed information of primary image.

In step S392, judge to strengthen the whether identical with the reference image appointed information of primary image with reference to image appointed information of image with reference to image setting unit 453.When in step S392, when judging that the reference image appointed information of enhancing image is identical with the reference image appointed information of primary image, in step S393, with reference to image setting unit 453, never mark is set as 1.

In step S394, judge that whether the never mark of the SPS supplied from the setup unit 431 of Figure 66 is as 1 with reference to image setting unit 453.When in step S394, judge never mark as 1 time, process terminates.

Meanwhile, when in step S394, judge never mark not as 1 time, with reference to image setting unit 453, the never mark of setting is supplied to lossless encoding unit 451, then process terminates.

Meanwhile, when in step S392, when judging that the reference image appointed information of enhancing image is different from the reference image appointed information of primary image, in step S396, with reference to image setting unit 453, never mark is set as 0.In step S397, judge that with reference to image setting unit 453 whether the never mark of the SPS supplied from setup unit 431 is as 1.

When in step S397, judge the never mark of SPS as 1 time, in step S398, with reference to image setting unit 453, the never mark of setting is supplied to lossless encoding unit 451, then process enters step S399.

Meanwhile, when in step S397, the never mark judging SPS not as 1 time, that is, when the never mark of SPS is identical with the never mark of setting, namely 0 time, skip the process of step S398, process enters step S399.

In step S399, judge that with reference to image setting unit 453 whether the RPS the reference image appointed information supplied from motion prediction/compensating unit 47 is identical with the RPS of SPS.When in step S399, when judging that the RPS of RPS with SPS supplied from motion prediction/compensating unit 47 is identical, in step S400, reference image setting unit 453 is 1 RPS flag settings.

In step S401, with reference to image setting unit 453, the index of the RPS of the SPS identical with the RPS supplied from motion prediction/compensating unit 47 is supplied to lossless encoding unit 451, then process enters step S404.

Meanwhile, when in step S399, when judging that the RPS of RPS with SPS supplied from motion prediction/compensating unit 47 is different, in step S402, reference image setting unit 453 is 0 RPS flag settings.

In step S403, with reference to image setting unit 453, the RPS supplied from motion prediction/compensating unit 47 is supplied to lossless encoding unit 451, then process enters step S404.

In step S404, judge to be included in marking whether for a long time as 1 in SPS with reference to image setting unit 453.When in step S404, when judging to be labeled as 1 for a long time, process enters step S405.

In step S405, be supplied to lossless encoding unit 451 what supply from motion prediction/compensating unit 47 with reference to the long-term reference image appointed information etc. among image appointed information with reference to image setting unit 453.Particularly, when the long-term long-term reference image appointed information with reference to the SPS among image appointed information supplied from motion prediction/compensating unit 47 is different, with reference to image setting unit 453, the index of identical information is supplied to lossless encoding unit 451.Subsequently, process terminates.

As mentioned above, in encoding device 400, due to setting never mark, therefore between Primary layer and enhancement layer, can share with reference to image appointed information.Thus, the amount of information of enhanced flow can be reduced, and improve code efficiency.

(Exemplary structures according to the decoding device of the 3rd embodiment)

Figure 74 is the block diagram of the Exemplary structures of the decoding device of the encoding stream of all each layer that graphic extension transmits from the encoding device 400 of Figure 65 according to the decoding of third embodiment of the present disclosure.

In each assembly in Figure 74 shown in diagram, the identical Reference numeral of the assembly identical with Figure 29 represents.By suitably the repetitive description thereof will be omitted.

The difference of the structure of the structure of the decoding device 470 of Figure 74 and the decoding device 90 of Figure 29 is to replace strengthening decoding unit 94, is provided with and strengthens decoding unit 471.

The enhancing decoding unit 471 of decoding device 470, according to the mode observing HEVC mode, is decoded from the enhanced flow of separative element 92 supply, thus generating enhanced images.Now, strengthen the reference image appointed information of decoding unit 471 with reference to the primary image supplied from basic decoding unit 93, be included in the never mark etc. in SPS or section head.Strengthen decoding unit 471 and export the enhancing image generated.

(strengthening the Exemplary structures of decoding unit)

Figure 75 is the block diagram of the Exemplary structures of the enhancing decoding unit 471 of graphic extension Figure 74.

In each assembly in Figure 75 shown in diagram, the identical Reference numeral of the assembly identical with Figure 30 represents.By suitably the repetitive description thereof will be omitted.

The difference of the structure of the structure of the enhancing decoding unit 471 of Figure 75 and the enhancing decoding unit 94 of Figure 30 is to replace decoding unit 112, is provided with decoding unit 491.

The reference image appointed information of the primary image that the decoding unit 491 strengthening decoding unit 471 supplies with reference to the basic decoding unit 93 from Figure 74, with the never mark be included in from the SPS or section head of extraction unit 111 supply, according to the mode observing HEVC mode, the coded data supplied from extraction unit 111 of decoding.Decoding unit 491 exports the image obtained by decoding, as enhancing image.

(Exemplary structures of decoding unit)

Figure 76 is the block diagram of the Exemplary structures of the decoding unit 491 of graphic extension Figure 75.

In each assembly in Figure 76 shown in diagram, the identical Reference numeral of the assembly identical with Figure 31 represents.By suitably the repetitive description thereof will be omitted.

The difference of the structure of the structure of the decoding unit 491 of Figure 76 and the decoding unit 112 of Figure 31 is to replace lossless decoding unit 132, is provided with lossless decoding unit 511, replaces, with reference to image setting unit 145, being provided with reference to image setting unit 512.

The lossless decoding unit 511 of decoding unit 491, to the coded data supplied from accumulation buffer 131, carries out losslessly encoding, thus obtains quantization parameter and coded message.Lossless decoding unit 511 is supplied to inverse quantization unit 133 quantization parameter.In addition, lossless decoding unit 511 is supplied to intraprediction unit 143 using the intraprediction mode information etc. as coded message, and motion vector, inter-frame forecast mode information etc. are supplied to motion compensation units 146.

In addition, lossless decoding unit 511 using as coded message never mark, RPS mark, be supplied to reference to image setting unit 512 with reference to the index etc. of image appointed information, RPS.In addition, lossless decoding unit 511 is supplied to switch 147 using as the intraprediction mode information of coded message or inter-frame forecast mode information.Lossless decoding unit 511 is supplied to self adaptation offset filter 137 using the skew filtering information as coded message, and filter factor is supplied to auto-adaptive loop filter 138.

The never mark be included in the SPS supplied from the extraction unit 111 of Figure 75 is kept with reference to image setting unit 512, and with reference to image appointed information.When supplying never mark from lossless decoding unit 511 and with reference to image appointed information, upgrade the never mark and reference image appointed information that remain on wherein with reference to image setting unit 512.

Subsequently, with reference to the never mark of image setting unit 512 according to maintenance, from the reference image appointed information reading primary image with reference to buffer 144, the reference image appointed information being defined as current slice with reference to image appointed information of the primary image read.In addition, with reference to image setting unit 512 according to never mark and the RPS mark that supplies from lossless decoding unit 511, the index supplied from lossless decoding unit 511 with reference to image appointed information or maintenance upgraded with reference to image appointed information be defined as current slice with reference to image appointed information.

(explanation of first example of the process of decoding device)

Except the enhancing Computer image genration process of the step S106 of Figure 33, the scalable decoding process of the decoding device 470 of Figure 74 is similar to the scalable decoding process of Figure 33.Thus, explanation is strengthened Computer image genration process below.

Figure 77 is the flow chart of the enhancing Computer image genration process of the enhancing decoding unit 471 that Figure 74 is described.

At the step S431 of Figure 77, strengthen the extraction unit 111 of decoding unit 471 from supplying from the enhanced flow of the separative element 92 of Figure 74, extract parameter set and the coded data of such as SPS or PPS and so on, then parameter set and coded data are supplied to decoding unit 491.

In step S432, the reference image setting unit 512 of decoding unit 491 carries out, from supply from the SPS of extraction unit 111, extracting the SPS extraction process of never mark etc.The details of SPS extraction process is with reference to Figure 78 explanation illustrated below.

In step S433, decoding unit 491 carries out the never mark with reference to being included in SPS or section head, from the reference image appointed information etc. that basic decoding unit 93 supplies, according to the mode observing HEVC mode, decode from the enhancing decoding process of the coded data of extraction unit 111 supply.Except the generating process of the step S136 of Figure 35, this enhancing decoding process is similar to the decoding process of Figure 35.Thus, below with reference to the Figure 79 illustrated, described generating process is described below.

Figure 78 is the flow chart of the details of the SPS extraction process of the step S432 that Figure 77 is described.

At the step S451 of Figure 78, with reference to image setting unit 512 from supplying from the SPS of extraction unit 111, extract never mark, and keep the never mark of extraction.In step S452, judge that whether the never mark extracted is as 1 with reference to image setting unit 512.

When in step S452, judge never mark not as 1 time, that is, when never mark is 0, in step S453, from SPS, extract RPS with reference to image setting unit 512, keep the RPS extracted.In step S454, from SPS, extract long-term mark with reference to image setting unit 512.

In step S455, judge to mark whether as 1 for a long time with reference to image setting unit 512, when judging to be labeled as 1 for a long time, process enters step S456.In step S456, from SPS, extract long-term reference image appointed information with reference to image setting unit 512, and keep the long-term reference image appointed information of extraction.Subsequently, process returns the step S432 of Figure 77, then enters step S433.

Meanwhile, when in step S452, judge never mark as 1 time, or when in step S455, judge long-term mark not as 1 time, process returns the step S432 of Figure 77, then enters step S433.

Figure 79 is the flow chart of details of the generating process in the enhancing decoding process of the step S433 that Figure 77 is described.Such as, generating process is carried out in units of cutting into slices.

At the step S471 of Figure 79, determine whether from lossless decoding unit 511 for giving never mark with reference to image setting unit 512.When in step S471, judge for when giving never mark, in step S472, with reference to image setting unit 512, the never mark be included in SPS kept is updated to the never mark supplied from lossless decoding unit 511.Subsequently, process enters step S473.

Meanwhile, when in step S471, judge that process enters step S473 for when giving never mark.

In step S473, judges that whether the never mark kept is as 1 with reference to image setting unit 512, when judge the never mark of maintenance as 1 time, process enters step S474.

In step S474, with reference to image setting unit 512 from reference to buffer 144, read the reference image appointed information of primary image.In step S475, with reference to image setting unit 512 the reference image appointed information being defined as current slice with reference to image appointed information of primary image.Subsequently, be supplied to motion compensation units 146 what determine with reference to image appointed information with reference to image setting unit 512, then process terminates.

Meanwhile, when in step S473, judge never mark not as 1 time, that is, when never mark is 0, process enters step S476.In step S476, judge that RPS marks whether as 1 with reference to image setting unit 512.When in step S476, when judging that RPS is labeled as 1, in step S477, obtain the index with reference to image appointed information supplied from lossless decoding unit 511 with reference to image setting unit 512.

In step S478, with reference to image setting unit 512 keep be included in SPS with reference in image appointed information, the reference image appointed information being defined as current slice with reference to image appointed information that the index obtained is assigned to, is then supplied to motion compensation units 146 what determine with reference to image appointed information.

In addition, now, when be included in SPS be labeled as 1 for a long time time, from lossless decoding unit 511 supply long-term with reference to image appointed information be also confirmed as current slice with reference to image appointed information, and be provided to motion compensation units 146.After the process of step S478, process terminates.

Meanwhile, when in step S476, judge RPS mark not as 1 time, in step S479, with reference to image setting unit 512 obtain from lossless decoding unit 511 supply with reference to image appointed information.Subsequently, with reference to image setting unit 512, the reference image appointed information be included in SPS kept is updated to this reference image appointed information.

In step S480, with reference to image setting unit 512 the reference image appointed information being defined as current slice with reference to image appointed information upgraded, and be supplied to motion compensation units 146 what determine with reference to image appointed information.Subsequently, process terminates.

After generating process terminates, remain on reference to the never mark in image setting unit 512 and be restored to reference to image appointed information and be included in never mark in SPS and with reference to image appointed information.

As mentioned above, by utilizing never mark, the reference image appointed information of decoding device 470 generating enhanced images, thus between Primary layer and enhancement layer, can share with reference to image appointed information.Thus, owing to sharing with reference to image appointed information between Primary layer and enhancement layer, therefore, it is possible to code efficiency improves ground decoding enhanced flow.

In addition, in superincumbent explanation, to RPS and long-term reference image appointed information, set public never mark, but independent never mark can be set.Below using second example as the 3rd embodiment, this situation is described.In addition, below, second example of the 3rd embodiment is called as " during the setting of another never mark ", and first example is called as " during public never mark setting ".

(second example of the grammer of the enhanced flow of SPS)

When Figure 80 is illustrated in the setting of another never mark, the diagram of the illustration grammer of the SPS set by the setup unit 431 of Figure 66.

As shown in diagram in the 3rd row of Figure 80, the RPS never mark (inter_layer_short_copy_flag) of serving as the never mark of RPS is configured to SPS.When the RPS of primary image is used as the RPS strengthening image, RPS never mark is 1, and when being not used as the RPS strengthening image as the RPS of primary image, RPS never mark is 0.

In addition, as shown in diagram in the 4th row to eighth row of Figure 80, when RPS never mark is 0, be similar to the SPS of basic stream, the information relevant to RPS is set to SPS.

As shown in diagram in the 9th row of Figure 80, long-term mark is set to SPS.As shown in the 10th row and the 11st row, when being labeled as 1 for a long time, the long-term never mark (inter_layer_long_copy_flag) for the long-term never mark with reference to image appointed information is served as in setting.

When primary image long-term with reference to image appointed information be used as to strengthen image long-term with reference to image appointed information time, long-term never mark is 1.Meanwhile, when primary image long-term with reference to image appointed information be not used as strengthen image long-term with reference to image appointed information time, long-term never mark is 0.

If the 12nd row at Figure 80 is to shown in diagram in the 17th row, when long-term never mark is 0, the long-term number (num_long_term_ref_pics_sps) with reference to image appointed information, and long-term reference image appointed information is set to SPS.

(second example of the grammer of the section head of enhanced flow)

Figure 81 is illustrated in the diagram of the illustration grammer of the section head of enhanced flow when another never mark sets.

As shown in diagram in the 5th row of Figure 81, when the RPS never mark of correspondence section is different from the RPS never mark be included in SPS, RPS never mark is set to the section head of enhanced flow.In addition, if the 6th row at Figure 81 is to shown in diagram in the 12nd row, when the RPS never mark of correspondence section is 0, the section head of basic stream is similar to, the index of setting RPS mark and RPS or RPS.

In addition, if the 13rd row at Figure 81 is to shown in diagram in the 17th row, when being labeled as 1 for a long time, long-term never mark is set to section situation, when long-term never mark is 0, sets long-term with reference to image appointed information.

(explanation of second example of the process of encoding device)

Except SPS setting process and never mark setting process, the ges forschung process when another never mark sets is similar to the ges forschung process of Figure 71.Thus, the following describes described SPS setting process and never mark setting process.

Figure 82 is the flow chart of the SPS setting process illustrated when another never mark sets.

At the step S501 of Figure 82, setup unit 431 is set to SPS RPS never mark.In step S502, setup unit 431 judges that whether the RPS never mark being set to SPS is as 1.

When in step S502, judge RPS never mark not as 1 time, that is, when RPS never mark is 0, in step S503, setup unit 431 is set to SPS RPS, and then process enters step S504.Meanwhile, when in step S502, judge RPS never mark as 1 time, process enters step S504.

In step S504, setup unit 431 long-term flag settings to SPS.In step S505, setup unit 431 judges to be set to marking whether for a long time as 1 of SPS.When in step S505, when judging to be labeled as 1 for a long time, in step S506, setup unit 431 is set to SPS long-term never mark.

In step S507, judge that whether the long-term never mark being set to SPS is as 1.When in step S507, judge long-term never mark not as 1 time, that is, when long-term never mark is 0, in step S508, setup unit 431 sets long-term with reference to image appointed information, and then process terminates.

Meanwhile, when in step S505, judge long-term mark not as 1 time, or when in step S507, judge long-term never mark as 1 time, process terminates.

Figure 83 is the flow chart of the never mark setting process illustrated when another never mark sets.Such as, in units of cutting into slices, carry out this never mark setting process.

The step S521 of Figure 83 and the process of S522 are similar to the step S390 of Figure 73 and the process of S391, thus the description thereof will be omitted.In addition, except replacing never mark by RPS never mark, replace with reference to outside image appointed information with RPS, the process of step S523-S534 is similar to the process of the step S392-S403 of Figure 73, thus the description thereof will be omitted.

When in step S525, judge the RPS never mark of SPS as 1 time, or after the process of step S526, S532 or S534, process enters step S535.

Whether identical with the long-term reference image appointed information of primary image in the long-term reference image appointed information with reference to image setting unit 453 judgement enhancing image of step S535, Figure 67.When in step S535, when judging that the long-term reference image appointed information of enhancing image is identical with the long-term reference image appointed information of primary image, in step S536, with reference to image setting unit 453, long-term never mark is set as 1.

In step S537, judge that whether the long-term never mark of the SPS supplied from the setup unit 431 of Figure 66 is as 1 with reference to image setting unit 453.When in step S537, judge the long-term never mark of SPS as 1 time, process terminates.

Meanwhile, when in step S537, the long-term never mark judging SPS not as 1 time, that is, when the long-term never mark of SPS is 0, in step S538, with reference to image setting unit 453, long-term never mark is supplied to lossless encoding unit 451.Subsequently, process terminates.

In addition, when in step S535, judge to strengthen image long-term with reference to image appointed information be different from primary image long-term with reference to image appointed information time, in step S539, with reference to image setting unit 453, long-term never mark is set as 0.

In step S540, judge that whether the long-term never mark of SPS is as 1 with reference to image setting unit 453.When in step S540, judge the long-term never mark of SPS as 1 time, in step S541, with reference to image setting unit 453, the long-term never mark of setting is supplied to lossless encoding unit 451, then process enters step S542.

Meanwhile, when in step S540, the long-term never mark judging SPS not as 1 time, that is, when the long-term never mark of SPS is 0, process enters step S542.

The process of step S542 and S543 is similar to the step S404 of Figure 73 and the process of S405, thus the description thereof will be omitted.After the process of step S543, process terminates.

(explanation of second example of the process of decoding device)

Except SPS extraction process and generating process, the scalable decoding process when another never mark sets is similar to the scalable decoding process when public never mark setting.Thus, described SPS extraction process and generating process will be described below.

Figure 84 is the flow chart of the SPS extraction process illustrated when another never mark sets.

At the step S561 of Figure 84, with reference to image setting unit 512 from supplying from the SPS of extraction unit 111, extract RPS never mark, and keep the RPS never mark of extraction.In step S562, judge that whether the RPS never mark extracted is as 1 with reference to image setting unit 512.

When in step S562, judge RPS never mark not as 1 time, that is, when RPS never mark is 0, in step S563, from SPS, extract RPS with reference to image setting unit 512, keep the RPS extracted.Subsequently, process enters step S564.

Meanwhile, when in step S562, judge RPS never mark as 1 time, process enters step S564.

In step S564, with reference to image setting unit 512 from SPS, extract and mark for a long time.

In step S565, what judges to extract with reference to image setting unit 512 marks whether as 1 for a long time, when judge extraction be labeled as 1 for a long time time, process enters step S566.In step S566, with reference to image setting unit 512 from SPS, extract long-term never mark, and keep the long-term never mark of extraction.

In step S567, judge that whether long-term never mark is as 1 with reference to image setting unit 512, when judge long-term never mark not as 1 time, that is, when judge long-term never mark as 0 time, process enters step S568.In step S568, with reference to image setting unit 512 from SPS, extract long-term reference image appointed information, and keep the long-term reference image appointed information of extraction, then process terminates.

Meanwhile, when in step S565, judge long-term mark not as 1 time, or when in step S567, judge long-term never mark as 1 time, process terminates.

Figure 85 is the flow chart of the generating process illustrated when another never mark sets.Such as, in units of cutting into slices, carry out described generating process.

At the step S581 of Figure 85, determine whether from lossless decoding unit 511 for giving RPS never mark with reference to image setting unit 512.When in step S581, judge for when giving RPS never mark, in step S582, with reference to image setting unit 512, the RPS never mark be included in SPS kept is updated to the RPS never mark supplied from lossless decoding unit 511.Subsequently, process enters step S583.

Meanwhile, when in step S581, when judging not supply RPS never mark, process enters step S583.

In step S583, determine whether from lossless decoding unit 511 for giving long-term never mark with reference to image setting unit 512.When in step S583, judge for when giving long-term never mark, in step S584, with reference to image setting unit 512, the long-term never mark be included in SPS kept is updated to the long-term never mark supplied from lossless decoding unit 511.Subsequently, process enters step S585.

Meanwhile, when in step S583, when judging not supply long-term never mark, process enters step S585.

Except replacing never mark by RPS never mark, replace with reference to image appointed information with RPS, outside the long-term reference image appointed information of uncertain current slice, the process of step S585-S592 is similar to the process of the step S473-S480 of Figure 79, thus the description thereof will be omitted.

After the process of step S587, S590 or S592, process enters step S593.

In step S593, judge that whether long-term never mark is as 1 with reference to image setting unit 512.When in step S593, judge long-term never mark as 1 time, in step S594, with reference to image setting unit 512 from reference to buffer 144, read the long-term with reference to image appointed information of primary image.

In step S595, with reference to image setting unit 512 the long-term long-term reference image appointed information being defined as current slice with reference to image appointed information of primary image.Subsequently, with reference to image setting unit 512, the long-term reference image appointed information determined is supplied to motion compensation units 146, then process terminates.

Meanwhile, when in step S593, judge long-term never mark not as 1 time, that is, when long-term never mark is 0, process enters step S596.In step S596, obtain the long-term index with reference to image appointed information supplied from lossless decoding unit 511 with reference to image setting unit 512.

In step S597, with reference to image setting unit 512 keep to be included in SPS long-term with reference in image appointed information, what the index of acquisition was assigned to long-term is defined as the long-term with reference to image appointed information of current slice with reference to image appointed information.In addition, equally the long-term reference image appointed information supplied from lossless decoding unit 511 is defined as the long-term reference image appointed information of current slice with reference to image setting unit 512.Subsequently, with reference to image setting unit 512, the long-term reference image appointed information of current slice is supplied to motion compensation units 146, then process terminates.

After generating process terminates, remain on reference to the RPS never mark in image setting unit 512, long-term never mark and reference image appointed information are restored to the RPS never mark be included in SPS, long-term never mark and reference image appointed information.

As mentioned above, when another never mark sets, when primary image RPS and long-term be used to strengthen image with reference to image appointed information is one of any time, used one need not be set redundantly.Thus, improve code efficiency.

Here, due to primary image RPS and long-term be used to strengthen image with reference to image appointed information is one of any, therefore exist and press the situation that the setting of ref_idx_framework pattern strengthens the reference image of image.In this case, because primary image is used as the long-term reference image strengthening image, even if therefore the RPS of primary image is identical with the RPS strengthening image, the long-term long-term reference image appointed information being also different from enhancing image with reference to image appointed information of primary image.

In addition, in this case, the long-term number with reference to image strengthening image is larger than primary image, and the size of frame memory 44 (141) increases.Thus in order to reduce described size, some the long-term reference images strengthening image are deleted in consideration, remove for it, and the reference image beyond the primary image that high-precision forecast is possible is deleted.Thus, in this case, the long-term long-term reference image appointed information being different from enhancing image with reference to image appointed information of primary image.

In addition, owing to strengthening the resolution of resolution higher than primary image of image, the short-term or the long-term reference image that therefore also there is enhancing image are deleted, to reduce the situation of the size of frame memory 44 (141).In this case, primary image short-term or long-term be different from the short-term that strengthens image or long-term with reference to image appointed information with reference to image appointed information.

In addition, due to when primary image is used as with reference to image, high-precision forecast may be carried out according to primary image, therefore also exist by deleting the short-term or long-term reference image that strengthen image, reducing the situation of the size of frame memory 44 (141).In this case, primary image short-term or long-term be different from the short-term that strengthens image or long-term with reference to image appointed information with reference to image appointed information.

As mentioned above, when pressing ref_idx_framework pattern, when setting the reference image strengthening image, the long-term long-term reference image appointed information being different from enhancing image with reference to image appointed information of primary image.

Thus, the redundancy of RPS can be avoided to set, so that replace in RPS and long-term reference image appointed information, set never mark respectively, by pressing ref_idx_framework pattern, it is invalid to make for the long-term never mark with reference to image appointed information, and never mark is set as 0.Below as the 3rd example of the 3rd embodiment, this situation is described.Below, the 3rd example of the 3rd embodiment is called as " when set model uses ".

(the illustration grammer of VPS)

Figure 86 is the diagram of the illustration grammer of the expansion (vps_extension) of the VPS of graphic extension the 3rd embodiment.

In addition, the syntactic class except VPS expansion of the VPS in the 3rd embodiment is similar to the VPS in the first and second embodiments.

As shown in diagram in the 4th row of Figure 86, whether the coded system of instruction primary image is the expansion that AVC mark (avc_base_layer_flag) of AVC mode is set to VPS.The coded system that AVC is marked at primary image is 1 when being AVC mode, and when the coded system of primary image is HEVC mode is 0.

In addition, as shown in diagram in the 7th row, the set model information (scalavility_mask) of the kind of the distinctive set model of scalable function of the set model with reference to image being used as to strengthen image is indicated to be set to the expansion of VPS.Such as, set model information is 1 when ref_idx_framework is used as the set model with reference to image strengthening image.

As mentioned above, the set model with reference to image that instruction strengthens image be whether the information of ref_idx_framework pattern as set model information, be set to VPS.Thus utilize set model information, it is invalid to make for the long-term never mark with reference to image appointed information.

(the 3rd example of the grammer of the SPS of enhanced flow)

When Figure 87 is illustrated in set model use, the diagram of the illustration grammer of the SPS set by the setup unit 431 of Figure 66.

As shown in diagram in the 3rd row of Figure 87, be similar to the example of Figure 68, never mark (inter_layer_copy_flag) is set to the SPS of enhanced flow.In addition, as shown in diagram in the 4th row to eighth row, be similar to the example of Figure 68, the information relevant to RPS is set to SPS.

In addition, as shown in diagram in the 9th row to the 17th row, when never mark is 0, or when set model information is 1, be similar to the example of Figure 68, be set to SPS to the long-term information relevant with reference to image appointed information.In other words, although never mark is 1, when not excessive set model information is 1, make never mark invalid, be set to SPS to the long-term information relevant with reference to image appointed information.

(the 3rd example of the grammer of the section head of enhanced flow)

When Figure 88 is illustrated in set model use, the diagram of the illustration grammer of the section head of enhanced flow.

As shown in diagram in the 5th row of Figure 88, when the never mark of correspondence section is different from the never mark be included in SPS, be similar to the example of Figure 69, never mark is set to the section head of enhanced flow.In addition, if the 6th row at Figure 88 is to shown in diagram in the 12nd row, the example of Figure 69 is similar to, when the never mark of correspondence section is 0, the index of setting RPS mark and RPS or RPS.

In addition, if the 13rd row at Figure 88 is to shown in diagram in the 16th row, when being labeled as 1 for a long time, when never mark is 0, or when set model information is 1, set long-term with reference to image appointed information.

(explanation of the 3rd example of the process of encoding device)

Except SPS setting process and never mark setting process, the ges forschung process when set model uses is similar to the ges forschung process of Figure 71.Thus, described SPS setting process and never mark setting process will be described below.

Figure 89 is the flow chart of the SPS setting process illustrated when set model uses.

Except replacing except RPS never mark by never mark, the process of the step S611-S613 of Figure 89 is similar to the process of the step S501-S503 of Figure 82, thus the description thereof will be omitted.

After the process of step S613, or when in step S612, judge never mark as 1 time, process enters step S614.

In step S614, setup unit 431 judges that whether never mark is as 0, or whether the set model information being set as VPS is 1.

When in step S614, judge that never mark is as 0, or when set model information is 1, process enters step S615.The process of step S615-S617 is similar to the process of the step S384-S386 of Figure 72, thus the description thereof will be omitted.

Meanwhile, when in step S614, judge never mark not as 0 time, when set model is not 1, that is, when never mark is 1, and when set model information is 0, process terminates.

Figure 90 is the flow chart of the never mark setting process illustrated when set model uses.In units of section, carry out never mark setting process.

The step S631 of Figure 90 and the process of S632 are similar to the step S390 of Figure 73 and the process of S391, thus the description thereof will be omitted.

Judge that whether the set model information being set to the VPS supplied from setup unit 431 is as 1 in the reference image setting unit 453 of step S633, Figure 67.When in step S633, judge set model information as 1 time, in step S634, judge that whether the RPS strengthening image identical with the RPS of primary image with reference to image setting unit 453.

When in step S634, when judging that the RPS of enhancing image is identical with the RPS of primary image, in step S635, reference image setting unit 453 is set as 1 never mark, and then process enters step S640.

Meanwhile, when in step S634, when judging that the RPS of enhancing image is different from the RPS of primary image, in step S636, reference image setting unit 453 is set as 0 never mark, and then process enters step S640.

In addition, when in step S633, judge set model information not as 1 time, that is, when set model information is 0, process enters step S637.The process of step S637-S639 is similar to the step S392 of Figure 73, the process of S393 and S396, thus the description thereof will be omitted.After step S638 or S639, process enters step S640.

In step S640, judge that with reference to image setting unit 453 whether the never mark of the SPS supplied from setup unit 431 is identical with the never mark of setting.When in step S640, when judging that the never mark of SPS is identical with the never mark of setting, process enters step S642.

Meanwhile, when in step S640, when judging that the never mark of SPS is different from the never mark of setting, in step S641, with reference to image setting unit 453, the never mark of setting is supplied to lossless encoding unit 451, then process enters step S642.

In step S642, judge that whether the never mark set is as 1 with reference to image setting unit 453.When in step S642, judge the never mark that sets as 1 time, in step S643, judge that whether set model information is as 1 with reference to image setting unit 453.

When in step S643, judge set model information not as 1 time, that is, when set model information is 0, process terminates.Meanwhile, when in step S643, judge set model information as 1 time, process enters step S649.

In addition, when in step S642, judge the never mark that sets not as 1 time, that is, when the never mark of setting is 0, process enters step S644.The process of step S644-S650 is similar to the process of the step S399-S405 of Figure 73, thus the description thereof will be omitted.

Whether, as mentioned above, when set model uses, when set model information is 1, it is invalid to make for the long-term never mark with reference to image appointed information, thus can be identical with the RPS strengthening image according to the RPS of primary image, setting never mark.Thus, due to set model information be 1 time, when primary image long-term with reference to image appointed information be different from strengthen image long-term with reference to image appointed information time, never mark is set to 1, thus between primary image and enhancing image, can RPS be shared.As a result, code efficiency is improved.

(explanation of the 3rd example of the process of decoding device)

Except SPS extraction process and generating process, the scalable decoding process when set model uses is similar to the scalable decoding process when public never mark setting.Thus, described SPS extraction process and generating process will be described below.

Figure 91 is the flow chart of the SPS extraction process illustrated when set model uses.

The process of the step S671-S673 of Figure 91 is similar to the process of the step S451-S453 of Figure 78, thus the description thereof will be omitted.

After the process of step S673, or when in step S672, judge never mark as 1 time, process enters step S674.

In step S674, judge to be included in from the never mark the SPS of extraction unit 111 supply whether as 0 with reference to image setting unit 512, or whether the set model information be included in VPS is 1.

When in step S674, judge that never mark is as 0, or when set model information is 1, process enters step S675.The process of step S675-S677 is similar to the process of the step S454-S4556 of Figure 78, thus the description thereof will be omitted.

Meanwhile, when in step S674, judge never mark not as 0, and when set model information is not 1, that is, when never mark is 1, when set model information is 0, process terminates.

Figure 92 is the flow chart of the generating process illustrated when set model uses.Such as, in units of cutting into slices, carry out generating process.

The step S691 of Figure 92 and the process of S692 are similar to the step S471 of Figure 79 and the process of S472, thus the description thereof will be omitted.Except replacing except RPS never mark by never mark, the process of step S693-S700 is similar to the process of the step S585-S592 of Figure 85, thus the description thereof will be omitted.

In step S701, judge to be included in set model information in the VPS extracted by separative element 92 with reference to image setting unit 512 whether as 1.When in step S701, judge set model information not as 1 time, that is, when set model information is 0, process enters step S702.

Except replacing except long-term never mark by never mark, the process of step S702-S706 is similar to the process of the step S593-S597 of Figure 85, thus the description thereof will be omitted.

Meanwhile, when in step S701, judge set model information as 1 time, process enters step S705.

In addition, in superincumbent explanation, the coded system of primary image is assumed that HEVC mode, but can be the coded system except HEVC mode, such as AVC mode.In this case, the reference image appointed information being difficult to shared primary image and the reference image appointed information strengthening image.Thus, only have when the coded system of primary image is identical with the coded system strengthening image, just set never mark, thus the reference image appointed information of primary image can be shared and strengthen the reference image appointed information of image.

Below as the 4th example of the 3rd embodiment, this situation is described.Here, the example when the setting of public never mark will be described, but, when another never mark sets or when set model uses, this is applicable equally.Below, the 4th example of the 3rd embodiment will be called as " when the public never mark based on coded system sets ".

(the 4th example of the grammer of the SPS of enhanced flow)

When Figure 93 is illustrated in the public never mark setting based on coded system, the diagram of the illustration grammer of the SPS set by the setup unit 431 of Figure 66.

As shown in diagram in the 2nd row and the 3rd row of Figure 93, when be set to Figure 86 VPS expansion AVC mark (avc_base_layer_flag) be 0 time, that is, when coded system is not AVC mode, never mark is set to SPS.In addition, as shown in diagram in the 4th row and the 14th row, when AVC is labeled as 1, or when never mark is 0, be similar to the SPS of basic stream, the information relevant to RPS, and be set to SPS to the long-term information relevant with reference to image appointed information.

(explanation of the 4th example of the process of encoding device)

Except SPS setting process and never mark setting process, the ges forschung process when the public never mark setting based on coded system is similar to the ges forschung process of Figure 71.

Except before the SPS setting process of Figure 72, the AVC that setup unit 431 judges to be set to VPS marks whether that the SPS setting process when the public never mark setting based on coded system is similar to the SPS setting process of Figure 72 as outside 1.When judge AVC mark not as 1 time, carry out the SPS setting process of Figure 72, and when judgement AVC be labeled as 0 time, process enters step S383.

In addition, except before the never mark setting process of Figure 73, the AVC judging to be set to the VPS supplied from setup unit 431 with reference to image setting unit 453 marks whether that the never mark setting process when the public never mark based on coded system sets is similar to the never mark setting process of Figure 73 as outside 1.When judge AVC mark not as 1 time, carry out the never mark setting process of Figure 73, and when judgement AVC be labeled as 1 time, process enters step S399.

(explanation of the 4th example of the process of decoding device)

Except SPS extraction process and generating process, the scalable decoding process when scalable decoding process when the public never mark setting based on coded system is similar in public never mark setting.

Except before the SPS extraction process of Figure 78, the AVC judging with reference to image setting unit 512 to be included in the VPS extracted by separative element 92 marks whether that the SPS extraction process when the public never mark based on coded system sets is similar to the SPS extraction process of Figure 78 as outside 1.When judge AVC mark not as 1 time, carry out the SPS extraction process of Figure 78, and when judgement AVC be labeled as 0 time, process enters step S453.

In addition, except before the generating process of Figure 79, judge that AVC marks whether that the generating process when the public never mark setting based on coded system is similar to the generating process of Figure 79 as outside 1 with reference to image setting unit 512.When judge AVC mark not as 1 time, carry out the generating process of Figure 79, and when judgement AVC be labeled as 1 time, process enters step S476.

< the 4th embodiment >

(Exemplary structures of the encoding device of the 4th embodiment)

In each assembly in Figure 94 shown in diagram, the identical Reference numeral of the assembly identical with Fig. 6 represents.By suitably the repetitive description thereof will be omitted.

The structure of encoding device 530 of Figure 94 and the difference of the structure of Fig. 6 are to replace strengthening coding unit 12, are provided with and strengthen coding unit 531.Between primary image and enhancing image, encoding device 530 shares RPS at least partially.

Particularly, from outside the enhancing coding unit 531 strengthening image input coding equipment 530.Strengthen coding unit 531 according to the mode observing HEVC mode, coding strengthens image.

In addition, strengthen the RPS that coding unit 531 utilizes the primary image supplied from basic coding unit 11, part RPS never mark is set as specifying information generated (with reference to image generation information) with reference to image.Subsequently, strengthen coding unit 531 and part RPS never mark etc. is added in coding result, thus generate coded data.In addition, part RPS never mark is the mark whether being used as the RPS strengthening image at least partially of the RPS of instruction primary image.

Strengthening coding unit 531 using comprising the encoding stream of coded data, SPS, PPS etc. as enhanced flow, being supplied to synthesis unit 13.

Here, encoding device 530 transmits the encoding stream of all each layers, but where necessary, can only transmit basic stream.

(strengthening the Exemplary structures of coding unit)

Figure 95 is the block diagram of the Exemplary structures of the enhancing coding unit 531 of graphic extension Figure 94.

The enhancing coding unit 531 of Figure 95 comprises setup unit 551 and coding unit 552.

The setup unit 551 strengthening coding unit 531 takes the circumstances into consideration to set the parameter set comprising part RPS never mark, such as SPS or PPS.Setup unit 551 is supplied to coding unit 552 the parameter set of setting.

Coding unit 552 receives as input signal, and from the enhancing image of the frame unit of outside input, according to the mode observing HEVC mode, coding strengthens image.In addition, coding unit 552 according to coding time use RPS, and from basic coding unit 11 supply RPS, setting section RPS never mark.

Coding unit 552 by the part RPS never mark according to setting, and is included in from the part RPS never mark the SPS of setup unit 551 supply, part RPS never mark etc. is added in coding result, generates coded data.Subsequently, coding unit 552, according to coded data and the parameter set from setup unit 551 supply, generates enhanced flow, and the enhanced flow generated is supplied to the synthesis unit 13 of Figure 94.

(Exemplary structures of coding unit)

Figure 96 is the block diagram of the Exemplary structures of the coding unit 552 of graphic extension Figure 95.

In each assembly in Figure 96 shown in diagram, the identical Reference numeral of the assembly identical with Figure 14 represents.By suitably the repetitive description thereof will be omitted.

Except replacing lossless encoding unit 36, accumulation buffer 37 and with reference to image setting unit 50, is provided with lossless encoding unit 571, accumulation buffer 452 and with reference to outside image setting unit 572, and the structure of the coding unit 552 of Figure 96 is different from the structure of Figure 14.

Lossless encoding unit 571 obtains intraprediction mode information from intraprediction unit 46.In addition, lossless encoding unit 571, from motion prediction/compensating unit 47, obtains the inter-frame forecast mode information, motion vector etc. that supply from motion prediction/compensating unit 47.In addition, lossless encoding unit 571, from reference to image setting unit 572, obtains RPS mark, part RPS never mark etc.In addition, lossless encoding unit 571 obtains skew filtering information from self adaptation offset filter 42, and obtains filter factor from auto-adaptive loop filter 43.

Lossless encoding unit 571, to the quantization parameter supplied from quantifying unit 35, carries out lossless coding.In addition, lossless encoding unit 571 pairs of intraprediction mode information or inter-frame forecast mode information one of any, motion vector, RPS marks, part RPS never mark, and skew filtering information and filter factor carry out lossless coding, as coded message.Lossless encoding unit 571 adds the coded message of lossless coding in the coefficient of lossless coding, thus generates coded data.Lossless encoding unit 571 supplies coded data, thus is accumulated in accumulation buffer 452.

Compare the RPS supplied from motion prediction/compensating unit 47 with reference to image setting unit 572, and be kept at reference to the RPS in buffer 49, and setting section RPS never mark.Subsequently, when the part RPS never mark set is different from the part RPS never mark the SPS being included in and supplying from the setup unit 551 of Figure 95, with reference to image setting unit 572, the part RPS never mark of setting is supplied to lossless encoding unit 571.

In addition, with reference to image setting unit 572 according to the RPS supplied from motion prediction/compensating unit 47, the RPS of primary image and part RPS never mark, generate the RPS being used for part RPS never mark.Subsequently, compare the RPS for part RPS never mark of generation and the RPS for being included in the part RPS never mark in SPS with reference to image setting unit 572, setting RPS mark, and RPS mark is supplied to lossless encoding unit 571.In addition, mark according to RPS with reference to image setting unit 572, the RPS for part RPS never mark generated is supplied to lossless encoding unit 571, or identical with corresponding RPS for appointment, the index for the RPS being included in the part RPS never mark in SPS is supplied to lossless encoding unit 571.

(first example of the grammer of the SPS of enhanced flow)

Figure 97 is the diagram of the illustration grammer of the SPS that graphic extension is set by the setup unit 551 of Figure 95.

As shown in diagram in the 3rd row of Figure 97, part RPS never mark (inter_layer_prediction_flag) is set to the SPS of enhanced flow.Part RPS never mark is 1 when the RPS at least partially as enhancing image of the RPS of primary image, and is 0 when all RPS of primary image are not used as the RPS strengthening image.

In addition, as shown in diagram in the 4th row, be similar to the SPS of basic stream, setting is included in the number (num_short_term_ref_pic_sets) of the RPS in SPS.In addition, as shown in diagram in the 5th row and the 6th row, the RPS (shrot_term_ref_pic_set (i, inter_layer_prediction_flag) of the part RPS never mark for being set to SPS is set.In addition, as shown in diagram in the 7th row to the 13rd row, be similar to the RPS of basic stream, set and the long-term information relevant with reference to image appointed information.

(first example of the grammer of the section head of enhanced flow)

As shown in diagram in the 5th row of Figure 98, when the part RPS never mark of correspondence section is different from the part RPS never mark be included in SPS, part RPS never mark is set to the section head of enhanced flow.As shown in diagram in the 6th row, be similar to the section head of basic stream, setting RPS mark.

In addition, as shown in diagram in the 7th row and eighth row, when RPS is labeled as 0, setting is used for the RPS of the part RPS never mark of corresponding section.As shown in diagram in the 9th row and the 10th row, when RPS is labeled as 1, set identical with the RPS for corresponding part RPS never mark of cutting into slices, for being included in the index of the RPS of the part RPS never mark in SPS.In addition, as shown in diagram in the 11st row and the 12nd row, be similar to the section head of basic stream, according to long-term mark, set long-term reference image appointed information.

(first example of the grammer of RPS)

As shown in diagram in the 3rd row of Figure 99, for the RPS for part RPS never mark, setting is with reference to information (inter_ref_pic_set_prediction_flag).Here, described reference information indicates except between identical layer, the presence or absence of the difference between different layers.In other words, when being 1 with reference to information, be used as at the RPS of front image or primary image the RPS strengthening image with reference to information instruction.Meanwhile, when being 0 with reference to information, be not used as at the RPS of front image and primary image the RPS strengthening image with reference to information instruction.

As shown in diagram in the 4th row to the 10th row, when reference information is 1, and when part RPS never mark is 0, namely, when being used as the RPS strengthening image at the RPS of front image, be set in front image appointed information (delta_idx_minus1), symbol (delta_rps_sign) and absolute value (abs_delta_rps_minus1).

In addition, as shown in diagram in the 11st row to the 14th row, when being 1 with reference to information, setting mark (used_by_curr_pic_lt_flag (used_by_curr_pic_flag)), when mark (used_by_curr_pic_flag) is 0, set mark (use_delta_flag) in addition.Meanwhile, as shown in diagram in the 15th row to the 25th row, when being 0 with reference to information, setting instruction is with reference to the number of image or the information of POC.

Thus, when being 1 with reference to information, front image appointed information (delta_idx_minus1), it is the RPS of the situation of 0 that symbol (delta_rps_sign) and absolute value (abs_delta_rps_minus1) are set to for wherein part RPS never mark.In addition, setting mark (used_by_curr_pic_lt_flag), and according to mark (used_by_curr_pic_flag), setting mark (use_delta_flag).When being 0 with reference to information, such as, instruction is set to for wherein part RPS never mark with reference to the number of image or the information of POC is the RPS of the situation of 0.

Simultaneously, for being the RPS of the situation of 1 for wherein part RPS never mark, setting mark (used_by_curr_pic_flag), and according to mark (used_by_curr_pic_flag), setting mark (use_delta_flag).

In other words, when part RPS never mark is 1, the image with reference to destination due to RPS is fixed to the primary image of same time, does not therefore set the information of specifying primary image.In addition in this case, because the RPS at least partially of primary image is used as the RPS strengthening image steadily, therefore do not set the symbol (delta_rps_sign) relevant to the difference of RPS and absolute value (abs_delta_rps_minus1) yet.As a result, code efficiency is improved.

In addition when part RPS never mark is 1, setting mark (used_by_curr_pic_flag) (use information).Thus, for each reference image of primary image, the RPS with reference to image can be specified whether to be used as strengthening the RPS of image.

In addition, when part RPS never mark is 1, with reference to information must be 1, thus do not exist wherein with reference to information be the situation of 0.

(explanation of first example of the process of encoding device)

Except SPS setting process and never mark setting process, the ges forschung process of the encoding device 530 of Figure 94 is similar to the ges forschung process of Figure 71.Thus, the following describes described SPS setting process and never mark setting process.

Figure 100 is the flow chart of the SPS setting process that encoding device 530 is described.

At the setup unit 551 of the step S721 of Figure 100, Figure 95, part RPS never mark is set to SPS.In step S722, whether setup unit 551 judging section RPS never mark is 1.When in step S722, when judging section RPS never mark is 1, in step S723, setup unit 551 presses the number (num_short_term_ref_pic_sets) of RPS, and the RPS for wherein part RPS never mark being the situation of 1 is set to SPS.

Particularly, setup unit 551 presses the number (num_short_term_ref_pic_sets) of RPS, is set to SPS for comprising the RPS be used as with reference to 1 of information and the part RPS never mark of mark (used_by_curr_pic_lt_flag).In addition, when mark (used_by_curr_pic_flag) is 0, mark (use_delta_flag) is also set to the RPS for part RPS never mark.

Meanwhile, when in step S722, when judging section RPS never mark is not 1, that is, when part RPS never mark is 0, process enters step S724.In step S724, setup unit 551 presses the number (num_short_term_ref_pic_sets) of RPS, and the RPS for wherein part RPS never mark being the situation of 0 is set to SPS.

Particularly, setup unit 551 presses the number (num_short_term_ref_pic_sets) of RPS, being used for comprising with reference to information, when being 1 with reference to information, be included in front image appointed information (delta_idx_minus1), symbol (delta_rps_sign), absolute value (abs_delta_rps_minus1) and mark (used_by_curr_pic_flag), and when being 0 with reference to information, the RPS comprising the part RPS never mark of instruction the reference number of information or the information of POC etc. is set to SPS.In addition, when mark (used_by_curr_pic_flag) is 0, mark (use_delta_flag) is also set to the RPS for part RPS never mark.

After the process of step S723 or S724, process enters step S725.The process of step S725-S727 is similar to the process of the step S384-S386 of Figure 72, thus the description thereof will be omitted.

Figure 101 is the flow chart of the details of the never mark setting process that encoding device 530 is described.Such as, in units of cutting into slices, carry out never mark setting process.

Except replacing with reference to except image appointed information with RPS, the step S741 of Figure 101 and the process of S742 are similar to the step S390 of Figure 73 and the process of S391, thus the description thereof will be omitted.

In step S743, judge whether all RPS strengthening image are included in the RPS of primary image with reference to image setting unit 572.When in step S743, when judging that all RPS of enhancing image are included in the RPS of primary image, in step S744, with reference to image setting unit 572, part RPS never mark is set as 1.

In addition, with reference to image setting unit 572 according to the enhancing RPS of image and the RPS of primary image, generating enhanced images be the RPS of the situation of 1 for wherein part RPS never mark.

Particularly, generating with reference to image setting unit 572 is used for wherein among the reference image utilizing the RPS of primary image to specify, strengthens the mark (used_by_curr_pic_flag) with reference to image identical with reference to image that the RPS of image specifies and be set to 1 with utilizing, and the part RPS never mark that the mark (used_by_curr_pic_flag) of different reference images is set to 0 is the RPS of the situation of 1.

Subsequently, in step S745, judge that whether the part RPS never mark of the SPS supplied from the setup unit 551 of Figure 95 is as 1 with reference to image setting unit 572.When in step S745, when judging section RPS never mark is 1, process enters step S746.

In step S746, with reference to image setting unit 572 judges for wherein part RPS never mark as 1 the RPS of SPS of situation whether with identical as the RPS of enhancing image of the situation of 1 for wherein part RPS never mark.When in step S746, when judging that the RPS of the RPS of SPS and enhancing image is identical, process enters step S747.

In step S747, with reference to image setting unit 572 serve as 1 of RPS mark with the identical with the RPS strengthening image of SPS, the index being the RPS of the situation of 1 for wherein part RPS never mark is supplied to lossless encoding unit 571, and then process terminates.

Meanwhile, when in step S745, when judging section RPS never mark is not 1, that is, when part RPS never mark is 0, process enters step S748.In step S748, be supplied to lossless encoding unit 571 with reference to image setting unit 572 0 of the part RPS never mark of serving as setting, then process enters step S749.

In addition, when in step S746, when judging that the RPS of the RPS of SPS and enhancing image is identical, process enters step S749.

In step S749, with reference to image setting unit 572 serving as 0 of RPS mark, and be supplied to lossless encoding unit 571 for the RPS that wherein part RPS never mark is the situation of 1, then process terminates.

Meanwhile, when in step S743, when judging that the RPS at least partially of enhancing image is not included in the RPS of primary image, in step S750, with reference to image setting unit 572, part RPS never mark is set as 0.In addition, with reference to image setting unit 572 according to the enhancing RPS of image and the RPS at front image, generating enhanced images be the RPS of the situation of 0 for wherein part RPS never mark.

In step S751, judge that with reference to image setting unit 572 whether the part RPS never mark of the SPS supplied from setup unit 551 is as 0.

When in step S751, judge the part RPS never mark of SPS as 0 time, process enters step S752.Meanwhile, when in step S751, the part RPS never mark judging SPS not as 0 time, that is, judge the part RPS never mark of SPS as 1 time, process enters step S754.

Except being that to be used to wherein part RPS never mark be that except the RPS of the situation of 0 replaces, the process of step S752-S755 is similar to the process of step S746-S749, thus the description thereof will be omitted to the RPS of the situation of 1 for wherein part RPS never mark.

As mentioned above, encoding device 530 setting section RPS never mark.Thus, even if the RPS that the RPS incomplete sum of primary image strengthens image is identical, if the RPS strengthening image is included in the RPS of primary image, so between Primary layer and enhancement layer, can RPS be shared.As a result, the amount of information of enhanced flow can be reduced, thus improve code efficiency.

In addition, encoding device 530 sets mark (used_by_curr_pic_flag), thus can specify the RPS of the part shared by Primary layer and enhancement layer.

(Exemplary structures of the decoding device of the 4th embodiment)

Figure 102 is the block diagram of the Exemplary structures of the decoding device of the encoding stream of all each layer that graphic extension transmits from the encoding device 530 of Figure 94 according to the decoding of fourth embodiment of the present disclosure.

In each assembly in Figure 102 shown in diagram, the identical Reference numeral of the assembly identical with Figure 29 represents.By suitably the repetitive description thereof will be omitted.

The difference of the structure of the structure of the decoding device 590 of Figure 102 and the decoding device 90 of Figure 29 is to replace strengthening decoding unit 94, is provided with and strengthens decoding unit 591.

The enhancing decoding unit 591 of decoding device 590, according to the mode observing HEVC mode, is decoded from the enhanced flow of separative element 92 supply, thus generating enhanced images.Now, strengthen the RPS of decoding unit 591 with reference to the primary image supplied from basic decoding unit 93, be included in the part RPS never mark etc. in SPS or section head.Strengthen decoding unit 591 and export the enhancing image generated.

(strengthening the Exemplary structures of decoding unit)

Figure 103 is the block diagram of the Exemplary structures of the enhancing decoding unit 591 of graphic extension Figure 102.

In each assembly in Figure 103 shown in diagram, the identical Reference numeral of the assembly identical with Figure 30 represents.By suitably the repetitive description thereof will be omitted.

The difference of the structure of the structure of the enhancing decoding unit 591 of Figure 103 and the enhancing decoding unit 94 of Figure 30 is to replace decoding unit 112, is provided with decoding unit 611.

The RPS of the primary image that the decoding unit 611 strengthening decoding unit 591 supplies with reference to the basic decoding unit 93 from Figure 102, with the part RPS never mark be included in from the SPS or section head of extraction unit 111 supply, according to the mode observing HEVC mode, the coded data supplied from extraction unit 111 of decoding.Decoding unit 611 exports the image obtained by decoding, as enhancing image.

(Exemplary structures of decoding unit)

Figure 104 is the block diagram of the Exemplary structures of the decoding unit 611 of graphic extension Figure 103.

In each assembly in Figure 104 shown in diagram, the identical Reference numeral of the assembly identical with Figure 31 represents.By suitably the repetitive description thereof will be omitted.

The difference of the structure of the structure of the decoding unit 611 of Figure 104 and the decoding unit 112 of Figure 31 is to replace lossless decoding unit 132, is provided with lossless decoding unit 631, and replaces, with reference to image setting unit 145, being provided with reference to image setting unit 632.

The lossless decoding unit 631 of decoding unit 611, to the coded data supplied from accumulation buffer 131, carries out losslessly encoding, obtains quantization parameter and coded message.Lossless decoding unit 631 is supplied to inverse quantization unit 133 quantization parameter.In addition, lossless decoding unit 631 is supplied to intraprediction unit 143 the intraprediction mode information etc. serving as coded message, and motion vector, inter-frame forecast mode information etc. are supplied to motion compensation units 146.

In addition, lossless decoding unit 631 is using the part RPS never mark as coded message, and RPS marks, and is supplied to reference to image setting unit 632 for the RPS of part RPS never mark or the index etc. of this RPS.In addition, lossless decoding unit 631 is supplied to switch 147 using as the intraprediction mode information of coded message or inter-frame forecast mode information.Lossless decoding unit 631 is supplied to self adaptation offset filter 137 using the skew filtering information as coded message, and filter factor is supplied to auto-adaptive loop filter 138.

The part RPS never mark be included in the SPS supplied from the extraction unit 111 of Figure 103 is kept with reference to image setting unit 632, and for the RPS of part RPS never mark.When from lossless decoding unit 631 supply portion RPS never mark, with during RPS for part RPS never mark, upgrade the part RPS never mark that keeps and the RPS for part RPS never mark with reference to image setting unit 632.

Subsequently, with reference to image setting unit 632 according to the part RPS never mark kept, from the RPS reading primary image with reference to buffer 144.In addition, with reference to image setting unit 632 according to RPS mark supply from lossless decoding unit 631, the RPS for part RPS never mark kept is obtained, or the RPS of the part RPS never mark of the index from lossless decoding unit 631 supply for keeping.

When part RPS never mark is 1, with reference to the RPS of image setting unit 632 according to primary image, and be the RPS of the situation of 1 for wherein part RPS never mark, the RPS of primary image determined the RPS being defined as current slice at least partially.Meanwhile, when part RPS never mark is 0, with reference to image setting unit 632, the RPS for wherein part RPS never mark being the situation of 0 is defined as the RPS of current slice.

(explanation of first example of the process of decoding device)

Except SPS extraction process and generating process, the scalable decoding process of the decoding device 590 of Figure 102 is similar to the scalable decoding process of decoding device 470.Thus, SPS extraction process and generating process will be gone on to say below.

Figure 105 is the flow chart of the details of the SPS extraction process that decoding device 590 is described.

The step S771 of Figure 105, Figure 104 with reference to image setting unit 632 from SPS, Extraction parts RPS never mark, and keep the part RPS never mark extracted.

In step S772, whether be 1 with reference to image setting unit 632 judging section RPS never mark.When in step S772, when judging section RPS never mark is 1, in step S773, with reference to the number (num_short_term_ref_pic_sets) of image setting unit 632 according to RPS, from SPS, extraction is used for wherein part RPS never mark is the RPS of the situation of 1, and keeps the RPS of extraction.

Meanwhile, when in step S772, when judging section RPS never mark is not 1, that is, when part RPS never mark is 0, process enters step S774.In step S774, with reference to the number (num_short_term_ref_pic_sets) of image setting unit 632 according to RPS, from SPS, extraction is used for wherein part RPS never mark is the RPS of the situation of 0, and keeps the RPS of extraction.

After the process of step S773 or S774, process enters step S775.The process of step S775-S777 is similar to the process of the step S454-S456 of Figure 78, thus the description thereof will be omitted.

Figure 106 is the flow chart of the details of the generating process that decoding device 590 is described.Such as, in units of cutting into slices, carry out generating process.

Except replacing except never mark by part RPS never mark, the process of the step S800-S802 of Figure 106 is similar to the process of the step S471-S473 of Figure 79, thus the description thereof will be omitted.

When in step S802, when judging section RPS never mark is 1, in step S803, with reference to image setting unit 632 from reference to buffer 144, read the RPS of primary image.

In step S804, judge that the RPS supplied from lossless decoding unit 631 marks whether as 1 with reference to image setting unit 632.

When in step S804, when judging that RPS is labeled as 1, in step S805, obtain the index of the RPS supplied from lossless decoding unit 631 with reference to image setting unit 632.

In step S806, with reference to image setting unit 632 according among the RPS kept, be the situation of 1 for wherein part RPS never mark, and the RPS that the index obtained is assigned to, from the RPS of primary image, determine the RPS of current slice.Particularly, with reference to image setting unit 632 only specify wherein be included in primary image RPS among, be mark (used_by_curr_pic_lt_flag) in the RPS of the situation of 1 for wherein part RPS never mark be the information of 1, be defined as the RPS of current slice.Subsequently, process terminates.

Meanwhile, when in step S804, judge RPS mark not as 1 time, that is, when RPS is labeled as 0, in step S807, with reference to image setting list, 632 to obtain what supply from lossless decoding unit 631 be the RPS of the situation of 1 for wherein part RPS never mark.According to this RPS, upgrade for being included in the RPS remained on reference to the part RPS never mark in the SPS in image setting unit 632.

In step S808, be similar to step S806, be the RPS of the situation of 1 according to what upgrade for wherein part RPS never mark with reference to image setting unit 632, from the RPS of primary image, determine the RPS of current slice.Subsequently, process terminates.

Meanwhile, when in step S802, when judging section RPS never mark is not 1, that is, when part RPS never mark is 0, process enters step S809.Except being that to be used to wherein part RPS never mark be except the RPS of the situation of 0 replaces to the RPS of the situation of 1 for wherein part RPS never mark, the process of step S809, S810 and step S812 is similar to the process of step S804, S805 and S807, thus the description thereof will be omitted.

After the process of step S810, in step S811, with reference to image setting unit 632 among the RPS kept being the situation of 0 for wherein part RPS never mark, and the RPS that the index obtained is assigned to is defined as the RPS of current slice.Subsequently, process terminates.

Meanwhile, after the process of step S812, in step S813, with reference to image setting unit 632 the RPS being defined as current slice for the wherein part RPS never mark RPS that is the situation of 0 upgraded.Subsequently, process terminates.

In addition, after generating process terminates, remain on to be restored to reference to the part RPS never mark in image setting unit 632 and RPS and be included in part RPS never mark in SPS and RPS.

As mentioned above, decoding device 590 utilizes part RPS never mark, the RPS of generating enhanced images.Thus, even if when the RPS of primary image is not completely identical with the RPS strengthening image, if the RPS strengthening image is included in the RPS of primary image, so also can share RPS between Primary layer and enhancement layer, thus code efficiency improves ground decoding enhanced flow.

In addition, decoding device 590 receives mark (used_by_curr_pic_flag), thus can identify the part of the RPS shared by Primary layer and enhancement layer.

In addition, in superincumbent explanation, being distinguished into for wherein part RPS never mark is the RPS of the situation of 1, and for wherein part RPS never mark be the RPS of the situation of 0 set RPS, but can not RPS be set with distinguishing.Below using second example as the 4th embodiment, this situation is described.Below, second example of the 4th embodiment is called as " during public RPS setting ".

(second example of the grammer of RPS)

Figure 107 is illustrated in the diagram of the illustration grammer of RPS when public RPS sets.

When public RPS setting, the RPS of setting Figure 107, as the RPS of the part RPS never mark of the section head of SPS or Figure 98 for Figure 97.

In the RPS of Figure 107, as shown in diagram in the 3rd row, be similar to the example of Figure 99, setting is with reference to information (inter_ref_pic_set_prediction_flag).In addition, as shown in diagram in the 4th row and the 5th row, when being 1 with reference to information, part RPS never mark is set to RPS.In other words, when being 0 with reference to information, part RPS never mark must be 0, thus only has when being 1 with reference to information, just setting section RPS never mark.

In addition, as shown in diagram in the 6th row to the 11st row, when part RPS never mark is 0, namely, when reference information is 1, and when part RPS never mark is 0, be similar to the example of Figure 99, front image appointed information (delta_idx_minus1), symbol (delta_rps_sign) and absolute value (abs_delta_rps_minus1) are set to RPS.

In addition, as shown in diagram in the 12nd row to the 15th row, when being 1 with reference to information, be similar to the example of Figure 99, mark (used_by_curr_pic_flag) and mark (use_delta_flag) be accordingly set to RPS with mark (used_by_curr_pic_flag).

In addition, as shown in diagram in the 16th row to the 27th row, when being 0 with reference to information, be similar to the example of Figure 99, instruction is set to RPS with reference to the number of image or the information of POC.

(explanation of second example of the process of encoding device)

Except SPS setting process and never mark setting process, the ges forschung process when public RPS setting is similar to the ges forschung process of Figure 71.Thus, described SPS setting process and never mark setting process will be described below.

Figure 108 is the flow chart of the SPS setting process illustrated when public RPS setting.

At the setup unit 551 of the step S831 of Figure 108, Figure 95, part RPS never mark is set to SPS.In step S832, setup unit 551 presses the number (num_short_term_ref_pic_sets) of RPS, and RPS is set to SPS.Particularly, setup unit 551 presses the number (num_short_term_ref_pic_sets) of RPS, wherein setting with reference to information, set the part RPS never mark when being 1 with reference to information, the RPS of the information corresponding with part RPS never mark to reference information with setting is set to SPS.

The process of step S833-S835 is similar to the process of the step S384-S386 of Figure 72, thus the description thereof will be omitted.

Figure 109 is the flow chart of the details of the never mark setting process illustrated when public RPS setting.Such as, in units of cutting into slices, carry out described never mark setting process.

The process of the step S851-S855 of Figure 109 is similar to the step S741-S744 of Figure 101 and the process of S750, thus the description thereof will be omitted.

After the process of step S854 or S855, process enters step S856.In step S856, judge that whether the part RPS never mark of the SPS supplied from the setup unit 551 of Figure 95 is identical with the part RPS never mark set among step S854 or step S855 with reference to image setting unit 572.

When in step S856, when judging section RPS never mark is mutually the same, in step S857, judge that whether the RPS of SPS is identical with the RPS strengthening image with reference to image setting unit 572.When in step S857, when judging that the RPS of the RPS of SPS and enhancing image is identical, process enters step S858.

In step S858, with reference to image setting unit 572 the index serving as 1 and RPS that RPS marks, be supplied to lossless encoding unit 571, then process terminates.

Meanwhile, when in step S586, when judging section RPS never mark is not mutually the same, in step S859, with reference to image setting unit 572, the part RPS never mark of setting is supplied to lossless encoding unit 571, then process enters step S860.

In addition, step S857 face to face, when judging that the RPS of SPS is identical with the RPS strengthening image, process enters step S860.

In step S860, with reference to image setting unit 572,0 and the RPS serving as RPS mark is supplied to lossless encoding unit 571, then process terminates.

(explanation of second example of the process of decoding device)

Except SPS extraction process and generating process, the scalable decoding process when public RPS setting is similar to the scalable decoding process of decoding device 470.Thus, described SPS extraction process and generating process will be described below.

The step S881 of Figure 110, Figure 104 with reference to image setting unit 632 from SPS, Extraction parts RPS never mark, and keep the part RPS never mark extracted.In step S882, from SPS, extract RPS with reference to image setting unit 632, and keep the RPS of extraction.

The process of step S883-S885 is similar to the process of the step S454-S456 of Figure 78, thus the description thereof will be omitted.

Except being the RPS of the situation of 1 for wherein part RPS never mark and being that except the RPS of the situation of 0 is replaced by RPS, the generating process when public RPS setting is similar to the generating process of Figure 106 for wherein part RPS never mark.

In addition, in the fourth embodiment, be similar to the 3rd embodiment, except the coded system except HEVC mode, such as AVC mode, can be used as the coded system of primary image.In this case, only have when the coded system of primary image is identical with the coded system strengthening image, just setting section RPS never mark, primary image and enhancing Image Sharing RPS at least partially.

The application > that < decodes to multi-view image coding and multi-view image

Above-mentioned a series of process can be applied to multi-view image coding and multi-view image decoding.The multi-view image coded system of Figure 111 graphic extension illustration.

As shown in diagram in Figure 111, multi-view image comprises the image of multiple viewpoint, and the image of certain viewpoint among described multiple viewpoint is designated as basic visual point image.The image of each viewpoint except basic visual point image is regarded as not substantially visual point image.When utilizing scalable function, when carrying out multi-view image coding, basic visual point image is encoded into Primary layer image, and not substantially visual point image is encoded into enhancing image.

When carrying out the multi-view image coding of Figure 111, in each viewpoint (same viewpoint), the difference of quantization parameter can be obtained.

(1) basic viewpoint:

(1-1) dQP (basic viewpoint)=Current_CU_QP (basic viewpoint)-LCU_QP (basic viewpoint)

(1-2) dQP (basic viewpoint)=Current_CU_QP (basic viewpoint)-Previsous_CU_QP (basic viewpoint)

(1-3) dQP (basic viewpoint)=Current_CU_QP (basic viewpoint)-Slice_QP (basic viewpoint)

(2) not substantially viewpoint:

(2-1) dQP (not substantially viewpoint)=Current_CU_QP (not substantially viewpoint)-LCU_QP (not substantially viewpoint)

(2-2) dQP (not substantially viewpoint)=CurrentQP (not substantially viewpoint)-PrevisousQP (not substantially viewpoint)

(2-3) dQP (not substantially viewpoint)=Current_CU_QP (not substantially viewpoint)-Slice_QP (not substantially viewpoint)

When carrying out multi-view image coding, in each viewpoint (different points of view), the difference of quantization parameter can be obtained.

(3) basic viewpoint/not substantially viewpoint:

(3-1) dQP (between viewpoint)=Slice_QP (basic viewpoint)-Slice_QP (not substantially viewpoint)

(3-2) dQP (between viewpoint)=LCU_QP (basic viewpoint)-LCU_QP (not substantially viewpoint)

(4) not substantially viewpoint/not substantially viewpoint:

(4-1) dQP (between viewpoint)=Slice_QP (not substantially viewpoint i)-Slice_QP (not substantially viewpoint j)

(4-2) dQP (between viewpoint)=LCU_QP (not substantially viewpoint i)-LCU_QP (not substantially viewpoint j)

In this case, can use in combination (1)-(4).Such as, in not substantially viewpoint, can consider in section level, obtain the technology (combination of 3-1 and 2-3) of the difference of the quantization parameter between basic viewpoint and not substantially viewpoint, with in LCU level, obtain the technology (combination of 3-2 and 2-1) of the difference of the quantization parameter between basic viewpoint and not substantially viewpoint.As mentioned above, because described difference is by repeated application, even if therefore carry out multi-vision-point encoding, also code efficiency can be improved.

Be similar to above-mentioned technology, for each dQP, the mark identifying whether to exist the dQP with non-zero value can be set.

< utilizes the another kind of illustration coding > of scalable function

As shown in diagram in Figure 112, in the coding utilizing scalable function, in each layer (identical layer), the difference of quantization parameter can be obtained.

(1) Primary layer:

(1-1) dQP (Primary layer)=Current_CU_QP (Primary layer)-LCU_QP (Primary layer)

(1-2) dQP (Primary layer)=Current_CU_QP (Primary layer)-Previsous_CU_QP (Primary layer)

(1-3) dQP (Primary layer)=Current_CU_QP (Primary layer)-Slice_QP (Primary layer)

(2) not substantially layer:

(2-1) dQP (not substantially layer)=Current_CU_QP (not substantially layer)-LCU_QP (not substantially layer)

(2-2) dQP (not substantially layer)=CurrentQP (not substantially layer)-PrevisousQP (not substantially layer)

(2-3) dQP (not substantially layer)=Current_CU_QP (not substantially layer)-Slice_QP (not substantially layer)

In addition, in each layer (different layers), the difference of quantization parameter can be obtained.

(3) Primary layer/not substantially layer:

(3-1) dQP (interlayer)=Slice_QP (Primary layer)-Slice_QP (not substantially layer)

(3-2) dQP (interlayer)=LCU_QP (Primary layer)-LCU_QP (not substantially layer)

(4) not substantially layer/not substantially layer:

(4-1) dQP (interlayer)=Slice_QP (not substantially layer i)-Slice_QP (not substantially layer j)

(4-2) dQP (interlayer)=LCU_QP (not substantially layer i)-LCU_QP (not substantially layer j)

In this case, can use in combination (1)-(4).Such as, in not substantially layer, consider in section level, obtain the technology (combination of 3-1 and 2-3) of the difference of the quantization parameter between Primary layer and not substantially layer, with in LCU level, obtain the technology (combination of 3-2 and 2-1) of the difference of the quantization parameter between Primary layer and not substantially layer.As mentioned above, because difference is by repeated application, even if therefore carry out ges forschung, also code efficiency can be improved.

< the 5th embodiment >

(explanation of the computer that this technology is applicable to)

The available hardware of above-mentioned a series of process or software perform.When performing described a series of process with software, the program forming described software is installed in computer.Here, the example of computer comprises the computer be incorporated in specialized hardware, comprises and is arranged on wherein each kind of program, can perform the general purpose personal computer of various function.

Figure 113 is that graphic extension utilizes program, performs the block diagram of the illustration hardware configuration of the computer of above-mentioned a series of process.

In a computer, central processing unit (CPU) 801, read-only memory (ROM) 802 and random access memory (RAM) 803 are interconnected by bus 804.

Input/output interface 805 is also connected to bus 804.Input unit 806, output unit 807, memory cell 808, communication unit 809 and driver 810 are connected to input/output interface 805.

Input unit 806 comprises keyboard, mouse, microphone etc.Output unit 807 comprises display, loud speaker etc.Memory cell 808 comprises hard disk, nonvolatile memory etc.Communication unit 809 comprises network interface.Driver 810 drives detachable media 811, such as disk, CD, magneto optical disk or semiconductor memory.

In the computer with described structure, CPU 801, by through input/output interface 805 and bus 804, is loaded into the program be kept in memory cell 808 in RAM 803, and performs described program, carry out a series of process described above.

Such as, the program that computer (CPU 801) performs can be recorded in the detachable media 811 as suit medium and be provided.In addition, by wired or wireless transmission medium, such as local area network (LAN) (LAN), internet or digital satellite broadcasting, provide described program.

In a computer, detachable media 811 is placed in driver 810, and by input/output interface 805, program can be installed in memory cell 808.In addition, program, by wired or wireless transmission medium, is received by communication unit 809, is installed in subsequently in memory cell 808.In addition, program can be pre-installed appropriately in ROM 802 or memory cell 808.

In addition, described program can be the order wherein illustrated in this specification, sequentially carries out the program processed, or can be concurrently, or where necessary when called (such as), carries out the program processed.

< the 6th embodiment >

The Exemplary structures > of < television set

The schematic construction of the television set that this technology of Figure 114 graphic extension is applicable to.Television set 900 comprises antenna 901, tuner 902, demultiplexer 903, decoder 904, video signal processing unit 905, display unit 906, audio signal processing unit 907, loud speaker 908 and exterior I/F unit 909.Television set 900 also comprises control unit 910, user I/F unit 911 etc.

Tuner 902 be tuned to utilize antenna 901 to receive broadcast singal in desired channel, carry out demodulation, subsequently the coded bit stream obtained exported to demultiplexer 903.

Demultiplexer 903, from coded bit stream, extracts video or the audio packet of the program of viewing object, and the data of the grouping of extracting is exported to decoder 904.Demultiplexer 903 is supplied to control unit 910 the packet of such as electronic program guides (EPG) and so on.In addition, when having carried out scrambling, demultiplexer etc. has been utilized to carry out descrambling.

Decoder 904 carries out the decoding process of decoded packet, and the Audio and Video data generated by decoding process are exported to video signal processing unit 905 and audio signal processing unit 907 respectively.

Video signal processing unit 905 pairs of video datas carry out noise Processing for removing or set corresponding Video processing to user.Video signal processing unit 905 generates the video data of the program on display unit 906 to be shown, the view data etc. corresponding to the process based on the application provided by network.Video signal processing unit 905 generates the video data for showing the menu screen for option, and makes video data superimposed on the video data of program.Video signal processing unit 905, according to the video data generated as mentioned above, generates drive singal, thus drives display unit 906.

Display unit 906, according to the drive singal provided from video signal processing unit 905, drives display device (such as, liquid crystal display etc.), thus program video etc. is shown.

Audio signal processing unit 907 pairs of voice datas carry out certain process of such as noise Processing for removing and so on, digital-to-analogue (D/A) conversion process is carried out to the voice data after process and amplifies process, and result data is supplied to loud speaker 908, so that output sound.

Exterior I/F unit 909 is the interfaces for being connected with external equipment or network, carries out transmission and the reception of such as video data or voice data and so on data.

User I/F unit 911 is connected with control unit 910.User I/F unit 911 comprises console switch, remote-control-signal reception unit etc., and the operation signal corresponding to the operation of user is supplied to control unit 910.

Control unit 910 comprises CPU, memory etc.The program performed by CPU preserved by memory, the various data required when CPU performs process, EPG data, the data etc. obtained by network.Preservation program in memory in certain timing by CPU, is read when such as television set 900 is activated and performs.CPU executive program, thus control unit, to cause television set 900 according to the operation element of user.

Television set 900 possesses the bus 912 connecting tuner 902, demultiplexer 903, video signal processing unit 905, audio signal processing unit 907, exterior I/F unit 909 etc. and control unit 910.

In the television set with said structure, decoder 904 possesses the function according to decoding device of the present disclosure (coding/decoding method).Thus, can share or predict the information relevant with reference to image to the image with hierarchy.

< the 7th embodiment >

(Exemplary structures of mobile phone)

The schematic construction of the mobile phone that this technology of Figure 15 graphic extension is applicable to.Mobile phone 920 comprises communication unit 922, audio codec 923, camera unit 926, graphics processing unit 927, multiplexing/separative element 928, recoding/reproduction unit 929, display unit 930 and control unit 931.These unit are interconnected by bus 933.

In addition, antenna 921 is connected to communication unit 922, and loud speaker 924 and microphone 925 are connected to audio codec 923.In addition, operating unit 932 is connected to control unit 931.

Mobile phone 920 is by various pattern, and such as voice calling mode and data communication mode, carry out the various operations of the transmission of such as audio signal and reception, the transmission of Email or view data and reception, image taking or data record and so on.

Under voice calling mode, the audio signal that microphone 925 produces is converted into voice data by audio codec 923, is compressed, is provided to communication unit 922 subsequently.Communication unit 922 carries out modulation treatment and the frequency conversion process of voice data, thus generates signal transmission.In addition, communication unit 922 is supplied to antenna 921 signal transmission, so that signal transmission is transmitted to base station (not shown).In addition, communication unit 922, to the Received signal strength received by antenna 921, is carried out amplification process, frequency conversion process and demodulation process, the voice data obtained is supplied to audio codec 923.Audio codec 923 uncompressed audio data, convert simulated audio signal to packed data, and simulated audio signal are exported to loud speaker 924.

In a data communication mode, when carrying out mail transmission, control unit 931 receives the text data inputted by manipulation operations unit 932, and the text of input is displayed on display unit 930.In addition, control unit 931, according to the user instruction inputted by operating unit 932, is generated mail data, and mail data is supplied to communication unit 922.Communication unit 922 carries out modulation treatment and the frequency conversion process of mail data, is then transmitted the signal transmission obtained by antenna 921.In addition, communication unit 922 carries out the amplification process of the Received signal strength received by antenna 921, frequency conversion process and demodulation process, thus recovers mail data.Described mail data is provided to display unit 930, so that Mail Contents is shown.

Mobile phone 920, by recoding/reproduction unit 929, is kept at the mail data received in storage medium.Storage medium is any rewritable storage medium.The example of storage medium comprises the detachable media of the semiconductor memory of such as RAM or onboard flash memory and so on, hard disk, disk, magneto optical disk, CD and such as USB (USB) memory or storage card and so on.

In a data communication mode, when transmitting view data, the view data generated by camera unit 926 is provided to graphics processing unit 927.Graphics processing unit 927 carries out the coded treatment of coded image data, thus generates coded data.

Multiplexing/separative element 928 according to certain mode, the multiplexing coded data generated by graphics processing unit 927 and the voice data supplied from audio codec 923, and result data is supplied to communication unit 922.Communication unit 922 carries out modulation treatment and the frequency conversion process of multiplex data, is then transmitted the signal transmission obtained by antenna 921.In addition, communication unit 922 carries out the amplification process of the Received signal strength received by antenna 921, frequency conversion process and demodulation process, thus recovers multiplex data.Described multiplex data is provided to multiplexing/separative element 928.Multiplexing/separative element 928 separation multiplexing data, are supplied to graphics processing unit 927 and audio codec 923 respectively coded data and voice data.Graphics processing unit 927 carries out the decoding process of decode encoded data, thus image data generating.View data is provided to display unit 930, so that the image received is shown.Audio codec 923 converts simulated audio signal to voice data, and simulated audio signal is supplied to loud speaker 924, thus exports the audio frequency received.

In the mobile phone with above structure, graphics processing unit 927 possesses the function according to encoding device of the present disclosure and decoding device (coding method and coding/decoding method).Thus, can share or predict the information relevant with reference to image to the image with hierarchy.

< the 8th embodiment >

(Exemplary structures of recording/reproducing apparatus)

The schematic construction of the recording/reproducing apparatus that this technology of Figure 116 graphic extension is applicable to.Such as, recording/reproducing apparatus 940 the voice data of broadcast program received and video data recording in the recording medium, and in the timing corresponding to the instruction of user, is supplied to user the data of record.In addition, such as, recording/reproducing apparatus 940 can obtain voice data or video data from another equipment, and the data of acquisition are recorded in the recording medium.In addition, recording/reproducing apparatus 940 is decoded and is exported record voice data in the recording medium or video data, so that in monitor etc., can carry out image display or voice output.

Recording/reproducing apparatus 940 comprises tuner 941, exterior I/F unit 942, encoder 943, hard disk drive (HDD) unit 944, CD drive 945, selector 946, decoder 947, screen display device (OSD) unit 948, control unit 949 and user I/F unit 950.

Tuner 941 be tuned to the desired channel of broadcast singal that received by antenna (not shown).The Received signal strength of tuner 941 demodulation desired channel, and the coded bit stream obtained is exported to selector 946.

Exterior I/F unit 942 one of is at least made up of IEEE 1394 interface, network interface, USB interface, flash interface etc.Exterior I/F unit 942 is the interfaces for being connected with external equipment, network, storage card etc., receives the data of video data or voice data and so on such as to be recorded.

Encoder 943 is according to certain mode, and the encode uncoded video data or voice data that provide from exterior I/F unit 942, export to selector 946 coded bit stream subsequently.

HDD unit 944 is recorded in the content-data of such as video or sound and so on, various program and other data in built-in hard disk, and when reproducing etc., the data of record are read from hard disk.

CD drive 945 is recorded in signal in the CD of loading, and from signals produced from optical disk.The example of CD comprises DVD CD (DVD-video, DVD-RAM, DVD-R, DVD-RW, DVD+R, DVD+RW etc.) and blue light (registered trade mark) CD.

When recording of video or sound, selector 946 coded bit stream that provides from tuner 941 is provided and the coded bit stream that provides from encoder 943 one of any, and the coded bit stream selected is supplied to one of HDD unit 944 and CD drive 945.In addition, when rendered video or sound, selector 946 is supplied to decoder 947 the coded bit stream exported from HDD unit 944 or CD drive 945.

Decoder 947 carries out the decoding process of decoding and coding bit stream.Decoder 947 is supplied to OSD unit 948 processing by carrying out decoding the video data generated.In addition, decoder 947 exports by carrying out the voice data processing and generate of decoding.

OSD unit 948 generates and is used for indication example as the video data of the menu screen for option, and exports described video data, so that overlaps from the video data of decoder 947 output.

User I/F unit 950 is connected to control unit 949.User I/F unit 950 comprises console switch, remote-control-signal reception unit etc., and the operation signal corresponding to the operation of user is supplied to control unit 949.

Control unit 949 is made up of CPU, memory etc.The program that CPU performs preserved by memory, and the various data required when CPU process.Preservation program in memory in certain timing by CPU, is read when such as recording/reproducing apparatus 940 is activated and performs.CPU executive program, thus control unit, to cause recording/reproducing apparatus 940 according to the operation element of user.

In the recording/reproducing apparatus with above structure, decoder 947 possesses the function according to decoding device of the present disclosure (coding/decoding method).Thus, can share or predict the information relevant with reference to image to the image with hierarchy.

< the 9th embodiment >

(Exemplary structures of imaging device)

The schematic construction of the imaging device that this technology of Figure 117 graphic extension is applicable to.Subject taken by imaging device 960, and the image of subject is shown on the display unit, or Imagery Data Recording in the recording medium.

Imaging device 960 comprises optics 961, image-generating unit 962, camera signals processing unit 963, image data processing unit 964, display unit 965, exterior I/F unit 966, memory cell 967, media drive 968, OSD unit 969 and control unit 970.In addition, user I/F unit 971 is connected to control unit 970.In addition, image data processing unit 964, exterior I/F unit 966, memory cell 967, media drive 968, OSD unit 969, control unit 970 etc. are interconnected by bus 972.

Optics 961 is made up of condenser lens, aperture device etc.Optics 961, in the image planes of image-generating unit 962, forms the optical imagery of subject.Image-generating unit 962 is made up of ccd image sensor or cmos image sensor, by opto-electronic conversion, generates the signal of telecommunication corresponding to the photoelectric image obtained, and the signal of telecommunication is supplied to camera signals processing unit 963.

Camera signals processing unit 963, to the signal of telecommunication supplied from image-generating unit 962, carries out various camera signals process, and such as flex point corrects, γ corrects and color correction.Camera signals processing unit 963 is supplied to image data processing unit 964 the view data through camera signals process.

Image data processing unit 964 carries out encoding the coded treatment of the view data provided from camera signals processing unit 963.Image data processing unit 964 is supplied to exterior I/F unit 966 or media drive 968 the coded data by carrying out coded treatment generation.In addition, image data processing unit 964 carries out decoding the decoding process of the coded data provided from exterior I/F unit 966 or media drive 968.Image data processing unit 964 is supplied to display unit 965 processing by carrying out decoding the view data generated.In addition, image data processing unit 964 carries out the process the view data supply display unit 965 provided from camera signals processing unit 963, or the display data obtained from OSD unit 969 are supplied to display unit 965, thus overlap is on the image data.

OSD unit 969 generates the menu screen comprising symbol, text or figure, or the display data of such as icon and so on, and the menu screen generated or display data are exported to image data processing unit 964.

External interface unit 966 is such as made up of USB I/O terminal etc., when print image, is connected with printer.In addition, if desired, driver is connected to exterior I/F unit 966, suitably loads detachable media, such as disk or CD, and where necessary, installs the computer program read from detachable media.In addition, exterior I/F unit 966 comprises and is connected to certain network, the network interface of such as LAN or internet.Such as, control unit 970 according to the instruction sent by user I/F unit 971, can read coded data from media drive 968, and by exterior I/F unit 966, the coded data read is supplied to another equipment connected by network.In addition, control unit 970, by exterior I/F unit 966, is obtained the coded data or view data that are provided from another equipment by network, and the coded data obtained or view data is supplied to image data processing unit 964.

As the recording medium driven by media drive 968, such as, the readable/detachable media that can write arbitrarily is used, such as disk, magneto optical disk, CD or semiconductor memory.In addition, recording medium can be tape unit, disk or storage card, has nothing to do with the kind of detachable media.Certainly, recording medium can be contact-free integrated circuit (IC) card etc.

In addition, media drive 968 can be combined with recording medium, thus forms the not portable recording medium of such as built-in HDD or solid-state drive (SSD) and so on.

Control unit 970 is made up of CPU.Memory cell 967 preserves the program that control unit 970 performs, the various data etc. required when control unit 970 processes.The program in memory cell 967 of being kept in certain timing by control unit 970, is read when such as imaging device 960 is activated and performs.Control unit 970 executive program, thus control unit, to cause imaging device 960 according to the operation element of user.

In the imaging device with above structure, image data processing unit 964 possesses the function according to encoding device of the present disclosure and decoding device (coding method and coding/decoding method).Thus, can share or predict the information relevant with reference to image to the image with hierarchy.

The application > of < ges forschung

(the first system)

Below, the embody rule example of the scalable encoded data utilizing ges forschung (utilizing the coding of scalable function) to generate is described.Such as, as shown in diagram in Figure 118, ges forschung is for the selection of data to be transmitted.

In data transmission system 1000 in Figure 118 shown in diagram, the scalable encoded data of Distributor 1002 reading and saving in scalable encoded data memory cell 1001, and through network 1003, scalable encoded data is distributed to terminal equipment, such as personal computer 1004, AV equipment 1005, flat-panel devices 1006 and mobile phone 1007.

Now, Distributor 1002, according to the ability of terminal equipment or communication environment, selects suitable high-quality coded data, and transmits the high-quality coded data selected.Although the data that Distributor 1002 delivery quality is unnecessarily high, terminal equipment also not necessarily obtains high quality graphic, and may occur to postpone or overflow.In addition, unnecessarily can occupy communication bandwidth, and unnecessarily increase the load of terminal equipment.On the other hand, although the data that Distributor 1002 delivery quality is unnecessarily low, terminal equipment also may not obtain the enough images of quality.Thus the scalable encoded data of Distributor 1002 reading and saving in scalable encoded data unit 1001, is suitable for the ability of terminal equipment or the coded data of communication environment as quality, and transmit the data read.

Such as, assuming that scalable encoded data memory cell 1001 is preserved utilize ges forschung, the scalable encoded data (BL+EL) 1011 of coding.Scalable encoded data (BL+EL) 1011 is the coded datas comprising Primary layer and enhancement layer, by Decoding Scalable coded data (BL+EL) 1011, can obtain the image of Primary layer and the image of enhancement layer.

The ability of the terminal that Distributor 1002 is transmitted to according to data or communication environment, select suitable layers, and read the data of selected layer.Such as, for personal computer 1004 or the flat-panel devices 1006 with high throughput, Distributor 1002 is from scalable encoded data memory cell 1001, read high-quality scalable encoded data (BL+EL) 1011, and former state transmits scalable encoded data (BL+EL) 1011.On the other hand, such as, for AV equipment 1005 or the mobile phone 1007 with reduction process ability, Distributor 1002 is from scalable encoded data (BL+EL) 1011, extract the data of Primary layer, and it is identical with scalable encoded data (BL+EL) 1011 to transmit content, but the scalable encoded data (BL) 1012 that quality is lower than scalable encoded data (BL+EL).

As mentioned above, utilize scalable encoded data, easily can adjust data volume, thus, the generation of delay or overflow can be prevented, and prevent the load of terminal equipment or communication media from unnecessarily increasing.In addition, the redundancy between each layer of scalable encoded data (BL+EL) 1011 is reduced, thus can reduce data volume, is less than data volume when independent data are used as the coded data of each layer.Thus, the memory block of scalable encoded data memory cell 1001 can more effectively be utilized.

In addition, such as personal computer 1004 can be used as terminal equipment to the various equipment of mobile phone 1007 and so on, thus the hardware performance of terminal equipment is different with equipment.In addition, because terminal equipment can perform various application, therefore software has various ability.In addition, all order wire road networks, comprise wired and/or wireless network, and such as internet or LAN (local area network (LAN)) can be used as the network 1003 serving as communication media, thus provides various data transmission capabilities.In addition, another kind of communication etc. can change.

In this, Distributor 1002 can be configured to before beginning transfer of data, with the terminal equipment in communication of the transmission destination as data, obtain the information relevant to the ability of terminal equipment, the hardware performance of such as terminal equipment, or the performance of the application (software) that terminal equipment performs, and the information relevant to communication environment, the such as available bandwidth of network 1003.Subsequently, Distributor 1002 according to the information obtained, can select suitable layers.

In addition, the extraction of layer can be carried out in terminal equipment.Such as, the scalable encoded data (BL+EL) 1011 that personal computer 1004 decodable code transmits, the display image of Primary layer or the image of enhancement layer.In addition, such as, personal computer 1004 can from the scalable encoded data (BL+EL) 1011 transmitted, extract the scalable encoded data (BL) 1012 of Primary layer, preserve the scalable encoded data (BL) 1012 of Primary layer, the scalable encoded data of Primary layer (BL) 1012 is sent to another equipment, the scalable encoded data (BL) 1012 of decoded base layer, thus the image of display Primary layer.

Certainly, the number of scalable encoded data memory cell 1001, the number of Distributor 1002, the number of network 1003 and the number of terminal equipment are arbitrary.Be described above wherein Distributor 1002 and transmit the example of data to terminal equipment, but example application is not limited thereto.Data transmission system 1000 can be applied to wherein when the coded data utilizing ges forschung to generate is transmitted to terminal equipment, according to ability or the communication environment of terminal equipment, selects suitable layers, and transmits any system of coded data.

(second system)

Such as, as shown in diagram in Figure 119, ges forschung is for utilizing the transmission of multiple communication media.

In data transmission system 1100 in Figure 119 shown in diagram, the scalable encoded data (BL) 1121 of Primary layer, by terrestrial broadcasting 1111, is transmitted in broadcasting station 1101.In addition, the arbitrary network 1112 of broadcasting station 1101 by being made up of the communication network including spider lines and/or wireless network, transmit the scalable encoded data (EL) 1122 of enhancement layer (such as, scalable encoded data (EL) 1122 is packed, transmits grouping as a result subsequently).

Terminal equipment 1102 has the receiving function receiving the terrestrial broadcasting 1111 broadcasted by broadcasting station 1101, thus receives the scalable encoded data (BL) 1121 of the Primary layer transmitted by terrestrial broadcasting 1111.Terminal equipment 1102 also has and carries out by network 1112 communication function that communicates, thus receives the scalable encoded data (EL) 1122 of the enhancement layer transmitted by network 1112.

Terminal equipment 1102 is such as according to the instruction etc. of user, the scalable encoded data (BL) 1121 of the Primary layer that decoding is obtained by terrestrial broadcasting 1111, obtain the image of Primary layer, preserve the image obtained, and the image obtained is sent to another equipment.

In addition, terminal equipment 1102 is such as according to the instruction etc. of user, be combined through the scalable encoded data (BL) 1121 of the Primary layer that terrestrial broadcasting 1111 obtains, with the scalable encoded data (EL) 1122 of the enhancement layer obtained by network 1112, obtain scalable encoded data (BL+EL), Decoding Scalable coded data (BL+EL), to obtain the image of enhancement layer, preserve the image obtained, and the image obtained is sent to another equipment.

As mentioned above, by different communication medias, the scalable encoded data of each layer can be transmitted.Thus, can scatteredload, and can prevent from postponing or the generation of overflow.

In addition, can, according to situation, be the communication media of every one deck selection for transmitting.Such as, can by having the communication media compared with large bandwidth, transmit the scalable encoded data (BL) 1121 of the larger Primary layer of data volume, by the communication media that bandwidth is narrower, transmit the scalable encoded data (EL) 1122 of the less enhancement layer of data volume.In addition, such as, according to the available bandwidth of network 1112, between network 1112 and terrestrial broadcasting 1111, the communication media of the scalable encoded data (EL) 1122 for transmitting enhancement layer can be switched.Certainly, this is equally applicable to the data of random layer.

Owing to as above controlling, the increase of the load in transfer of data can be suppressed further.

Certainly, the number of plies is arbitrary, and the number for the communication media transmitted also is arbitrary.In addition, the number as the terminal equipment 1102 of Data dissemination destination is also arbitrary.Be described above the example of broadcasting from broadcasting station 1101, but example application is not limited thereto.Data transmission system 1100 can be applicable to wherein in units of layer, and the coded data generated utilizing ges forschung is divided into two or more parts, and by any system that many circuits transmit.

(the 3rd system)

As shown in diagram in Figure 120, ges forschung is used for the storage of coded data.

In imaging system 1200 in Figure 120 shown in diagram, subject 1211 taken by imaging device 1201, ges forschung is carried out to the view data obtained, and scalable encoded data (BL+EL) 1221 is supplied to scalable encoded data memory device 1202.

Scalable encoded data memory device 1202, with the quality corresponding to situation, preserves the scalable encoded data (BL+EL) 1221 provided from imaging device 1201.Such as when normal, scalable encoded data memory device 1202, from scalable encoded data (BL+EL) 1221, extracts the data of Primary layer, and preserves the data extracted, low as quality, and the scalable encoded data (BL) 1222 of the little Primary layer of data volume.On the other hand, such as, when paying close attention to, scalable encoded data memory device 1202 former state preservation quality is high, and the scalable encoded data that data volume is large (BL+EL) 1221.

Thus scalable encoded data memory device 1202 can only have when necessary, just preserves image in high quality.Thus, while the reduction suppressing the image value caused by deterioration, the increase of data volume can be suppressed, and improve the service efficiency of memory block.

Such as, imaging device 1201 is monitoring cameras.When monitored object (such as invader) does not appear on the image of shooting (time normal), the content of the image of shooting may be inessential, thus, the minimizing of preference for data amount, thus view data (scalable encoded data) is preserved on low quality ground.On the other hand, when monitored object is as subject 1211, when appearing in the image of shooting (during concern), the content of the image of shooting may be important, thus, prioritizes image quality, thus preserve view data (scalable encoded data) in high quality.

By utilizing scalable encoded data memory device 1202 analysis image, when can judge normal or when paying close attention to.In addition, imaging device 1201 can carry out described judgement, and result of determination is sent to scalable encoded data memory device 1202.

In addition, so criterion when still paying close attention to when closing normal is arbitrary standards, the content as the image of criterion is arbitrary.Certainly, the condition except the content of image can be criterion.Such as, according to the size of the sound of record or waveform, can switch, every the scheduled time, can switch, or can according to external command, the instruction of such as user, switches.

Illustrate above wherein when normal and pay close attention to time two states between, carry out the example switched, but, the number of state is arbitrary.Such as, can when such as normal, low when paying close attention to, 3 kinds during concern and when showing great attention to and so on or between more kinds of state, switch.Here, the upper limit number of the state be switched depends on the number of plies of scalable encoded data.

In addition, imaging device 1201 according to state, can determine the number of plies of ges forschung.Such as, time normal, it is low that imaging device 1201 can generate quality, and the scalable encoded data (BL) 1222 of the little Primary layer of data volume, and the scalable encoded data of Primary layer (BL) 1222 is supplied to scalable encoded data memory device 1202.In addition, such as, during concern, it is high that imaging device 1201 can generate quality, and the scalable encoded data (BL+EL) 1221 of the Primary layer that data volume is large, and the scalable encoded data of Primary layer (BL+EL) 1221 is supplied to scalable encoded data memory device 1202.

Be described above the example of monitoring camera, but, the purposes of imaging system 1200 is arbitrary, is not limited to monitoring camera.

In this manual, describe wherein various information, be such as multiplexed in encoding stream with reference to image appointed information and weighting information generated, then send the example of decoding side from coding staff to.But, the technology transmitting information is not limited to this example.Such as, information can be used as the independent data associated with coded bit stream and is transmitted or record, and is not multiplexed in coded bit stream.Here, term " association " refers to when decoding, and the image (or a part for image, such as section and block) comprised in the bitstream can get up with the informational linkage corresponding to described image.In other words, by the transmission path different from image (or bit stream), described information is transmitted.In addition, information can be recorded in the recording medium (or different recording regions of identical recordings medium) different from image (or bit stream).In addition, can arbitrary unit, such as in units of multiframe, in units of frame, or in units of a part for frame, described information associated with each other and image (or bit stream).

This technology is applicable to receive at the network medium by such as satellite broadcasting, cable TV, internet or mobile phone and so on, on such as CD, disk or flash memory and so on recording medium process as MPEG or H.26x in, the encoding device used during the bit stream utilizing the orthogonal transform of such as discrete cosine transform and so on and motion compensation to compress or decoding device.

In addition, in this manual, describe wherein according to the mode observing HEVC mode, carry out the example of Code And Decode, but, the application of this technology is not limited to this example.This technology is applicable to encoding device and the decoding device of any alternate manner, as long as described encoding device is current to be encoded image layered, and utilize inter prediction encoding image, so that as in spatial scalability or SNR scalability, primary image and enhancing image correspond to each other one to one, and described decoding device carries out corresponding operation.

In addition, the embodiment of this technology is not limited to above embodiment, in the scope not departing from main idea of the present disclosure, can make various change.

Such as, this technology can have one of them function is shared also co-treatment by network cloud computing structure by multiple equipment.

The step illustrated above with reference to flow chart can be performed by individual equipment, or can be performed with sharing by multiple equipment.

In addition, when multiple pack processing is containing time in one step, the multiple process comprised in one step can be performed by individual equipment, or can be performed with sharing by multiple equipment.

In addition, the first embodiment can be combined with the second embodiment.In this case, share between the layers or predict with reference to image appointed information and weighted information.

This technology also can have following structure.

(1) encoding device, comprising:

Setup unit, described setup unit utilizes the first reference image information of serving as the information relevant to the first reference image used when encoding the first scalable image of the image with hierarchy, second reference image information of the information relevant with the second reference image serving as image scalable to second, setting is used for the reference image generation information of the generation of the first reference image information;

Coding unit, described coding unit utilizes first with reference to image, coding the first scalable image, thus generates coded data; With

Transmission unit, described transmission unit transmits the coded data generated by coding unit, and the reference image generation information set by setup unit.

(2) according to the encoding device described in (1),

Wherein when first is identical with reference to image information with second with reference to image information, described setup unit setting instruction second is used as the information of the first reference image information with reference to image information, as described with reference to image generation information.

(3) according to the encoding device described in (1) or (2),

Wherein when first is different from the second reference image information with reference to image information, described setup unit setting first is with reference to the difference between image information and the second reference image information, with instruction from the second reference image information and described difference, the information of prediction first reference image information, as reference image generation information.

(4) according to one of any described encoding device in (1)-(3),

Wherein transmission unit transmits the information of specifying ground floor.

(5) according to the encoding device described in (2),

Wherein first is the information of appointment first with reference to image with reference to image information, and

Second reference image information is the information of appointment second with reference to image.

(6) according to the encoding device described in (5),

Wherein setup unit is with sequence parameter set (SPS) for unit, and setting is with reference to image generation information.

(7) according to the encoding device described in (5) or (6),

Wherein when serve as the information of specifying first of short-term the reference image first is identical with reference to image appointed information with serve as the information of specifying second of short-term the reference image second with reference to image appointed information, second reference image appointed information of setup unit setting instruction short-term is used as the information of the first reference image appointed information of short-term, as the reference image generation information of short-term, and

When long-term first is identical with reference to image appointed information with long-term second with reference to image appointed information, setup unit setting indicates the second long-term reference image appointed information to be used as the information of the first long-term reference image appointed information, as long-term reference image generation information.

(8) according to the encoding device described in (5) or (6),

Wherein when serve as the information of specifying first of short-term the reference image first is identical with reference to image appointed information with serve as the information of specifying second of short-term the reference image second with reference to image appointed information, setup unit is according to the set model of the first reference image, setting expression second is used as the information of the first reference image appointed information, as reference image generation information with reference to image appointed information.

(9) according to one of any described encoding device in (5)-(8),

Wherein setup unit is according to the coded system of the second scalable image, and setting is with reference to image generation information.

(10) according to the encoding device described in (5) or (6),

Wherein when serve as specify first of short-term with reference to image information all first to be included in reference to image appointed information serve as specify second of short-term with reference to image information second with reference to image appointed information in time, some second reference image appointed information of setup unit setting instruction short-term are used as the information of the first reference image appointed information of short-term, as reference image generation information.

(11) according to the encoding device described in (10),

Wherein when all first reference image appointed information of short-term are included in the second reference image appointed information of short-term, setup unit is for each second reference image, second reference image appointed information of setting instruction short-term is used as the use information of the first reference image appointed information of short-term, and

Transmission unit transmits the described use information set by setup unit.

(12) according to one of any described encoding device in (1)-(4), also comprise,

Utilize and comprise the weighted information of weight coefficient, weighting first with reference to the weighting processing unit of image,

Wherein the first reference image information is the weighted information of the first reference image,

Second reference image information is the weighted information of the second reference image, and

Coding unit utilizes by first of weighting processing unit weighting with reference to image, coding the first scalable image.

(13) according to the encoding device described in (12),

Wherein when second is weighted with reference to image, weighting processing unit weighting first is with reference to image.

(14) coding method undertaken by encoding device, comprising:

Setting procedure, described setting procedure utilizes the first reference image information of serving as the information relevant to the first reference image used when encoding the first scalable image of the image with hierarchy, second reference image information of the information relevant with the second reference image serving as image scalable to second, setting is used for the reference image generation information of the generation of the first reference image information;

Coding step, described coding step utilizes first with reference to image, coding the first scalable image, thus generates coded data; With

Transmitting step, described transmitting step transmits the coded data generated in the encoding step, and the reference image generation information set in setting procedure.

(15) decoding device, comprising:

Receiving element, described receiving element receives the first reference image information utilizing and serve as the information relevant to the first reference image used when encoding the first scalable image of the image with hierarchy, second of the information relevant with the second reference image serving as image scalable to second generates with reference to image information, for the reference image generation information of the generation of the first reference image information, and the coded data of the first scalable image;

Generation unit, described generation unit utilizes with reference to image generation information, generates first with reference to image information; With

Decoding unit, the first reference image information that described decoding unit generates according to generation unit, utilizes first with reference to image, the coded data of decoding the first scalable image.

(16) according to the decoding device described in (15),

Wherein receiving element receives instruction second is used as the first reference image information information with reference to image information, as reference image generation information, and

The reference image generation information that generation unit receives according to receiving element, generates second with reference to image information, as first with reference to image information.

(17) according to the decoding device described in (15) or (16),

Wherein receiving element receives the difference between the first reference image information and the second reference image information, and instruction is from the second reference image information and described difference, the information of prediction first reference image information, as reference image generation information, and

The information that generation unit receives according to receiving element, is added second with reference to image information and the difference utilizing receiving element to receive, thus generates the addition value obtained as a result, as first with reference to image information.

(18) according to one of any described decoding device in (15)-(17),

Wherein receiving element receives the information of specifying ground floor, and

The information of the appointment ground floor that generation unit receives according to receiving element, generates first with reference to image information.

(19) according to the decoding device described in (16),

(20) according to the decoding device described in (19),

Wherein receiving element receives the reference image generation information set for unit with sequence parameter set (SPS).

(21) according to the decoding device described in (19) or (20),

Wherein receiving element receives the reference image generation information of short-term and long-term reference image generation information, the reference image generation information of described short-term indicates to serve as specifies second of the information of second of short-term the reference image to be used as serving as first of the information of the first reference image of appointment short-term with reference to image appointed information with reference to image appointed information, described reference image generation information for a long time indicates the second long-term reference image appointed information to be used as the first long-term reference image appointed information

The reference image generation information of the short-term that generation unit receives according to receiving element, generate second of short-term with reference to image appointed information, as the first reference image appointed information of short-term, and according to the long-term reference image generation information that receiving element receives, generate the second long-term reference image appointed information, as long-term first with reference to image appointed information.

(22) according to the decoding device described in (19) or (20),

Wherein receiving element receives and to set with reference to the set model of image according to first, the second reference image appointed information of the information of the second reference image serving as appointment short-term is indicated to be used as serving as the information of the first reference image appointed information of the information of the first reference image of specifying short-term, as reference image generation information, and

The reference image generation information that generation unit receives according to receiving element, generate second of short-term with reference to image appointed information, as the first reference image appointed information of short-term, and according to reference image generation information and set model, generate the second long-term reference image appointed information, as long-term first with reference to image appointed information.

(23) according to one of any described decoding device in (19)-(22),

Wherein receiving element receives the reference image generation information set according to the coded system of the second scalable image.

(24) according to the decoding device described in (19) or (20),

Wherein receiving element receives the information indicating some second reference image appointed information of the information of the second reference image serving as appointment short-term to be used as serving as the first reference image appointed information of the information of the first reference image of specifying short-term, as reference image generation information

The reference image generation information that generation unit receives according to receiving element, generates some second reference image appointed information of short-term, as the first reference image appointed information of short-term.

(25) according to the decoding device described in (24),

Wherein receiving element is for each second with reference to image, and the second reference image appointed information receiving instruction short-term is used as the use information of the first reference image appointed information of short-term, and

Generation unit according to receiving element receive with reference to image generation information and the information of use, some generating short-term second with reference to image appointed information, as short-term first with reference to image appointed information.

(26) according to one of any described decoding device in (15)-(18), also comprise,

Weighting processing unit utilize generation unit to generate first with reference to image information, weighting first with reference to image, and

Decoding unit utilizes by first of weighting processing unit weighting with reference to image, the coded data of decoding the first scalable image.

(27) according to the decoding device described in (26),

(28) coding/decoding method undertaken by decoding device, comprising:

Receiving step, described receiving step receives the first reference image information utilizing and serve as the information relevant to the first reference image used when encoding the first scalable image of the image with hierarchy, second of the information relevant with the second reference image serving as image scalable to second generates with reference to image information, for the reference image generation information of the generation of the first reference image information, and the coded data of the first scalable image;

Generation step, described generation step utilizes with reference to image generation information, generates first with reference to image information; With

Decoding step, described decoding step with reference to image information, utilizes first with reference to image according to generate in generation step first, the coded data of decoding the first scalable image.

Reference numerals list

10 encoding devices

14 transmission units

33 arithmetic elements

50 with reference to image setting unit

90 decoding devices

91 receiving elements

135 adder units

163 generation units

180 encoding devices

221 motion predictions/compensating unit

223 weight setting unit

280 decoding devices

323 motion compensation units

354 generation units

Claims

1. an encoding device, comprising:

Setup unit, described setup unit utilizes the first reference image information of serving as the information relevant to the first reference image used when encoding the first scalable image of the image with hierarchy, second reference image information of the information relevant with the second reference image serving as image scalable to second, sets the reference image generation information for generating the first reference image information;

Coding unit, described coding unit utilizes first with reference to the scalable image of Image Coding first, and generates coded data; And

2. according to encoding device according to claim 1,

3. according to encoding device according to claim 1,

Wherein when first is different from the second reference image information with reference to image information, described setup unit setting first is with reference to the difference between image information and the second reference image information, with instruction according to second with reference to image information and described difference prediction first information with reference to image information, as with reference to image generation information.

4. according to encoding device according to claim 1,

Wherein transmission unit transmits the information of specifying ground floor.

5. according to encoding device according to claim 1,

6., according to encoding device according to claim 1, also comprise,

Utilize the weighted information weighting first comprising weight coefficient with reference to the weighting processing unit of image,

7. according to encoding device according to claim 6,

8. the coding method undertaken by encoding device, comprising:

Setting procedure, described setting procedure utilizes the first reference image information of serving as the information relevant to the first reference image used when encoding the first scalable image of the image with hierarchy, second reference image information of the information relevant with the second reference image serving as image scalable to second, sets the reference image generation information for generating the first reference image information;

Coding step, described coding step utilizes first with reference to the scalable image of Image Coding first, and generates coded data; And

9. a decoding device, comprising:

Receiving element, described receiving element receives the first reference image information utilizing and serve as the information relevant to the first reference image used when encoding the first scalable image of the image with hierarchy, second of the information relevant with the second reference image serving as image scalable to second generates with reference to image information, for generating the reference image generation information of the first reference image information, and the coded data of the first scalable image;

Generation unit, described generation unit utilizes with reference to image generation information, generates first with reference to image information; And

Decoding unit, the first reference image information that described decoding unit generates based on generation unit, utilizes first with reference to image, the coded data of decoding the first scalable image.

10. according to decoding device according to claim 9,

What generation unit received based on receiving element generates second with reference to image information, as first with reference to image information with reference to image generation information.

11. according to decoding device according to claim 9,

Wherein receiving element receives first with reference to image information and second with reference to the difference between image information, and instruction according to second with reference to image information and described difference prediction first information with reference to image information, as with reference to image generation information, and

The information that generation unit receives based on receiving element, is added second with reference to image information and the difference utilizing receiving element to receive, thus generates the addition value obtained as a result, as first with reference to image information.

12. according to decoding device according to claim 9,

The information of the appointment ground floor that generation unit receives based on receiving element, generates first with reference to image information.

13. according to decoding device according to claim 9,

14., according to decoding device according to claim 9, also comprise,

Second reference image information is the weighted information of the second reference image,

15. according to decoding device according to claim 14,

16. 1 kinds of coding/decoding methods undertaken by decoding device, comprising:

Receiving step, described receiving step receives the first reference image information utilizing and serve as the information relevant to the first reference image used when encoding the first scalable image of the image with hierarchy, second of the information relevant with the second reference image serving as image scalable to second generates with reference to image information, for generating the reference image generation information of the first reference image information, and the coded data of the first scalable image;

Generation step, described generation step utilizes with reference to image generation information, generates first with reference to image information; And

Decoding step, described decoding step with reference to image information, utilizes the coded data of the scalable image of the first reference image decoding first based on generate in generation step first.