WO2015053111A1 - Dispositif et procédé de décodage, dispositif et procédé de codage - Google Patents

Dispositif et procédé de décodage, dispositif et procédé de codage Download PDF

Info

Publication number
WO2015053111A1
WO2015053111A1 PCT/JP2014/075799 JP2014075799W WO2015053111A1 WO 2015053111 A1 WO2015053111 A1 WO 2015053111A1 JP 2014075799 W JP2014075799 W JP 2014075799W WO 2015053111 A1 WO2015053111 A1 WO 2015053111A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
unit
layer
loop filter
encoding
Prior art date
Application number
PCT/JP2014/075799
Other languages
English (en)
Japanese (ja)
Inventor
佐藤　数史
Original Assignee
ソニー株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ソニー株式会社 filed Critical ソニー株式会社
Publication of WO2015053111A1 publication Critical patent/WO2015053111A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/80—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
    • H04N19/82—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation involving filtering within a prediction loop
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/117—Filters, e.g. for pre-processing or post-processing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
    • H04N19/159—Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/187—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a scalable video layer
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/59—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards

Definitions

  • the present disclosure relates to a decoding device and a decoding method, and an encoding device and an encoding method, and in particular, a decoding device, a decoding method, and an encoding device that can improve the encoding efficiency of scalable encoding. And an encoding method.
  • MPEG Motion Picture Experts Group phase
  • MPEG Motion Experts Group phase
  • orthogonal transformation such as discrete cosine transformation and motion compensation using redundancy unique to image information
  • MPEG2 was mainly intended for high-quality encoding suitable for broadcasting, but it did not support encoding methods with a lower code amount (bit rate) than MPEG1, that is, a higher compression rate. With the widespread use of mobile terminals, the need for such an encoding system is expected to increase in the future, and the MPEG4 encoding system has been standardized accordingly. Regarding the MPEG4 image coding system, the standard was approved as an international standard in December 1998 as ISO / IEC 449 14496-2.
  • ITUHEVC High Efficiency Video Coding
  • JCTVC Joint Collaboration Team-Video Coding
  • an image of a first layer before loop filter processing or an image of the first layer after loop filter processing is selected, and the selected image of the first layer is used.
  • the second layer image is encoded, and encoded data is generated.
  • a program to be executed by a computer is transmitted through a transmission medium or recorded on a recording medium, Can be provided.
  • FIG. 1 is a diagram for explaining spatial scalable coding.
  • the encoding apparatus transmits only the encoded data of the base image to the decoding apparatus having a low processing capability, so that the decoding apparatus can generate a low SNR image.
  • the encoding device transmits the encoded data of the base layer and the enhancement image to the decoding device having high processing capability, so that the decoding device decodes the base layer and the enhancement image to generate a high SNR image. can do.
  • bit-depth scalability in which an image is hierarchized by the number of bits.
  • an 8-bit video image is used as a base image
  • a 10-bit video image is used as an enhancement image and encoded.
  • the setting unit 51 of the enhancement coding unit 32 sets parameter sets such as VPS, SPS, and PPS as necessary.
  • the setting unit 51 supplies the set parameter set to the encoding unit 52.
  • the orthogonal transform unit 74 performs orthogonal transform on the residual information from the calculation unit 73 by a predetermined method, and supplies the generated orthogonal transform coefficient to the quantization unit 75.
  • the adding unit 81 functions as a decoding unit, adds the residual information supplied from the inverse orthogonal transform unit 80 and the prediction image supplied from the prediction image selection unit 89, and adds the locally decoded enhancement image. obtain.
  • the adding unit 81 sets the residual information supplied from the inverse orthogonal transform unit 80 as a locally decoded enhancement image.
  • the adding unit 81 supplies the locally decoded enhancement image to the deblocking filter 82 and also supplies the enhancement image to the frame memory 85 for accumulation.
  • the adaptive offset filter 83 performs an adaptive offset (SAO (Sample-adaptive-offset)) process that mainly removes ringing, as necessary, on the enhancement image after the deblocking filter process supplied from the deblocking filter 82. .
  • SAO Sample-adaptive-offset
  • the intra prediction unit 87 performs intra prediction in all candidate intra prediction modes in units of PU (Prediction Unit). Specifically, the intra prediction unit 87 reads out pixels around the PU as reference pixels from the frame memory 85 via the switch 86 for all candidate intra prediction modes. The intra prediction unit 87 performs intra prediction using the reference pixel, and generates a predicted image.
  • PU Prediction Unit
  • D is the difference (distortion) between the original image and the decoded image
  • R is the generated code amount including even the coefficient of orthogonal transformation
  • ⁇ is the Lagrange undetermined multiplier given as a function of the quantization parameter QP.
  • the motion prediction / compensation unit 88 refers to the inter prediction mode information, the corresponding motion vector, and the reference image candidate. Reference image specifying information as an image is output to the lossless encoding unit 76.
  • the predicted image selection unit 89 Based on the cost function values supplied from the intra prediction unit 87 and the motion prediction / compensation unit 88, the predicted image selection unit 89 has a smaller corresponding cost function value among the optimal intra prediction mode and the optimal inter prediction mode. Are determined as the optimum prediction mode. Then, the predicted image selection unit 89 supplies the predicted image in the optimal prediction mode to the calculation unit 73 and the addition unit 81. Further, the predicted image selection unit 89 notifies the intra prediction unit 87 or the motion prediction / compensation unit 88 of selection of the predicted image in the optimal prediction mode.
  • the upsampling unit 93 has a two-dimensional linear interpolation adaptive filter similar to the motion prediction / compensation unit 88.
  • the up-sampling unit 93 performs the interpolation filter process on the base image after the loop filter process supplied from the image selection unit 91 by the two-dimensional linear interpolation adaptive filter, thereby performing the base image after the loop filter process. Upsample.
  • the upsampling unit 93 supplies the high-resolution image generated by the upsampling to the frame memory 85 as the base image after the upsampling.
  • CU is defined as a coding unit. Details of this CU are described in Non-Patent Document 1.
  • the predetermined conditions are a condition that at least one of the orthogonal transform coefficients of the block P and the block Q is a non-zero orthogonal transform coefficient, a condition that the reference pictures of the blocks P and Q are different, and the motion vectors of the block P and the block Q are It is a condition that they are different, or a condition that the difference motion vectors of block P and block Q differ by 1.0 or more in units of integer pixels.
  • the third pattern is a pattern in which the pixel 135 adjacent to the upper left of the pixel 130 and the pixel 136 adjacent to the lower right of the pixel 130 are adjacent pixels, as shown in FIG.
  • the fourth pattern is a pattern in which the pixel 137 adjacent to the upper right of the pixel 130 and the pixel 138 adjacent to the lower left are adjacent pixels.
  • an offset is calculated for each category and transmitted to the decoding device as offset information.
  • the sign of the offset is fixed for each category, and information regarding the sign of the offset is not transmitted.
  • the positive and negative of the offsets of the second and third categories are fixed to positive
  • the positive and negative of the offsets of the fourth and fifth categories are fixed to negative.
  • FIG. 12 is a block diagram illustrating a configuration example of the image selection unit 91 in FIG.
  • the frame memory 141 stores a base image before loop filter processing that is decoded for use as a reference image when the base image is encoded.
  • the frame memory 142 stores the base image after the loop filter processing decoded for use as a reference image when the base image is encoded.
  • FIG. 13 is a diagram for explaining interpolation filter processing by the motion prediction / compensation unit 88.
  • step S17 the transmission unit 34 transmits the encoded stream of all layers supplied from the synthesis unit 33 to a decoding device to be described later.
  • step S35 when it is determined in step S35 that the optimal prediction mode is not the optimal inter prediction mode, that is, when the optimal prediction mode is the optimal intra prediction mode, the prediction image selection unit 89 performs prediction generated in the optimal intra prediction mode.
  • the intra prediction unit 87 is notified of image selection.
  • FIG. 19 is a flowchart for explaining the details of the base conversion process in step S32 of FIG.
  • step S61 of FIG. 19 the generation unit 152 (FIG. 12) of the image selection unit 91 converts the base image used for encoding in units of slices into a base image before loop filter processing or a loop based on a user instruction or the like The base image after filtering is determined.
  • the generation unit 152 supplies a selection signal indicating selection of the determined base image to the selector 151. Further, the generation unit 152 generates information indicating the presence / absence of the determined loop filter processing of the base image as loop filter information.
  • step S73 the deblocking filter 82 determines, BS value, and, based on a threshold ⁇ and the threshold t c, whether to perform deblock filter processing. If it is determined to perform the deblock filter processing in step S73, in step S74, the deblocking filter 82, BS value, and, based on a threshold ⁇ and the threshold t c, to determine the strength of the deblocking filter processing or the like .
  • step S73 when it is determined in step S73 that the deblocking filter process is not performed, the process returns to step S44 in FIG. 18 and proceeds to step S45.
  • the separating unit 162 separates the base stream from the encoded streams of all layers supplied from the receiving unit 161 and supplies the base stream to the base decoding unit 163, and separates the enhancement stream and supplies the enhancement stream to the enhancement decoding unit 164.
  • the enhancement decoding unit 164 decodes the enhancement stream supplied from the demultiplexing unit 162 by a method according to the HEVC method, and generates an enhancement image. At this time, the enhancement decoding unit 164 refers to the base image before the loop filter processing supplied from the base decoding unit 163 or the base image after the loop filter processing. The enhancement decoding unit 164 outputs the enhancement image after the loop filter processing.
  • the accumulation buffer 201 of the decoding unit 182 receives and accumulates encoded data from the extraction unit 181 of FIG.
  • the accumulation buffer 201 supplies the accumulated encoded data to the lossless decoding unit 202.
  • the lossless decoding unit 202 performs lossless decoding such as variable length decoding and arithmetic decoding corresponding to the lossless encoding of the lossless encoding unit 76 of FIG. 6 on the encoded data from the accumulation buffer 201, Obtain quantized coefficients and encoding information.
  • the lossless decoding unit 202 supplies the quantized coefficient to the inverse quantization unit 203. Further, the lossless decoding unit 202 supplies intra prediction mode information as encoded information to the intra prediction unit 213, and supplies inter prediction mode information, motion vectors, reference image specifying information, and the like to the motion compensation unit 214.
  • the screen rearrangement buffer 209 stores the enhancement image supplied from the adaptive loop filter 208 in units of frames.
  • the screen rearrangement buffer 209 rearranges the stored enhancement images in frame units for encoding in the original display order and supplies them to the D / A conversion unit 210.
  • the upsampling unit 218 includes a two-dimensional linear interpolation adaptive filter similar to the motion compensation unit 214.
  • the up-sampling unit 218 up-samples the base image after the loop filter processing supplied from the image selection unit 216 in units of PUs using a two-dimensional linear interpolation adaptive filter.
  • the upsampling unit 218 supplies the high-resolution image generated by the upsampling to the frame memory 211 as the base image after the upsampling.
  • the two-dimensional linear interpolation adaptive filter of the motion compensation unit 214 and the two-dimensional linear interpolation adaptive filters of the upsampling units 217 and 218 may be shared. Further, the upsampling unit 217 and the upsampling unit 218 may be shared.
  • the base decoding unit 163 includes a frame memory 231 and a frame memory 232.
  • the lossless decoding unit 202 instructs the switch 215 to select the intra prediction unit 213 when the encoded information does not include inter prediction mode information, and if the inter prediction mode information is included, the lossless decoding unit 202 instructs the switch 215 to select a motion compensation unit.
  • the selection of 214 is instructed.
  • the lossless decoding unit 202 supplies offset information as encoded information to the adaptive offset filter 207 and supplies filter coefficients to the adaptive loop filter 208. Further, the lossless decoding unit 202 supplies loop filter information as encoded information to the image selection unit 216.
  • step S135 the motion compensation unit 214 determines whether or not the inter prediction mode information is supplied from the lossless decoding unit 202. If it is determined in step S135 that the inter prediction mode information has been supplied, the process proceeds to step S136.
  • step S144 the D / A conversion unit 210 D / A converts the enhancement image in units of frames supplied from the screen rearrangement buffer 209 and outputs the enhancement image. Then, the process returns to step S115 in FIG. 25 and ends.
  • the acquisition unit 241 supplies a selection signal indicating selection of the base image after the loop filter process to the selector 242.
  • step S166 the frame memory 211 stores the base image after upsampling supplied from the upsampling unit 217 or the upsampling unit 218.
  • the base image stored in the frame memory 211 is output to the motion compensation unit 214. Then, the process returns to step S132 in FIG. 26 and proceeds to step S133.
  • the loop filter information is set in units of slices, but may be set in units of sequences or pictures.
  • the loop filter information is set, for example, in SPS_extension () of the enhancement image.
  • the loop filter information is set in units of pictures, the loop filter information is set in, for example, enhancement image PPS_extension () or VPS_extension (). SPS_extension (), PPS_extension (), VPS_extension (), etc. in which the loop filter information is set are included in the enhancement stream as a parameter set.
  • the setting unit 251 of the enhancement encoding unit 32 sets VPS_extension () including layer information representing the highest time layer of the enhancement image encoded using the base image before the loop filter processing. .
  • the setting unit 251 sets VPS, SPS, PPS, and the like.
  • the setting unit 251 supplies the set parameter set such as VPS_extension (), VPS, SPS, and PPS to the encoding unit 252.
  • dotted lines between the squares indicate dependency relationships (reference relationships) within the same hierarchy.
  • the pictures in the upper temporal hierarchy depend on the pictures in the lower temporal hierarchy. That is, the picture of sublayer 2 (Sublayer2) refers to the picture of sublayer 1 or the picture of sublayer 0.
  • the sublayer 1 picture refers to the sublayer 0 picture.
  • the sublayer 0 picture refers to the sublayer 0 picture as appropriate.
  • the enhancement layer sub-layer 2 enhancement image is encoded using the base image at the same time after the loop filter processing. That is, at the time of encoding the enhancement image of the sublayer 2 that is the temporal hierarchy above the sublayer 1 represented by the hierarchy information, the base image after the loop filter processing is selected and up-sampled.
  • step S184 in FIG. 34 is the same as the enhancement encoding process in FIGS. 17 and 18 except for the base conversion process in step S32. Accordingly, only the base conversion process will be described below.
  • the lossless decoding unit 301 of the decoding unit 182 performs variable length decoding or arithmetic decoding corresponding to the lossless encoding of the lossless encoding unit 262 of FIG. 29 on the encoded data from the accumulation buffer 201.
  • quantized coefficients and encoded information are obtained.
  • the lossless decoding unit 301 supplies the quantized coefficient to the inverse quantization unit 203.
  • the lossless decoding unit 301 also supplies intra prediction mode information as encoded information to the intra prediction unit 213, and supplies inter prediction mode information, motion vectors, reference image specifying information, and the like to the motion compensation unit 214.
  • the image selection unit 302 is configured similarly to the image selection unit 261 in FIG.
  • the image selection unit 302 acquires hierarchical information included in VPS_extension () in the parameter set supplied from the extraction unit 181 in FIG.
  • the image selection unit 302 selects the base image before the loop filter process or the base image after the loop filter process supplied from the base decoding unit 163 of FIG. 21 based on the hierarchy information and the time hierarchy of the encoded data to be decoded. select.
  • the image selection unit 302 supplies the selected base image before the loop filter processing to the upsampling unit 217, and supplies the base image after the loop filter processing to the upsampling unit 218.
  • the second embodiment of the decoding device to which the present disclosure is applied is similar to the second embodiment of the encoding device to which the present disclosure is applied, based on the time hierarchy of the enhancement image, and the loop filter processing The previous base image and the base image after the loop filter processing are selected. Therefore, it is possible to decode a coded stream that is scalable coded so that coding efficiency is improved by the second embodiment of the coding apparatus to which the present disclosure is applied.
  • the motion prediction / compensation unit 322 detects motion vectors of all candidate inter prediction modes with fractional pixel accuracy based on the enhanced resolution image and the reference image candidate. Then, the motion prediction / compensation unit 322 performs compensation processing on the reference image candidate based on the motion vector, and generates a predicted image candidate.
  • the change of the index assignment as the reference image specifying information can be performed by the syntax ref_pic_lists_modification ().
  • the syntax ref_pic_lists_modification () the reference image specifying information that uses the base image before up-sampling before the loop filter processing as the reference image, and the reference image specifying information that uses the base image after the up-sampling loop filter processing as the reference image It can be made smaller than
  • step S222 the up-sampling unit 92 up-samples the base image before the loop filter processing supplied from the base encoding unit 31 and supplies the base image to the frame memory 321.
  • the upsampling unit 93 upsamples the base image after the loop filter processing supplied from the base encoding unit 31 and supplies the upsampled base image to the frame memory 321.
  • the motion prediction / compensation unit 322 performs motion prediction / compensation processing in all inter prediction modes that are candidates in PU units.
  • the motion prediction / compensation unit 322 includes both the base image before the up-sampling before the loop filter processing and the base image after the up-sampling after the loop filter processing as reference image candidates.
  • the motion prediction / compensation unit 322 calculates cost function values for all candidate inter prediction modes based on enhancement images and prediction image candidates supplied from the screen rearrangement buffer 72.
  • the motion prediction / compensation unit 322 determines the inter prediction mode that minimizes the cost function value as the optimal inter prediction mode, and sets the corresponding prediction image candidate as the prediction image. Then, the motion prediction / compensation unit 322 supplies the cost function value of the optimal inter prediction mode and the predicted image to the predicted image selection unit 89.
  • steps S224 to S242 Since the processing of steps S224 to S242 is the same as the processing of steps S34 to S52 of FIGS. 17 and 18, the description thereof will be omitted.
  • the third embodiment of the encoding device to which the present disclosure is applied is based on the prediction result in which the base image before up-sampling before the loop filter processing and the base image after the loop filter processing are used as reference image candidates. Based on this, one of those base images is selected. Accordingly, even when both a texture region having a high frequency component and a flat region exist in the same picture, an optimal base image is selected with high accuracy as a reference image. Specifically, in the texture area, the base image before the loop filter processing after upsampling is selected as the reference image, and in the flat area, the base image after the loop filter processing after upsampling is selected as the reference image. . As a result, encoding efficiency is improved.
  • the third embodiment of the decoding device to which the present disclosure is applied is similar to the third embodiment of the encoding device to which the present disclosure is applied, or the base image before the loop filter processing after upsampling or The base image after the loop filter processing is selected. Therefore, it is possible to decode a coded stream that is scalable coded so that coding efficiency is improved by the third embodiment of the coding apparatus to which the present disclosure is applied.
  • FIG. 41 shows an example of a multi-view image encoding method.
  • dQP (base view) Current_CU_QP (base view)-LCU_QP (base view) (1-2)
  • dQP (base view) Current_CU_QP (base view)-Previsous_CU_QP (base view) (1-3)
  • dQP (base view) Current_CU_QP (base view)-Slice_QP (base view)
  • non-base-view (2-1)
  • dQP (non-base view) Current_CU_QP (non-base view)-LCU_QP (non-base view) (2-2)
  • dQP (non-base view) Current QP (non-base view)-Previsous QP (non-base view) (2-3)
  • FIG. 42 shows another example of scalable coding.
  • the input unit 606 includes a keyboard, a mouse, a microphone, and the like.
  • the output unit 607 includes a display, a speaker, and the like.
  • the storage unit 608 includes a hard disk, a nonvolatile memory, and the like.
  • the communication unit 609 includes a network interface or the like.
  • the drive 610 drives a removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.
  • the program executed by the computer may be a program that is processed in time series in the order described in this specification, or in parallel or at a necessary timing such as when a call is made. It may be a program for processing.
  • the video signal processing unit 905 performs noise removal, video processing according to user settings, and the like on the video data.
  • the video signal processing unit 905 generates video data of a program to be displayed on the display unit 906, image data by processing based on an application supplied via a network, and the like.
  • the video signal processing unit 905 generates video data for displaying a menu screen for selecting an item and the like, and superimposes the video data on the video data of the program.
  • the video signal processing unit 905 generates a drive signal based on the video data generated in this way, and drives the display unit 906.
  • the external interface unit 909 is an interface for connecting to an external device or a network, and transmits and receives data such as video data and audio data.
  • the control unit 910 is configured using a CPU (Central Processing Unit), a memory, and the like.
  • the memory stores a program executed by the CPU, various data necessary for the CPU to perform processing, EPG data, data acquired via a network, and the like.
  • the program stored in the memory is read and executed by the CPU at a predetermined timing such as when the television device 900 is activated.
  • the CPU executes each program to control each unit so that the television device 900 operates in accordance with the user operation.
  • the television device 900 includes a bus 912 for connecting the tuner 902, the demultiplexer 903, the video signal processing unit 905, the audio signal processing unit 907, the external interface unit 909, and the control unit 910.
  • FIG. 45 illustrates a schematic configuration of a mobile phone to which the present technology is applied.
  • the cellular phone 920 includes a communication unit 922, an audio codec 923, a camera unit 926, an image processing unit 927, a demultiplexing unit 928, a recording / reproducing unit 929, a display unit 930, and a control unit 931. These are connected to each other via a bus 933.
  • the image data generated by the camera unit 926 is supplied to the image processing unit 927.
  • the image processing unit 927 performs encoding processing of image data and generates encoded data.
  • the external interface unit 942 includes at least one of an IEEE 1394 interface, a network interface unit, a USB interface, a flash memory interface, and the like.
  • the external interface unit 942 is an interface for connecting to an external device, a network, a memory card, and the like, and receives data such as video data and audio data to be recorded.
  • the HDD unit 944 records content data such as video and audio, various programs, and other data on a built-in hard disk, and reads them from the hard disk during playback.
  • the decoder 947 is provided with the function of the decoding apparatus (decoding method) of the present application. For this reason, it is possible to decode an image that has been scalable-encoded so as to improve the encoding efficiency.
  • the camera signal processing unit 963 performs various camera signal processing such as knee correction, gamma correction, and color correction on the electrical signal supplied from the imaging unit 962.
  • the camera signal processing unit 963 supplies the image data after the camera signal processing to the image data processing unit 964.
  • the scalable encoded data storage unit 1001 stores scalable encoded data (BL + EL) 1011 encoded in a scalable manner.
  • the scalable encoded data (BL + EL) 1011 is encoded data including both a base layer and an enhancement layer, and is data that can obtain both a base image and an enhancement image by decoding.
  • the terminal apparatus 1102 has a reception function of the terrestrial broadcast 1111 broadcast by the broadcast station 1101 and receives base layer scalable encoded data (BL) 1121 transmitted via the terrestrial broadcast 1111.
  • the terminal apparatus 1102 further has a communication function for performing communication via the network 1112, and receives enhancement layer scalable encoded data (EL) 1122 transmitted via the network 1112.
  • BL base layer scalable encoded data
  • EL enhancement layer scalable encoded data
  • the scalable encoded data storage device 1202 stores the scalable encoded data (BL + EL) 1221 supplied from the imaging device 1201 with quality according to the situation. For example, in the normal case, the scalable encoded data storage device 1202 extracts base layer data from the scalable encoded data (BL + EL) 1221, and the base layer scalable encoded data ( BL) 1222. On the other hand, for example, in the case of attention, the scalable encoded data storage device 1202 stores scalable encoded data (BL + EL) 1221 with high quality and a large amount of data.
  • whether it is normal time or attention time may be determined by the scalable encoded data storage device 1202 analyzing an image, for example.
  • the imaging apparatus 1201 may make a determination, and the determination result may be transmitted to the scalable encoded data storage device 1202.
  • the imaging apparatus 1201 may determine the number of layers for scalable coding according to the state. For example, in a normal case, the imaging apparatus 1201 may generate base layer scalable encoded data (BL) 1222 with low quality and a small amount of data, and supply the scalable encoded data storage apparatus 1202 to the scalable encoded data storage apparatus 1202. For example, when attention is paid, the imaging device 1201 generates scalable encoded data (BL + EL) 1221 having a high quality and a large amount of data, and supplies the scalable encoded data storage device 1202 to the scalable encoded data storage device 1202. May be.
  • BL base layer scalable encoded data
  • BL + EL scalable encoded data
  • the monitoring camera has been described as an example.
  • the use of the imaging system 1200 is arbitrary and is not limited to the monitoring camera.
  • the video set 1300 includes a module group such as a video module 1311, an external memory 1312, a power management module 1313, and a front-end module 1314, and a related group such as a connectivity 1321, a camera 1322, and a sensor 1323. And a device having a function.
  • the 51 is a processor that executes an application related to image processing.
  • the application executed in the application processor 1331 not only performs arithmetic processing to realize a predetermined function, but also can control the internal and external configurations of the video module 1311 such as the video processor 1332 as necessary. .
  • the video processor 1332 is a processor having a function related to image encoding / decoding (one or both of them).
  • the broadband modem 1333 is a processor (or module) that performs processing related to wired or wireless (or both) broadband communication performed via a broadband line such as the Internet or a public telephone line network.
  • the broadband modem 1333 digitally modulates data to be transmitted (digital signal) to convert it into an analog signal, or demodulates the received analog signal to convert it into data (digital signal).
  • the broadband modem 1333 can digitally modulate and demodulate arbitrary information such as image data processed by the video processor 1332, a stream obtained by encoding the image data, an application program, setting data, and the like.
  • the RF module 1334 is a module that performs frequency conversion, modulation / demodulation, amplification, filter processing, and the like on an RF (Radio RF Frequency) signal transmitted and received via an antenna. For example, the RF module 1334 generates an RF signal by performing frequency conversion or the like on the baseband signal generated by the broadband modem 1333. Further, for example, the RF module 1334 generates a baseband signal by performing frequency conversion or the like on the RF signal received via the front end module 1314.
  • RF Radio RF Frequency
  • the application processor 1331 and the video processor 1332 may be integrated into a single processor.
  • the external memory 1312 is a module having a storage device that is provided outside the video module 1311 and is used by the video module 1311.
  • the storage device of the external memory 1312 may be realized by any physical configuration, but is generally used for storing a large amount of data such as image data in units of frames. For example, it is desirable to realize it with a relatively inexpensive and large-capacity semiconductor memory such as DRAM (Dynamic Random Access Memory).
  • the front end module 1314 is a module that provides the RF module 1334 with a front end function (a circuit on a transmitting / receiving end on the antenna side). As illustrated in FIG. 51, the front end module 1314 includes, for example, an antenna unit 1351, a filter 1352, and an amplifying unit 1353.
  • Connectivity 1321 is a module having a function related to connection with the outside.
  • the physical configuration of the connectivity 1321 is arbitrary.
  • the connectivity 1321 has a configuration having a communication function other than the communication standard supported by the broadband modem 1333, an external input / output terminal, and the like.
  • the connectivity 1321 may include a data (signal) transmission destination device.
  • the drive 1321 reads and writes data to and from a recording medium such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory (not only a removable medium drive, but also a hard disk, SSD (Solid State Drive) NAS (including Network Attached Storage) and the like.
  • the connectivity 1321 may include an image or audio output device (a monitor, a speaker, or the like).
  • the sensor 1323 includes, for example, a voice sensor, an ultrasonic sensor, an optical sensor, an illuminance sensor, an infrared sensor, an image sensor, a rotation sensor, an angle sensor, an angular velocity sensor, a velocity sensor, an acceleration sensor, an inclination sensor, a magnetic identification sensor, an impact sensor, It is a module having an arbitrary sensor function such as a temperature sensor.
  • the data detected by the sensor 1323 is supplied to the application processor 1331 and used by an application or the like.
  • the present technology can be applied to the video processor 1332 as described later. Therefore, the video set 1300 can be implemented as a set to which the present technology is applied.
  • the video processor 1332 receives the video signal and the audio signal and encodes them in a predetermined method, decodes the encoded video data and audio data, A function of reproducing and outputting an audio signal.
  • the video processor 1332 includes a video input processing unit 1401, a first image enlargement / reduction unit 1402, a second image enlargement / reduction unit 1403, a video output processing unit 1404, a frame memory 1405, and a memory control unit 1406.
  • the video processor 1332 includes an encoding / decoding engine 1407, video ES (ElementaryElementStream) buffers 1408A and 1408B, and audio ES buffers 1409A and 1409B.
  • the video input processing unit 1401 acquires a video signal input from, for example, the connectivity 1321 (FIG. 51) and converts it into digital image data.
  • the first image enlargement / reduction unit 1402 performs format conversion, image enlargement / reduction processing, and the like on the image data.
  • the second image enlargement / reduction unit 1403 performs image enlargement / reduction processing on the image data in accordance with the format of the output destination via the video output processing unit 1404, or is the same as the first image enlargement / reduction unit 1402. Format conversion and image enlargement / reduction processing.
  • the video output processing unit 1404 performs format conversion, conversion to an analog signal, and the like on the image data, and outputs the reproduced video signal to, for example, the connectivity 1321 (FIG. 51).
  • the memory control unit 1406 receives the synchronization signal from the encoding / decoding engine 1407, and controls the writing / reading access to the frame memory 1405 according to the access schedule to the frame memory 1405 written in the access management table 1406A.
  • the access management table 1406A is updated by the memory control unit 1406 in accordance with processing executed by the encoding / decoding engine 1407, the first image enlargement / reduction unit 1402, the second image enlargement / reduction unit 1403, and the like.
  • the video ES buffer 1408A buffers the video stream generated by the encoding / decoding engine 1407 and supplies the buffered video stream to the multiplexing unit (MUX) 1412.
  • the video ES buffer 1408B buffers the video stream supplied from the demultiplexer (DMUX) 1413 and supplies the buffered video stream to the encoding / decoding engine 1407.
  • the demultiplexing unit (DMUX) 1413 demultiplexes the bit stream in which the video stream and the audio stream are multiplexed by a method corresponding to the multiplexing by the multiplexing unit (MUX) 1412. That is, the demultiplexer (DMUX) 1413 extracts the video stream and the audio stream from the bit stream read from the stream buffer 1414 (separates the video stream and the audio stream). That is, the demultiplexer (DMUX) 1413 can convert the stream format by demultiplexing (inverse conversion of the conversion by the multiplexer (MUX) 1412).
  • the demultiplexing unit (DMUX) 1413 obtains a transport stream supplied from, for example, the connectivity 1321 and the broadband modem 1333 (both in FIG. 51) via the stream buffer 1414 and demultiplexes the transport stream. Can be converted into a video stream and an audio stream. Further, for example, the demultiplexer (DMUX) 1413 obtains the file data read from various recording media by the connectivity 1321 (FIG. 51) via the stream buffer 1414 and demultiplexes it, for example. It can be converted into a video stream and an audio stream.
  • the stream buffer 1414 buffers the bit stream.
  • the stream buffer 1414 buffers the transport stream supplied from the multiplexing unit (MUX) 1412 and, for example, at the predetermined timing or based on a request from the outside, for example, the connectivity 1321 or the broadband modem 1333 (whichever Are also supplied to FIG.
  • MUX multiplexing unit
  • the stream buffer 1414 buffers the file data supplied from the multiplexing unit (MUX) 1412 and, for example, connectivity 1321 (FIG. 51) or the like at a predetermined timing or based on an external request or the like. To be recorded on various recording media.
  • MUX multiplexing unit
  • connectivity 1321 FIG. 51
  • the stream buffer 1414 buffers the transport stream acquired through, for example, the connectivity 1321 and the broadband modem 1333 (both of which are shown in FIG. 51), and performs reverse processing at a predetermined timing or based on an external request or the like.
  • the data is supplied to a multiplexing unit (DMUX) 1413.
  • DMUX multiplexing unit
  • a transport stream input from an external network to the video processor 1332 via the connectivity 1321 or the broadband modem 1333 (both in FIG. 51) is buffered in the stream buffer 1414 and then demultiplexed (DMUX). 1413 is demultiplexed.
  • file data read from various recording media in the connectivity 1321 (FIG. 51) and inputted to the video processor 1332 is buffered in the stream buffer 1414 and then demultiplexed by the demultiplexer (DMUX) 1413. It becomes. That is, the transport stream or file data input to the video processor 1332 is separated into a video stream and an audio stream by the demultiplexer (DMUX) 1413.
  • FIG. 53 illustrates another example of a schematic configuration of the video processor 1332 (FIG. 51) to which the present technology is applied.
  • the video processor 1332 has a function of encoding and decoding video data by a predetermined method.
  • the eyelid image processing engine 1514 performs predetermined image processing such as filter processing for improving image quality on the image data under the control of the control unit 1511.
  • the codec engine 1516 includes, for example, MPEG-2 video 1541, AVC / H.2641542, HEVC / H.2651543, HEVC / H.265 (Scalable) 1544, as function blocks for processing related to the codec.
  • HEVC / H.265 (Multi-view) 1545 and MPEG-DASH 1551 are included.
  • the multiplexing / demultiplexing unit (MUX DMUX) 1518 can convert the data format by multiplexing / demultiplexing.
  • the multiplexing / demultiplexing unit (MUX DMUX) 1518 multiplexes the bitstream, thereby transporting the transport stream, which is a bit stream in a transfer format, or data in a file format for recording (file data).
  • the transport stream which is a bit stream in a transfer format, or data in a file format for recording (file data).
  • file data file format for recording
  • image data and other data are exchanged between the processing units in the video processor 1332 using, for example, the internal memory 1515 and the external memory 1312.
  • the power management module 1313 controls power supply to the control unit 1511, for example.
  • the method for transmitting such information is not limited to such an example.
  • these pieces of information may be transmitted or recorded as separate data associated with the encoded bitstream without being multiplexed into the encoded bitstream.
  • the term “associate” means that an image (which may be a part of an image such as a slice or a block) included in the bitstream and information corresponding to the image can be linked at the time of decoding. Means. That is, information may be transmitted on a transmission path different from that of the image (or bit stream).
  • This disclosure receives bitstreams compressed by orthogonal transform such as discrete cosine transform and motion compensation, such as MPEG, H.26x, etc., via network media such as satellite broadcasting, cable TV, the Internet, and mobile phones.
  • orthogonal transform such as discrete cosine transform and motion compensation
  • the present invention can be applied to an encoding device or a decoding device that is used when processing on a storage medium such as an optical, magnetic disk, or flash memory.
  • the present disclosure can take a cloud computing configuration in which one function is shared by a plurality of devices via a network and is processed jointly.
  • the plurality of processes included in the one step can be executed by being shared by a plurality of apparatuses in addition to being executed by one apparatus.
  • An image selection unit that selects an image of a first layer before loop filter processing or an image of the first layer after loop filter processing;
  • a decoding device comprising: a decoding unit that decodes encoded data of an image in a second layer using the image in the first layer selected by the image selection unit.
  • An upsampling unit that upsamples the image of the first layer selected by the image selection unit;
  • the decoding apparatus according to (1), wherein the decoding unit is configured to decode the encoded data using the first layer image upsampled by the upsampling unit.
  • the decoding device according to (1) or (2), wherein the loop filter process is a deblock filter process.
  • the loop filter process is an adaptive offset filter process.
  • An image selection unit that selects an image of a first layer before loop filter processing or an image of the first layer after loop filter processing;
  • An encoding apparatus comprising: an encoding unit that encodes a second layer image using the first layer image selected by the image selection unit and generates encoded data.
  • An upsampling unit that upsamples the image of the first layer selected by the image selection unit; The encoding unit according to (11), wherein the encoding unit is configured to encode the second layer image using the first layer image up-sampled by the up-sampling unit.
  • Device (13) The encoding device according to (11) or (12), wherein the loop filter process is a deblock filter process.
  • An up-sampling unit that up-samples the first layer image before the loop filter processing and the first layer image after the loop filter processing;
  • the image selection unit predicts the first layer image before the loop filter processing upsampled by the upsampling unit and the first layer image after the loop filter processing as reference image candidates. Based on the result, the image of the first layer before the loop filter processing upsampled by the upsampling unit or the image of the first layer after the loop filter processing is selected (11 ) To (14).
  • a transmission unit that transmits an index that identifies the image of the first layer selected by the image selection unit;
  • the index for specifying the first layer image after the loop filter processing is configured to be smaller than the index for specifying the first layer image before the loop filter processing.
  • the encoding device An image selection step of selecting an image of the first layer before the loop filter processing or an image of the first layer after the loop filter processing;
  • An encoding method comprising: an encoding step of encoding an image of a second layer using the image of the first layer selected by the processing of the image selection step and generating encoded data.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

　La présente invention concerne un dispositif et un procédé de décodage, ainsi qu'un dispositif et un procédé de codage, susceptibles d'améliorer le rendement de codage d'un codage extensible. Une unité de sélection d'images sélectionne une image de base avant traitement par filtre à boucle ou une image de base après traitement par filtre à boucle. Une unité d'addition décode des données codées d'une image d'enrichissement en utilisant l'image de base sélectionnée par l'unité de sélection d'images. La présente invention peut être appliquée, par exemple, dans un dispositif de décodage qui décode des images qui ont été soumises à un codage extensible tel qu'un codage extensible spatial dans un schéma de codage vidéo à haut rendement (HEVC).
PCT/JP2014/075799 2013-10-11 2014-09-29 Dispositif et procédé de décodage, dispositif et procédé de codage WO2015053111A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2013213779 2013-10-11
JP2013-213779 2013-10-11

Publications (1)

Publication Number Publication Date
WO2015053111A1 true WO2015053111A1 (fr) 2015-04-16

Family

ID=52812930

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2014/075799 WO2015053111A1 (fr) 2013-10-11 2014-09-29 Dispositif et procédé de décodage, dispositif et procédé de codage

Country Status (1)

Country Link
WO (1) WO2015053111A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10337923B2 (en) 2017-09-13 2019-07-02 Qualcomm Incorporated Directional interpolation and cross-band filtering for hyperspectral imaging

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006191576A (ja) * 2005-01-04 2006-07-20 Samsung Electronics Co Ltd イントラｂｌモードを考慮したデブロックフィルタリング方法、及び該方法を用いる多階層ビデオエンコーダ／デコーダ
JP2013093792A (ja) * 2011-10-27 2013-05-16 Sony Corp 画像処理装置および方法
JP2013187868A (ja) * 2012-03-09 2013-09-19 Sharp Corp 画像復号装置、画像符号化装置、およびデータ構造

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006191576A (ja) * 2005-01-04 2006-07-20 Samsung Electronics Co Ltd イントラｂｌモードを考慮したデブロックフィルタリング方法、及び該方法を用いる多階層ビデオエンコーダ／デコーダ
JP2013093792A (ja) * 2011-10-27 2013-05-16 Sony Corp 画像処理装置および方法
JP2013187868A (ja) * 2012-03-09 2013-09-19 Sharp Corp 画像復号装置、画像符号化装置、およびデータ構造

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
DANNY HONG(ET AL. ET AL.: "Scalability Support in HEVC", JOINT COLLABORATIVE TEAM ON VIDEO CODING (JCT-VC) OF ITU-T SG 16 WP3 AND ISO/IEC JTC1/SC29/WG11 JCTVC-F290, ITU-T, 22 July 2011 (2011-07-22), pages 1 - 15 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10337923B2 (en) 2017-09-13 2019-07-02 Qualcomm Incorporated Directional interpolation and cross-band filtering for hyperspectral imaging

Similar Documents

Publication Publication Date Title
JP6780761B2 (ja) 画像符号化装置および方法
JP6287035B2 (ja) 復号装置および復号方法
WO2015053115A1 (fr) Dispositif de décodage, procédé de décodage, dispositif de codage et procédé de codage
WO2014002896A1 (fr) Dispositif et procédé de codage, dispositif et procédé de décodage
WO2015137145A1 (fr) Dispositif et procédé de codage d'image, dispositif et procédé de décodage d'image
WO2015098561A1 (fr) Dispositif de décodage, procédé de décodage, dispositif de codage et procédé de codage
WO2015053116A1 (fr) Dispositif de décodage, procédé de décodage, dispositif de codage, et procédé de codage
JP2015005899A (ja) 復号装置および復号方法、並びに、符号化装置および符号化方法
WO2015053112A1 (fr) Dispositif et procédé de décodage, dispositif et procédé de codage
WO2015098559A1 (fr) Dispositif de décodage, procédé de décodage, dispositif de codage, et procédé de codage
KR102338669B1 (ko) 화상 처리 장치 및 방법, 및 기록 매체
JP6477930B2 (ja) 符号化装置および符号化方法
WO2014208326A1 (fr) Dispositif et procédé de codage d'image, et dispositif et procédé de décodage d'image
WO2014002900A1 (fr) Dispositif et procédé de traitement d'images
WO2015053111A1 (fr) Dispositif et procédé de décodage, dispositif et procédé de codage
WO2014050732A1 (fr) Dispositif et procédé de codage, et dispositif et procédé de décodage
WO2015098563A1 (fr) Dispositif et procédé de codage d'image et dispositif et procédé de décodage d'image
JP6402802B2 (ja) 画像処理装置および方法、プログラム、並びに記録媒体
WO2014162916A1 (fr) Appareil et procédé de codage d'image, et appareil et procédé de décodage d'image
JP2015050738A (ja) 復号装置および復号方法、並びに、符号化装置および符号化方法
WO2014156705A1 (fr) Dispositif et procédé de décodage, et dispositif et procédé d'encodage
WO2014203762A1 (fr) Dispositif de décodage, procédé de décodage, dispositif de codage, et procédé de codage
WO2014156707A1 (fr) Dispositif et procédé de codage d'image et dispositif et procédé de décodage d'image

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14852291

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 14852291

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP