EP1862010A1 - Procede hierarchique de codage/decodage video pour une variabilite d'echelle complete et appareil correspondant - Google Patents

Procede hierarchique de codage/decodage video pour une variabilite d'echelle complete et appareil correspondant

Info

Publication number
EP1862010A1
EP1862010A1 EP06732697A EP06732697A EP1862010A1 EP 1862010 A1 EP1862010 A1 EP 1862010A1 EP 06732697 A EP06732697 A EP 06732697A EP 06732697 A EP06732697 A EP 06732697A EP 1862010 A1 EP1862010 A1 EP 1862010A1
Authority
EP
European Patent Office
Prior art keywords
region
interest
layer
recited
regions
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
EP06732697A
Other languages
German (de)
English (en)
Other versions
EP1862010A4 (fr
Inventor
Jung-Won Kang
Hae-Chul Choi
Jae-Gon 203-402 Saemmeori Apt. KIM
Jin-Woo Hong
Yong-Man 816-1102 Yeolmae Maeul Apt Site 8 RO
Tae-Meon Bae
Yong-Ju Jung
Cong-Thang Info. And Communication Uni. TRUONG
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Electronics and Telecommunications Research Institute ETRI
Korea Advanced Institute of Science and Technology KAIST
Original Assignee
Electronics and Telecommunications Research Institute ETRI
Research and Industrial Cooperation Group
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Electronics and Telecommunications Research Institute ETRI, Research and Industrial Cooperation Group filed Critical Electronics and Telecommunications Research Institute ETRI
Publication of EP1862010A1 publication Critical patent/EP1862010A1/fr
Publication of EP1862010A4 publication Critical patent/EP1862010A4/fr
Withdrawn legal-status Critical Current

Links

Classifications

    • A—HUMAN NECESSITIES
    • A41—WEARING APPAREL
    • A41C—CORSETS; BRASSIERES
    • A41C3/00—Brassieres
    • A41C3/08—Brassieres combined with other garments
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/59—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
    • A—HUMAN NECESSITIES
    • A41—WEARING APPAREL
    • A41B—SHIRTS; UNDERWEAR; BABY LINEN; HANDKERCHIEFS
    • A41B9/00—Undergarments
    • A41B9/16—Shoulder-straps forming part of the undergarments
    • A—HUMAN NECESSITIES
    • A41—WEARING APPAREL
    • A41C—CORSETS; BRASSIERES
    • A41C3/00—Brassieres
    • A41C3/0007—Brassieres with stay means
    • A—HUMAN NECESSITIES
    • A41—WEARING APPAREL
    • A41C—CORSETS; BRASSIERES
    • A41C3/00—Brassieres
    • A41C3/02—Brassieres with front closures
    • A—HUMAN NECESSITIES
    • A41—WEARING APPAREL
    • A41C—CORSETS; BRASSIERES
    • A41C3/00—Brassieres
    • A41C3/12—Component parts
    • A41C3/122—Stay means
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/187—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a scalable video layer
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/20—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding
    • H04N19/29—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding involving scalability at the object level, e.g. video object layer [VOL]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
    • H04N19/33—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability in the spatial domain
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/46—Embedding additional information in the video signal during the compression process
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards

Definitions

  • the present invention relates to a scalable video encoding/decoding method and an apparatus thereof; and more particularly, to a method of hierarchically encoding and decoding multi-regions having various locations and different resolutions according to a resolution for providing complete scalability of a spatial domain.
  • JPEG 2000 standard [ISO/IEC JTCl 15444-2: JPEG 2000 Image Coding System; Extension, 2004] supports different regions in an image to encode at various bit rates. That is, a predetermined region in an image is allowed to be encoded at a higher bit rate compared to other regions in the image.
  • Such an encoding scheme has been a main issue for last several years.
  • the JPEG 2000 standard allows a decoder to independently decode a region of interest (ROI) with a spatial scalability by defining a predetermined region in an image as a region of interest (ROI) and encoding the ROI using a scaling based method before encoding other regions.
  • the encoding method introduced in the JPEG 2000 standard may not be applied into a MPEG based video encoding scheme because the encoding method of the JPEG 2000 standard is designed for encoding a still image.
  • a MPEG based video encoding method is a general method for encoding a video image.
  • an object based encoding scheme introduced in MPEG-4 [ISO/IEC JTCI 14496-2: Coding of Audio-Visual Objects, Part 2, 1998] provides a coding method that satisfies a condition of independently coding regions in an image having more than one regions.
  • a plurality of regions in the image is a two- dimensional region having a predetermined shape.
  • Such regions are encoded independently or encoded through motion estimation, residual coding or shaped-adaptive DCT (SADCT).
  • SADCT shaped-adaptive DCT
  • a MPEG-4 object oriented encoding method may not be used for an image having a plurality of regions having different resolutions.
  • H.262/MPEG-2 Visual [ISO/IEC JTCl 13818-2: Generic Coding of Moving Pictures and Associated Audio Information, Part 2; Video, 1994], H.263 [ITU-T: Video Coding for Low Bit-rate Communication, 1995 (version 1), 1998 (version 2)], and MPEG-4 Visual [ISO/IEC JTCl 14496-2: Coding of Audio-Visual Objects, Part 2, 1998] support an same image to be decoded into images with different region resolutions using a layer based coding method in order to achieve a spatial resolution adaptive encoding.
  • Such a layer based coding method encodes an image produced at a lower resolution than a resolution of an original image into a base layer and encodes an image produced at a higher spatial resolution using information about the coded image into a next layer.
  • demands of various motion picture services increase according to introduction of diverse video reproducing terminals such as a personal data assistant (PDA) , a mobile phone, a digital multimedia broadcasting (DMB) phone, a standard definition (SD) TV and a high definition (HD) TV.
  • PDA personal data assistant
  • DMB digital multimedia broadcasting
  • SD standard definition
  • HD high definition
  • an object of the present invention to provide an encoding/decoding apparatus for providing a complete scalability on a spatial domain to independently decode regions having various locations and different resolutions in an image, and a method thereof.
  • an encoder for encoding a video image including: an overlapped region (OR) detector for receiving coding region information about a plurality of regions of interest (ROI) in the video image to encode and detecting overlapped regions (OR) in the ROI regions; a region arranger for arranging the video image, the regions of interest and the detected overlapped regions into a plurality of layers according to a resolution; and a region encoder for encoding the video image, the regions of interest and the detected overlapped regions according to a resolution of a corresponding layer arranged at the region arranger.
  • the coding region information may include information about locations of the regions of interest in the video image and a coding resolution of the regions of interest.
  • the region encoder may perform an inter-layer coding that encodes regions in a unit of a block using a region arranged in a lower layer.
  • An interpolation may be performed after deciding a pixel value in an outside of a region of interest through an extrapolation if an interpolation is required to perform using a pixel in an outside of a region of interest in the inter-layer coding.
  • a flag may be added into a coded stream where the flag denotes that a region of interest is in the video image.
  • the intra-layer coding may be performed without performing the interpolation when the interpolation requires a pixel in an outside of a region of interest.
  • a decoder including: decoding region extractor for receiving selection information for selecting a region to decode, and extracting region information required for decoding a region of interest (ROI) corresponding to the selected region from a coded stream including coding information about a plurality of regions of interest; and a region decoder for receiving the extracted region information and recovering an image signal of a region of interest in a video image by performing a decoding based on the received region information.
  • the selection information may include information about location of the decoding region in the video image and a decoding resolution.
  • the decoding region extractor may extract a coded stream of a region of interest corresponding to the selected region from the coded stream, and extract information of related regions in a lower layer from the extracted coded stream for decoding the region of interest.
  • An inter-layer decoding may be performed in a unit of a block using a region of a lower layer having regions of a lower resolution.
  • An intra-layer decoding may be performed on a block located at a region of interest boundary in the inter-layer decoding.
  • Each of the regions of interest may be configured of small rectangular regions for supporting an interactive decoding.
  • an encoding method for providing a spatial scalability including the steps of: a) receiving information about locations and resolutions of a plurality of regions of interest for encoding an input video image; b) arranging the regions of interest into corresponding layers according to a resolution; and c) encoding the arranged region of interest in a block unit, an intra-layer coding is performed on a block that requires an interpolation to perform using a pixel in an outside of a region of interest when an inter-layer coding is performed using region of interest information in a lower layer.
  • the motion information of a block that requires an interpolation to perform using a pixel in an outside of a region of interest may be predicted in an integer pixel unit when the motion prediction/compensation coding uses ROI information in a same layer having a temporal correlation.
  • An interpolation may be performed after deciding an external pixel value for a block requiring an interpolation to perform using a pixel in an outside of a region of interest when a motion prediction/compensation coding is performed using region of interest information of a same layer having a temporal correlation.
  • the region of interest location information may be expressed as scan numbers assigned to macro blocks for a video image.
  • a method for decoding including the steps of: a) receiving information about a location and a resolution of a decoding region in an input video image; b) extracting region of interest information corresponding to the decoding region from a transmitted coded stream; and c) decoding the region of interest using the extracted information.
  • Intra-layer decoding is performed on a block requiring an interpolation to perform using a pixel in an outside of a region of interest when an inter layer decoding is performed using region of interest information of a lower layer.
  • the motion information of a block that requires an interpolation to perform using a pixel in an outside of a region of interest may be predicted in a unit of an integer pixel when a motion prediction/compensation decoding is performed using region of interest information of a same layer having a temporal correlation.
  • An interpolation may be performed after deciding an external pixel value through an extrapolation for a block that requires an interpolation to perform using a pixel in an outside of a region of interest when an inter-layer decoding is performed using region of interest information of a lower layer.
  • An interpolation may be performed after deciding an external pixel value through an extrapolation for a block that requires an interpolation using a pixel in an outside of a region of interest when a motion prediction/compensation decoding is performed using region of interest information of a same layer having a temporal correlation.
  • the decoding method may further include determining whether a region of interest of a video image in a coded stream through a flag denoting existence of a region of interest included in the coded stream.
  • the location information of the region of interest may be expressed as scan numbers assigned to macro blocks in a video image.
  • a video encoding/decoding apparatus provides a complete scalability of a spatial domain by defining a region of interest (ROI) in a video image. Also, the video encoding/decoding apparatus according to the present invention provides an improved coding rate by encoding video image in consideration of spatial redundancy among a plurality of regions of interest.
  • the encoding/decoding apparatus and method thereof according to the present invention easily obtains region information for decoding regions of interest through a coded stream which has an effective syntax structure designed to effectively transmit information related to a plurality of region of interest.
  • a method of adding a ROI flag into the syntax structure of the coded steam is disclosed in the present invention. Therefore, a decoder is enabled to recognize existence of an independently decodable ROI and to active a related function thereof.
  • the present invention discloses a method of processing a region at a ROI boundary when a region requires information of other regions to reference. Therefore, the decoder is allowed to independently decode a ROI according to the present invention. Especially, the present invention enables the decoder to easily decode a ROI without requiring additional elements by adding a restriction condition that prevents an interpolation from being performed when the interpolation requires a pixel in an outside of an ROI to use.
  • Fig. 1 is a block diagram illustrating an image processing system for providing a spatial scalability in accordance with a preferred embodiment of the present invention
  • Fig. 2 is a block diagram illustrating the encoder 110 of Fig. 1 for encoding an input video image and regions in the image
  • Fig. 3 is a flowchart for describing an operation of an overlapped region detector 210 in accordance with a preferred embodiment of the present invention
  • Fig. 4 is a flowchart for describing an operation for detecting overlapped regions in all of regions at the step S310 in Fig. 3;
  • Fig. 5 is a diagram showing arrangement of regions such as a video image, a region and an overlapped region according to layers when the multi-layer based coding method is employed in accordance with a preferred embodiment of the present invention
  • Fig. 6 is a diagram illustrating arrangement of regions such as a video image, a region and an overlapped region according to layers when the one-layer based coding method is used in accordance with a preferred embodiment of the present invention
  • Fig. 7 is a flowchart illustrating a method of arranging regions into corresponding layers in a multilayer based encoding method in accordance with a preferred embodiment of the present invention
  • Fig. 8 is a flowchart of a method of arranging regions into corresponding layers in a one-layer based encoding method in accordance with a preferred embodiment of the present invention
  • Fig. 9 is a block diagram of a region encoder 230 employing a multi-layer encoding method in accordance with a preferred embodiment of the present invention.
  • Fig. 10 is a block diagram of a region encoder 230 employing a one-layer based coding method in accordance with a preferred embodiment of the present invention
  • Fig. 11 is a flowchart for describing an operation of a region encoder 230 employing a multi-layer based encoding method in accordance with a preferred embodiment of the present invention
  • Fig. 12 is a flowchart showing a method of deciding a coding mode for encoding a block of a selected region when a multi-layer based encoding method is employed in accordance with a preferred embodiment of the present invention
  • Fig. 13 is a flowchart showing a method of deciding a coding mode for encoding a block of a selected region when a one-layer based encoding method is employed in accordance with a preferred embodiment of the present invention
  • Fig. 14 is a flowchart of a method of deciding a coding mode for encoding a block of a selected region when a one-layer based coding method is employed in accordance with a preferred embodiment of the present invention
  • Fig. 15 is a view illustrating a method of interpolation used in an inter-layer coding mode and a motion prediction/ compensation mode in accordance with a preferred embodiment of the present invention
  • Fig. 16 shows a SVC bit stream having a flag denoting existence of a ROI in an image in accordance with a first embodiment of the present invention
  • Fig. 17 shows a SVC bit stream having a flag denoting existence of a ROI in an image in accordance with a second embodiment of the present invention
  • Fig. 18 shows a SVC bit stream having a flag denoting existence of a ROI in an image in accordance with a third embodiment of the present invention
  • Fig. 19 is a block diagram showing coding dependency between regions in the inter-layer coding when a multilayer based coding method is employed;
  • Fig. 20 is a block diagram showing coding dependency between regions in the inter-layer coding when a one-layer based coding method is employed;
  • Fig. 21 shows a syntax structure of a coded stream including information about related regions in a lower layer in accordance with a preferred embodiment of the present invention
  • Fig. 22 is a block diagram of the decoder 120 of Fig. 1 for decoding a coded stream in accordance with a preferred embodiment of the present invention
  • Fig. 23 is a flowchart showing operations of the region decoder employing a multi-layer based decoding method in accordance with a preferred embodiment of the present invention.
  • Fig. 24 is a flowchart showing a method of decoding a coded stream for blocks of a selected region when a multilayer based decoding method is applied in accordance with a preferred embodiment of the present invention
  • Fig. 25 is a flowchart showing operations of the region decoder employing a one-layer based decoding method in accordance with a preferred embodiment of the present invention
  • Fig. 26 is a flowchart showing a decoding of a coded stream of a block of a selected region when a one-layer based decoding is applied in accordance with a preferred embodiment of the present invention. Best Mode for the Invention
  • Fig. 1 is a block diagram illustrating an image processing system for providing a spatial scalability in accordance with a preferred embodiment of the present invention.
  • the image processing system includes an encoder 110, a decoder 20, a user interface 130 and a display 140.
  • the encoder 110 receives video images and region of interest (ROI) information and outputs coded bit-streams by performing an encoding operation according to the present invention.
  • the region of interest (ROI) denotes a desired region in a video image to encode
  • the ROI information denotes information about the ROI.
  • the ROI information includes information about a location of a ROI and a coding resolution.
  • the information may additionally include a region index of a ROI.
  • the coded bit-streams outputted from the encoder 110 are transmitted to the decoder 120 through a channel.
  • the decoder 120 receives decoding region information from the user interface unit 130.
  • the decoding region information denotes information about a decoding region selected by a user through the user interface 130, and the decoding region is a target region to decode.
  • the decoder 120 restores an image signal of a decoding region by performing a decoding operation according to the present invention using the received decoding region information.
  • the decoding region information includes information about a location of a decoding region.
  • the decoding region information may additionally include information about a decoding resolution which is a desired resolution to decode.
  • the display 140 receives the restored image signal from the decoder 120 and displays an image of the region selected by the user.
  • Fig. 2 is a block diagram illustrating the encoder 110 of Fig. 1 for encoding an input video image and regions in the image.
  • the encoder 110 includes an overlapped region (OR) detector 210 for receiving the video image and the ROI information and detecting an overlapped region (OR) in the video image and the regions, a region arranging unit 220 for arranging regions of interest in an image including the detected OR into corresponding layers and a region encoder 230 for generating a coded stream by encoding the input video image and regions.
  • OR overlapped region
  • region arranging unit 220 for arranging regions of interest in an image including the detected OR into corresponding layers
  • a region encoder 230 for generating a coded stream by encoding the input video image and regions.
  • the overlapped region (OR) detector 210 detects an overlapped region (OR) in regions using information about locations of the regions of interest and defines a new region index for the detected overlapped region.
  • the OR detector 210 defines an overlapped region detected between a first region 1 and a second region 2 as an ORl .
  • An overlapped region detected between a second region 2 and a third region 3 is defined as an OR2.
  • an overlapped region between a third region 3 and a fourth region 4 also includes a region overlapped with the second region 2.
  • the overlapped region detector 210 detects a region having a same spatial resolution which is overlapped in the second region 2 and the third region 3 at first and then defines the detected overlapped region as an OR2.
  • the overlapped region detector 210 selects the overlapped region having different spatial resolutions detected from the region 3 and the region 4 with the region defined as the OR2 excluded and defines the selected overlapped region as an 0R3. That is, the overlapped region detected from the region 3 and the region 4 is configured of the OR2 and the 0R3.
  • Fig. 3 is a flowchart for describing an operation of an overlapped region detector 210 in accordance with a preferred embodiment of the present invention.
  • the overlapped region detector 210 detects overlapped regions in all of regions in an image using ROI information at step S310.
  • the overlapped region detector 210 also detects overlapped regions in the detected overlapped regions at step S320.
  • the overlapped region detector 210 defines a new region index for the detected overlapped region at step S330.
  • the detected overlapped regions detected from regions having different spatial resolutions may include an overlapped region detected from regions having a same spatial resolution such as the 0R2.
  • the overlapped region having different spatial resolutions detected from the region 3 and the region 4 is selected by excluding the overlapped region from regions having the same spatial resolution such as the 0R2.
  • the selected overlapped region is defined as a new region index such as the OR3.
  • Fig. 4 is a flowchart for describing a method for detecting overlapped regions in all of regions at the step S310 in Fig. 3.
  • the shown method of Fig. 4 is equivalently applied into the step S320 for detecting an overlapped region from the detected overlapped regions.
  • two regions are selected using ROI information which denotes information about a region to encode at step S410. Then, it determines whether the two selected regions include a region overlapped or not at step S420. If the two selected regions include the overlapped region at step S420, it determined whether the two selected regions have a same spatial resolution or not at step S430. If the two selected regions have the same spatial resolution, the detected region is defined as an OR of the two selected regions at step S460.
  • the two selected regions determines whether a region having a higher spatial resolution completely includes a region having a lower spatial resolution or not at step S440. If not, a region having the lower spatial resolution overlapped with the region having the higher spatial resolution is defined as the OR at step S450. Then, a pair of regions is selected from unselected regions in the image, and the steps S420 to S460 are repeatedly performed.
  • the region arranging unit 220 arranges regions and overlapped regions according to a resolution.
  • the region arranging unit 220 treats the input video image as a one region having the highest resolution.
  • the present invention is divided into two different methods according to processing of the overlapped region (OR).
  • the encoder 110, the decoder 120 and the display 140 perform different operations in the image processing system of Fig. 1.
  • the two methods are a multi-layer based coding method and a one-layer based coding method.
  • the multi-layer based coding method composes a new layer when there is an overlapped region, arranges the overlapped layer at the new layer and uses information about a plurality of related lower layers to encode one layer.
  • the one-layer based coding method dose not compose a new layer for the overlapped region and uses information about a one-level lower layer to encode a layer.
  • Fig. 5 is a diagram showing arrangement of regions such as a video image, a region and an overlapped region according to layers when the multi-layer based coding method is employed in accordance with a preferred embodiment of the present invention
  • Fig. 6 is a diagram illustrating arrangement of regions such as a video image, a region and an overlapped region according to layers when the one-layer based coding method is used in accordance with a preferred embodiment of the present invention.
  • a second region 2 and a third region 3 have a same resolution, and a fourth region 4 has a higher resolution than the second region 2 and the third region 3. Also, a first region 1 has a higher resolution than the fourth region 4.
  • the input video image is treated as a region having the highest resolution.
  • a first layer 1, a second layer 2, a third layer 3, a fourth layer 4 and a fifth layer 5 denote that each layer has a corresponding layer indexes 1 to 5 , respectively.
  • a layer having an upper layer index denotes a region having a higher resolution.
  • a region having a higher resolution is arranged to an upper layer, and a region having a lower resolution is arranged at a lower layer. Also, regions having a same resolution are arranged at a same layer.
  • the input video image is treated as one region having the highest resolution and is arranged at the highest layer.
  • a method of arranging the overlapped region is different according to the multi-layer based encoding method and the one-layer based encoding method.
  • an OR layer is newly defined in order to arrange the overlapped region between a layer where regions having a same resolution are arranged and a one-level lower layer thereof. Then, the overlapped region is arranged at the OR layer. On the contrary, the overlapped region is arranged at a layer where a region having a same resolution compared to the overlapped region in the one-layer based encoding method as shown in Fig. 6.
  • FIG. 7 is a flowchart illustrating a method of arranging regions into corresponding layers in a multilayer based encoding method in accordance with a preferred embodiment of the present invention. Referring to Fig. 7, it selects a region having a lowest spatial resolution among regions of interest which are coding regions including the input video image and the overlapped region at step S710, and an index of the current layer is initialized as 1 at step S720. Then, it determines whether any overlapped region having a same spatial resolution compared to the currently-selected region is existed on not at step S730.
  • the overlapped region is arranged at the current layer and regions which are not overlapped region are arranged at a one-level upper layer at steps S750 and S760. On the contrary, if there is no overlapped region existed, regions having a same resolution are arranged at the current layer at step S760. Then, it determines whether there is a region having a higher spatial resolution than the currently selected region at step SIlO. If there is the region having the higher spatial resolution, it selects a region having a lowest resolution among un-arranged regions at step S780, and a layer index of the selected region increases by one at step S790. Then, the steps of arranging the regions are repeatedly performed.
  • Fig. 8 is a flowchart of a method of arranging regions into corresponding layers in a one-layer based encoding method in accordance with a preferred embodiment of the present invention. Referring to Fig. 8, it selects a region having a lowest spatial resolution among regions of interest which are coding regions including the input video image and the overlapped region at step S810, and the index of the current layer is initialized as one at step S820. The regions having the same spatial resolution are arranged at the current layer at step S830. Then, the layer index increases by one at step S840 and it determines whether there are any regions having a higher spatial resolution compared to the currently selected region at step S850. If the region having the higher spatial resolution is existed, it selects a region having a lowest resolution among the un-arranged regions at step S860. Then, the region arranging steps are repeatedly performed.
  • the region encoder 230 generates a coded stream by encoding the input video image and regions .
  • the region encoder 230 has a different configuration and performs different operations according to the method of processing the overlapped region (OR). Therefore, the region decoder 230 will be described according to the multi-layer encoding method and the one-layer encoding method.
  • Fig. 9 is a block diagram of a region encoder 230 employing a multi-layer encoding method in accordance with a preferred embodiment of the present invention.
  • the region encoder 230 performs a coding operation on regions in layers according to methods shown in Figs. 11 and 12.
  • the video image is treated as one region for the coding operation.
  • the region information includes information about regions to encode with the overlapped region and information about arrangement of regions into the layers.
  • the region encoder 230 includes down-sampling units 910 and 930, a layerl encoder 920, a Iayer2 encoder 240, a Iayer3 encoder 950 for encoding regions of corresponding layers and a stream MUX 960.
  • the layerl, Iayer2 and Iayer3 encoders 920, 940 and 950 receive the region information and decide a coding mode for encoding a block of the selected region according to the method shown in Fig. 12.
  • the encoders 920, 940 and 950 After encoding the block of the selected region according to the decided coding mode, the encoders 920, 940 and 950 output coded streams to a stream MUX 960.
  • the layer 1 encoder dose not select an inter-layer coding mode. That is, the layer 1 encoder receives a video image which is down-sampled by the down-sampling unit 910 to have a resolution suitable to the first layer 1, selects one of a motion prediction/compensation mode and an intra mode and outputs the coded stream by encoding the block of the selected region according to the selected mode.
  • the encoder When the encoder performs an inter-layer coding, the encoder up- samples motion information, texture information and motion compensated residual information of a region overlapped with the block among regions of a lower layer and uses the up-sampled information. Meanwhile, the stream MUX 960 receives the coded streams from the encoders 920, 940 and 950 and multiplexes the received encoded streams.
  • Fig. 10 is a block diagram of a region encoder 230 employing a one-layer based coding method in accordance with a preferred embodiment of the present invention.
  • the region encoder 230 when the region encoder 230 receives a video image and region information from the region arranging unit 220, the region encoder 230 performs a coding operation on regions of each layer according to a method shown in Figs. 13 and 14.
  • the inputted region information includes information about the coding region including the overlapped region and information about arrangement of regions into corresponding layers.
  • Layer encoders 1020, 1040 and 1050 receive the region information and decide a coding mode for a block of a selected region according to a method shown in Fig. 14.
  • the layer encoders 1020, 1040 and 1050 encodes the block of the selected region according to the selected mode and outputs the coded stream to a stream MUX 1060.
  • the layer 1 encoder 1020 which is an encoder for a first layer 1 is operated identically to that in Fig. 9.
  • the layer 2 encoder 1040 and the layer 3 encoder 1050 perform an inter-layer coding
  • the layer 2 encoder 1040 and the layer 3 encoder 1050 up-sample motion information, texture information and motion compensated residual information of a region of a one-level lower layer and use the up-sampled information differently from those in Fig. 9 which use region information in a plurality of lower layers.
  • the stream MUX 1060 receives coded streams outputted from the layer encoders 1020, 1040 and 1050 and multiplexes the received coded streams.
  • Fig. 11 is a flowchart for describing an operation of a region encoder 230 employing a multi-layer based encoding method in accordance with a preferred embodiment of the present invention.
  • a region of a lowest layer is selected and coded at steps SlIlO and S1120 when information about arrangement of regions including video image and overlapped region into corresponding layers is inputted from the region encoder 230.
  • it determines whether other regions are existed in the same layer or not at step S1130. If the other regions are existed, the other regions are selected and encoded at step S1140.
  • regions in an upper layer are selected and encoded at steps S1150 and S1160.
  • the coding operation in the step S1120 follows a block-based video coding scheme and uses one of an inter- layer coding using a lower layer's region information and an intra-layer coding using a same layer's region information.
  • the inter-layer coding is a coding method introduced in MPEG-4 standard [ISO/IEC 14496-2 (1998)]. Such a inter-layer coding up-samples motion information, texture information and motion compensated residual information of a lower layer and uses the up-sampled information.
  • the intra-layer coding is a coding method introduced in MPEG-4 AVC [ISO/IEC 14496-10: Advanced Video Coding, 2003] and is classified into a motion prediction/compensation mode and an intra mode. That is, the region encoder 230 encodes a block of a selected region using one of the inter-layer coding mode, the motion prediction/compensation mode and the intra mode and outputs a coded stream.
  • Fig. 12 is a flowchart showing a method of deciding a coding mode for encoding a block of a selected region when a multi-layer based encoding method is employed in accordance with a preferred embodiment of the present invention.
  • the coding of the selected region in the step S1120 follows a block-based video coding scheme, and the block of the selected region is encoded according to one of the inter-layer coding mode, the motion prediction/compensation mode and the intra mode.
  • the inter-layer coding mode it must decide which region in a lower layer is used for each block at first.
  • step S1210 determines whether there is an overlapped region between a block of a selected region and regions of a lower layer using the region arrangement information transferred from the region arranging unit 220 at step S1210. If there is the overlapped region, a region in a layer having the highest layer index among regions having overlapped region is selected at step S1220. Then, it determines whether the selected region is defined as an overlapped region (OR) or not at step S1230. If the selected region is defined at the OR, an inter-layer coding is performed using the defined OR at step S1240. On the contrary, if the selected region is not defined, an inter- layer coding is performed using a region in a layer having the highest layer index among the regions having an overlapped region at step S1250.
  • OR overlapped region
  • an intra-layer coding of a motion prediction/compensation mode and an intra mode is performed on the block of the selected region without regard to whether the block of the selected region has an overlapped region with regions of the lower layer at step S1260.
  • it selects one that minimizes a bit rate among the three coding modes at step S1270.
  • the inter-layer coding mode is discarded when one of the three coding modes is selected.
  • Fig. 13 is a flowchart showing a method of deciding a coding mode for encoding a block of a selected region when a one-layer based encoding method is employed in accordance with a preferred embodiment of the present invention.
  • Regions of a lowest layer are selected and encoded at steps S1310 and S1320 when information about arranging regions including a video image and an overlapped region into corresponding layers is inputted from the region arranging unit 220.
  • After encoding it determines whether other regions are existed in a same layer or not at step S1330. If other regions are existed, the regions are selected and encoded at step S1340.
  • the process determines whether there is a region not overlapped in the regions of the current layer and the regions of a one- level lower layer at step S1350. If there is a region not overlapped, an intermediate region is composed for a region of the current layer that is not spatially matched with a region of the one-level lower layer at step S1360. The intermediate region is a reference region when the inter- layer coding is performed at an upper layer. Meanwhile, if there is no region not overlapped at step S1350, a region in an upper layer is selected and encoded at steps S1370 and S1380.
  • the intermediate region composed in the step S1360 is a region configured of motion information, texture information and motion compensated residual information of regions of a one-level lower layer suitably to the spatial resolution of the current layer. Therefore, the coding is not performed on the intermediate region.
  • the intermediate region is used when the inter-layer coding is performed at the upper layer. Since the coding of the present invention follows the block-based video coding scheme, block based motion information, texture information and motion compensated residua information are required to configure the intermediate region.
  • the motion information, texture information and motion compensated residual information which are up-sampled suitable to a spatial resolution of a current layer through interpolation may be used to compose a new intermediate region at the upper layer through same interpolation if the upper layer do not include a region matched with the intermediate region.
  • the motion information includes a motion vector and corresponding block' s coding mode such as the inter-layer coding mode, the motion prediction/compensation mode and the intra mode.
  • the encoding in the step S1320 follows a block-based video coding scheme identically to the encoding employing the multi-layer based coding method and uses one of the inter-layer coding using a lower layer's region information and the inter-layer coding using a same layer's region information.
  • the inter-layer coding is a coding method introduced in MPEG-4 standard [ISO/IEC 14496-2 (1998)], and the inter-layer coding up-samples motion information, texture information and motion compensated residual information of a lower layer and uses the up-sampled information.
  • the intra-layer coding is an encoding method introduced in MPEG-4 AVC [ISO/IEC 14496-10; Advanced Video Coding, 2003] and classified into a motion prediction /compensation mode and an intra mode.
  • the region encoder 230 encodes a block of a selected region based on one of the inter-layer coding mode, the motion prediction/compensation mode and the intra mode and outputs the coded stream.
  • Fig. 14 is a flowchart of a method of deciding a coding mode for encoding a block of a selected region when a one-layer based coding method is employed in accordance with a preferred embodiment of the present invention.
  • the coding performed on the selected region in the step S1320 follows a block-based video coding scheme and outputs a coded stream that is coded based on one of the inter-layer coding mode, the motion prediction/ compensation mode and the intra mode.
  • the inter-layer coding mode it must decide what region in a lower layer is used for each block.
  • the inter-layer coding is performed using the region overlapped with a region of the one-level lower layer at step S1420. If there is no region overlapped, it determines whether the block of the selected region is overlapped with an intermediate region of a one-level lower layer and the inter-layer coding is performed using the intermediate region at step S1450 if the block is overlapped with the intermediate region.
  • the block is encoded by an intra-layer coding of the motion prediction/ compensation mode and the intra mode without regard to whether the block of the selected region is overlapped with the regions of the lower layer.
  • one that minimizes a bit rate is selected using the results of coding according to the three modes at step S1270.
  • the inter-layer coding mode is discarded when one of the three modes is selected at the step S1430.
  • Fig. 15 is a view illustrating interpolation used for the inter-layer coding mode performed in the steps of S1240, S1250, S1420 and S1450 and the motion prediction/ compensation mode performed in the steps . of S1260 and S1460 in accordance with a preferred embodiment of the present invention.
  • a small gray rectangular region denotes a pixel of an original image.
  • a half- pixel value between the pixels is generated through interpolation.
  • MPEG-4 AVC Advanced video coding
  • SVC scalable video coding
  • Eq. 1 is a filter equation using the interpolation of the motion prediction/compensation mode and the inter-layer coding mode in the present invention.
  • the interpolation is basically a method of expanding a resolution of an image through a half pixel interpolation.
  • Eq. 1 is one of filter equations applicable to the interpolation of the present invention in a view of computation amount and performance.
  • I(x,y) denotes a pixel value of a (x,y) coordinate in an input image
  • 0(x,y) denotes a pixel value of a (x,y) coordinate in an output image after interpolation is performed.
  • the region of interest In a view of performing such an interpolation, the region of interest (ROI) must not be decoded using information about other regions. Therefore, pixel values in the outside of the ROI must not use when the interpolation is performed for a ROI boundary region in the region encoder 230. In order not to use the pixel values in the outside of the ROI, two methods are disclosed in the present invention.
  • a half pixel interpolation is performed after deciding pixel values in the outside of the boundary region through an extrapolation before performing the interpolation on the ROI boundary.
  • the extrapolation may be one of a zero order extrapolation or a method of substituting of a predetermined constant.
  • Eq. 2 is a filter equation employing the zero order extrapolation with regard to the ROI boundary.
  • a decoder must know that the ROI in the image signal transmitted through a channel when the half pixel interpolation is performed after deciding the pixel value in the outside of the boundary region through the extrapolation in the ROI boundary. Therefore, a coded stream outputted from the region encoder must includes a flag such as roi_flag (roi_enable, boundary_handling, or multiple_roi_flat ) in order to notice that the transmitted bit-stream includes a ROI that can be independently decoded.
  • the decoder determines whether a ROI is defined or not through the value of the flag after the decoder receives a bit stream having such a flag.
  • Figs. 16 to 18 are diagrams for describing adding of the flag denoting existence of ROI in an image into a coded stream in accordance with a preferred embodiment of the present invention.
  • Fig. 16 shows a SVC bit stream having a roi_flag (roi_enable, boundary_handling, or multiple_roi_ flat) 1610 when a ROI is generated in a slice group map type 2 ( slice_group_map_type 2).
  • roi_flag roi_enable, boundary_handling, or multiple_roi_ flat
  • roi_flag roi_enable, boundary_handling, or multiple_roi_flat
  • slice_group _map_type a new slice group map type for ROI. That is, the slice group map type is generated as an integer greater than 6 by expanding a current slice group map type ( slice_group_map_ type), and the roi_flag (roi_enable, boundary_handling, or multiple_roi_ flat) is disabled (1710) before determining the slice group map type. Then, the flag is enabled (1720) in the new slice group map type for the ROI.
  • Fig. 18 shows a generation of a ROI as a slice group map type 2 ( slice_group_map_type 2) (1820) as like as Fig. 16.
  • slice_group_map_type 2 slice_group_map_type 2
  • a roi_flag (roi_enable, boundary_handling, or multiple_roi_flat ) is enabled, geometric information such as the number of ROIs and coordinates is read through variables num_rois_minusl , roi_top_left and roi_bottom_right, and an interpolation, that is, an up-sampling, is performed with regard to a ROI boundary for a slice group boundary matched to the boundary of the ROI.
  • the location of the ROI may be expressed using a number assigned to a macro block where the ROI ends and a number assigned to a macro block where the ROI begins after assigning a rester scan number to macro blocks of entire image without using coordinates.
  • a second method for decoding a ROI not using other regions' information in the decoder is a method of adding a restriction when performing a decoding in the motion prediction/compensation mode and the inter-layer coding mode.
  • the interpolation is not performed if the interpolation is required to perform with reference to pixels in the outside of the ROI boundary in the motion prediction/compensation mode performed in the steps S1260 and S1460. Therefore, motion information referring to the ROI boundary is predicted in a unit of the integer pixel.
  • the inter-layer prediction is not performed on a block located at the ROI boundary in the inter-layer coding mode performed in the steps S1240, S1250, S1420 and S1450. Therefore, a block located at the ROI boundary is encoded using a coding mode that uses information of a current layer, not the lower layer. That is, the motion prediction/compensation mode or the intra mode is used.
  • Figs. 19 and 20 are block diagrams showing coding dependency between regions in the inter-layer coding when a multi-layer based coding method is employed and when a one- layer based coding method is employed.
  • a region where an arrow begins is required to encode a region where an arrow points.
  • the coding dependency with the multi-layer encoding method applied will be described with reference to Figs. 19 and 12.
  • Blocks in a region 1910 having no regions overlapped with regions in a lower layer are encoded based on a mode minimizing a bit rate between the motion prediction/compensation mode and an intra mode.
  • Blocks in a region 1920 having regions overlapped with regions in the lower layer are encoded based on a mode minimizing a bit rate among the inter-layer coding mode, the motion prediction/compensation mode and the intra mode.
  • a first region 1 has a region overlapped with a second region 2 and the overlapped region is defined as the ORl
  • blocks in the region having regions overlapped with regions in the lower layer are encoded using the ORl when the inter-layer coding mode is applied.
  • the coding dependency with the one-layer encoding method applied will be described with reference to Figs. 20 and 14.
  • blocks in a region 2010 having no regions overlapped with regions in the lower layer are coded using a mode minimizing a bit rate among the motion prediction/compensation mode and the intra mode as like as the case of using ⁇ the multi-layer encoding method.
  • blocks in a region 2020 having regions overlapped with region in the lower layer are encoded using a mode minimizing a bit rate among the inter-layer coding mode, the motion prediction/compensation mode and the intra mode.
  • the overlapped region is defined as the ORl in two-level lower layer that is the first layer 1.
  • an intermediate region 2030 is composed by up-sampling motion information, texture information and motion compensated residual information of the ORl in a second layer (layer 2). Then, blocks in a region 2020 having a region overlapped with a region of a lower layer in a third layer 3 are encoded using the information of the intermediate region 2030.
  • Fig. 21 shows a syntax structure of a coded stream including information about related regions in a lower layer in accordance with a preferred embodiment of the present invention.
  • information about regions in a lower layer used for coding a selected region is added into a coded stream.
  • a coded stream of a selected region includes a layer index 2120 of a layer having the selected region, a number of related regions 2130 used for coding the selected region, region information 2140 related to regions used for encoding, and a video signal 2150 which is a coded region video signal.
  • the region information 2140 related the regions used for encoding the selected region includes information of each of related regions 2160, 2170 and 2180 as many as the number of related regions used for encoding. Also, the region information 2140 includes a region index 2171 of a region used for the encoding, a layer index 2172 of a layer having the region used for the encoding, a horizontal axis location (H_org) 2173 of a region used for encoding the selected region, a vertical axis location (V_org) 2174 of a region used for encoding the selected region, a horizontal length (width) 2175 of a region used for the encoding, and a vertical length (height) 2176 of a region used for the encoding.
  • H_org horizontal axis location
  • V_org vertical axis location
  • a macro block number may be used to express a location of the region instead of using coordinates. That is, the location of the region may be expressed by a number assigned to a macro block where the region begins and another number assigned a macro block where the region ends after assigning numbers to the macro blocks of the entire image as a rester scan order. Meanwhile, a same region index is assigned to regions having a same location in an input video image although the spatial resolutions of the regions are different. Accordingly, the decoder may decode a predetermined region in various spatial resolutions by extracting coded streams of regions having same region index in each layer.
  • Fig. 22 is a block diagram of the decoder 120 of Fig. 1 for decoding a coded stream in accordance with a preferred embodiment of the present invention.
  • the decoder 120 receives decoding region information selected by a user from a user interface unit 130 and recovers an image signal by performing a decoding according to the present invention on a coded stream transmitted from the encoder 110.
  • the decoding region information includes information about a location of a region selected by a user to decode.
  • the decoder 120 includes a decoding region extractor 2210 and a region decoder 2220.
  • the decoding region extractor 2210 receives the decoding region information from the user interface unit 130 and extracts from the coded stream transmitted through a channel or composing information for decoding the decoding region. That is, the decoding region extractor
  • the decoding region extractor 2210 reads the number of regions 2130 in a lower layer for decoding the ROI having the selected region and the related region information 2140 from the coded stream and composes index of regions required for ROI decoding. Also, the decoding region extractor 2210 composes information about each of the related regions by extracting regions required for decoding the ROI from the coded stream.
  • an interactive decoding allowing the decoder 120 to select regions and decode the selected region if the encoder 110 encodes the entire image by dividing the entire image to a plurality of small regions not overlapped one another. Such an encoding may be performed on the entire image in a plurality of layers.
  • the interactive decoding may be embodied as follows.
  • a region is composed with very small rectangular regions through a slice group map type 2 in a current scalable video coding (SVC).
  • SVC current scalable video coding
  • the small rectangular regions corresponding to a region selected by a user are decoded.
  • the decoding region extractor 2210 composes a ROI using regions corresponding to the region selected by a user. Then, the decoding region extractor 2210 composes indexes of regions required for decoding by reading the number of related regions 2130 in the lower layer for decoding the ROI region and the related region information 2140 from the coded stream. Then, the decoding region extractor 2210 extracts necessary region information for decoding the ROI from the coded stream.
  • the region decoder 2220 receives the region information for decoding the ROI from the decoding region extractor 2210 and performs the decoding to recover the image signal.
  • operations of the region decoding 2220 will be described according to a multi-layer based coding method and a one-layer based coding method.
  • Fig. 23 is a flowchart showing operations of the region decoder employing a multi-layer based decoding method in accordance with a preferred embodiment of the present invention.
  • the decoding at the step S2320 follows a block-based vide coding scheme and uses a decoding method corresponding to one of coding methods of an inter-layer coding using region information of a lower layer and an intra-layer coding using region information of a same layer.
  • the intra-layer coding is divided into a motion prediction/ compensation mode and an intra mode. That is, the region decoder 2220 decodes each block of the selected region based on one of the inter-layer decoding, the motion prediction/compensation decoding and the intra mode decoding.
  • the motion prediction/compensation decoding and the intra mode decoding are introduced in MPEG-4 AVC [ISO/IEC 14496-10: Advanced Video Coding, 2003].
  • the inter-layer decoding is introduced in MPEG-4 standard [ISO/IEC 14496-2 (1998)], and the inter-layer decoding up- samples motion information, texture information and motion compensated residual information of the lower layer and uses the up-sampled information.
  • Fig. 24 is a flowchart showing a method of decoding a coded stream for blocks of a selected region when a multilayer based decoding method is applied in accordance with a preferred embodiment of the present invention.
  • the decoding of the selected region performed in the step S2320 follows a block-based decoding scheme, and a coded stream of each block is decoded based on one of the inter-layer decoding, the motion prediction/compensation decoding and the intra mode decoding. Referring to Fig. 24, it determines a coding mode of a decoding block at step S2410 and S2440.
  • the coding mode of the decoding block is the inter-layer coding mode, it selects a region in a layer having a highest layer index among the regions of lower layers overlapped with the current decoding block at step S2420 and an inter-layer decoding is performed on the selected region at step S2430. If the coding mode of the decoding block is the motion prediction/compensation mode, a motion prediction/compensation decoding is performed on the block. If the coding mode of the decoding block is not either of the motion prediction/compensation mode and the inter-layer coding mode, the intra mode decoding is performed on the block.
  • Fig. 25 is a flowchart showing operations of the region decoder employing a one-layer based decoding method in accordance with a preferred embodiment of the present invention .
  • regions of a lowest layer are selected among regions related for decoding at step S2510, and the selected regions are decoded at step S2550 if the selected region is in a highest layer. If the selected region is not in the highest layer, it determines whether any regions in a current layer are not overlapped with regions in a one- level lower layer at step S2330. If there is no such a region, the selected region is decoded at step S2550. If there is such a region, an intermediate region is composed for regions in the lower layer not matched with the regions in the current layer and the decoding is performed at steps S2540 and S2550.
  • step S2560 After decoding, it determines whether other regions are in the same layer or not at step S2560. If there is the region in the same layer, the region is selected and decoded at step S2570. After decoding the regions in the same layer, regions in an upper layer are selected and decoded at steps S2580, S2590 and S2520.
  • the intermediate region composed in the step S2540 is a region composed by performing interposition on motion information, texture information and motion compensated residual information of the regions in an one-level lower layer.
  • the intermediate region is used to perform the inter-layer decoding at the upper layer. Since the decoding according to the present invention follows a block based decoding, the intermediate region is configured of unit blocks.
  • the motion information includes motion vectors and information about a coding mode such as an inter-layer coding mode, a motion prediction/compensation mode and the intra mode .
  • each block of the selected region is decoded based on one of the inter-layer decoding, the motion prediction/compensation decoding and the intra mode decoding.
  • the motion prediction/compensation decoding and the intra mode decoding are introduced in MPEG-4 AVC [ISO/IEC 14496-10: Advanced Video Coding, 2003].
  • the inter-layer decoding is introduced in MPEG-4 standard [ISO/IEC 14496-2 (1998)].
  • Such inter-layer decoding up- samples the motion information, the texture information and the motion compensated residual information of the lower layer and uses the up-sampled information.
  • Fig. 26 is a flowchart showing a decoding of a coded stream of a block of a selected region when a one-layer based decoding is applied in accordance with a preferred embodiment of the present invention.
  • the decoding employing the one-layer based decoding is basically identical to the decoding with the multi-based decoding. However, the one-layer based decoding is different from the multi-layer based decoding when the inter-layer decoding is performed. In this case, only one-level lower layer is referenced. If there is no overlapped region in a one- level lower layer in the one-layer based decoding, an intermediate region is composed as shown in the step S2540 in order to reference only one-level lower layer. After composing the intermediate region, the decoding is performed.
  • a coding mode of a decoding block is determined at steps S2610 and S2640. If the coding mode of the decoding block is the inter-layer coding mode, the inter-layer decoding is performed at steps S2620 and S2630 using only a one-level lower layer having regions overlapped with the current decoding block or the intermediate region composed at the step S2540. If the coding mode of the decoding block is the motion prediction/ compensation mode, the motion prediction/compensation decoding is performed on the decoding block. If the coding mode of the decoding block is not either of the motion prediction/compensation mode and the inter-layer coding mode, the intra-mode decoding is performed on the block.
  • the method of performing a half pixel interpolation for the motion estimation decoding performed at the steps S2450 and S2650 and the method of up-sampling in the inter- layer decoding performed at steps S2430 and S2630 are identical to those in the steps S1260, S1460, S1240, S1250, S1420 and S1450 described with reference to Fig. 15.
  • the encoder 110 uses one of two processing methods for the ROI boundary region, and the decoder 120 also performs different operations according to the process method used in the encoder 110.
  • the decoder 120 determines whether an independently decodable ROI region is defined on not through the flag such as roi_flag (roi_enable, boundary_ handling or multiple_roi_flag) included in the coded stream transmitted through a channel. If the independently decodable ROI is defined, the region decoder 220 enables a function for decoding the bit stream coded with regard to the ROI boundary region, for example, a filter shown in Eq. 2.
  • the encoder 110 may add a restriction condition for encoding a block at a ROI boundary to use the prediction/compensation mode or the intra mode which uses only information about a current layer without performing an interpolation on the boundary region with reference to a pixel in the outside of the ROI boundary region.
  • the region decoder 2220 estimates the motion information in an integer unit with reference to ROI boundary in the motion prediction/compensation decoding and uses a decoding method that only uses information of the current layer such as the motion prediction/ compensation decoding or the intra mode instead of using the inter-layer decoding.
  • the above described method according to the present invention can be embodied as a program and stored on a computer readable recording medium.
  • the computer readable recording medium is any data storage device that can store data which can be thereafter read by the computer system.
  • the computer readable recording medium includes a read-only memory (ROM), a random-access memory (RAM), a CD-ROM, a floppy disk, a hard disk and an optical magnetic disk.

Abstract

La présente invention a trait à un procédé hiérarchique de codage/décodage vidéo pour une variabilité d'échelle complète et un appareil correspondant. L'appareil pour le codage d'une image vidéo comporte: un détecteur de zones de chevauchement (OR) pour la réception d'information de zones de codage concernant une pluralité de zones d'intérêt (ROI) dans l'image vidéo à coder et pour la détection de zones de chevauchement (OR) dans les zones d'intérêt; un dispositif d'agencement de zones pour l'agencement de l'image vidéo, des zones d'intérêt et des zones de chevauchement détectées en une pluralité de couches selon une résolution; et un codeur de zones pour le codage de l'image vidéo, des zones d'intérêt et des zones de chevauchement détectées selon une résolution d'une couche correspondante agencée au niveau du dispositif d'agencement de zones. L'information de zones de codage peut inclure une information d'emplacements des zones d'intérêt dans l'image vidéo et une résolution de codage des zones d'intérêt. L'appareil de codage/décodage selon l'invention fournit une variabilité d'échelle complète d'un domaine spatial en définissant une zone d'intérêt dans une image vidéo. En outre, l'appareil de codage/décodage selon l'invention fournit un taux de codage amélioré grâce au codage de l'image vidéo en tenant compte de la redondance spatiale parmi une pluralité de zones d'intérêt.
EP06732697A 2005-03-25 2006-03-24 Procede hierarchique de codage/decodage video pour une variabilite d'echelle complete et appareil correspondant Withdrawn EP1862010A4 (fr)

Applications Claiming Priority (7)

Application Number Priority Date Filing Date Title
KR20050025166 2005-03-25
KR20050030117 2005-04-11
KR20050065471 2005-07-19
KR20050097863 2005-10-18
KR20050117649 2005-12-05
KR20060002908 2006-01-10
PCT/KR2006/001097 WO2006112620A1 (fr) 2005-03-25 2006-03-24 Procede hierarchique de codage/decodage video pour une variabilite d'echelle complete et appareil correspondant

Publications (2)

Publication Number Publication Date
EP1862010A1 true EP1862010A1 (fr) 2007-12-05
EP1862010A4 EP1862010A4 (fr) 2011-08-17

Family

ID=37115308

Family Applications (1)

Application Number Title Priority Date Filing Date
EP06732697A Withdrawn EP1862010A4 (fr) 2005-03-25 2006-03-24 Procede hierarchique de codage/decodage video pour une variabilite d'echelle complete et appareil correspondant

Country Status (3)

Country Link
EP (1) EP1862010A4 (fr)
KR (1) KR100728222B1 (fr)
WO (1) WO2006112620A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2041976A2 (fr) * 2006-07-12 2009-04-01 Nokia Corporation Signalisation d'informations de scalabilité de région d'intérêt dans des fichiers multimédia
CN101494785B (zh) * 2008-12-19 2012-05-09 无锡云视界科技有限公司 一种h.264感兴趣区域编码的方法

Families Citing this family (28)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8340177B2 (en) 2004-07-12 2012-12-25 Microsoft Corporation Embedded base layer codec for 3D sub-band coding
US8243820B2 (en) 2004-10-06 2012-08-14 Microsoft Corporation Decoding variable coded resolution video with native range/resolution post-processing operation
US9071847B2 (en) 2004-10-06 2015-06-30 Microsoft Technology Licensing, Llc Variable coding resolution in video codec
US7956930B2 (en) 2006-01-06 2011-06-07 Microsoft Corporation Resampling and picture resizing operations for multi-resolution video coding and decoding
US8059721B2 (en) 2006-04-07 2011-11-15 Microsoft Corporation Estimating sample-domain distortion in the transform domain with rounding compensation
US8711925B2 (en) 2006-05-05 2014-04-29 Microsoft Corporation Flexible quantization
US8428144B2 (en) 2006-09-07 2013-04-23 Lg Electronics Inc. Method and apparatus for decoding/encoding of a video signal
AU2007318376B2 (en) 2006-11-09 2010-07-01 Lg Electronics Inc. Method and apparatus for decoding/encoding a video signal
JP2009538086A (ja) * 2006-11-17 2009-10-29 エルジー エレクトロニクス インコーポレイティド ビデオ信号のデコーディング／エンコーディング方法及び装置
WO2008069503A1 (fr) * 2006-12-04 2008-06-12 Electronics And Telecommunications Research Institute Appareil et procédé de traitement dynamique d'informations extensibles dans un processus de codage vidéo extensible
KR100805805B1 (ko) * 2006-12-04 2008-02-21 한국전자통신연구원 스케일러블 비디오 스트림의 동적 스케일러블 정보 처리장치 및 그 방법
US8107571B2 (en) 2007-03-20 2012-01-31 Microsoft Corporation Parameterized filters and signaling techniques
US8243797B2 (en) 2007-03-30 2012-08-14 Microsoft Corporation Regions of interest for quality adjustments
JP2009089356A (ja) * 2007-09-10 2009-04-23 Fujifilm Corp 画像処理装置、画像処理方法、およびプログラム
US8953673B2 (en) 2008-02-29 2015-02-10 Microsoft Corporation Scalable video coding and decoding with sample bit depth and chroma high-pass residual layers
US8711948B2 (en) 2008-03-21 2014-04-29 Microsoft Corporation Motion-compensated prediction of inter-layer residuals
US8897359B2 (en) 2008-06-03 2014-11-25 Microsoft Corporation Adaptive quantization for enhancement layer video coding
US9571856B2 (en) 2008-08-25 2017-02-14 Microsoft Technology Licensing, Llc Conversion operations in scalable video encoding and decoding
US8213503B2 (en) 2008-09-05 2012-07-03 Microsoft Corporation Skip modes for inter-layer residual video coding and decoding
WO2011105817A2 (fr) * 2010-02-23 2011-09-01 삼성전자 주식회사 Procédé et dispositif destinés à coder/décoder une image prédictive dans une structure de codage en couches
KR102111768B1 (ko) * 2012-01-19 2020-05-15 삼성전자주식회사 계층적 부호화 단위에 따라 스캔 순서를 변경하는 비디오 부호화 방법 및 장치, 비디오 복호화 방법 및 장치
WO2017041271A1 (fr) * 2015-09-10 2017-03-16 Mediatek Singapore Pte. Ltd. Modélisation de contexte efficace pour le codage d'un bloc de données
ES2963845T3 (es) * 2016-07-14 2024-04-02 Koninklijke Kpn Nv Codificación de vídeo
JP6936018B2 (ja) * 2017-02-21 2021-09-15 ソニーセミコンダクタソリューションズ株式会社 映像送信装置および映像受信装置
KR102119300B1 (ko) * 2017-09-15 2020-06-04 서울과학기술대학교 산학협력단 360도 영상 부호화 장치 및 방법, 이를 수행하기 위한 기록 매체
WO2020000333A1 (fr) * 2018-06-29 2020-01-02 SZ DJI Technology Co., Ltd. Procédé et appareil de traitement d'image
KR102628058B1 (ko) * 2018-08-01 2024-01-22 삼성에스디에스 주식회사 관심 영역 검출 방법 및 그 장치
CN113038127B (zh) * 2021-03-30 2022-03-22 福州大学 一种基于roi的实时视频多路并行传输码率控制方法

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6097842A (en) * 1996-09-09 2000-08-01 Sony Corporation Picture encoding and/or decoding apparatus and method for providing scalability of a video object whose position changes with time and a recording medium having the same recorded thereon

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CA2127151A1 (fr) * 1993-09-21 1995-03-22 Atul Puri Codage et decodage video a gradation spatiale
US6023301A (en) * 1995-07-14 2000-02-08 Sharp Kabushiki Kaisha Video coding device and video decoding device
JP3247836B2 (ja) * 1996-05-16 2002-01-21 シャープ株式会社 画像符号化装置及び画像復号装置
US6493387B1 (en) * 2000-04-10 2002-12-10 Samsung Electronics Co., Ltd. Moving picture coding/decoding method and apparatus having spatially scalable architecture and signal-to-noise ratio scalable architecture together
US6507618B1 (en) * 2000-04-25 2003-01-14 Hewlett-Packard Company Compressed video signal including independently coded regions
KR100446235B1 (ko) * 2001-05-07 2004-08-30 엘지전자 주식회사 다중 후보를 이용한 움직임 벡터 병합 탐색 방법
US6757434B2 (en) 2002-11-12 2004-06-29 Nokia Corporation Region-of-interest tracking method and device for wavelet-based video coding
KR100543700B1 (ko) * 2003-01-30 2006-01-20 삼성전자주식회사 영상의 중복 부호화 및 복호화 방법 및 장치

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6097842A (en) * 1996-09-09 2000-08-01 Sony Corporation Picture encoding and/or decoding apparatus and method for providing scalability of a video object whose position changes with time and a recording medium having the same recorded thereon

Non-Patent Citations (6)

* Cited by examiner, † Cited by third party
Title
REICHEL J ET AL: "Joint Scalable Video Model JSVM 0", JOINT VIDEO TEAM (JVT) OF ISO/IEC MPEG & ITU-T VCEG(ISO/IEC JTC1/SC29/WG11 AND ITU-T SG16 Q6), XX, XX, 17 January 2005 (2005-01-17), pages 1-73, XP002345849, *
SCHELKENS P ET AL: "Wavelet-based compression of medical images: Protocols to improve resolution and quality scalability and region-of-interest coding", FUTURE GENERATIONS COMPUTER SYSTEMS, ELSEVIER SCIENCE PUBLISHERS. AMSTERDAM, NL, vol. 15, no. 2, 11 March 1999 (1999-03-11) , pages 171-184, XP004222986, ISSN: 0167-739X, DOI: DOI:10.1016/S0167-739X(98)00061-2 *
See also references of WO2006112620A1 *
THANG T C ET AL: "CE4: On signaling ROI boundary handling", ITU STUDY GROUP 16 - VIDEO CODING EXPERTS GROUP -ISO/IEC MPEG & ITU-T VCEG(ISO/IEC JTC1/SC29/WG11 AND ITU-T SG16 Q6), XX, XX, no. JVT-R059, 15 January 2006 (2006-01-15) , XP030006326, *
THANG T C ET AL: "Spatial Scalability Multiple ROIs for Surv", ITU STUDY GROUP 16 - VIDEO CODING EXPERTS GROUP -ISO/IEC MPEG & ITU-T VCEG(ISO/IEC JTC1/SC29/WG11 AND ITU-T SG16 Q6), XX, XX, no. JVT-O037, 14 April 2005 (2005-04-14), XP030005983, *
YU T ET AL: "A REGION-OF-INTEREST BASED TRANSMISSION PROTOCOL FOR WAVELET-COMPRESSED MEDICAL IMAGES", PROCEEDINGS OF SPIE, SPIE, USA, vol. 3078, 22 April 1997 (1997-04-22), pages 56-64, XP008002445, ISSN: 0277-786X, DOI: DOI:10.1117/12.271755 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2041976A2 (fr) * 2006-07-12 2009-04-01 Nokia Corporation Signalisation d'informations de scalabilité de région d'intérêt dans des fichiers multimédia
EP2041976A4 (fr) * 2006-07-12 2012-06-20 Nokia Corp Signalisation d'informations de scalabilité de région d'intérêt dans des fichiers multimédia
US8442109B2 (en) 2006-07-12 2013-05-14 Nokia Corporation Signaling of region-of-interest scalability information in media files
CN101494785B (zh) * 2008-12-19 2012-05-09 无锡云视界科技有限公司 一种h.264感兴趣区域编码的方法

Also Published As

Publication number Publication date
EP1862010A4 (fr) 2011-08-17
KR100728222B1 (ko) 2007-06-13
KR20060103226A (ko) 2006-09-28
WO2006112620A1 (fr) 2006-10-26

Similar Documents

Publication Publication Date Title
EP1862010A1 (fr) Procede hierarchique de codage/decodage video pour une variabilite d'echelle complete et appareil correspondant
KR20210061991A (ko) 스케일러블 비디오 코딩 및 디코딩 방법과 이를 이용한 장치
US7586985B2 (en) Method and apparatus for encoding/decoding video signal using reference pictures
KR100657268B1 (ko) 컬러 영상의 신축적 부호화, 복호화 방법 및 장치
US7903735B2 (en) Method of effectively predicting multi-layer based video frame, and video coding method and apparatus using the same
CN112806003A (zh) 使用mpm列表的基于帧内预测的图像编码方法及其装置
US7929606B2 (en) Method and apparatus for encoding/decoding video signal using block prediction information
US8660180B2 (en) Method and apparatus for scalably encoding and decoding video signal
KR102467326B1 (ko) 영상 코딩 시스템에서 서브 블록 단위의 움직임 예측에 기반한 영상 디코딩 방법 및 장치
WO2007058470A1 (fr) Procede de codage scalable de video permettant de faire varier en temps reel la scalabilite du flux binaire, et codec l'utilisant
EP4120680B1 (fr) Procédé de décodage d'image basé sur une prédiction cclm dans un système de codage d'image, et dispositif correspondant
WO2006078142A1 (fr) Procede et systeme de codage-decodage d'un signal video au moyen de donnees de prediction en blocs
KR20070074452A (ko) 영상신호의 엔코딩/디코딩시의 레이어간 예측 방법
KR100883603B1 (ko) 기준 픽쳐를 이용하여 비디오 신호를 디코딩하는 방법 및장치
WO2014009603A1 (fr) Procédé et appareil de codage vidéo
CN108718411B (zh) 图像解码方法和使用其的装置
KR102521647B1 (ko) 영상 코딩 시스템에서 인트라 예측 관련 정보를 사용하는 영상 디코딩 방법 및 그 장치
JP2007228560A (ja) 動画像符号化方法および動画像符号化装置
Correia et al. Enhancement method for multiple description decoding of depth maps subject to random loss
Ricordel et al. Livrable D3. 4 of the PERSEE project: 2D coding tools final report
PERSEE 2D coding tools final report

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20070918

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC NL PL PT RO SE SI SK TR

DAX Request for extension of the european patent (deleted)
RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: ELECTRONICS AND TELECOMMUNICATIONS RESEARCH INSTIT

Owner name: KOREA ADVANCED INSTITUTE OF SCIENCE AND TECHNOLOGY

A4 Supplementary search report drawn up and despatched

Effective date: 20110718

RIC1 Information provided on ipc code assigned before grant

Ipc: H04N 7/26 20060101ALI20110712BHEP

Ipc: H04N 7/24 20110101AFI20110712BHEP

17Q First examination report despatched

Effective date: 20140121

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN

18D Application deemed to be withdrawn

Effective date: 20140603