US20080170793A1 - Image encoding device and image encoding method - Google Patents

Image encoding device and image encoding method Download PDF

Info

Publication number
US20080170793A1
US20080170793A1 US11926593 US92659307A US2008170793A1 US 20080170793 A1 US20080170793 A1 US 20080170793A1 US 11926593 US11926593 US 11926593 US 92659307 A US92659307 A US 92659307A US 2008170793 A1 US2008170793 A1 US 2008170793A1
Authority
US
Grant status
Application
Patent type
Prior art keywords
encoding
unit
color
prediction
process
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US11926593
Inventor
Yoshihisa Yamada
Shunichi Sekiguchi
Yoshimi Moriya
Kazuo Sugimoto
Yuichi Idehara
Kohtaro Asai
Tokumichi Murakami
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Mitsubishi Electric Corp
Original Assignee
Mitsubishi Electric Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103Selection of coding mode or of prediction mode
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/12Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/186Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H04N19/436Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation using parallelised computational arrangements
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/46Embedding additional information in the video signal during the compression process
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television, VOD [Video On Demand]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/236Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a Uniform Resource Locator [URL] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream

Abstract

There are provided an encoding device, a decoding device, an encoding method, and a decoding method that enhance optimality in a case of encoding moving image signals having no sample ratio distinction between color components such as a 4:4:4 format. In a case of conducting compression encoding by inputting digital moving image signals of the 4:4:4 format, there are prepared a first encoding process of encoding three color component signals of the input moving image signals in a common encoding mode, and a second encoding process of encoding the three color component signals of the input moving image signals in respective independent encoding modes. The encoding process is executed by selecting any one of the first encoding process and the second encoding process, and the compression data contains an identification signal for specifying which process is selected.

Description

    BACKGROUND OF THE INVENTION
  • [0001]
    1. Field of the Invention
  • [0002]
    The present invention relates to a digital image signal encoding device, a digital image signal decoding device, a digital image signal encoding method, and a digital image signal decoding method used for an image compression encoding technology or a compressed image data transmission technology.
  • [0003]
    2. Description of the Related Art
  • [0004]
    An international standard video encoding system such as MPEG or ITU-TH.26x (e.g., “Information Technology Coding of Audio-Visual Objects Part 10: Advanced Video Coding,” ISO/IEC 14496-10, 2003 (hereinafter, referred to as Non-Patent Document 1)) has conventionally been premised on use of a standardized input signal format called a 4:2:0 format. The 4:2:0 format is a format where a color moving image signal of RGB or the like is converted into a luminance component (Y) and two chrominance components (Cb, Cr), and the number of chrominance component samples is reduced to half of luminance components both in horizontal and vertical directions. The chrominance component is inferior to the luminance component in visibility. Accordingly, the conventional international standard video encoding system has been based on the premise that the amount of original information to be encoded is reduced by downsampling chrominance components before encoding is executed as mentioned above. On the other hand, recent increases in resolution and gradation of a video display have been accompanied by studies on a system for performing encoding by maintaining the number of samples equal to that of luminance components without downsampling chrominance components. A format where the numbers of luminance and chrominance component samples are completely equal is called a 4:4:4 format. The conventional 4:2:0 format has been limited to Y, Cb, and Cr color space definitions because of the premise of downsampling of chrominance components. In the case of the 4:4:4 format, however, because there is no sample ratio distinction between color components, R, G, and B can be directly used in addition to Y, Cb, and Cr, and a plurality of color space definitions can be used. An example of a video encoding system targeting the 4:4:4 format is, Woo-Shik Kim, Dae-Sung Cho, and Hyun Mun Kim, “INTER-PLANE PREDICTION FOR RGB VIDEO CODING,” ICIP 2004, October 2004 (hereinafter, referred to as Non-Patent Document 2). Non-Patent Document 2 proposes an approach of reducing a size of data which is to be encoded by performing prediction between different color components by employing a correlation remaining between the color components. However, a degree of the correlation between the color components varies depending on types of video contents and color spaces, and the prediction might adversely affects in terms of encoding efficiency. Further, the signal processing is required over a plurality of color components so such a problem arises in that parallel processing efficiency deteriorates in, for example, executing a real time processing of video signals having an extremely high resolution as a digital cinema picture (having 4000×2000 pixels).
  • SUMMARY OF THE INVENTION
  • [0005]
    In a high 4:2:0 profile encoding the 4:2:0 format of MPEG-4 advanced video coding (hereinafter, referred to as AVC) of the Non-Patent Document 1, in a macroblock area composed of luminance components of 16×16 pixels, corresponding chrominance components are 8×8 pixel blocks for both Cb and Cr. In motion compensation prediction of the high 4:2:0 profile, block size information which becomes a unit of motion compensation prediction only for the luminance components, reference image information used for prediction, and motion vector information of each block are multiplexed, and motion compensation prediction is carried out for chrominance components by the same information as that of the luminance components. The 4:2:0 format has characteristics in color space definition that almost all pieces of structure information (texture information) of an image is integrated into a luminance component, distortion visibility is lower for a chrominance component than for the luminance component, and a contribution to video reproducibility is small, and prediction and encoding of the high 4:2:0 profile are based on such characteristics of the 4:2:0 format. However, in the case of the 4:4:4 format, three color components equally hold texture information. The system for performing motion compensation prediction in inter prediction mode depending only on one component and based on reference image information and motion vector information is not necessarily an optimal method in the 4:4:4 format where the color components make equal contributions in representing a structure of an image signal.
  • [0006]
    As described by way of the above related art, it is an object of the present invention to provide an encoding device, a decoding device, an encoding method, and a decoding method that enhance optimality in a case of encoding moving image signals having no sample ratio distinction between color components such as a 4:4:4 format.
  • [0007]
    According to the present invention, in a case of conducting compression encoding by inputting digital moving image signals of the 4:4:4 format, there are prepared a first encoding process of encoding three color component signals of the input moving image signals in a common encoding mode and a second encoding process of encoding the three color component signals of the input moving image signals in respective independent encoding modes. The encoding process is executed by selecting any one of the first encoding process and the second encoding process, and the compression data contains an identification signal for specifying which process is selected.
  • [0008]
    Further, in a case of decoding the compression data of the digital moving image signals of the 4:4:4 format there are prepared a first decoding process of decoding the three color component signals in the common encoding mode, and a second decoding process of decoding the three color component signals in the respective independent encoding modes. The decoding process is executed by decoding the identification signal specifying whether the three color component signals are encoded from the compression data in the common encoding mode or in the respective independent encoding modes, and employing any one of the first decoding process and the second decoding process in accordance with the identification signal.
  • [0009]
    According to the present invention, in a case of performing encoding that uses multiple color spaces without being limited to the fixed color spaces of Y, Cb, Cr, etc., it is possible to flexibly select inter prediction mode information to be used for the respective color components, and to conduct an optimal encoding process even in a case where there are diverse definitions of the color spaces.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • [0010]
    In the accompanying drawings:
  • [0011]
    FIG. 1 is an explanatory diagram showing a hierarchical structure of video signals consisting of a sequence, a group of pictures (GOP), a picture, a slice, and a macroblock;
  • [0012]
    FIG. 2 is an explanatory diagram showing a structure of macroblock encoded data;
  • [0013]
    FIG. 3 is an explanatory diagram showing three color components structuring the macroblock in a case of a “common encoding process”;
  • [0014]
    FIG. 4 is an explanatory diagram showing the three color components structuring the macroblock in a case of an “independent encoding process”;
  • [0015]
    FIG. 5 is an explanatory diagram showing a time-directional motion prediction reference relation between the pictures in “common encoding” and “independent encoding”;
  • [0016]
    FIG. 6 is an explanatory diagram showing an example structure of a bit stream according to a first embodiment of the present invention;
  • [0017]
    FIG. 7 is an explanatory diagram showing an example structure of the bit stream of slice data according to the first embodiment of the present invention;
  • [0018]
    FIG. 8 is a schematic structural diagram of an encoding device according to the first embodiment of the present invention;
  • [0019]
    FIG. 9 is an internal configuration diagram of a first picture encoding unit 5;
  • [0020]
    FIGS. 10( a) to 10(h) are an explanatory diagrams showing seven types of block sizes with each of which motion compensation prediction is conducted;
  • [0021]
    FIG. 11 is an internal structural diagram of a second picture encoding unit 7;
  • [0022]
    FIG. 12 is a schematic structural diagram of a decoding device according to the first embodiment of the present invention;
  • [0023]
    FIG. 13 is an internal structural diagram of a first picture decoding unit 302;
  • [0024]
    FIG. 14 is an internal structural diagram of a second picture decoding unit 304; and
  • [0025]
    FIG. 15 is an explanatory diagram showing an example of how picture data for the three color components is allocated.
  • DESCRIPTION OF THE EMBODIMENTS First Embodiment
  • [0026]
    A first embodiment will exemplify an encoding device that efficiently encodes video signals in a 4:4:4 format which is not restricted by a specified color space, and a decoding device that receives encoded bit streams generated by the encoding device decodes the image signals. The encoding device in the first embodiment receives the video signals including three color components such as RGB, XYZ and YCbCr, and outputs the bit streams by conducting compression encoding. As illustrated in FIG. 1, the input video signals are expressed as time-series data of screen information (hereinafter, referred to as picture) defined by a frame-by-frame basis or a field-by-field basis through time-sampling. A data unit, on which the pictures are arranged in time series, is referred to as sequence. The sequence may be divided into some groups of pictures (GOPs). The GOPs are utilized for applications such as assuring execution of decoding from an arbitrary head GOP without depending on other GOPs and assuring a random access to the bit stream. The picture is further divided into square blocks called macroblocks, and video compression is performed on a macroblock basis in a way that applies a prediction/transform/quantization process.
  • [0027]
    Further, a unit on which a plurality of macroblocks are aggregated is called a slice. The slice is a data unit on which the encoding and decoding can be independently executed in each slice. For example, when processing the video signals having a resolution equal to or higher than a high definition television (HDTV) in real time, the single picture is sliced into a plurality of slices, a period of computing time is reduced by encoding/decoding the respective slices in parallel, and the bit streams are transmitted via a line having a high error ratio. In this case, the slice is utilized for such an application that if a decoded image is disturbed because of a certain slice being destructed as affected by an error, a proper decoding process is restored from the next slice. Generally, the prediction using signal dependency on the neighboring slice cannot be applied at a boundary between the slices, and hence encoding performance declines as the number of slices increases. Such a characteristic, however, exists that flexibility of the parallel processing and the error resilience increase.
  • [0028]
    The video compression is carried out by applying the prediction/transform/quantization process on the macroblock basis, and therefore the encoded data of the macroblock multiplexed on the bit stream consists of roughly two types of information. One type of information is a category called side information different from the video signal itself such as parameters for a prediction mode, motion prediction information, and quantization, and this category of information is generically called a macroblock header. Another type of information is information of the video signal itself. According to the first embodiment of the present invention, the video signal to be encoded is compressed data of a prediction error signal acquired as a result of conducting the prediction/transform/quantization based on the macroblock header information. The video signal is expressed in a transform-coefficient-quantized format and therefore will be called coefficient-encoded data, hereinafter. FIG. 2 illustrates how the pieces of macroblock encoded data are arranged in the first embodiment. In FIG. 2, the macroblock header contains all items of side information anterior to the coefficient-encoded data, such as the encoding/prediction mode information of a macroblock type/submacroblock type/intra prediction mode etc., the motion prediction information of a reference image identifying number/motion vector etc., a quantization parameter with respect to the transform coefficient, and a valid transform coefficient existence/absence determining flag on an 8×8 block basis.
  • [0029]
    The first embodiment of the present invention will exemplify: an encoding device that encodes the three color components, selectively based on the common macroblock header or based on the independent macroblock header according to every color component; and a decoding device that executes a video decoding process by receiving the bit streams obtained as an output of the encoding device, and selecting, based on the identification information extracted from the bit stream by decoding the bit stream, whether the three color components have been encoded based on the common macroblock header or based on the independent macroblock header for each color component. The first embodiment of the present invention will particularly exemplify, with reference to specific drawings, configurations and operations of the encoding device and the decoding device that perform encoding and decoding by switching over on a sequence basis an operation of encoding and decoding the three color component signals based on the common macroblock header, and an operation of encoding and decoding those color signals based on the independent macroblock header according to every color component. The encoding device and the decoding device are thus constructed, thereby enabling the color components to be encoded by selecting a case of encoding the color components in accordance with the color space in which the input video signals are defined and with a statistical characteristic of the video signals by use of the common prediction parameters, and a case of encoding the color components by the independent prediction parameters. It is therefore possible to optimally encode the video signals in the 4:4:4 format.
  • [0030]
    Hereinafter, the process of encoding the three color component signals of one frame or one field by the common macroblock header is referred to as “common encoding process,” and a process of encoding the three color component signals of one frame or one field by the individual independent macroblock header is referred to as “independent encoding process.” Similarly, the process of decoding the image data from the bit streams to which the three color component signals of one frame or one field are encoded by the common macroblock header is referred to as “common decoding process,” and the process of decoding the image data from the bit streams to which the three color component signals of one frame or one field are encoded by the individual independent macroblock header is referred to as “independent decoding process.” In the common encoding process, the three color components of one frame or one field are collectively defined as one picture and are divided into the macroblocks each consisting of an aggregation of the three color components (FIG. 3). In FIG. 3 and the following description, the three color components are called C0, C1, and C2 components. On the other hand, in the independent encoding process, the input video signals of one frame or one field are separated into the three color components each defined as a picture, and each picture is divided into the macroblocks consisting of the single color component (FIG. 4). In other words, the macroblock as a target of the common encoding process includes samples (pixels) of the three color components of C0, C1, and C2, but, the macroblock as a target of the independent encoding process includes only the samples (pixels) of any one component of the C0, C1, and C2 components.
  • [0031]
    FIG. 5 illustrates a motion prediction reference relation in a time direction between the pictures in the encoding device/decoding device according to the first embodiment of the present invention. In this example, the data unit indicated by a bold vertical line represents the picture, and a relation between the picture and an access unit is indicated by a dotted line. In the case of the common encoding/decoding process, as described above, one picture is the data representing the video signals consisting of the aggregations of the three color components. In the case of the independent encoding/decoding process, one picture is the video signals consisting of any one color component. The access unit is a minimum data unit for providing time stamp aiming at, for example, synchronizing the video signals with audio/sound information etc. In the case of the common encoding/decoding process, one access unit includes the data for one picture. On the other hand, in the case of the independent encoding/decoding process, one access unit includes three pictures. This is because, in the case of the independent encoding/decoding process, regeneration video signals for one frame are not obtained till the pictures of all the three color components at the same display time become available. Note that numbers attached to the top of the respective pictures indicate a time-directional encoding/decoding process sequence (corresponding to frame_num of AVC) of the pictures. In FIG. 5, an arrow between the pictures indicates a reference direction of the motion prediction. To be specific, in the case of the independent encoding/decoding process, the motion prediction reference between the pictures included in the same access unit and the motion prediction reference between the different color components are not performed but the encoding/decoding process are performed while performing the prediction reference by limiting the pictures of the color components of C0, C1, and C2 to the signals of the same single color component. With this structure, in the case of the independent encoding/decoding process according to the first embodiment of the present invention, each color component can be encoded and decoded without depending on the encoding/decoding process of other color components at all, thereby facilitating the parallel processing.
  • [0032]
    The identification information indicating whether the encoding based on the common encoding process is carried out or the encoding based on the independent encoding process is carried out will hereinafter be referred to as common encoding/independent encoding identification signal 1.
  • [0033]
    FIG. 6 shows one example of a structure of the bit stream generated by the encoding device according to the first embodiment of the present invention and serving as a target of a receiving/decoding process by the decoding device according to the first embodiment. FIG. 6 illustrates the structure of the bit stream from the sequence to a slice level. First, a higher-order header of the sequence level (sequence parameter set in the case of AVC) is multiplexed on the common encoding/independent encoding identification signal 1. AUD represents an Access Unit Delimiter NAL unit which is a unique NAL unit for identifying an access unit delimiter in the AVC. The AUD is information specifying the head of the access unit and can be, without depending on the data format of the AUD of the AVC, applied to an arbitrary data format if conformable to its purpose. For example, the AUD corresponds to a picture start code according to the MPEG-2 standards and to a VOP start code according to the MPEG-4 standards.
  • [0034]
    When the common encoding/independent encoding identification signal 1 indicates “common encoding process,” the access unit contains the encoded data for one picture. The picture at this time is the data representing the video signals for one frame or one field, which consist of the aggregations of the three color components as described above. The actual video encoded data is multiplexed on the bit stream on the slice basis of FIG. 1. On the other hand, when the common encoding/independent encoding identification signal 1 indicates the “independent encoding process,” one picture is the video signal for any one color component in one frame or one field, and one access unit includes the three pictures. In this case, the slice is defined with respect to the picture of each color component.
  • [0035]
    FIG. 7 illustrates structures of the bit streams of the slice data in the case of the common encoding process and in the case of the independent encoding process, respectively. In the bit stream encoded by the independent encoding process, in order to attain an effect that will be described later, a color component identification flag 2 (color_channel_idc) is provided to a header field at the head of the slice data such that the slice is distinguishable as to which color component picture the slice data received by the decoding device belongs to. The slices having the same values are grouped by identification flag 2. In other words, none of encoding/decoding dependency (e.g., motion prediction reference, context modeling/generation probability learning of CABAC, etc.) is provided between the slices having different values of the color component identification flag 2. This rule ensures the independence of the individual picture within the access unit in the case of the independent encoding process. Further, frame num (encoding/decoding processing sequence of the picture to which the slice belongs) multiplexed on each slice header takes the same value in the full-color component pictures within one access unit.
  • [0036]
    FIG. 8 illustrates an outline of the configuration of the encoding device according to the first embodiment of the present invention. In FIG. 8, a first picture encoding unit 5 executes the common encoding process, and second picture encoding units 7 a to 7 c (provided for the three color components) execute the independent encoding process.
  • [0037]
    Input video signals 3 are supplied to any one of the first picture encoding unit 5, a color component separating unit 6, and the second picture encoding units 7 a to 7 c via a switch (SW) 4. The switch 4 is activated by the common encoding/independent encoding identification signal 1 and supplies the input video signals 3 to a designated path.
  • [0038]
    The common encoding/independent encoding identification signal 1 is a signal that, when the input video signal takes the 4:4:4 format, is multiplexed on the sequence parameter set and selects the common encoding process and the independent encoding process on the sequence basis. The common encoding/independent encoding identification signal 1 is multiplexed on the sequence parameter set in a bit stream 11 as information that designates which process has been employed to generate the bit stream 11. This structure enables the decoding device, which receives the bit stream 11, to execute the common decoding process when the bit stream 11 is generated by use of the common encoding process by decoding the common encoding/independent encoding identification signal 1 in the sequence parameter set and checking the value thereof, and to execute the independent decoding process when the bit stream 11 is generated by use of the independent encoding process.
  • [0039]
    If the common encoding/independent encoding identification signal 1 indicates “common encoding process,” in the first picture encoding unit 5, the input video signals 3 are divided into the macroblocks in the form of aggregating the samples of the three color components as illustrated in FIG. 3, the encoding process on the macroblock basis is executed, and the encoded data is output as a bit stream 8. The encoding process by the first picture encoding unit 5 will be described later.
  • [0040]
    If the common encoding/independent encoding identification signal 1 designates “independent encoding process,” the input video signals 3 are separated into the signals of the color components of C0, C1, and C2 by the color component separating unit 6 and are supplied to the second picture encoding units 7 a to 7 c corresponding to the respective color components. In the second picture encoding units 7 a to 7 c, the signals separated according to every color component are divided into the macroblocks taking the format shown in FIG. 4, and the encoding process on the macroblock basis is executed, whereby the signals are output as bit streams 9 a to 9 c. The encoding process by the second picture encoding units 7 a to 7 c will be described later.
  • [0041]
    In a multiplexing unit 10, the common encoding/independent encoding identification signal 1 is added to the sequence parameter set and is multiplexed on the bit stream 11. In the multiplexing unit 10, any one of the bit stream 8 and the bit streams 9 a to 9 c, corresponding to the value of the common encoding/independent encoding identification signal 1, is selected, and the selected bit stream is multiplexed on the bit stream 11.
  • [0042]
    Further, though the details will be described later, quantization weighting coefficient information (12 a to 12 c) for quantization parameters used in a picture encoding process, especially a quantization/inverse quantization process, is provided for every three color components and input to the respective picture encoding units in which the quantization process matching a characteristic of every color component is executed. The quantization weighting coefficients 12 a to 12 c are also sent to the multiplexing unit 10 to be multiplexed on the sequence parameter set in order for the decoding device to use the same values of the coefficients 12 a to 12 c as those employed in the encoding process.
  • [0043]
    Further, an intra only encoding instruction signal 13 is input to the picture encoding units 5 and 7 a to 7 c, thereby controlling the encoding process. The intra only encoding instruction signal 13 is a signal for instructing whether the picture encoding unit executes the time-directional prediction process based on a motion compensation prediction. If the intra only encoding instruction signal 13 indicates “intra only encoding,” the encoding closed within the picture is carried out with respect to all of the pictures of the input video signals 3 without performing the time-directional prediction based on the motion compensation prediction. Further, at this time, an intra loop deblocking filter is simultaneously disabled (details will be described later) within the picture encoding unit. If the intra only encoding instruction signal 13 indicates “non intra only encoding,” an inter-encoding process using an intra picture/inter-picture correlation is executed with respect to the pictures of the input video signals 3 by also employing the time-directional prediction based on the motion compensation prediction. The multiplexing unit 10 adds the intra only encoding instruction signal 13 to the sequence parameter set and thus multiplexes the sequence parameter set with the bit stream 11. With this multiplexing, the decoding device receiving the bit stream 11 decodes the intra only encoding instruction signal 13 contained in the sequence parameter set and checks its value, whereby it can be recognized whether the bit stream 11 is intra only-encoded or not. Therefore, if the bit stream 11 is intra only-encoded, the intra loop deblocking filter process can be made unnecessary, and a computing amount of the decoding device can be reduced.
  • [0044]
    The intra encoding process of the AVC requires approximately a two-fold to ten-fold encoding amount as compared with the inter-encoding process, and hence the data encoded by “intra only encoding” has a data size that is considerably larger than the data encoded by “non intra only encoding.”
  • [0045]
    The conventional decoding device provides an upper limit to the data size enabling the decoding process to be executed, and reduces an operation speed and a required memory size in the device to the greatest possible degree, thereby scheming to stabilize the operation. Therefore, in the case of “intra only encoding,” there is a possibility that the data exceeding the set upper limit might be input, resulting in a problem of being disabled to determine whether the stable operation can be performed or not.
  • [0046]
    This being the case, a flag for showing whether the encoded data is less than or exceeds a predetermined amount, is provided in the sequence parameter set. A determining process is done based on the flag, and, if the encoded data is smaller than the predetermined amount, even the conventional decoding device is assumed to be capable of processing and therefore the decoding process is executed. If the encoded data exceeds the predetermined amount, it is assumed that the conventional decoding device may not execute the stable process, and consequently a process such as alarming can be taken.
  • [0047]
    Moreover, image size information 14 of the input video signal 3 is input to the picture encoding units 5 and 7 a to 7 c, and is configured to control the encoding process. The image size information 14 is information representing the number of intra picture macroblocks of the input video signal 3, which is controlled to set an upper limit value of the number of the macroblocks contained in the slice if a value of the information 14 is larger than a predetermined threshold value, and to prevent the slice from containing a larger number of macroblocks than the upper limit value. The image size information 14 is added to the sequence parameter set, and the sequence parameter set is multiplexed on the bit stream 11. With this multiplexing, if the picture size of the input video signal 3 is large (i.e., a spatial resolution is high), both of the encoding device and the decoding device can specify unit which can be a parallel processed and can assign tasks smoothly.
  • [0048]
    Hereinafter, the operations of the first and second picture encoding units will be explained in detail.
  • Outline of Operation of First Picture Encoding Unit
  • [0049]
    FIG. 9 illustrates an internal configuration of the first picture encoding unit 5. In FIG. 9, it is assumed that the input video signal 3 takes the 4:4:4 format and is encoded on the macroblock basis in the format of FIG. 3, where the macroblock consists of the aggregation of the three color components. The internal processing differs depending on the value of the intra only encoding instruction signal 13.
  • [0050]
    (1) Case of Intra Only Encoding Instruction Signal 13 Indicating “Non Intra Only Encoding”
  • [0051]
    The prediction unit 15 selects a reference image from among motion compensation prediction reference image data stored in the memory 16, and executes a motion compensation prediction process on the macroblock basis. The memory 16 is stored with a plurality of pieces of reference image data each consisting of the three color components over a plurality of points of time of the most recent past or the past and the future. The prediction unit 15 performs the motion prediction by selecting an optimal reference image on the macroblock basis from among those reference images. As to the allocation of the reference image data in the memory 16, the data may be stored separately plane-sequentially according to every color component, and the samples of the respective color components may also be stored dot-sequentially. Seven types of block sizes subjected to the motion compensation prediction are provided. First, as illustrated in FIGS. 10( a) to 10(d), any size of 16×16, 16×8, 8×16, and 8×8 is selected on the macroblock basis. Further, when the 8×8 size is selected, as illustrated in FIGS. 10( e) to 10(h), any size of 8×8, 8×4, 4×8, and 4×4 is selected for every 8×8 block. In the common encoding process executed by the first picture encoding unit 5, the motion compensation prediction block size common to the three color components is selected and applied.
  • [0052]
    The prediction unit 15 executes the motion compensation prediction process on the macroblock basis with respect to all or part of the block sizes/sub-block sizes of FIG. 10, a motion vector in a predetermined search range, and one or more pieces of usable reference images, thereby outputting prediction overhead information 17 containing the motion vector and a reference image index used for the prediction and a predicted image 33. A subtracter 18 obtains a prediction difference signal 19 for every block that serves as a motion compensation prediction unit from the predicted image 33 and the input video signal 3. an encoding mode determining unit 20 carries out a selecting process from the prediction process executed by the prediction unit 15, and outputs the selected prediction difference signal 19 and a macroblock type/sub-macroblock type 21. All items of macroblock header information such as the macroblock type, the sub-macroblock type, the reference image index, and the motion vector are determined as the header information common to the three color components. The macroblock header information is used in common for encoding the three color components and is multiplexed on the bit stream 8. In a case of evaluating the optimality of the prediction efficiency, for the purpose of restraining the computing amount, only a prediction error amount for a certain predetermined color component (e.g., the G-component of R, G, and B, and the Y-component of Y, Cb, and Cr) may also be evaluated, and the prediction error amounts for all of the color components may also be comprehensively evaluated in order to acquire the optimal prediction performance, though the computing amount is increased.
  • [0053]
    Similarly, the prediction unit 15 executes the intra prediction. When the prediction unit 15 executes the intra prediction, intra prediction mode information is output to the signal 17.
  • [0054]
    Hereinafter, unless there is no particular distinction between the intra prediction and the motion compensation prediction, the signal 17 is called the prediction overhead information. Note that the reference image data for conducting the intra prediction involves using a local decoded image 23 before being subjected to the process of a deblocking filter 22 (though not illustrated, the local decoded image 23 is used as the reference image data for the intra prediction and is therefore temporarily stored in the prediction unit 15 and the like). An intra prediction mode common to the three color components is selected and applied to the common encoding process executed by the first picture encoding unit 5. As for the intra prediction, the prediction error amount only for the predetermined color component may be evaluated, and the prediction error amounts for all of the color components may also be comprehensively evaluated. Finally, the encoding mode determining unit 20 selects, by evaluating in terms of the prediction efficiency or the encoding efficiency, whether the macroblock type is set to the intra prediction or the inter prediction.
  • [0055]
    A transformation unit 24 transforms the prediction difference signal 19 and outputs the transformed signal as a transform coefficient to a quantization unit 25. In this case, a block size serving as a unit for conducting the transform may be selected from among a plurality of candidate sizes such as 4×4 and 8×8. In the case of making the transformation block size selectable, the block size selected when encoded is reflected in a value of the transformation block size designation flag 26, and the flag 26 is multiplexed on the bit stream 8. The quantization unit 25 quantizes the transform coefficient to be input on the basis of the quantization parameter 27 and the quantization weighting coefficients 12 a to 12 c, and outputs the quantized result as a quantized transform coefficient 28 to a variable length encoding unit 29 and a inverse quantization unit 30.
  • [0056]
    The process of the quantization unit 25 will be described. The transform coefficient transformed into a signal of a frequency domain from that of a spatial domain by the transformation unit 24 is separated into a low frequency domain where distortion is easy to be caught by eyes in terms of visual characteristics of a man and a high frequency domain where the distortion is hard to be detected. Thus, every frequency domain is weighted. Fine quantization is conducted in the low frequency domain, while coarse quantization is conducted in the high frequency domain, whereby the quantization process adapted to the visual characteristics of the man can be realized. The quantization weighting coefficients 12 a to 12 c are the weighting parameters given in every frequency domain. 16 weighting parameters are used for transforming the 4×4 block size, and 64 weighting parameters are used for transforming the 8×8 block size. As described above, the quantization weighting coefficients 12 a to 12 c are multiplexed on the sequence parameter set. In the case of executing the “common encoding process,” however, the same quantization weighting coefficient is used for the three color components. Accordingly, there is no necessity of multiplexing the three coefficients 12 a, 12 b, and 12 c, and it is sufficient to multiplex only one coefficient. The quantization unit 25 executes the quantization process in which the transform coefficients of the three color components are weighted by use of the quantization weighting coefficients 12 a to 12 c, thereby obtaining the quantized transform coefficient 28.
  • [0057]
    The quantized transform coefficient 28 for the three color components is entropy-coded by the variable length encoding unit 29 using measures such as Huffman coding and arithmetic coding.
  • [0058]
    Further, the quantized transform coefficient 28 is restored to a local decoded prediction difference signal 32 via the inverse quantization unit 30 and a inverse transform unit 31. Then, in an adder 34, the signal 32 is added with a prediction image 33 generated based on the selected macroblock type/sub-macroblock type 21 and the prediction overhead information 17, thereby generating the local decoded image 23. The local decoded image 23 is, after being subjected to a block distortion removing process in the deblocking filter 22, stored in the memory 16 in order to be employed for the subsequent motion compensation prediction process.
  • [0059]
    The quantized transform coefficient 28, the macroblock type/sub-macroblock type 21, the prediction overhead information 17, and the quantization parameter 27, which are input to the variable length encoding unit 29, are arranged and shaped according to a predetermined syntax of the macroblock encoded data, and are packetized (also called NAL unitization in the AVC) on the unit of the slice data consisting of one macroblock or the aggregation of the plurality of macroblocks in the format shown in FIG. 3. Then, the data packets are output as the bit stream 8.
  • [0060]
    (2) Case of Intra Only Encoding Instruction Signal 13 Indicating “Intra Only Encoding”
  • [0061]
    The prediction unit 15 executes only the intra prediction process described in the item (1). When the intra prediction is executed, the intra prediction mode information is output to the prediction overhead information 17. Note that the reference image data for conducting the intra prediction involves using the local decoded image 23 before being subjected to the process of the deblocking filter 22 (though not illustrated, the local decoded image 23 is used as the reference image data for the intra prediction and is therefore temporarily stored in the prediction unit 15 and the like). The intra prediction mode common to the three color components is selected and applied to the common encoding process executed by the first picture encoding unit 5. The encoding mode determining unit 20 selects the intra prediction mode by evaluating in terms of the prediction efficiency or the encoding efficiency.
  • [0062]
    The encoding mode determining unit 20 outputs the selected prediction difference signal 19 to the transformation unit 24. The transformation unit 24 transforms the prediction difference signal 19 to be input and outputs the transformed signal as a transform coefficient to the quantization unit 25. In this case, the block size serving as the unit for conducting the transform may be selected from among the plurality of candidate sizes such as 4×4 and 8×8. The AVC is configured so that a prediction target block of the intra prediction mode is adjusted to the transformation block size. In the case of making the transformation block size selectable, the block size selected when encoded is reflected in the value of the transformation block size designation flag 26, and the flag 26 is multiplexed on the bit stream 8. The quantization unit 25 quantizes the transform coefficient to be input on the basis of the quantization parameter 27 and the quantization weighting coefficients 12 a to 12 c, and outputs the quantized result as the quantized transform coefficient 28 to the variable length encoding unit 29. The example of the process is as described above.
  • [0063]
    The quantized transform coefficient 28 for the three color components is entropy-coded by the variable length encoding unit 29 using the measures such as Huffman coding and arithmetic coding.
  • [0064]
    Further, the quantized transform coefficient 28 is restored to the local decoded prediction difference signal 32 via the inverse quantization unit 30 and the inverse transform unit 31. Then, in the adder 34, the signal 32 is added with the prediction image 33 generated based on the prediction overhead information 17, thereby generating the local decoded image 23. If the intra only encoding instruction information 13 indicates “intra only encoding,” the motion compensation prediction is not executed, and hence the deblocking filter 22 executes neither the process nor the writing of the data as the reference image to the memory 16. With this structure, access to the memory and the arithmetic operation needed for the process of the deblocking filter can be reduced.
  • [0065]
    The quantized transform coefficient 28, the macroblock type/sub-macroblock type 21 (fixed to the intra mode), the prediction overhead information 17, and the quantization parameter 27, which are input to the variable length encoding unit 29, are arranged and shaped according to the predetermined syntax of the macroblock encoded data, and are packetized (also called the NAL unitization in the AVC) on the unit of the slice data consisting of one macroblock or the aggregation of the plurality of macroblocks in the format shown in FIG. 3. Then, the data packets are output as the bit stream 8.
  • [0066]
    Note that the number of the macroblocks included in the slice is restricted not by the value of the intra only encoding instruction information 13 but by the image size information 14. The image size information 14 is input to the variable length encoding unit 29. The variable length encoding unit 29 sets, based on the image size information 14, an upper limit value of the number of the macroblocks included in the slice. The variable length encoding unit 29 counts the number of the encoded macroblocks in advance, and, when the number of the macroblocks included in the slice reaches the upper limit value, closes the packet of the slice data. Subsequent macroblocks are packetized as new slice data.
  • [0067]
    Moreover, the first picture encoding unit 5 according to the first embodiment of the present invention does not provide the color component identification flag 2 in the slice data because of its being recognizable from the common encoding/independent encoding identification signal 1 that all pieces of the slice data in the sequence can be defined as C0, C1, and C2 aggregated slices (i.e., the slices each consisting of the aggregation of the three color component information).
  • Outline of Operation of Second Picture Encoding Unit
  • [0068]
    FIG. 11 illustrates an internal configuration of the second picture encoding unit 7 a. In FIG. 11, it is assumed that an input video signal 3 a is input on the macroblock basis, the macroblock consisting of the samples of the C0 component in the format shown in FIG. 4. The second picture encoding units 7 b, 7 c have absolutely the same internal configuration except for handling input video signals 3 b (C1 component) and 3 c (C2 component) in place of the input video signal 3 a. Therefore, an operation of the second picture encoding unit will hereinafter be described by way of a representative example of the second picture encoding units 7 a.
  • [0069]
    (3) Case of Intra Only Encoding Instruction Signal 13 Indicating “Non Intra Only Encoding”
  • [0070]
    A prediction unit 115 selects a reference image from among motion compensation prediction reference image data stored in a memory 116, and executes a motion compensation prediction process on the macroblock basis. The memory 116 can be stored with a plurality of pieces of reference image data each consisting of the single color component over a plurality of points of time such as the most recent past or the past and the future. The prediction unit 115 performs the motion prediction by selecting an optimal reference image on the macroblock basis from among those reference images. Note that each of the second picture encoding units 7 a to 7 c can be configured so as to employ only the reference image data for each target color component and not to access the reference image data of other color components. Hence, the memory 116 may take not the configuration of providing the memories for the three color components, respectively, but a configuration of aggregating the memories into one memory. Further, in the sequence where the second picture encoding units 7 a to 7 c execute the encoding process, the first picture encoding unit 5 does not operate, and therefore the memory 116 is configured to be shared with the memory 16. Seven types of block sizes subjected to the motion compensation prediction are provided. First, as illustrated in FIGS. 10( a) to 10(d), any size of 16×16, 16×8, 8×16, and 8×8 is selected on the macroblock basis. Further, when the 8×8 size is selected, as illustrated in FIGS. 10( e) to 10(h), any size of 8×8, 8×4, 4×8, and 4×4 is selected for every 8×8 block. In the independent encoding process executed by the second picture encoding unit 7, the motion compensation prediction block sizes for the C0 to C2 components are individually selected and applied.
  • [0071]
    The prediction unit 115 executes the motion compensation prediction process on the macroblock basis with respect to all or part of the block sizes/sub-block sizes of FIG. 10, the motion vector in the predetermined search range, and one or more pieces of usable reference images, thereby outputting prediction overhead information 117 containing the motion vector and a reference image index used for the prediction and a predicted image 133. A subtracter 118 obtains a prediction difference signal 119 for every block that serves as a motion compensation prediction unit from the predicted image 133 and the input video signal 13 a. an encoding mode determining unit 120 carries out the selecting process from the prediction process executed by the prediction unit 115, and outputs the selected prediction difference signal 119 and a macroblock type/sub-macroblock type 121. All items of macroblock header information such as the macroblock type, the sub-macroblock type, the reference image index, and the motion vector are determined as the header information for the input video signal 3 a. Then, the macroblock header information is used for encoding and multiplexed on a bit stream 9 a. In a case of evaluating the optimality of the prediction efficiency, a prediction error amount with respect to the input video signal 3 a serving as the encoding process target is evaluated.
  • [0072]
    Similarly, the prediction unit 115 also executes the intra prediction. When the intra prediction is executed, intra prediction mode information is output to the signal 117.
  • [0073]
    The intra prediction mode in the respective color components of the input video signals 3 is selected and applied individually for the respective color components. Note that the reference image data for conducting the intra prediction involves using a local decoded image 123 before being subjected to the process of the deblocking filter 22 (though not illustrated, the local decoded image 123 is used as the reference image data for the intra prediction and is therefore temporarily stored in the prediction unit 115). As to the intra prediction, the prediction error amount with respect to the input video signal 3 a serving as the encoding processing target is evaluated. Finally, the encoding mode determining unit 120 selects, by evaluating in terms of the prediction efficiency or the encoding efficiency, whether the macroblock type is set to the intra prediction or the inter prediction.
  • [0074]
    A transformation unit 124 transforms the prediction difference signal 119 and outputs the transformed signal as a transform coefficient to a quantization unit 125. In this case, a block size serving as a unit for conducting the transform may be selected from among 4×4 and 8×8. In the AVC, a prediction target block in the intra prediction mode is configured to be adjusted to the transformation block size. In the case of making the transformation block size selectable, the block size selected when encoded is reflected in a value of a transformation block size designation flag 126, and the flag 126 is multiplexed on the bit stream 9 a. The quantization unit 125 quantizes the transform coefficient to be input on the basis of a quantization parameter 127 and the quantization weighting coefficient 12 a, 12 b, or 12 c, and outputs the quantized result as a quantized transform coefficient 128 to a variable length encoding unit 129.
  • [0075]
    Next, the process of the quantization unit 125 will be described. The transform coefficient transformed into a signal of the frequency domain from that of the spatial domain by the transformation unit 124 is separated into the low frequency domain where the distortion is easy to be caught by eyes in terms of the visual characteristics of the man and the high frequency domain where the distortion is hard to be detected. Therefore, every frequency domain is weighted. The fine quantization is conducted in the low frequency domain, while the rough quantization is conducted in the high frequency domain, whereby the quantization process adapted to the visual characteristics of the man can be realized. Each of the quantization weighting coefficients 12 a, 12 b, 12 c is the weighting parameter given in every frequency domain. 16 weighting parameters are used for transforming the 4×4 block size, and 64 weighting parameters are used for transforming the 8×8 block size. As described above, the quantization weighting coefficients 12 a, 12 b, 12 c are multiplexed on the sequence parameter set. In the case of executing the “independent encoding process,” however, the quantization weighting coefficients different among the three color components can be used. Accordingly, all of the three coefficients 12 a, 12 b, and 12 c may be multiplexed, and, in the case of using the same value, only one coefficient may also be multiplexed together with information indicating this purpose. The quantization unit 125 executes the weighted quantization process on the transform coefficients of the three color components by using each of the quantization weighting coefficient 12 a or 12 b or 12 c, thereby obtaining the quantized transform coefficient 128. The quantized transform coefficient 128 is entropy-coded by a variable length encoding unit 129 using measures such as Huffman coding and arithmetic coding.
  • [0076]
    Further, the quantized transform coefficient 128 is restored to a local decoded prediction difference signal 132 via a inverse quantization unit 130 and a inverse transform unit 131. Then, an adder 134 adds the local decoded prediction difference signal 132 to the prediction image 133 generated based on the selected macroblock type/sub-macroblock type 121 and the prediction overhead information 117, thereby generating a local decoded image 123. The local decoded image 123 is stored, after being subjected to the block distortion removing process in a deblocking filter 122, in the memory 116 in order to be employed for the subsequent motion compensation prediction process. The quantized transform coefficient 128, the macroblock type/sub-macroblock type 121, the prediction overhead information 117 and the quantization parameter 127, which are input to the variable length encoding unit 129, are arranged and shaped according to the predetermined syntax of the macroblock encoded data, and are packetized (also called NAL unitization in the AVC) on the unit of the slice data consisting of one macroblock or the aggregation of the plurality of macroblocks in the format shown in FIG. 4. Then, the data packets are output as the bit stream 9 a.
  • [0077]
    (4) Case of Intra Only Encoding Instruction Signal 13 Indicating “Intra Only Encoding”
  • [0078]
    The prediction unit 115 executes only the intra prediction process described in the item (3). When executing the intra prediction, the intra prediction mode information is output to the prediction overhead information 117. Note that the reference image data for conducting the intra prediction involves using the local decoded image 123 before being subjected to the process of the deblocking filter 122 (though not illustrated, the local decoded image 123 is used as the reference image data for the intra prediction and is therefore temporarily stored in the prediction unit 115). The intra prediction in the AVC is as described above and executed with respect to the input video signal 3 a. Therefore, the intra prediction mode in the respective color components of the input video signals 3 is selected and applied individually for the respective color components. The encoding mode determining unit 120 selects the intra prediction mode by evaluating in terms of the prediction efficiency or the encoding efficiency.
  • [0079]
    The encoding mode determining unit 120 outputs the selected prediction difference signal 119 to the transformation unit 124. The transformation unit 124 transforms the received prediction difference signal 119 and outputs the transformed signal as a transform coefficient to the quantization unit 125. In this case, the block size serving as the unit for conducting the transform may be selected from among the plurality of candidate sizes such as 4×4 and 8×8. The AVC is configured so that the prediction target block in the intra prediction mode is adjusted to the transformation block size. In the case of making the transformation block size selectable, the block size selected when encoded is reflected in a value of the transformation block size designation flag 126, and the flag 126 is multiplexed on the bit stream 9 a. The quantization unit 125 quantizes the transform coefficient to be input on the basis of the quantization parameter 127 and the quantization weighting coefficient 12 a, and outputs the quantized result as a quantized transform coefficient 128 to the variable length encoding unit 129. The example of the process is given as described above.
  • [0080]
    The quantized transform coefficient 128 is entropy-coded by a variable length encoding unit 129 using measures such as Huffman coding and arithmetic coding.
  • [0081]
    The quantized transform coefficient 128 is restored to the local decoded prediction difference signal 132 via the inverse quantization unit 130 and the inverse transform unit 131. Then, the adder 134 adds the signal 132 to the prediction image 133 generated based on the prediction overhead information 117, thereby generating the local decoded image 123. If the intra only encoding instruction information 113 indicates “intra only encoding,” the motion compensation prediction is not executed, and hence the deblocking filter 122 executes neither the process nor the write of the data as the reference image to the memory 116. With this configuration, the memory access and the arithmetic operation needed for the process of the deblocking filter can be reduced.
  • [0082]
    The quantized transform coefficient 128, the macroblock type/sub-macroblock type 121 (fixed to the intra mode), the prediction overhead information 117 and the quantization parameter 127, which are input to the variable length encoding unit 129, are arranged and shaped according to the predetermined syntax of the macroblock encoded data, and are packetized (also called the NAL unitization in the AVC) on the unit of the slice data consisting of one macroblock or the aggregation of the plurality of macroblocks in the format shown in FIG. 4. Then, the data packets are output as the bit stream 9 a.
  • [0083]
    The second picture encoding units 7 a to 7 c according to the first embodiment encode all pieces of slice data in the sequence with the single color component slice (i.e., the C0 slice or the C1 slice or the C2 slice) through the common encoding/independent encoding identification signal 1. Hence, the color component identification flag 2 is invariably multiplexed on the head of the slice data, whereby the decoding device can recognize which slice corresponds to which picture data in the access unit. For example, the second picture encoding unit 7 a sets “0” as the value of the color component identification flag 2, the second picture encoding unit 7 b sets “1” as the value of the color component identification flag 2, and the second picture encoding unit 7 c sets “2” as the value of the color component identification flag 2, and each of the flag values is attached to the head of the slice data. Accordingly, even when serially multiplexing the bit streams 9 a to 9 c on the bit stream 11, the decoding device can readily recognize which slice therein corresponds which encoded data of the C0 or C1 or C2 component. In other words, the second picture encoding units 7 a to 7 c can transmit the bit stream output any time when the data for one slice is accumulated without accumulating the respective bit stream outputs for one picture.
  • [0084]
    Note that the number of the macroblocks included in the slice is restricted not by the value of the intra only encoding instruction information 13 but by the image size information 14. The image size information 14 is input to the variable length encoding unit 129. The variable length encoding unit 129 sets, based on the image size information 14, an upper limit value of the number of the macroblocks included in the slice. The variable length encoding unit 129 previously counts the number of the encoded macroblocks, and, just when the number of the macroblocks included in the slice reaches the upper limit value, closes the packet of the slice data. Subsequent macroblocks are packetized as new slice data. It should be noted that the image size information 14 has the same value with respect to the C0, C1, C2 components in the case of the 4:4:4 format, and it may therefore be enough to multiplex only one piece of data in the sequence parameter set.
  • [0085]
    Further, the first picture encoding unit 5 and the second picture encoding units 7 a to 7 c are different in terms of whether to treat the macroblock header information as the information common to the three color components or as the information of the single color component, and the bit stream structure of the slice data. Accordingly, the transformation unit 24, the inverse transform unit 31, the quantization unit 25, the inverse quantization unit 30 and the deblocking filter 22 of FIG. 9 are realized in a way that repeats the arithmetic operations, for the three color components, of the transformation unit 124, the inverse transform unit 131, the quantization unit 125, the inverse quantization unit 130 and the deblocking filter 122 of FIG. 11. Hence, part of the internal configurations of the first picture encoding unit 5 and the second picture encoding units 7 a to 7 c can also be realized by common function blocks. Accordingly, it is feasible to actualize implementation of multiform encoding devices in a way that repeatedly operates, e.g., the same circuits a plurality of times by properly combining the components of FIGS. 9 and 11 without being limited to the completely independent encoding processing unit as in FIG. 8. Further, as described above, if the arrangement of the memory 16 in the first picture encoding unit 5 is provided plane-sequentially, the reference image storage memory can be shared between the first picture encoding unit 5 and the second picture encoding units 7 a to 7 c.
  • [0086]
    FIG. 12 illustrates an outline of the configuration of the decoding device according to the first embodiment. In FIG. 12, a common decoding process is executed by a first picture decoding unit 302, and an independent decoding process is executed by a color component determining unit 303 and second picture decoding units 304 (provided for the three color components).
  • [0087]
    The bit stream 11 is divided on a NAL unit basis by a higher-order header analyzing unit 300. Higher-order header information such as the sequence parameter set and the picture parameter set is decoded and stored in a higher-order header memory 305 to which the first picture decoding unit 302, the color component determining unit 303 and the second picture decoding units 304 can refer. The common encoding/independent encoding identification signal 1, the quantization weighting coefficients 12 a to 12 c, the intra only encoding instruction information 13 and the image size information 14, which are multiplexed on the sequence basis, are retained as part of the sequence parameter set in the higher-order header memory 305.
  • [0088]
    The decoded common encoding/independent encoding identification signal 1 is supplied to a switch (SW) 301. The switch 301, if the common encoding/independent encoding identification signal 1 indicates “being subjected to the common encoding process,” supplies all of the slice NAL units in the sequence as the bit stream 8 to the first picture decoding unit 302. The switch 301, if the common encoding/independent encoding identification signal 1 indicates “being subjected to the independent encoding process,” supplies all of the slice NAL units in the sequence to the color component determining unit 303. Detailed operations of the first and second picture decoding units will be described later.
  • [0089]
    The color component determining unit 303 analyzes a value of the color component identification flag 2 shown in FIG. 7 from among the received slice NAL units, then recognizes which color component picture in the present access unit the slice NAL units correspond to, and distributes the slice NAL units as bit streams 9 a to 9 c to the second picture decoding units 304 a to 304 c. The configuration of this type of decoding device has such an effect that even when receiving the bit streams into which the slices are interleaved and encoded for every color component in the access unit, it is readily determined which slice belongs to which color component picture, and the encoded bit stream can be properly decoded.
  • Outline of Operation of First Picture Decoding Unit
  • [0090]
    FIG. 13 illustrates an internal configuration of the first picture decoding unit 302. The first picture decoding unit 302 receives the bit stream 11 output from the encoding device illustrated in FIG. 8, for example, in the format of the bit stream 8 built up by the slices each consisting of the aggregation of the three color components of C0, C1, and C2. The first picture decoding unit 302 restores the output video frame by executing the decoding process on the macroblock basis. The macroblock consists of the samples (pixels) of the three color components illustrated in FIG. 3.
  • [0091]
    A variable length decoding unit 310 receives the bit stream 8, decodes the bit stream 8 according to a predetermined rule (syntax) and extracts, from the bit stream 8, the quantized transform coefficient 28 for the three color components and the macroblock header information (the macroblock type/sub-macroblock type 21, the prediction overhead information 17, the transformation block size designation flag 26 and the quantization parameter 27) used in common to the three color components.
  • [0092]
    Further, the quantized transform coefficient 28 is input together with the quantization parameter 27 to the inverse quantization unit 30 that executes the same process as the first picture encoding unit 5, thereby performing the inverse quantization process is performed. In this case, the quantization weighting coefficients 12 a to 12 c used for the respective color components are employed by referring to the higher-order header memory 305. Note that if the three quantization weighting coefficients 12 a to 12 c take the same value, the decoder is not necessarily required to internally have the coefficients as three pieces of data, and one piece of data can be employed in common. Subsequently, the output thereof is input to the inverse transform unit 31 that executes the same process as the first picture encoding unit 5, and the output is restored to the local decoded prediction difference signal 32 (if the transformation block size designation flag 26 exists in the bit stream 8, this flag 26 is referred to in the inverse quantization process and in the inverse transform process). On the other hand, the prediction unit 311, referring to the prediction overhead information 17 in the prediction unit 15 of the first picture encoding unit 5, includes only the process of generating the predicted image 33, and the macroblock type/sub-macroblock type 21 and the prediction overhead information 17 are input to the prediction unit 311, thereby obtaining the predicted image 33 for the three color components.
  • [0093]
    If the macroblock type indicates that the macroblock type is the intra prediction, the predicted image 33 for the three color components is obtained from the prediction overhead information 17 according to the intra prediction mode information. If the macroblock type indicates that the macroblock type is the inter prediction, the predicted image 33 for the three color components is obtained from the prediction overhead information 17 according to the motion vector and the reference image index. The adder 34 adds the local decoded prediction difference signal 32 to the predicted image 33, thereby acquiring a temporary decoded image 323 for the three color components. The temporary decoded image 323 is used for the motion compensation prediction of the macroblock, and is therefore, after executing the block distortion removing process on the temporary decoded image samples for the three color components by use of the deblocking filter 22 that executes the same process as the first picture encoding unit 5, output and stored as a decoded image 313 in a memory 312. The memory 312 stores a plurality of sets of reference image data each consisting of the three color components over a plurality of points of time. The prediction unit 311 generates the predicted image by selecting the reference image indicated by the reference image index extracted out of the bit stream 8 on the macroblock basis from the reference image data. As to the arrangement of the reference image data in the memory 312, these pieces of data may be stored separately and plane-sequentially for every color component, and the samples (pixels) of the respective color components may also be stored dot-sequentially. The decoded image 313 is defined as a color video frame containing the three color components.
  • [0094]
    Further, the first picture decoding unit 302 can be configured as follows. If the intra only encoding instruction information 13 stored in the higher-order header memory 305 indicates “intra only encoding,” the reference image becomes unnecessary because the motion compensation prediction process is not needed. So, the process performed in the deblocking filter 22 is skipped, and writing the reference image to the memory 312 is not performed. This configuration enables the reduction in the memory access and the arithmetic operation required for the process of the deblocking filtering. Even in the case of the “intra only encoding,” however, the deblocking filtering or similar post-processing filtering can be carried out as the post-processing for displaying the decoded image.
  • Outline of Operation of Second Picture Decoding Unit
  • [0095]
    FIG. 14 illustrates an internal configuration of the second picture decoding unit 304. The second picture decoding unit 304 receives any one of bit streams 9 a to 9 c formed of C0, C1, or C2 slice NAL unit, to which the bit stream 11 output from the encoding device illustrated in FIG. 8, for example, is allocated by the color component determining unit 303. The second picture decoding unit 304 executes the decoding process on the macroblock basis. The macroblock consisting of the samples of the single color component illustrated in FIG. 4, thereby restoring the output video frame.
  • [0096]
    A variable length decoding unit 410 receives the bit stream 9, decodes the bit stream 9 according to a predetermined rule (syntax), and extracts the quantized transform coefficient 128 for the single color component and the macroblock header information applied to the single color component (the macroblock type/sub-macroblock type 121, the prediction overhead information 117, the transformation block size designation flag 126 and the quantization parameter 127). The quantized transform coefficient 128 is input together with the quantization parameter 127 to the inverse quantization unit 130 that executes the same process as the second picture encoding unit 5, thereby performing the inverse quantization process. As the quantization weighting coefficient employed in this case, based on the color component identification flag 2 decoded by the variable length decoding unit 410, one quantization weighting coefficient corresponding to the color component concerned is selected from the quantization weighting coefficients 12 a to 12 c in the higher-order header memory 305, and the selected quantization weighting coefficient is referred to. Subsequently, the output of the inverse quantization unit 130 is input to the inverse transform unit 131 that executes the same process as the second picture encoding unit 7, and is restored to the local decoded prediction difference signal 132 (if the transformation block size designation flag 126 exists in the bit stream 9, this flag 126 is referred to in the inverse quantization process and a reverse orthogonal transform process).
  • [0097]
    On the other hand, a prediction unit 411 includes only the process of generating the predicted image 133, referring to the prediction overhead information 117 in the prediction unit 115 of the second picture encoding unit 7, and receives the macroblock type/sub-macroblock type 121 and the prediction overhead information 117, thereby obtaining the predicted image 133 for the single color component. If the macroblock type indicates the intra prediction, the predicted image 133 for the single color component is obtained from the prediction overhead information 117 according to the intra prediction mode information. If the macroblock type indicates that macroblock type is the inter prediction, the predicted image 133 for the single color component is obtained from the prediction overhead information 117 according to the motion vector and the reference image index. The adder 134 adds the local decoded prediction difference signal 132 to the predicted image 133, thereby acquiring a temporary decoded image 423 for the single color component. The temporary decoded image 423 is used for the subsequent motion compensation prediction of the macroblock. Therefore, after executing the block distortion removing process on the temporary decoded image samples for the single color component by use of the deblocking filter 122 that executes the same process as the second picture encoding unit 7, the temporary decoded image 423 is output as a decoded image 413 to be stored in a memory 412. The decoded image 413 includes only the samples of the single color component, and is, as illustrated in FIG. 5, formed as a color video frame by binding the respective outputs of the second picture decoding units 304 for other color components.
  • [0098]
    Further, the second picture decoding unit 304 can be configured as follows. If the intra only encoding instruction information 113 stored in the higher-order header memory 305 indicates that intra only encoding instruction information 113 is “intra only encoding,” the reference image becomes unnecessary because the motion compensation prediction process is not needed. So the process performed in the deblocking filter 122 is skipped, and writing the reference image to the memory 412 is not performed. This configuration enables the reduction in the memory access and the arithmetic operation required for the process of the deblocking filtering. Even in the case of the “intra only encoding,” however, the deblocking filtering or similar post-processing filtering can be carried out as the post-processing for displaying the decoded image.
  • [0099]
    As apparent from the above description, the first picture decoding unit 302 and the second picture decoding unit 304 are different in terms of whether to treat the macroblock header information as the information common to the three color components or as the information of the single color component, and the bit stream structure of the slice data. Hence, the basic decoding process blocks such as the prediction unit, the inverse transform unit and the inverse quantization unit of FIGS. 13 and 14 can be realized by the function blocks common to the first picture decoding unit 302 and the second picture decoding unit 304. Accordingly, it is possible to actualize implementation of multiform decoding devices by properly combining the basic components of FIGS. 13 and 14 without being limited to the completely independent encoding processing unit as illustrated in FIG. 12. Further, if the arrangement of the memory 312 in the first picture decoding unit 302 is provided plane-sequentially, the structures of the memory 312 and the memory 412 can be made common to the first picture decoding unit 302 and the second picture decoding unit 304.
  • [0100]
    The first picture decoding unit and the second picture decoding unit in the first embodiment have been described such that the decoding units receive the bit stream 11 output by the encoding device. However, the bit stream input to the first picture decoding unit and the second picture decoding unit are not limited to the bit stream output by the encoding device, and bit streams read from storage mediums such as a hard disc and a DVD, and bit streams read from a server and transmitted via a network may be input.
  • [0101]
    Note that the encoding device and the decoding device according to the above embodiment, in the case of the independent encoding process, as shown in FIG. 15, can achieve the same effect also by arranging the picture data for the three color components in sequence and treating these pieces of data as one set of picture data. At this time, the arrangement is made so that, corresponding to a single color pixel array, three pieces of data are connected in the vertical direction, with respect to color video signals with H pixels in the horizontal direction and V-lines in the vertical direction, and the image size information 14 is set to have H pixels in the horizontal direction and (V×3) lines in the vertical direction. It should be noted that in the encoding device and the decoding device according to the first embodiment, in order that the signals of the respective components may be treated as the independent pictures, the dependency among the color components are eliminated in the encoding/decoding processes at the boundaries the bold dotted line of FIG. 15 between the respective color components. The following conditions are given by way of an example of eliminating the dependency.
      • When encoded, in the macroblocks positioned at the boundaries between the respective color components, the motion vector search does not employ the neighboring pixels of other color components, and an off-plane search is conducted so as to extend edge point pixels of the self-color component in the same way as the process for the plane boundary. When decoded, in the macroblocks positioned at the boundaries between the color components, if the motion vector deviates from the image area of the self-color component (if the off-plane search is conducted), instead of employing the pixel signals of other colors, the predicted image is generated by extending the edge point pixels of the self-color component as deemed to be a plane boundary.
      • The deblocking filtering process is not performed between the blocks positioned at the boundaries between the respective color components.
      • In the case of using a learning-oriented arithmetic coding for the variable length encoding/variable length decoding processes in the encoding/decoding processes of the macroblocks of the color components, a probability model is provided individually for every color component, and a learning process is independently executed for every color component.
  • [0105]
    The independence of the signal of each color component is acquired by applying these conditions, and, the first, second and third picture encoding/decoding units can independently execute the processes, respectively.
  • [0106]
    Further, restrictions are imposed so that a slice is inhibited from being defined across the boundaries of the respective color components, one slice is prevented from containing the encoded data of the plurality of different color components, and the head macroblock data of each color component invariably becomes the head macroblock of the slice data.
  • [0107]
    Still further, a configuration for specifying which slice data belongs to which color component may involve explicitly specifying the color component to which the slice belongs by defining the color component identification flag “color_channel_idc” and attaching the flag to the head field of the slice data. Another configuration may involve using not “color_channel_idc” but the head macroblock address of each slice data and the image size information 14, and recognizing which slice data belongs to which color component. For example, when a horizontal pixel count W=1920 and a vertical pixel count V=1080, “0”, “8160” and “16320” are given as the head macroblock addresses of the C0, C1, and C2 components, so, the macroblocks having the macroblock addresses “0-8159” are allocated to the C0 component, the macroblocks having the addresses “8160-16319” are allocated to the C1 component, and the macroblocks having the addresses “16320-24479” are allocated to the C2 component.
  • [0108]
    With the configurations, the picture/access unit structure for the common encoding process/independent encoding process can be made common, thereby enhancing the efficiency of the random access and the editing operation.

Claims (5)

  1. 1. An image encoding device for generating bit streams by subjecting a color image in a 4:4:4 format to compression encoding, the compression encoding being performed by converting and quantizing the color image on a block basis, wherein
    the image encoding device multiplexes, on the bit streams, first identification information indicating whether or not to independently encode signals of respective color components, and, in a case where the first identification information indicates that the signals of respective color components are independently encoded, multiplexes, on the bit streams, second information indicating whether or not to intra encode all the color components and controlling an operation of a deblocking filter operating on each boundary between blocks serving as units of the conversion and quantization.
  2. 2. An image encoding device for performing compression encoding based on an input of digital moving image signals in a 4:4:4 format, comprising:
    a first encoding unit for subjecting three color component signals of the input digital moving image signals, to an encoding process in a common encoding mode and performing a quantization process by using a common quantization matrix; and
    a second encoding unit for subjecting the three color component signals of the input digital moving image signals, respectively, to an encoding process in independent encoding modes and performing a quantization process by using independent quantization matrices, wherein
    the image encoding device performs the encoding process by selecting one of the first encoding unit and the second encoding unit and sets an identification signal in the compressed data, the identification signal indicating which one of the first encoding unit and the second encoding unit is selected.
  3. 3. An image encoding device for performing compression encoding based on an input of digital moving image signals in a 4:4:4 format, comprising:
    a first encoding unit for subjecting three color component signals of the input digital moving image signals, to an encoding process in a common encoding mode and performing a quantization process by using a common quantization matrix; and
    a second encoding unit for subjecting the three color component signals of the input digital moving image signals, respectively, to an encoding process in independent encoding modes and performing a quantization process by using independent quantization matrices, wherein:
    the image encoding device performs the encoding process by selecting one of the first encoding unit and the second encoding unit, and sets an identification signal in the compressed data, the identification signal indicating which one of the first encoding unit and the second encoding unit is selected;
    the image encoding device further sets an intra only encoding instruction signal in the compressed data, the intra only encoding instruction signal indicating whether to perform all the encoding processes only in an intra encoding mode or in combination of the intra encoding mode and an inter-encoding mode; and
    in a case of performing the encoding processes only in the intra encoding mode, the process performed in a deblocking filter is skipped.
  4. 4. An image encoding method for performing compression encoding based on an input of digital moving image signals in a 4:4:4 format, the image encoding method comprising:
    a step of performing an encoding process by selecting one of a first encoding step and a second encoding step, the first encoding step for subjecting three color component signals of the input digital moving image signals to an encoding process in a common encoding mode and performing a quantization process by using a common quantization matrix and the second encoding step for subjecting the three color component signals of the input digital moving image signals, respectively, to an encoding process in independent encoding modes and performing a quantization process by using independent quantization matrices; and
    a step of setting an identification signal in the compressed data, the identification signal indicating which one of the first encoding step and the second encoding step is selected.
  5. 5. An image encoding method for generating bit streams by subjecting a color image in a 4:4:4 format to compression encoding, the compression encoding being performed by converting and quantizing the color image on a block basis, wherein
    first identification information indicating whether or not to independently encode signals of respective color components is multiplexed on the bit streams, and, in a case where the first identification information indicates that the signals of respective color components are independently encoded, second information indicating whether or not to intra encode all the color components and controlling an operation of a deblocking filter operating on each boundary between blocks serving as units of the conversion and quantization is multiplexed on the bit streams.
US11926593 2007-01-12 2007-10-29 Image encoding device and image encoding method Abandoned US20080170793A1 (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
JP2007-004652 2007-01-12
JP2007004652 2007-01-12
JP2007-028711 2007-02-08
JP2007028711A JP2008193627A5 (en) 2007-02-08

Publications (1)

Publication Number Publication Date
US20080170793A1 true true US20080170793A1 (en) 2008-07-17

Family

ID=39473045

Family Applications (3)

Application Number Title Priority Date Filing Date
US11926848 Abandoned US20080170624A1 (en) 2007-01-12 2007-10-29 Image encoding device and image encoding method
US11926593 Abandoned US20080170793A1 (en) 2007-01-12 2007-10-29 Image encoding device and image encoding method
US11926881 Active 2032-10-04 US8718139B2 (en) 2007-01-12 2007-10-29 Image decoding device and image decoding method

Family Applications Before (1)

Application Number Title Priority Date Filing Date
US11926848 Abandoned US20080170624A1 (en) 2007-01-12 2007-10-29 Image encoding device and image encoding method

Family Applications After (1)

Application Number Title Priority Date Filing Date
US11926881 Active 2032-10-04 US8718139B2 (en) 2007-01-12 2007-10-29 Image decoding device and image decoding method

Country Status (3)

Country Link
US (3) US20080170624A1 (en)
EP (1) EP1947860B1 (en)
CA (1) CA2615299C (en)

Cited By (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080219347A1 (en) * 2007-03-07 2008-09-11 Tsuyoshi Nakamura Moving picture coding method, moving picture decoding method, moving picture coding device, and moving picture decoding device
US20100254620A1 (en) * 2008-10-10 2010-10-07 Daisuke Iwahashi Image decoding apparatus and image decoding method
US20100322317A1 (en) * 2008-12-08 2010-12-23 Naoki Yoshimatsu Image decoding apparatus and image decoding method
WO2011126282A3 (en) * 2010-04-05 2012-03-08 Samsung Electronics Co., Ltd. Method and apparatus for encoding video by using transformation index, and method and apparatus for decoding video by using transformation index
US20120207210A1 (en) * 2009-10-13 2012-08-16 Canon Kabushiki Kaisha Method and device for processing a video sequence
US20120320993A1 (en) * 2011-06-14 2012-12-20 Google Inc. Apparatus and method for mitigating the effects of packet loss on digital video streams
CN102934432A (en) * 2010-04-05 2013-02-13 三星电子株式会社 Method and apparatus for encoding video by using transformation index, and method and apparatus for decoding video by using transformation index
US20130343453A1 (en) * 2011-03-10 2013-12-26 Nippon Telegraph And Telephone Corporation Quantization control apparatus and method, and quantization control program
US8819525B1 (en) 2012-06-14 2014-08-26 Google Inc. Error concealment guided robustness
US8838680B1 (en) 2011-02-08 2014-09-16 Google Inc. Buffer objects for web-based configurable pipeline media processing
US8907821B1 (en) 2010-09-16 2014-12-09 Google Inc. Apparatus and method for decoding data
US9042261B2 (en) 2009-09-23 2015-05-26 Google Inc. Method and device for determining a jitter buffer level
US9049450B2 (en) 2010-04-05 2015-06-02 Samsung Electronics Co., Ltd. Method and apparatus for encoding video based on internal bit depth increment, and method and apparatus for decoding video based on internal bit depth increment
US9078015B2 (en) 2010-08-25 2015-07-07 Cable Television Laboratories, Inc. Transport of partially encrypted media
US9113138B2 (en) 2010-01-07 2015-08-18 Kabushiki Kaisha Toshiba Moving-picture encoding apparatus and moving-picture decoding apparatus
US9154799B2 (en) 2011-04-07 2015-10-06 Google Inc. Encoding and decoding motion via image segmentation
US9262670B2 (en) 2012-02-10 2016-02-16 Google Inc. Adaptive region of interest
US9369736B2 (en) 2010-04-05 2016-06-14 Samsung Electronics Co., Ltd. Low complexity entropy-encoding/decoding method and apparatus
US9510019B2 (en) 2012-08-09 2016-11-29 Google Inc. Two-step quantization and coding method and apparatus
US9706205B2 (en) 2011-02-10 2017-07-11 Velos Media, Llc Image processing device and image processing method
RU2627101C2 (en) * 2010-04-09 2017-08-03 Мицубиси Электрик Корпорейшн Moving image coding and decoding devices
US9924161B2 (en) 2008-09-11 2018-03-20 Google Llc System and method for video coding using adaptive segmentation

Families Citing this family (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101366091B1 (en) * 2006-03-28 2014-02-21 삼성전자주식회사 Method and apparatus for encoding and decoding image
US8200959B2 (en) * 2007-06-28 2012-06-12 Cisco Technology, Inc. Verifying cryptographic identity during media session initialization
US8417942B2 (en) * 2007-08-31 2013-04-09 Cisco Technology, Inc. System and method for identifying encrypted conference media traffic
CN101389021B (en) * 2007-09-14 2010-12-22 华为技术有限公司;清华大学 Video encoding/decoding method and apparatus
US8837598B2 (en) * 2007-12-28 2014-09-16 Cisco Technology, Inc. System and method for securely transmitting video over a network
US20090169001A1 (en) * 2007-12-28 2009-07-02 Cisco Technology, Inc. System and Method for Encryption and Secure Transmission of Compressed Media
KR101503269B1 (en) * 2010-04-05 2015-03-17 삼성전자주식회사 Method and apparatus for determining intra prediction mode of image coding unit, and method and apparatus for determining intra predion mode of image decoding unit
JP5832519B2 (en) 2010-04-13 2015-12-16 サムスン エレクトロニクス カンパニー リミテッド Video encoding method and apparatus based on coding units according to a tree structure, as well as a video decoding method and apparatus
JP6056122B2 (en) * 2011-01-24 2017-01-11 ソニー株式会社 The image encoding device and an image decoding apparatus and method and program
US8634474B2 (en) * 2011-05-03 2014-01-21 Texas Instruments Incorporated CABAC macroblock rewind and end of slice creation to control slice size for video encoders
RU2604669C2 (en) 2011-06-28 2016-12-10 Самсунг Электроникс Ко., Лтд. Method and apparatus for predicting chroma components of image using luma components of image
US9451284B2 (en) 2011-10-10 2016-09-20 Qualcomm Incorporated Efficient signaling of reference picture sets
US9510020B2 (en) 2011-10-20 2016-11-29 Qualcomm Incorporated Intra pulse code modulation (IPCM) and lossless coding mode deblocking for video coding
US9930360B2 (en) * 2011-11-08 2018-03-27 Texas Instruments Incorporated Method, system and apparatus for intra-refresh in video signal processing
US9106936B2 (en) 2012-01-25 2015-08-11 Altera Corporation Raw format image data processing
JP5624576B2 (en) * 2012-03-14 2014-11-12 株式会社東芝 Image compression controller and image compression apparatus
US9473684B2 (en) * 2013-06-25 2016-10-18 Canon Kabushiki Kaisha Image processing apparatus, image processing method and program, and imaging apparatus
US9280694B2 (en) 2014-04-10 2016-03-08 Anki, Inc. Decoding machine-readable optical codes with aesthetic component

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6608935B2 (en) * 1998-06-26 2003-08-19 Sony Corporation Picture encoding method and apparatus, picture decoding method and apparatus and furnishing medium
US20040240549A1 (en) * 2003-05-30 2004-12-02 Lsi Logic Corporation Method and/or apparatus for reducing the complexity of H.264 B-frame encoding using selective reconstruction
US6999511B1 (en) * 1999-02-23 2006-02-14 International Business Machines Corporation Dynamically switching quant matrix tables within an MPEG-2 encoder
US20060233251A1 (en) * 2005-04-18 2006-10-19 Samsung Electronics Co., Ltd. Moving picture coding/decoding method and apparatus
US20070230585A1 (en) * 2006-03-28 2007-10-04 Samsung Electronics Co., Ltd. Method, medium, and system encoding and/or decoding an image
US20080089410A1 (en) * 2004-01-30 2008-04-17 Jiuhuai Lu Moving Picture Coding Method And Moving Picture Decoding Method

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6067118A (en) * 1997-12-16 2000-05-23 Philips Electronics North America Corp. Method of frame-by-frame calculation of quantization matrices
KR101136629B1 (en) 2004-01-30 2012-04-23 파나소닉 주식회사 Picture coding and decoding method, apparatus, and program thereof
EP1753242A2 (en) 2005-07-18 2007-02-14 Matsushita Electric Industrial Co., Ltd. Switchable mode and prediction information coding
KR100995226B1 (en) 2005-07-22 2010-11-17 미쓰비시덴키 가부시키가이샤 Image encoding device, image decoding device, image encoding method, image decoding method, computer readable recording medium having image encoding program recorded therein, and computer readable recording medium having image decoding program recorded therein
CN101218831B (en) 2005-09-20 2010-07-21 三菱电机株式会社 Image decoding method and device

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6608935B2 (en) * 1998-06-26 2003-08-19 Sony Corporation Picture encoding method and apparatus, picture decoding method and apparatus and furnishing medium
US6999511B1 (en) * 1999-02-23 2006-02-14 International Business Machines Corporation Dynamically switching quant matrix tables within an MPEG-2 encoder
US20040240549A1 (en) * 2003-05-30 2004-12-02 Lsi Logic Corporation Method and/or apparatus for reducing the complexity of H.264 B-frame encoding using selective reconstruction
US20080089410A1 (en) * 2004-01-30 2008-04-17 Jiuhuai Lu Moving Picture Coding Method And Moving Picture Decoding Method
US20060233251A1 (en) * 2005-04-18 2006-10-19 Samsung Electronics Co., Ltd. Moving picture coding/decoding method and apparatus
US20070230585A1 (en) * 2006-03-28 2007-10-04 Samsung Electronics Co., Ltd. Method, medium, and system encoding and/or decoding an image

Cited By (40)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080219347A1 (en) * 2007-03-07 2008-09-11 Tsuyoshi Nakamura Moving picture coding method, moving picture decoding method, moving picture coding device, and moving picture decoding device
US8300692B2 (en) * 2007-03-07 2012-10-30 Panasonic Corporation Moving picture coding method, moving picture decoding method, moving picture coding device, and moving picture decoding device
US9924161B2 (en) 2008-09-11 2018-03-20 Google Llc System and method for video coding using adaptive segmentation
US20100254620A1 (en) * 2008-10-10 2010-10-07 Daisuke Iwahashi Image decoding apparatus and image decoding method
US20100322317A1 (en) * 2008-12-08 2010-12-23 Naoki Yoshimatsu Image decoding apparatus and image decoding method
US9042261B2 (en) 2009-09-23 2015-05-26 Google Inc. Method and device for determining a jitter buffer level
US20120207210A1 (en) * 2009-10-13 2012-08-16 Canon Kabushiki Kaisha Method and device for processing a video sequence
US9532070B2 (en) * 2009-10-13 2016-12-27 Canon Kabushiki Kaisha Method and device for processing a video sequence
US9392282B2 (en) 2010-01-07 2016-07-12 Kabushiki Kaisha Toshiba Moving-picture encoding apparatus and moving-picture decoding apparatus
US9154779B2 (en) 2010-01-07 2015-10-06 Kabushiki Kaisha Toshiba Moving-picture encoding apparatus and moving-picture decoding apparatus
US9148659B2 (en) 2010-01-07 2015-09-29 Kabushiki Kaisha Toshiba Moving-picture encoding apparatus and moving-picture decoding apparatus
US9143777B2 (en) 2010-01-07 2015-09-22 Kabushiki Kaisha Toshiba Moving-picture encoding apparatus and moving-picture decoding apparatus
US9294768B2 (en) 2010-01-07 2016-03-22 Kabushiki Kaisha Toshiba Moving-picture encoding apparatus and moving-picture decoding apparatus
US9113138B2 (en) 2010-01-07 2015-08-18 Kabushiki Kaisha Toshiba Moving-picture encoding apparatus and moving-picture decoding apparatus
US9113139B2 (en) 2010-01-07 2015-08-18 Kabushiki Kaisha Toshiba Moving-picture encoding apparatus and moving-picture decoding apparatus
US9294781B2 (en) 2010-04-05 2016-03-22 Samsung Electronics Co., Ltd. Method and apparatus for encoding video by using transformation index, and method and apparatus for decoding video by using transformation index
US9049460B1 (en) 2010-04-05 2015-06-02 Samsung Electronics Co., Ltd. Method and apparatus for encoding video by using transformation index, and method and apparatus for decoding video by using transformation index
US9049450B2 (en) 2010-04-05 2015-06-02 Samsung Electronics Co., Ltd. Method and apparatus for encoding video based on internal bit depth increment, and method and apparatus for decoding video based on internal bit depth increment
WO2011126282A3 (en) * 2010-04-05 2012-03-08 Samsung Electronics Co., Ltd. Method and apparatus for encoding video by using transformation index, and method and apparatus for decoding video by using transformation index
US9042460B1 (en) 2010-04-05 2015-05-26 Samsung Electronics Co., Ltd. Method and apparatus for encoding video by using transformation index, and method and apparatus for decoding video by using transformation index
US8982961B2 (en) 2010-04-05 2015-03-17 Samsung Electronics Co., Ltd. Method and apparatus for encoding video by using transformation index, and method and apparatus for decoding video by using transformation index
US9866875B2 (en) 2010-04-05 2018-01-09 Samsung Electronics Co., Ltd. Low complexity entropy-encoding/decoding method and apparatus
US9602845B2 (en) 2010-04-05 2017-03-21 Samsung Electronics Co., Ltd. Low complexity entropy-encoding/decoding method and apparatus
CN102934432A (en) * 2010-04-05 2013-02-13 三星电子株式会社 Method and apparatus for encoding video by using transformation index, and method and apparatus for decoding video by using transformation index
US9369736B2 (en) 2010-04-05 2016-06-14 Samsung Electronics Co., Ltd. Low complexity entropy-encoding/decoding method and apparatus
US9042459B1 (en) 2010-04-05 2015-05-26 Samsung Electronics Co., Ltd. Method and apparatus for encoding video by using transformation index, and method and apparatus for decoding video by using transformation index
RU2627101C2 (en) * 2010-04-09 2017-08-03 Мицубиси Электрик Корпорейшн Moving image coding and decoding devices
US9078015B2 (en) 2010-08-25 2015-07-07 Cable Television Laboratories, Inc. Transport of partially encrypted media
US8907821B1 (en) 2010-09-16 2014-12-09 Google Inc. Apparatus and method for decoding data
US8838680B1 (en) 2011-02-08 2014-09-16 Google Inc. Buffer objects for web-based configurable pipeline media processing
US9967564B2 (en) 2011-02-10 2018-05-08 Velos Media, Llc Image processing device and image processing method
US9706205B2 (en) 2011-02-10 2017-07-11 Velos Media, Llc Image processing device and image processing method
US9986241B2 (en) 2011-02-10 2018-05-29 Velos Media, Llc Image processing device and image processing method
US20130343453A1 (en) * 2011-03-10 2013-12-26 Nippon Telegraph And Telephone Corporation Quantization control apparatus and method, and quantization control program
US9906803B2 (en) * 2011-03-10 2018-02-27 Nippon Telegraph And Telephone Corporation Quantization control apparatus and method, and quantization control program
US9154799B2 (en) 2011-04-07 2015-10-06 Google Inc. Encoding and decoding motion via image segmentation
US20120320993A1 (en) * 2011-06-14 2012-12-20 Google Inc. Apparatus and method for mitigating the effects of packet loss on digital video streams
US9262670B2 (en) 2012-02-10 2016-02-16 Google Inc. Adaptive region of interest
US8819525B1 (en) 2012-06-14 2014-08-26 Google Inc. Error concealment guided robustness
US9510019B2 (en) 2012-08-09 2016-11-29 Google Inc. Two-step quantization and coding method and apparatus

Also Published As

Publication number Publication date Type
US8718139B2 (en) 2014-05-06 grant
CA2615299C (en) 2013-12-10 grant
CA2615299A1 (en) 2008-07-12 application
US20080170627A1 (en) 2008-07-17 application
EP1947860A2 (en) 2008-07-23 application
EP1947860B1 (en) 2017-03-01 grant
EP1947860A3 (en) 2010-08-18 application
US20080170624A1 (en) 2008-07-17 application

Similar Documents

Publication Publication Date Title
US7324595B2 (en) Method and/or apparatus for reducing the complexity of non-reference frame encoding using selective reconstruction
US20060197777A1 (en) Color space scalable video coding and decoding method and apparatus for the same
US20060078048A1 (en) Deblocking filter
US20120106652A1 (en) Apparatus and Method for High Efficiency Video Coding Using Flexible Slice Structure
US20020122491A1 (en) Video decoder architecture and method for using same
US20090190655A1 (en) Moving picture encoding apparatus
EP1909508A1 (en) Image encoding device, image decoding device, image encoding method, image decoding method, image encoding program, image decoding program, computer readable recording medium having image encoding program recorded therein, and computer readable recording medium having image decoding program recorded
US20040258162A1 (en) Systems and methods for encoding and decoding video data in parallel
US20100014763A1 (en) Image coding apparatus and image decoding apparatus
US20070160299A1 (en) Moving image coding apparatus, moving image decoding apparatus, control method therefor, computer program, and computer-readable storage medium
US20080123977A1 (en) Image encoder and image decoder, image encoding method and image decoding method, image encoding program and image decoding program, and computer readable recording medium recorded with image encoding program and computer readable recording medium recorded with image decoding program
US20130272430A1 (en) Constraints and unit types to simplify video random access
US20080123947A1 (en) Image encoding device, image decoding device, image encoding method, image decoding method, image encoding program, image decoding program, computer readable recording medium having image encoding program recorded therein
US20090123066A1 (en) Image encoding device, image decoding device, image encoding method, image decoding method, image encoding program, image decoding program, computer readable recording medium having image encoding program recorded therein,
US20090034857A1 (en) Image encoder and image decoder, image encoding method and image decoding method, image encoding program and image decoding program, and computer readable recording medium recorded with image encoding program and computer readable recording medium recorded with image decoding program
US20080165849A1 (en) Image encoder and image decoder, image encoding method and image decoding method, image encoding program and image decoding program, and computer readable recording medium recorded with image encoding program and computer readable recording medium recorded with image decoding program
US20090003717A1 (en) Image encoding device, image decoding device, image encoding method and image decoding method
US20080137744A1 (en) Image encoder and image decoder, image encoding method and image decoding method, image encoding program and image decoding program, and computer readable recording medium recorded with image encoding program and computer readable recording medium recorded with image decoding program
US20080130990A1 (en) Image encoder and image decoder, image encoding method and image decoding method, image encoding program and image decoding program, and computer readable recording medium recorded with image encoding program and computer readable recording medium recorded with image decoding program
US8509551B2 (en) Image encoder and image decoder, image encoding method and image decoding method, image encoding program and image decoding program, and computer readable recording medium recording with image encoding program and computer readable recording medium recorded with image decoding program
US20080123736A1 (en) Image encoding method and image decoding method, image encoder and image decoder, and image encoded bit stream and recording medium
WO2000033583A1 (en) Efficient macroblock header coding for video compression
EP1906676A1 (en) Moving picture coding apparatus, moving picture decoding apparatus, and stream data
US20080130747A1 (en) Image encoder and image decoder, image encoding method and image decoding method, image encoding program and image decoding program, and computer readable recording medium recorded with image encoding program and computer readable recording medium recorded with image decoding program
US20080130989A1 (en) Image encoder and image decoder, image encoding method and image decoding method, image encoding program and image decoding program, and computer readable recording medium recorded with image encoding program and computer readable recording medium recorded with image decoding program

Legal Events

Date Code Title Description
AS Assignment

Owner name: MITSUBISHI ELECTRIC CORPORATION, JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:YAMADA, YOSHIHISA;SEKIGUCHI, SHUNICHI;MORIYA, YOSHIMI;AND OTHERS;REEL/FRAME:020345/0156;SIGNING DATES FROM 20071101 TO 20071125