BRPI0306560B1 - IMAGE CODING DEVICE, IMAGE DECODING DEVICE AND METHODS - Google Patents

Description

(54) Title: IMAGE ENCODING EQUIPMENT, IMAGE DECODING EQUIPMENT AND METHODS (51) lnt.CI .: H04N 19/112 (30) Unionist Priority: 01/10/2002 JP 2002-289303 (73) Holder ( es): GODO KAISHA IP BRIDGE 1 (72) Inventor (s): SHINYA KADONO; SATOSHI KONDO; KIYOFUMI ABE (85) National Phase Start Date: 06/01/2004 ···· ·· • · · · • · ··· ·· ···· ··

Descriptive Report of the Invention Patent for IMAGE ENCODING EQUIPMENT, IMAGE DECODING EQUIPMENT AND METHODS.

TECHNICAL FIELD

The present invention relates to an encoding apparatus and a decoding apparatus for encoding and decoding moving images, in particular an image decoding apparatus and an image decoding apparatus for performing a motion estimate using factors of motion. weighting and its methods, v 10 FUNDAMENTALS OF TECHNIQUE

Recently, with the arrival of the multimedia era, which fully handles audio, image, and other content and the like, it is

It is now possible to obtain and transmit the information carried by existing information media, that is, newspapers, periodicals, TVs, radios and telephones and other means using a single terminal. Generally speaking, multimedia refers to something that is represented in association not only with characters, but also with graphics, audio and especially images and the like together. However, in order to include the existing information media mentioned above in the scope of multimedia, 20 it appears as a prerequisite to represent such information in digital form.

However, when estimating the amount of information contained in each of the aforementioned media as the amount of digital information, the amount of information per character requires 1 = 2 bytes, while the audio requires more than 64 Kbits (phone quality ) per second and when it comes to mobile images, it requires more than 100 Mbits (current TV reception quality). Therefore, it is not feasible to manipulate the vast information directly in digital form through the information media mentioned above. For example, a videophone has already been • put to practical use through the Integrated Services Digital Network „30 (ISDN) with a transmission rate of 64 Kbps = 1.5 Mbps, however, it is not practical to transmit the mobile image captured in TV screen or recorded by a TV camera.

• ·· ·· »· · ··· ··· 4 • · · · • · · · · ···· · · · · · · · · · · · · · ····

This therefore requires information compression techniques, and for example, mobile image compression techniques that meet the H.261 and H.263 standards internationally standardized by the ITU-T (International Telecommunication Union, Telecommunication Standardization Sector) are used in the case of the videophone. According to information compression techniques that meet the MPEG-1 standard, image information as well as music information can be stored on a standard music CD (Compact Disc).

MPEG (Moving Picture Experts Group) is an international standard for compressing mobile image signals and MPEG-1 is a standard that compresses mobile image signals to 1.5 Mbps, that is, compressing information from mobile signals. TV in approximately one hundredth. The transmission rate within the scope of the MPEG-1 standard is limited primarily to approximately 1.5 Mbps, therefore MPEG-2, which has been standardized with the intention that the needs of high quality images, allows for data transmission of mobile image signals at a rate of 2 = 15 Mbps. In the present circumstances, a working group (ISO / IEC JTC1 / SC29 / WG 11) that commanded the standardization of MPEG-1 and MPEG-2 standardized MPEG-4 which achieves a compression rate that goes beyond that achieved by MPEG -1 and MPEG-2, performs the encoding / decoding operations on an object basis as well as a new function required by the multimedia era (see reference, for example, to the specifications of MPEG-1, MPEG-2 and MPEG-4 produced by ISO). MPEG-4 not only performs a highly efficient encoding method for a low bit rate, but it also introduces powerful error resistance techniques that can minimize a degradation of screen quality even when an error is encountered in a transmission line. . Also, ISO / IEC and ITU work together to standardize MPEG-4 AVC / ITU H.264 as the next generation image encoding method.

The encoding of mobile images, in general, compresses the volume of information by reducing redundancy both in the temporal direction

··· ··· ··· ·· ···· ·· · «• · · · · · · • · · · · ··· • · · · · · · · · · · · · · · · · ·· ♦ »· ·· ·· ·« ···· · «·· ··· ··· · • · · • · · · · ·« ·· · as in space. Therefore, inter-image forecast coding, which aims to reduce temporal redundancy, estimates a movement and generates a predictive image on a block-by-block basis with reference to the previous and subsequent images compared to a current image to be encoded, and then encodes a differential value between the predictive image obtained and the current image. Here, the term image represents a single canvas, while it represents a frame when used in a progressive image context as well as a frame or field in the context of an interlaced image. The interlaced image here is an image in which a single frame consists of two fields that have different times. In the process of encoding and decoding the interlaced image, three modes are possible: manipulating a single frame or as a frame, as two fields or as a frame structure or a field structure depending on a block in the frame.

Figure 1 is a diagram showing an example of image types and how the images refer to each other. The hatched images in Figure 1 are images to be stored in a memory as these are referred to by other images. As for the arrows used in Figure 1, the arrowhead points to a reference image that moves away from an image that refers to the reference image. Here, the images are in order of presentation.

IO (Image 0) is an intracoded image (Image I) which is encoded independently from other images (namely without referring to other images). P4 (Image 4) and P7 (Image 7) are advanced prediction encoded images (P image) that are predictively encoded with reference to images I located temporally before the current image or other P images. Β1 ~ B3 ~ (images 1 ~ 3), B5 (Image 5) and B6 (Image 6) are encoded images of bidirectional prediction (image B) that are predictively encoded with reference to other images both temporally before and subsequent to the current image.

Figure 2 is a diagram showing another example of the types of images and how the images relate to each other. The difference between

between Figure 2 and Figure 1 is that a temporal position of the images referred to by a B image is not limited to the images that are temporally located before and after the B image. For example, B5 can refer to two arbitrary IO images ( Image 0), P3 (Image 3) and P6 (Image 6). Namely, IO and P3, located temporally before, can be used as reference images. Such a reference method has been confirmed in the MPEG-4 AVC / H.264 specification since September 2001. Thus, a range for selecting an optimal predictive image is extended and the compression ratio can be improved through this.

Figure 3 is a diagram showing an example of an image data flow structure. As shown in Figure 3, the flow includes a common information area such as a header or the like and a GOP (Image Group) area. The GOP area includes a common information area such as a header or the like and a plurality of image areas. The image area includes a common information area such as a header or the like and a plurality of section data areas. The section data area includes a common information area such as a header and a plurality of macroblock data areas.

In the common image information area, the weighting factor required to perform a weighted prediction to be mentioned later is described according to the reference image respectively.

When transmitting data not in a bit stream that has successive streams, but in a packet that is a unit consisting of pieces of data, the header part and the data part which excludes the header part can be transmitted separately . In this case, the header part and the data part may not be included in a single bit stream. In the case of using a packet, however, even when the header part and the data part are not transmitted in sequence, the data part and the header part are transmitted respectively in a different packet. Although these are not transmitted in a stream of

bits, the concept is the same as when using a bit stream as described in Figure 3.

The following describes the weighted prediction processing performed by the conventional image encoding method.

Figures 4A and 4B are standard diagrams showing cases of weighted prediction execution on a frame-by-frame basis.

When referring to a single frame, as shown in Figure 4A, a pixel Q value in a predictive image in relation to a current block to be encoded can be calculated using a weighted prediction equation as shown in equation (1) below, where a pixel value within a reference block in the 1st reference frame number, Frame i, is represented as P0. When referring to two frames, as shown in Figure 4B, the pixel value Q in the predictive image can be calculated using an equation for weighted prediction as shown in equation (2) below, where the respective pixel values within the reference blocks in ^Q i and j ^are numbers of reference frames, Frame i and Frame j a, are represented as P0 and P1.

Q = (P0xW0 + D) / W2 Q = (P0xW0 + P1xW1 + D) / W2 (1) (2)

Here, WO and W1 represent weighting factors while W2 represents a normalization factor and D represents a polarized component (CG component).

Figures 5A and 5B are standard diagrams showing cases of weighted prediction processing on a field by field basis.

When referring to a single frame (namely, two fields) as shown in Figure 5A, the pixel values Qa and Qb in the predictive images in relation to a current block can be calculated using the equations for weighted prediction as shown in the equations (3 ) and (4) below, where the pixel values within the respective reference blocks in the respective fields of 2xi + 1 and 2xi, composing the i ° frame number (Table i) which is for reference, are represented as POa and POb .

• · · · · · · · · · ·· · · ··. · · ····

When referring to two frames, as shown in Figure 5B, the pixel values Qa and Qb can be calculated by using the equations for weighted prediction as shown in equations (5) and (6) below, where the pixel values within the respective reference blocks in the 2xi + 1 field,

2xi, 2xj + 1 and 2xj, composing the i ° and j ° number of frames (Table i and Table j) are represented respectively as POa, POb, P1a and P1b.

Qa = (P0axW0a + Da) / W2a (3)

Qb = (P0bxW0b + Db) / W2b (4)

Qa = (P0axW0a + P1 axW1 to + Da) / W2a (5)

Qb = (P0bxW0b + P1 bxW1 b + Db) / W2b (6)

Here, WOa, WOb, W1a and W1b represent weighting factors while W2 represents a normalization factor and Da and Db represent polarized components.

Figure 6 is a block diagram showing a functional structure 15 of a conventional image encoding apparatus 100. Image encoding apparatus 100 performs a compression encoding (for example, a variable length encoding) for an image signal. inserted image Vin and outputs an encoded Str image signal which is a bit stream converted by compression encoding, and includes a motion estimation unit ME, a motion compensation unit MC, a subtraction unit Sub, a unit of motion orthogonal transformation T, a quantization unit Q, an inverse quantization unit IQ, an inverse orthogonal transformation unit IT, an Add addition unit, a PicMem image memory, a SW key and a VLC variable length coding unit.

The image signal Vin is inserted into the subtraction unit Sub and the motion estimation unit ΜΕ. The subtraction unit Sub calculates a differential value between the inserted image signal Vin and the predictive image and outputs the result to the orthogonal transformation unit T. The orthogonal transformation unit T turns the differential value into a frequency coefficient and then outputs -o for the quantization unit Q. The quantization unit Q quantizes the entered frequency coefficient and

issues a quantized value for the VLC variable-length coding unit.

The inverse quantization unit IQ reconstructs the quantized value as a frequency coefficient by inversely quantizing it and outputs it to the IT inverse orthogonal transformation unit. The IT inverse orthogonal transformation unit performs an inverse frequency conversion in the frequency coefficient in order to obtain a differential pixel value and output it to the Add addition unit. The Add addition unit adds the differential pixel value to the image predictive signal emitted from the MC motion compensation unit and obtains a decoded image. The SW switch is ON when instructed to store the decoded image, and the decoded image is stored in the PicMem image memory.

The ME motion estimation unit, in which the image signal Vin is inserted is a macroblock by macroblock basis, determines the decoded images stored in the PicMem image memory for search, and by estimating an image area according to the image signal that is closest to the inserted image signal, determines a motion vector MV that indicates the area. The motion vector estimate is operated using a block that is a unit made by further dividing a macroblock. As multiple images can be used as reference images, identification numbers (image number index) to identify the images used for reference are required for each block. It is thus possible to identify the reference images by matching the image numbers assigned to each of the images in the PicMem image memory with the reference images using the Index image number.

The MC motion compensation unit removes an image area necessary to generate a predictive image of a decoded image stored in the PicMem image memory using the Index image number. The MC motion compensation unit then determines a final predictive image obtained by running, for the pixel values in the image area obtained, a conversion processing • · ·· · ·· · · • · ·· · ·· · · ··· ·· · ·· ·· • · · · · · · · · · · · · ·

pixel value such as interpolation processing operated on weighted prediction using the weighting factors associated with the Index image number.

Figure 7 is a block diagram showing an outline of a functional structure of the variable length coding unit VLC in the conventional image coding apparatus 100 shown in Figure 6. The variable length coding unit VLC includes a coding unit of MV 101, a quantized value coding unit 102, a weighting factor coding unit 103, an index coding unit 104, an AFF (Adaptive Field Frame) identification information coding unit 105 and a unit multiplexing 106.

The MV encoding unit 101 encodes a motion vector, while the quantized value encoding unit 102 encodes a quantized value Qcoef. The weight factor encoding unit 103 encodes a Weight weighting factor, while the index encoding unit 104 encodes an Index image number. The AFF identification information encoding unit 105 encodes an AFF identification signal (the AFF identification signal (AFF) will be mentioned later). The multiplexing unit 106 multiplexes each of the coded signals emitted from the MV 101 coding unit, the quantized value coding unit 102, the weighting factor coding unit 103, the index coding unit 104, and the unit encoding identification information AFF 105 and then outputs an encoded Str image signal.

Figure 8 is a block diagram showing a functional structure of a conventional image decoding apparatus 200.

The image decoding apparatus 200 for decoding the encoded image signal Str, which has been encoded by the image encoding apparatus 100 described above includes a variable length decoding unit VLD, a motion compensation unit MC, an addition unit Add, a PicMem image memory, a • · · · · · • ··· ··· ··· · · ·· · ·· ·· ·

inverse quantization unit IQ and an inverse orthogonal transformation unit IT.

When the encoded Str image signal is inserted, the VLD variable length decoding unit demultiplexes the encoded Str image signal inserted into a differential motion vector MV that is encoded, an index indicating an image number and a weighting factor Weight and send them to the MC motion compensation unit. The variable length decoding unit VLD then decodes the quantized encoded Qcoef value included in the inserted encoded Str image signal and outputs it to the inverse quantization unit IQ.

The MC motion compensation unit removes an image area needed to generate a predictive image of a decoded image stored in the PicMem image memory using the motion vector and the Index image number which are output from the variable length decoding unit VLD. The MC motion compensation unit then generates a predictive image by performing pixel value conversion processing such as interpolation processing in the weighted prediction using the Weighting factor for the obtained image.

The inverse quantization unit IQ quantizes the quantized value in reverse and reconstructs it as a frequency coefficient and sends it to the inverse orthogonal transformation unit IT. The IT inverse orthogonal transformation unit performs an inverse frequency conversion in the frequency coefficient in order to obtain a differential pixel value and output it to the Add addition unit. The Add addition unit adds the differential pixel value to the image predictive signal emitted from the MC motion compensation unit and obtains a decoded image. The decoded image is stored in the PicMem image memory to be used for reference in inter-image prediction. The decoded image is output as a Vout decoded image signal.

Figure 9 is a block diagram showing an outline of a functional structure of a length decoding unit

variable VLD in the conventional image decoding device 200 shown in Figure 8.

The variable length decoding unit VLD includes a demultiplexing unit 201, an MV decoding unit 202, a quantized value decoding unit 203, a weighting factor decoding unit 204, an index decoding unit 205, and an AFF 206 identification signal decoding unit.

When the encoded Str image signal is inserted into the variable length decoding unit VLD the demultiplexing unit 201 demultiplexes the inserted encoded image signal Str and emits respectively as follows: the encoded differential motion vector MV for the MV decoding unit 202; the quantized value Qcoef encoded for the quantized value decoding unit 203; the weighting factor Weight encoded for the weighting factor decoding unit 204; the encoded image number for the index decoding unit 205 and the encoded AFF identification signal (AFF) (abbreviated as AFF in the following description) for the AFF identification signal decoding unit 206.

The MV decoding unit 202 decodes the encoded differential vector and outputs a motion vector MV.

Similarly, the quantized value decoding unit 203 decodes the quantized value, the weighting factor decoding unit 204 decodes the Weighting factor, the index decoding unit 205 decodes the number of the Index image figure and the decoding the AFF 206 identification signal decodes the AFF respectively and then outputs them.

Conventional coding using weighted prediction, however, is performed on an image-by-image basis assuming that a block is encoded / decoded for the same image (a frame or one of the two fields). Therefore, only a set of weighting factors can be encoded / decoded in the image.

Therefore, although the conventional image coding apparatus has the potential to improve the efficiency in motion estimation, only a single weighting factor can be transmitted on a block-by-block basis thereby the prediction efficiency is low even when the field / frame change takes place on a block-by-block basis, and thereby the compression ratio cannot be improved. DESCRIPTION OF THE INVENTION

The present invention was conceived in view of the aforementioned circumstances and aims to provide an image encoding / decoding method that can handle the weighting factors appropriately even when the field / frame change takes place on a block-by-block basis.

In order to achieve the above objective, the image encoding apparatus according to the present invention encodes an interlaced image on a block-by-block basis, and comprises: a storage unit operable to store an image that is either a frame or a decoded field after being encoded, as a reference image; a predictive imaging unit operable to read the reference image from the storage unit and generate a predictive image based on pixel values in the reference image, using one of i) a frame weighting factor to encode the interlaced image on a frame by frame basis and ii) a field weighting factor to encode the interlaced image on a field by field basis; a signal encoding unit operable to encode, on a block-by-block basis, a differential value between an inserted image and the predictive image generated by the predictive imaging unit, either on a frame-by-frame basis or on a basis field by field; an operable weighting factor encoding unit to encode the frame weighting factor taken from the frame weighting factor and a field weighting factor, when the signal encoding unit encodes the differential value on a block basis by adaptive block or on a frame-by-frame basis or on a field-based basis

field; and an operable multiplexing unit for multiplexing the differential value encoded by the signal encoding unit as well as the frame weighting factor encoded by the weighting factor encoding unit and outputting the multiplexed differential value and the frame weighting factor, as an encoded signal.

Consequently, the image encoding apparatus according to the present invention abbreviates the field weighting factor, encodes only a frame weighting factor and transmits it to an image decoding device, regardless of whether the frame / field change is performed or not on a block-by-block basis when performing a weighted prediction for a moving image. Therefore, transmission efficiency can be improved.

In order to achieve the above objective, the image decoding apparatus according to the present invention decodes, on a block-by-block basis, a signal encoded according to an image which is either a single frame or a single field, and comprises: a signal decoding unit operable to decode the encoded signal either on a frame-by-frame basis or on a field-by-field basis, when the encoded signal is encoded changing adaptably between the frame-by-frame base and the base field by field; a storage unit operable to store at least one decoded image; a predictive image generation unit operable to extract, from the encoded signal, a frame weighting factor to decode the encoded signal on the frame by frame basis, generate a field weighting factor to decode the encoded signal on the field base by field, based on the frame weighting factor, and generate a predictive image based on the pixel values in the decoded image stored in the storage unit, using the extracted frame weighting factor and the generated field weighting factor, when the encoded signal is encoded changing adaptably between the frame base by frame and the field base by field; and an operable addition unit to add the image obtained in the decoding performed by the

• · · · · · · · ··· ··· · ♦ · ♦ ·· · · · ·· · ·

decoding the signal in the predictive image generated by the predictive image generation unit, outputting the added image as a decoded image, and storing the decoded image in the storage unit.

Consequently, the image decoding apparatus according to the present invention generates the field weighting factor based on the frame weighting factor even when the change of frame / field on a block-by-block basis takes place and the weighting factor field is not transmitted. This performs adaptive frame / field change on a block-by-block basis and improves transmission efficiency.

In order to achieve the above objective, the image encoding method according to the present invention encodes an interlaced image inserted with reference to at least one decoded image, and comprises the steps of: generating a predictive image using a weighted prediction equation by predetermined weighting factors, with reference to the decoded image: generate a first signal encoded by encoding a differential image between the inserted interlaced image and the predictive image adaptively either on a field by field basis or on a frame by frame basis; generating a decoded image by decoding said encoded signal and adding the decoded encoded signal to the differential image; and generating a second coded signal by coding the predetermined weighting factors in the respective modes, on a field-by-field basis or on a frame-by-frame basis, when the differential image between the inserted interlaced image and the predictive image is adaptively encoded or on the basis frame by frame or field by field basis.

The weighting factors operated on a field by field basis can be the weighting factors for both a first field and a second field.

In order to achieve the above objective, the image encoding method according to the present invention encodes an inter14 • · * ♦ ···· · · · • «· · · · ····· ··· · • · ·· 4 · · · * · · · ··

loop inserted with reference to at least one decoded image, and comprises the steps of: generating a predictive image using a prediction equation weighted by predetermined weighting factors, with reference to the decoded image; generating a first signal encoded by the adaptive encoding of a differential image between the inserted interlaced image and the predictive image either on a frame-by-frame basis or on a field-by-field basis; generate a second coded signal to encode the identification information that indicates whether to encode the predetermined weighting factors either on the field-by-field basis or on the frame-by-frame basis or to encode the predetermined weighting factors or on the field-by-field basis or frame by frame basis; and generating a third signal encoded by encoding the predetermined weighting factors according to the identification information.

In order to achieve the above objective, the image decoding method according to the present invention decodes an encoded signal, which is an encoded inserted interlaced image, with reference to at least one decoded image, and comprises, when the interlaced image inserted is adaptively encoded either on a frame by frame basis or on a field by field basis, the steps of: obtaining the weighting factors encoded on a field by field basis or on a frame by frame basis by decoding the signal encoded; generate a predictive image using a prediction equation weighted by weighting factors, with reference to the decoded image25; generate a differential image by decoding the encoded signal either on a frame by frame basis or on a field by field basis; and generate a decoded image by adding the predictive image to the differential image.

The weighting factors encoded on a field by field basis can be the weighting factors for both a first field and a second field.

In order to achieve the above objective, the decoding method15

image processing according to the present invention decodes an encoded signal, which is an encoded inserted interlaced image, with reference to at least one decoded image, and comprises, when a differential image between the inserted interlaced image and the predictive image is encoded adaptively or on a frame-by-frame basis or on a field-by-field basis, the steps of: obtaining identification information that indicates whether to decode the encoded signal on either a field-by-field basis or a frame-by-frame basis either on a field-by-field basis or on a frame-by-frame basis; obtain both weighting factors by decoding the encoded signal, when the identification information obtained indicates that the weighting factors are decoded in both modes, on a field by field basis or on a frame by frame basis; estimate one of the coded weighting factors based on the weighting factor, which is another weighting factor decoded using the coded signal, when the identifying information indicates that the weighting factors are decoded in any way, or on a field basis per field or on a frame-by-frame basis; generate a predictive image using a prediction equation weighted by weighting factors, with reference to the decoded image; generate a differential image by decoding the encoded signal either on a field by field basis or on a frame by frame basis; and generate a decoded image by adding the differential image to the predictive image.

In order to achieve the above objective, the image encoding apparatus according to the present invention encodes an interlaced image inserted with reference to at least one decoded image, and comprises: an operable unit for generating a predictive image using a prediction equation weighted by predetermined weighting factors, with reference to the decoded image; a unit operable to generate a first signal encoded by the adaptive encoding of a differential image between the inserted interlaced image and the predictive image either on a frame-by-frame basis or on a basis ·· • ··· • · ··· ·· · · • · · ·

field by field; an operable unit for generating a decoded image by decoding the encoded signal and adding the decoded encoded signal to the differential image; a unit operable to generate a second signal encoded by coding the predetermined weighting factors in respective modes, on a field by field basis and on a frame by frame basis, when the differential image between the inserted interlaced image and the predictive image is adaptively coded either on a frame-by-frame basis or on a field-by-field basis.

In order to achieve the above objective, an image encoding apparatus according to the present invention encodes an interlaced image inserted with reference to at least one decoded image, and comprises: an operable unit for generating a predictive image using a prediction equation weighted by predetermined weighting factors, with reference to the decoded image; a unit operable to generate a first signal encoded by the adaptive encoding of a differential image between the inserted interlaced image and the predictive image either on a frame by frame basis or on a field by field basis; an operable unit for generating a decoded image by decoding the encoded signal and adding the decoded encoded signal to the differential image; a unit operable to generate a second coded signal to generate the identification information that indicates whether to encode the predetermined weighting factors either on the field-by-field basis or on the frame-by-field basis or on the field-by-field basis or on the basis of the frame per frame; and an operable unit for generating a third signal encoded by encoding the predetermined weighting factors according to the identification information.

In order to achieve the above objective, the image decoding apparatus according to the present invention decodes an encoded signal, which is an encoded inserted interlaced image, with reference to at least one decoded image, and comprises, when the interlaced image inserted is adaptively coded either on a frame-by-frame basis or on a field-by-field basis: an operating unit

• · • · · · · · · · · · · · · · · · · · · · · · · · · · · · ····························vel on a field-by-field basis or on a frame-by-frame basis, by decoding the encoded signal; an operable unit to generate a predictive image using a prediction equation weighted by weighting factors, with reference to the decoded image; a unit operable to generate a differential image by decoding the encoded signal either on a frame by frame basis or on a field by field basis and generating a decoded image by adding the differential image to the predictive image.

In order to achieve the above objective, the image decoding apparatus according to the present invention decodes an encoded signal, which is an encoded inserted interlaced image, with reference to at least one decoded image, and comprises, when the interlaced image inserted is adaptively coded either on a frame-by-frame basis or on a field-by-field basis: an operable unit for obtaining identification information that indicates whether to decode the weighting factors in both modes, on a field-by-field basis how much and on a frame by frame basis, decode the weighting factors in any way, on a field by field basis or on a frame by frame basis; an operable unit to obtain both weighting factors when the identification information obtained indicates that the weighting factors must be decoded in both modes, on a field by field basis and on a frame by frame basis; an operable unit to estimate one of the coded weighting factors based on the weighting factor, which is another weighting factor decoded using the coded signal, and generating a predictive image using a weighting factor weighted prediction equation, with reference to decoded image, when the identification information obtained indicates that the weighting factors must be decoded in any way, on a field by field basis or on a frame by frame basis; an operable unit for generating a differential image by decoding the encoded signal either on a frame-by-frame basis or on a base • · ···· ·· • ·

field by field; and an operable unit for generating a decoded image by adding the differential image to the predictive image.

In order to achieve the above objective, the present invention can be carried out as an image encoding method and / or an image decoding method that has the characteristic composition units included in each of the above devices as steps and also as a program that includes all the steps included in these methods. The program can be stored on a ROM included in a device with which the above methods can be performed as well as distributed via a storage medium such as a CD-ROM or similar and a transmission medium such as a communication network or similar.

As for additional information on the technical basis of this application, Japanese Patent Application Number 2002-289303 filed on October 1, 2002, is incorporated here by reference.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other objectives, advantages and characteristics of the invention will be apparent from the following description taken in conjunction with the accompanying drawings that illustrate a specific embodiment of the invention. In the drawings:

Figure 1 is a diagram showing an example of image types and their reference relationship.

Figure 2 is a diagram showing another example of the types of images and their reference relationship.

Figure 3 is a diagram showing an example of an image data flow structure.

Figure 4A is a pattern diagram for performing weighted prediction processing with reference to a single frame.

Figure 4B is a pattern diagram for performing weighted prediction processing with reference to two frames.

Figure 5A is a pattern diagram for performing weighted prediction processing with reference to a first or second field that corresponds to a predictive image in relation to a • · • · · · · · · ·

• · · · · · · · · · · · · · · · · · · · ···· Current image to be encoded.

Figure 5B is a pattern diagram for performing weighted prediction processing with reference to both the first and second fields that correspond to the predictive image.

Figure 6 is a block diagram showing a functional structure of a conventional image encoding device.

Figure 7 is a block diagram showing an outline of a functional structure of a variable length coding unit in the conventional image coding apparatus.

Figure 8 is a block diagram showing a functional structure of a conventional image decoding device.

Figure 9 is a block diagram showing an outline of a functional structure of a variable length encoding unit in the conventional image decoding apparatus.

Figure 10 is a block diagram showing an outline of a functional structure of a variable length coding unit according to a first embodiment.

Figure 11 is a block diagram showing an outline of a functional structure of a variable length decoding unit according to the first embodiment.

Figure 12A is a detailed example of a header data structure included in a common information area in an image area according to the first embodiment.

Figure 12B is an example of a case in which only one field weighting factor is transmitted as an image weighting factor, which has no AFF.

Figure 12C is an example of a case in which the field and frame cannot be changed on a block-by-block basis since the picture frame encoding information indicates 1 and AFF indicates 0.

Figure 13 is a flowchart showing an encoding processing sequence in relation to the weighting factors operated • ·

• · · · · · by a variable length decoding unit when the picture frame encoding information indicates 1 and an image is encoded on a frame by frame basis, according to the first modality.

Figure 14A is a detailed example of a header data structure included in a common information area in an image area according to a variation of the first modality.

Figure 14B is an example of a case in which only one field weighting factor is transmitted as an image weighting factor, which has no AFF since the image frame encoding information indicates 0 which always determines image code. field.

Figure 14C is an example of a case in which a field and frame cannot be changed on a block-by-block basis since the picture frame encoding information indicates 1 and AFF indicates * 15 ca0.

Figure 15 is a flowchart showing an encoding processing sequence in relation to the weighting factors operated by a variable length decoding unit when the picture frame coding information indicates 1 and an image is encoded on a frame basis by according to the variation of the first modality.

Figure 16 is a flowchart showing an encoding processing sequence in relation to the weighting factors when the picture frame encoding information operated by the variable length encoding unit25 shown in Figure 11 indicates 1 and an image is encoded in a frame by frame. Figure 17A is a detailed example of a header data structure included in a common information area in an image area according to a second modality, in which the AFF is set to

1 and the field factor presence / absence information is set to 1.

Figure 17B is a diagram similar to Figure 17A in which the AFF

• ·· · • · · · · · · ··· ··; · · · · ·; · · · · · · · · · ·.

• · · · · · is set to 1 and the field factor presence / absence information is set to 0.

Figure 17C is an example in which the field / frame change does not happen on a block-by-block basis since the AFF is set to 0.

Figure 18 is a flowchart showing a coding processing sequence in relation to the weighting factors operated by the variable length coding unit according to the second modality.

Figure 19 is a flowchart showing a decoding processing sequence in relation to the weighting factors operated by the variable length decoding unit according to the second modality.

Figure 20A is a diagram showing an example of an image area data structure, in which the AFF is set to 1 and the frame factor presence / absence information is set to 1, according to a third modality .

Figure 20B is a diagram similar to Figure 20A in which the AFF is set to 1 and the frame factor presence / absence information is set to 0.

Figure 20C is an example in which the field / frame change does not happen on a block-by-block basis since the AFF is set to 0.

Figure 21 is a flowchart showing a coding processing sequence in relation to the weighting factors operated by a variable length coding unit according to the third modality.

Figure 22 is a flowchart showing a decoding processing sequence in relation to the weighting factors operated by a variable length decoding unit according to the third modality.

Figures 23A ~ 23C are illustrations for performing the • · ···· · · method

• image encoding and the image decoding method according to the first, second and third modes using a program recorded on a recording medium such as a floppy disk.

Figure 23A is an illustration showing a physical format of the floppy disk which is a main body of the recording medium.

Figure 23B is an illustration showing the total appearance of the flexible disk, a cross-sectional structure and the flexible disk itself.

Figure 23C is an illustration showing a structure for recording / playing the program on the FD floppy disk.

Figure 24 is a block diagram showing a complete structure of a content delivery system to perform a content delivery service.

Figure 25 is a diagram showing an example of a cell phone.

Figure 26 is a block diagram showing an internal cell phone structure.

Figure 27 is a block diagram showing the complete structure of a digital transmission system.

BEST WAY TO PERFORM THE INVENTION

The following describes the embodiments according to the present invention in detail with reference to the diagrams.

FIRST MODE

The functional structure of the image coding apparatus for carrying out the image coding method according to the present embodiment is the same as that of the conventional image coding apparatus 100 mentioned above, except for the variable length coding unit VLC. Similarly, the functional structure of the image decoding apparatus for carrying out the image decoding method according to the present modality is the same as that of the conventional image decoding apparatus 200 mentioned above, except for the variable length decoding unit VLD .

Therefore, the following focuses mainly on descriptions of

4Í \ ί '* .í ....... · v f

.....} a VLC variable length encoding unit and a VLD variable length decoding unit which are different from conventional ones.

Figure 10 is a block diagram showing an outline of a functional structure of the variable length coding unit according to the present embodiment. As shown in Figure 10, the VLC variable-length coding unit includes an MV 101 coding unit, a quantized value coding unit 102, a field weighting factor coding unit 11, a factor coding unit weighting frame 12, an index encoding unit 104, a weighting factor mode determination unit 13, an AFF identification information encoding unit 105, keys 14, 15 and a multiplexing unit 106. The the same reference identifications are placed for the same functional structures as those of the conventional variable length coding unit VLC, and the explanation is hereby abbreviated.

Keys 14 and 15 control ON / OFF by determining the destination of the Weighted weighting factor entered, or for the field weighting factor coding unit 11 or for the frame weighting factor coding unit 12, based on result of the determination made by the weighting factor mode determination unit 13.

The field weighting factor coding unit 11 encodes the Weighted weighting factor as a field weighting factor while the frame weighting factor coding unit 12 encodes it as a frame weighting factor.

The weighting factor 13 determination unit performs frame / field determination based on the AFF value and that of the Weighting factor Weight and then reports keys 14, 15 and multiplexing unit 106 of the determination result.

Figure 11 is a block diagram showing an outline of

• ·· ·· ··· * · ·· ·· a functional structure of the variable length decoding unit VLD according to the present modality. As shown in Figure 11, the VLD variable length decoding unit includes a demultiplexing unit 21, an MV decoding unit 202, a quantized value decoding unit 203, a field weighting factor decoding unit 22, a frame weighting factor decoding unit 23, a weighting factor generating unit 24, an index decoding unit 205, an AFF 206 identification information decoding unit and keys 26 ~

28. The same reference IDs are placed for the same functional structures as those of the conventional variable length decoding unit VLD, and the explanation is hereby abbreviated.

>

The demultiplexing unit 21 demultiplexes the inserted encoded Str image signal and outputs the demultiplexed signals respectively as follows: the encoded MV motion vector for the MV decoding unit 202; the quantized value Qcoef encoded for the quantized value decoding unit 203; the Weighted weighting factor encoded for the field weighting factor decoding unit 22 or 22 for the weighting factor decoding unit 23, and for the weighting factor generating unit 24; the Index image number for the index decoding unit 205 and the encoded AFF for the identification information decoding unit of AFF 206.

The field weighting factor decoding unit decodes the Weighted weighting factor entered as a field weighting factor. The frame weighting factor decoding unit 23 decodes the Weighted weighting factor entered as a frame weighting factor.

The weighting factor generation unit 24 generates a field weighting factor based on a frame weighting factor, if necessary. It is a case, for example, in which the change of wed • · ··· ·· • · • · dro / field on a block-by-block basis takes place and it is necessary to generate a field weighting factor based on a factor frame weighting since a field weighting factor is not coded.

Figures 12A, 12B and 12C are diagrams showing examples of an image area data structure according to the present embodiment. Figure 12A is a detailed example of a header data structure within a common information area in the image area. In the example in Figure 12A, the header includes the image frame encoding information which indicates whether the image is encoded on a frame by frame basis or on a field by field basis. For example, when the picture frame encoding information indicates 1, the header still includes an “AFF” flag that indicates whether the change between a field and a frame on a block-by-block basis happens or not. When the AFF indicates 1, for example, it indicates that the change between a field and a frame takes place. As shown in Figure 12A, when the AFF indicates 1, both the field weighting factor and the frame weighting factor are transmitted. The field weighting factor includes a first field weighting factor and a second field weighting factor.

When the image frame encoding information indicates 0, the image is encoded on a field by field basis, so it is impossible to switch between field and frame on a block-by-block basis. Consequently, as shown in Figure 12B, the header does not include AFF and only the field weighting factor is transmitted as an image weighting factor. In the case of Figure 12C, where the picture frame encoding information is 1 and the AFF indicates 0, it is impossible to switch between field and frame on a block-by-block basis. Therefore, only the frame weighting factor is transmitted as an image weighting factor.

Figure 13 is a flowchart showing an encoding processing sequence in relation to the weighting factors operated by the VLD variable length decoding unit when in • ·· ···· ··

image frame encoding formations indicate 1 ”and the image is encoded on a frame by frame basis, in accordance with the present modality.

First, when the AFF value indicates 1 and frame / field change 5 is operated on a block-by-block basis (Yes in S10), the AFF that indicates change on a block-by-block basis happens is coded (S13) , and then the frame weighting factor and the field weighting factor are coded (S14, S15).

When the AFF value is 0 and no quarter / field changes take place on a block-by-block basis (Not in S10), the AFF value 0 indicating no change takes place on a block-by-block basis is coded (S11 ) and the image weighting factor is coded (S12).

VARIATION

Figures 14A, 124 and 14C are diagrams showing examples of an image area data structure according to a variation of the present embodiment. Figure 14A is a detailed example of a header data structure within a common information area in the image area. In the example in Figure 14A, the header includes image frame encoding information which indicates whether the image is encoded on a frame-by-frame basis or on a field-by-field basis. For example, when the picture frame encoding information indicates 1 (this means that the picture is encoded on a frame by frame basis), the header still includes an AFF flag that indicates whether the change between a field and a frame in a block-by-block basis happens or not. When AFF indicates 1, for example, it indicates that the change between a field and a frame takes place on a block-by-block basis. As shown in Figure 14A, when the AFF indicates 1, the frame weighting factor is transmitted, and the field weighting factor appropriates the frame weighting factor for coding processing.

When the picture frame encoding information in27 • ·· • φ • ··· • φ ·

indicate 0 this indicates that the image is encoded on a field by field basis. In this case, changing the frame / field on a block-by-block basis does not happen. Therefore, it is impossible to switch between field and frame on a block-by-block basis. Consequently, when the header does not include AFF as shown in Figure 14B, it means that only the field weighting factor is transmitted as an image weighting factor. In the case of Figure 14C, where the image frame coding information indicates 1 and the AFF indicates 0, the change of frame / field on a block-by-block basis does not happen and thereby the image is always encoded on a base frame by frame. Therefore, only the frame weighting factor is transmitted as an image weighting factor.

Figure 15 is a flowchart showing a coding processing sequence in relation to the weighting factors operated by the VLC variable length coding unit when the picture frame coding information indicates 1 and the picture is encoded on a frame basis per table, according to the variation of the present modality.

First, when the AFF value is 1 and the frame / field change is operated on a block-by-block basis (Yes in S10), the AFF that indicates a change on a block-by-block basis is coded (S13) and the frame weighting factor is coded (S15).

When the AFF value is 0 and no change of frame / field occurs on a block-by-block basis (Not on S10), the AFF value 0 indicating no change on a block-by-block basis is coded (S11) and either the field weighting factor or the frame weighting factor that corresponds to a coding unit of a block is coded as an image weighting factor (S12) based on the image frame coding information.

Figure 16 is a flowchart showing a decoding processing sequence in relation to the weighting factors when the picture frame coding information indicates 1 and the image

• · ♦ · ··· * ♦ ···· ·· gem processed by the variable length decoding unit VLD shown in Figure 11 is encoded on a frame by frame basis. This flowchart also corresponds to the coding processing sequence described in Figure 13.

First, the VLD variable-length decoding unit decodes the AFF (S20). When the AFF value is 1 indicating that the frame / field change is operated on a block-by-block basis (Yes in S21), the VLD variable length decoding unit decodes the frame weighting factor (S23) and generates a field weighting factor based on this (for example, appropriating a frame weighting factor) (S24).

On the other hand, when the AFF value is 0 indicating that the frame / field change on a block-by-block basis does not happen (S21: No), the variable length decoding unit VLD deco15 differs or the weighting factor of field as an image weighting factor or the field weighting factor (S22).

Thus, by using the image encoding / decoding method according to the present modality, the change of frame / field on a block-by-block basis is performed, the prediction efficiency is improved, which eventually brings an improvement of the rate of compression. Furthermore, even when the field weighting factor is not coded, the variable length decoding unit VLD generates the field weighting factor based on the frame weighting factor so that the field / frame change on a base blocking happens without any problem.

SECOND MODE

The present modality describes an example of a case in which a data structure of an image area is different from that illustrated in the first modality.

Figures 17A, 17B and 17C are diagrams showing examples of an image area data structure according to the present embodiment. These diagrams also show a structure of

·· ·· ·· · «detailed data of a header included in a common information area in an image area. The present embodiment illustrates an example of a header structure from which a field weighting factor can be taken when the frame encoding information indicates 1 and the image is encoded on a frame by frame basis.

As shown in Figures 17A and 17B, the header includes the field factor presence / absence information as well as the AFF. The field factor presence / absence information is a signal that indicates whether the header has a field weighting factor or not. For example, the flag is set to 1 when the header has the field weighting factor and is set to 0 when the field weighting factor is removed.

Figure 17A is a case in which both the AFF and the field factor presence / absence information are set to 1 and the field weighting factor is transmitted. The field weighting factor includes the first field weighting factor and the second field weighting factor as in the case of the first modality described above.

Figure 17B is a case in which the AFF is set to 1 and the field factor presence / absence information is set to 0.

Figure 17C is a case in which the field / frame change on a block-by-block basis does not happen since the AFF is set to 0.

Figure 18 is a flowchart showing a coding processing sequence in relation to the weighting factors operated by the variable length coding unit VLC according to the present modality.

First, when the AFF value indicates 1 and the frame / field change on a block-by-block basis is operated (Yes on S10), the VLC variable-length coding unit encodes the AFF indi30 • tf · tf •

• tf ···· tftf • · · • · I • · · «·· ·· • · · · ············ ·· ·· ···· ··

indicating that the change on a block-by-block basis takes place (S31).

Furthermore, the variable length coding unit

VLC determines whether or not a field weighting factor can be generated based on a frame weighting factor (S32), and when this is possible, it encodes the information that indicates the generation of the field weighting factor, and the factor weighting table (S36, S37). When the field weighting factor is not generated based on the frame weighting factor, the variable length coding unit VLC encodes the information that indicates the presence / absence of the field weighting factor10 as well as the weighting factor of frame and the field weighting factor (S33 ~ S35).

On the other hand, when the AFF value is 0 and the frame / field change on a block-by-block basis is not operated (Not on ►

S10), the subsequent processing is the same as in the illustrated flow chart - 15 in Figure 15 (S11, S12).

Figure 19 is a flowchart showing a decoding processing sequence in relation to the weighting factors operated by the VLD variable length decoding unit described in Figure 11. This flowchart also corresponds to the encoding processing sequence described in Figure 18.

First, when the AFF value is 1 indicating that the frame / field change on a block per block basis is operated (Yes on S21), the VLD variable length decoding unit decodes the AFF (S20) and then decodes the information indicating the presence / absence of the field weighting factor (S41).

Then, the variable length decoding unit VLD determines whether the field weighting factor is found or not (S42), decodes the frame weighting factor when the field weighting factor is not found (S45) and generates the factor field weighting based on the frame weighting factor (S46). When the field weighting factor is found, the VLD variable length decoding unit decodes both the frame weighting factor and • · · · · ···· ··· ··· · • · · · · · ·

the field weighting factor (S43, S44).

On the other hand, when the AFF value is 0 indicating that the frame / field change on a block-by-block basis is not operated (Not on S21), the image weighting factor is decoded (S22).

Thus, using the image encoding / decoding method according to the present modality, the change of frame / field on a block-by-block basis is performed. Furthermore, the field weighting factor can be generated based on the frame weighting factor even when the field weighting factor is removed.

THIRD MODE

The present modality describes a case in which the data structure of the image area is different from that illustrated in the first modality.

Figures 20A, 20B and 20C are diagrams showing examples of the data structure of the image area according to the present embodiment. These show a detailed header data structure when the picture frame encoding information included in a common information area in an image area indicates 1 and the image is encoded on a frame by frame basis. This modality illustrates an example of the header structure from which the frame weighting factor is taken.

As shown in Figures 20A and 20B the header includes the presence / absence information of the frame factor as well as the AFF. The frame factor presence / absence information is a flag that indicates whether or not the header includes a frame weighting factor. For example, the flag is set to 1 when the frame weighting factor is found and is set to 0 when the frame weighting factor is removed.

Figure 20A is a case in which both the AFF and the presence / absence of frame factor information are set to 1 and the frame weighting factor is transmitted. Figure 20B is a case in which the AFF is set to 1 and the factor identification information • · · · · ···· ··· ··· · · ·· · ·· ·· · ·· ·· · Frames are set to 0. Figure 20C is a case in which the field / frame change on a block-by-block basis is not operated since the AFF is set to 0.

Figure 21 is a flowchart showing a coding processing sequence in relation to the weighting factors operated by the variable length coding unit VLC according to the present modality.

First, the VLC variable-length coding unit encodes the AFF indicating that the change on a block-by-block basis is operated (S51) when the AFF value is 1 and the frame / field change on a block-by-block basis happens (Yes in S10).

Furthermore, the variable length coding unit VLC determines whether or not to generate a frame weighting factor based on a field weighting factor (S52). When the frame weighting factor is generated based on a field weighting factor, the variable length encoding unit VLC encodes the information that indicates the generation of the frame weighting factor and the field weighting factor (S56 , S57). When the frame weighting factor is not generated based on the field weighting factor (Not in S52), the variable length encoding unit VLC encodes the information that indicates the presence / absence of the frame weighting factor as well as the field weighting factor and frame weighting factor (S53 ~ S55).

On the other hand, when the AFF value is 0 and the change of frame / field on a block-by-block basis does not happen (Not in S10), the same encoding processing as described in Figure 15 is performed (S11, S12) .

Figure 22 is a flowchart showing a decoding processing sequence in relation to the weighting factors operated by the VLD variable length decoding unit illustrated in Figure 11. This diagram also corresponds to the encoding processing sequence described in Figure 21.

First, when the AFF value is 1 indicating that the frame / field change on a block-by-block basis is operated (Yes on S21), the VLD variable length decoding unit first decodes the AFF (S20) and then the information indicating the presence / absence of the frame weighting factor (S61).

Then, the variable length decoding unit VLD determines whether the frame weighting factor is found or not (S62), decodes the field weighting factor (S65) when the frame weighting factor is not found (Yes in S62 ) and generates a frame weighting factor based on the field weighting factor (S66). When the frame weighting factor is not found (Not in S62), both the field weighting factor and the frame weighting factor are decoded (S63, S64).

On the other hand, when the AFF value is 0 indicating that the frame / field change on a block-by-block basis is not operated (Not on S21), the VLD variable-length decoding unit decodes the image weighting factor (S22).

Thus, by using the image encoding / decoding method according to the present modality, the field / frame change on a block-by-block basis is performed. In addition, the frame weighting factor can be generated based on the field weighting factor even when the frame weighting factor is removed. FOURTH MODALITY

Furthermore, the processing shown in each of the above modalities can easily be performed on an independent computer system by recording the program to perform the method described in each of the above modalities on a storage medium such as a floppy disk or the like.

Figure 23 is an illustration for performing the encoding / decoding method described in each of the above modalities on the computer system using the program recorded on the storage media such as a floppy disk or the like.

Figure 23B shows the total appearance of a floppy disk, its cross-sectional structure and the floppy disk itself while Figure 23A shows an example of a physical format of the floppy disk as a main body of the storage medium. A flexible disk FD is contained in a housing F with a plurality of Tr tracks formed concentrically from the periphery to the inside on the surface of the disk, and each track is divided into 16 sectors Se in the angular direction. Thus, the program is stored in an area designated for it on the FD floppy disk.

Figure 23C shows a structure for writing and reading the program on the FD floppy disk. When the program is written to the FD floppy disk, the Cs computer system writes to the program via a floppy disk drive. When the encoding device and the decoding device are built into the computer system using the program on the floppy disk, the program is read from the floppy disk and then transferred to the computer system by the floppy disk drive.

The above explanation is made on the assumption that a storage medium is a floppy disk, but the same processing can also be performed using an optical disk. In addition, the storage medium is not limited to a floppy disk or an optical disk, but any other medium such as a Cl card and a ROM cassette capable of recording a program can be used.

FIFTH MODE

The following is a description for the applications of the image encoding / decoding method illustrated in the above mentioned modalities and a system using them.

Figure 24 is a block diagram showing a complete configuration of an ex100 content supply system to perform a content delivery service. The area for providing the communication service is divided into cells of desired size, and cell points ex107 ~ ex110, which are fixed wireless stations, and placed in the respective cells.

This ex100 content supply system is connected to • ·· · ·· ·· • ·· · ·· · · ·· · · ·· ··

devices such as a computer ex111, a PDA (Personal Digital Assistant) ex112, a camera ex113, a mobile phone ex114 and a mobile phone with a camera ex115 via, for example, the Internet ex101, an Internet service provider ex102, a telephone network ex104, as well as cell points ex107 ~ ex110.

However, the ex100 content supply system is not limited to the configuration shown in Figure 24 and can be connected to a combination of any of them. Also, each device can be connected directly to the telephone network ex104, not via cell points ex107 ~ ex110.

The ex113 camera is a device capable of recording video just like a digital video camera. The ex114 mobile phone can be a mobile phone from any of the following systems: a PDC (Personal Digital Communication) system, a CDMA (Code Division Multiple Access) system, a W-CDMA (Division Multiple Access system) Code - Broadband) or a GSM system (Global System for Mobile Communication); a PHS (Personal Portable Telephone System) or similar.

An ex103 streaming server is connected to the ex113 camera via the ex104 phone network and also to cell point 109, which performs a live or similar distribution using the ex113 camera based on the coded data transmitted from the user. Anyone from the ex113 camera, from the server which transmits the data and the like can encrypt the data. The mobile image data recorded by an ex116 camera can be transmitted to the streaming server ex103 via the computer ex111. In this case, either the camera ex116 or the computer ex111 can encode the moving image data. An LSI ex117 included in the computer ex111 and the camera ex116 performs encoding processing. The software for encoding and decoding the images can be integrated into any type of storage media (such as a CD-ROM, a floppy disk and a hard disk) which is a recording medium which is readable by the computer ex111 or similar. Furthermore, the mobile phone with an ex115 camera can transmit the mobile image data. These data

mobile image is the data encoded by the LSI included in the ex115 mobile phone.

The ex100 content supply system encodes the content (such as live music video) recorded by a user using the camera ex113, camera ex116 or similar in the same way as shown in the above mentioned modalities and transmits it to the server streaming server ex103, while the streaming server ex103 delivers streamed content data to customers at their request. Customers include computer ex111, PDA ex112, camera ex113, mobile phone ex114 and so on capable of decoding the aforementioned encrypted data. In the ex100 content supply system, customers can thus receive and reproduce the encrypted data, and can also receive, decode and reproduce the data in real time in order to perform a personal transmission.

When each device in this system performs encoding or decoding, the image encoding device or the image decoding device shown in the above mentioned modalities can be used.

A cell phone will be explained as an example of such a device.

Figure 25 is a diagram showing the mobile phone ex115 using the image encoding / decoding method explained in the above mentioned modalities. The ex115 mobile phone has an ex201 antenna for communicating with the cell point ex110 via radio waves, an ex203 camera unit such as a CCD camera capable of recording moving and still images, an ex202 display unit such as an image display. liquid crystal to display data such as decoded and similar images recorded by the ex203 camera unit or received by the ex201 antenna, a body unit that includes a set of ex204 operation keys, an ex208 audio output unit such as a speaker speaker to output audio, an ex205 audio input unit such as a microphone to insert audio, an ex207 storage media

for storing encrypted or decoded data such as moving or still image data recorded by the camera, incoming email data and moving or still image data, and an ex206 connector unit for connecting ex207 storage media to the phone mobile ex115. The ex207 storage media stores in itself an element of instant memory, a type of EEPROM (Electrically Erasable and Programmable Read Only Memory) which is an electrically erasable and rewritable non-volatile memory in a plastic housing such as an SD card.

In the following, the ex115 mobile phone will be explained with reference to Figure 26. On the ex115 phone, an ex311 main control unit, designed to completely control each main body unit which contains the ex202 display unit as well as the operation ex204, is mutually connected to an ex310 power supply circuit unit, an ex304 operation input control unit, an ex312 image encoding unit, an ex303 camera interface unit, an LCD control unit ( Liquid Crystal Display) ex302, an ex309 image decoding unit, an ex308 multiplexing / demultiplexing unit, an ex307 reading / writing unit, an ex306 modem circuit unit and an ex305 audio processing unit via a bar synchronous ex313.

When an end-of-call key or a power key is switched ON by user operation, the ex310 power source circuit unit supplies the respective units with power from a battery pack in order to activate the digital mobile phone with a ex115 camera as a ready state.

On the ex115 mobile phone, the ex305 audio processing unit converts the audio signals received by the ex205 audio input unit in talk mode to digital audio data under the control of the ex311 main control unit which includes a CPU, a ROM and a RAM, the ex306 modem circuit unit performs expansion spectrum processing for digital audio data, and the unit

communication circuit ex301 performs a digital to analog conversion and a frequency conversion to the data, in order to transmit them through the ex201 antenna. Also, on the ex115 mobile phone, the ex301 communication circuit unit amplifies the data received by the ex201 antenna in talk mode and performs frequency conversion and analog to digital conversion of the data, the ex306 modem circuit unit performs processing of the inverse expansion spectrum of the data, and the ex305 audio processing unit converts them into analog audio data in order to output them via the ex208 audio output unit.

Furthermore, when transmitting an e-mail in data communication mode, the text data of the e-mail entered by operating the main body ex204 operation keys is sent to the main control unit ex311 via the input control unit of operation ex304. In the main control unit ex311, after the modem circuit unit ex306 performs the expansion spectrum processing of the text data and the communication circuit unit ex301 performs the digital to analog conversion and the frequency conversion of the text data, data is transmitted to cell point ex110 via antenna ex201.

When image data is transmitted in data communication mode, the image data recorded by the camera unit ex203 is supplied to the image coding unit ex312 via the camera interface unit ex303. When they are not transmitted, it is also possible to display the image data recorded by the ex203 camera unit directly on the ex202 display unit via the ex303 camera interface unit and the ex302 LCD control unit.

The image encoding unit ex312, which includes the image encoding apparatus as explained in the present invention, compresses and encodes the image data supplied from the camera unit ex203 using the encoding method employed by the image encoding apparatus as shown in the first modality in order to transform39

them in encoded image data, and send them to the multiplexing / demultiplexing unit ex308. At this time, the ex115 mobile phone sends the audio received by the ex205 audio input unit during recording with the ex203 camera unit to the ex308 multiplex / demuplex unit as digital audio data via the ex305 audio processing unit.

The multiplexing / demultiplexing unit ex308 multiplexes the encoded image data supplied from the ex312 image encoding unit and the audio data supplied from the ex305 audio processing unit, using a predetermined method, then the ex306 modem circuit unit performs the expansion spectrum processing of the multiplexed data obtained as a result of the multiplexing, and finally the communication circuit unit ex301 performs the digital to analog conversion and the frequency transformation of the data for transmission via the ex201 antenna.

As for receiving data from a mobile image file which is connected to a web page or similar in data communication mode, the ex306 modem circuit unit performs reverse expansion spectrum processing of the data received from the cell point ex110 through the ex201 antenna, and sends the multiplexed data obtained as a result of processing the reverse expansion spectrum.

In order to decode the multiplexed data received via the ex201 antenna, the multiplexing / demultiplexing unit ex308 demultiplexes the multiplexed data into an encoded stream of image data and that of audio data, and supplies the encoded image data to the image unit. image decoding ex309 and the audio data for the audio processing unit ex305, respectively through the synchronous bar ex313.

Next, the image decoding unit ex309, which includes the image decoding apparatus as explained in the aforementioned invention, decodes the encoded flow of the image data using the decoding method corresponding to the encoding method40 • · ·· · · ·

as shown in the aforementioned modalities for generating the reproduced mobile image data, and supplies this data to the ex202 display unit via the ex302 LCD control unit, and thus the image data included in the mobile image file connected to a web page, for example, are displayed. At the same time, the ex305 audio processing unit converts the audio data into analog audio data, and supplies this data to the ex208 audio output unit, and thus the audio data included in the mobile image file connected to a Web page, for example, are reproduced.

The present invention is not limited to the aforementioned system since digital land-based or satellite transmission has been in the news lately and at least either the image encoding device or the image decoding device described in the above mentioned modalities can be incorporated in a digital broadcast system as shown in Figure 27. More specifically, an encoded stream of video information is transmitted from an ex409 broadcast station to or communicated with an ex410 broadcast satellite via radio waves. Upon receipt, the transmission satellite ex410 transmits radio waves for transmission. Then, an ex406 home antenna with a satellite broadcast receiving function receives radio waves, and an ex401 television (receiver) or ex407 receiver / decoder (STB) decodes a coded bit stream for reproduction. The image decoding apparatus as shown in the aforementioned embodiments can be implemented in the reproduction apparatus ex403 to read and decode the encoded stream recorded on an ex402 storage medium which is a recording medium such as a CD or DVD. In this case, the reproduced moving image signals are displayed on an ex404 monitor. It is also conceivable to implement the image decoding device in the ex407 receiver / decoder connected to an ex405 cable for cable television or an ex406 antenna for satellite and / or ground-based transmission in order to reproduce them on an ex408 monitor of the ex401 television. The ima41 decoding device

gem can be incorporated into the television, not the receiver / decoder. Also, an ex412 car that has an ex411 antenna can receive signals from the ex410 satellite or cell point ex107 to reproduce mobile images on a display device such as an ex413 car navigation system placed on the ex412 car.

Furthermore, the image encoding apparatus as shown in the above-mentioned modalities can encode the image signals and record them on a storage medium. As a concrete example, an ex420 recorder such as a DVD recorder for recording image signals on a DVD disc ex421, a disc recorder for recording them on a hard disk can be cited. These can be recorded on an ex422 SD card. When the ex420 recorder includes the image decoding device as shown in the above mentioned modalities, the image signals recorded on the DVD disc ex421 or on the SD card ex422 can be reproduced for display on the monitor ex408.

As for the structure of the ex413 car navigation system, the structure without the ex203 camera unit, the ex303 camera interface unit and the ex312 image coding unit, of the components shown in Figure 26, is conceivable. The same applies to the computer ex111, the television (receiver) ex401 and others.

In addition, three types of implementations can be designed for a terminal such as the ex114 mobile phone; a sending / receiving terminal implemented with either an encoder or a decoder, a sending terminal implemented with only one encoder, and a receiving terminal implemented with only a decoder.

As described above, it is possible to use the image encoding method and the image decoding method described in the above mentioned modalities for any of the above mentioned devices and systems, and by using these methods, the effects described in the above mentioned modalities can be used. obtained.

From the invention thus described, it will be obvious that the modalities of the invention can be varied in many ways. Such variations should not

be seen as a departure from the spirit and scope of the invention, and all such modifications as would be obvious to someone skilled in the art are intended to be included in the scope of the following claims.

Thus, with the image encoding / decoding method according to the present invention, it is possible to carry out the field / frame change on a block basis and to improve the prediction efficiency and the compression rate.

Furthermore, with the image encoding / decoding method according to the present invention, the field weighting factor is generated based on the frame weighting factor, therefore, the field weighting factor can be taken from the data to be transmitted and thereby the transmission efficiency can be improved. Consequently, its practical value is high.

INDUSTRIAL APPLICABILITY

The present invention can be applied to the image encoding apparatus, the image decoding apparatus and its methods for performing a motion estimate by changing frames / fields on a block-by-block basis and is especially useful for the encoding apparatus image as described above to perform a motion estimate using weighting factors.

Claims (5)

1. Image encoding apparatus for encoding an interlaced image on a block-by-block basis comprising: a storage unit operable to store an image which is either a frame or a decoded field after being encoded, as a reference image; a predictive image generation unit operable to read a reference image from the storage unit and generate a predictive image based on pixel values in the reference image, using one of (i) a frame weighting factor for encode the interlaced image on a frame by frame basis and (ii) a field weighting factor to encode the interlaced image on a field by field basis; and a signal coding unit operable to encode in 15 a block-by-block basis a differential value between an inserted image and the predictive image generated by the predictive image generation unit, either on a frame-by-frame basis or on a field-by-field basis; characterized by the fact that it still includes: 20 an operable weighting factor coding unit to encode the frame weighting factor taken from the frame weighting factor and the field weighting factor, when the signal coding unit encodes the differential value on a block basis by block adaptively or on the basis of frame by frame or on the basis of field by field; and a multiplexing unit operable to multiplex the differential value encoded by the signal encoding unit as well as the frame weighting factor encoded by the weighting factor encoding unit, and outputting the encoded differential value and the frame weighting factor , as an encoded signal. 2. Image decoding device to decode, on a block-by-block basis, a signal encoded according to an image 870180026650, of 03/04/2018, p. 5/12 gem that is either a frame or a field, the image decoding apparatus comprising: a signal decoding unit operable to decode the encoded signal either on a frame by frame basis or on a base 5 from field to field, when the encoded signal is encoded changing adaptably between the base of frame by frame and the base of field by field; a storage unit operable to store at least one decoded image; 10 characterized by the fact that it still includes: a predictive image generation unit operable to extract, from the encoded signal, a frame weighting factor to decode the encoded signal on a frame by frame basis, generate a field weighting factor to decode the encoded signal on the basis of field 15 per field, based on the frame weighting factor, and generate a predictive image based on the pixel values in the decoded image stored in the storage unit, using the extracted frame weighting factor and the generated field weighting factor, when the encoded signal is encoded changing adaptably between the frame base 20 per frame and the field base per field; and an operable addition unit for adding the image obtained in decoding performed by the signal decoding unit to the predictive image generated by the predictive image generation unit, to output the added image as a decoded image and to store the decoded image in the unit of storage. 3. Image encoding method to encode an interlaced image on a block-by-block basis, the encoding method comprising the steps of: generate the predictive image for reading a reference image from an operable storage unit to store at least one reference image, and generate a predictive image based on the pixel values in the reference image, using one of ( i) Petition 870180026650, of 04/03/2018, p. 6/12 a frame weighting factor to encode the interlaced image on a frame by frame basis and (ii) a field weighting factor to encode the interlaced image on a field by field basis; and encode, on a block-by-block basis, a differential value 5 between an inserted image and the predictive image generated in the step of generating the predictive image, either on a frame by frame basis or on a field by field basis; characterized by the fact that it still includes: encode the frame weighting factor, taken from the 10 frame weighting and field weighting factor, when the differential value is coded on a block-by-block basis, changing adaptably between the frame-by-frame basis and the field-by-field basis in the coding step; and multiplex the differential value encoded in the step of encoding the 15 signal and the encoded frame weighting factor in the step of encoding the weighting factor, and outputting the multiplexed differential value and the frame weighting factor, as an encoded signal. 4. Image decoding method to decode, on a block-by-block basis, a signal encoded according to an image that is either a frame or a field, the image decoding method comprising the steps of: decode the encoded signal either on a frame-by-frame basis or on a field-by-field basis, when the encoded signal is encoded changing adaptably between the frame-by-frame base and the 25 field base per field; characterized by the fact that it still includes: generate the predictive image to extract, from the encoded signal, a frame weighting factor to decode the encoded signal on a frame by frame basis, generate a field weighting factor to de30 encode the encoded signal on a field by field basis , on a field by field basis, based on the frame weighting factor, and generate a predictive image based on the pixel values in a decoding image 870180026650, from 03/04/2018, p. 7/12 each stored in a storage unit, using the extracted frame weighting factor and the generated field weighting factor, when the encoded signal is encoded changing adaptably between the frame base per frame and the field base per field ; and 5 add the image obtained in the decoding performed in the step of decoding the signal in the predictive image generated in the step of generating the predictive image, outputting the added image as a decoded image and storing the decoded image in the storage unit. Petition 870180026650, of 04/03/2018, p. 12/8 1/27 • · · · · · · · • ··· ··· ··· · · • · ·· · ·· ·· · 4/27 fN O + O X o CL V σ PO pixel value < σι • MM LL ÍN% Q + X T-l CL + The £ X o Q. II σ QC Mσι LL 5/27 • · · · · · · · · · • · · · · ···· ··· ··· ♦ • · · · · ·· · · ··· · ·