JPH08186818A - Image coder and image decoder - Google Patents

Abstract

PURPOSE: To attain efficient compression coding by allowing a sequential scanning image to be received by a receiver for receiving an interlaced scanning image. CONSTITUTION: The image coder is provided with an image separate section 1 inputting a sequential scanning image and separating the image into 1st and 2nd interlaced scanning images, a 1st image coding section 2 applying compression coding to the separated 1st interlaced scanning image, a difference device 3 obtaining a difference between an image by decoding a coded image by the 1st image coding section 2 and a 2nd interlaced scanning image separated by the image separate section 1, and a 2nd image coding section 4 coding the difference. The image decoder is provided with 1st and 2nd decoding sections 5, 6 decoding received coding data, an adder 7 adding the decoded data, and an image multiplexing section 8 multiplexing the 1st and 2nd interlaced scanning decode data to generate a sequential scanning image.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image encoding device and a decoding device used for compressing and encoding progressively scanned digital images by digital broadcasting or the like.

[0002]

2. Description of the Related Art A digital image signal has an enormous amount of information, and high efficiency coding is indispensable for transmission and recording. In recent years, various image compression coding techniques have been proposed, and among them, there is also a coding method that realizes a function of receiving an HDTV and a standard TV from one transmission line in response to a user's request.

With reference to the drawings, an M which compresses and decodes the above-mentioned two images having different spatial resolutions.
An image encoding and decoding apparatus of the PEG system will be described.

FIG. 9 is a block diagram of a conventional MPEG image coding apparatus. In FIG. 9, 10 is a first image encoding unit, 11 is a motion detection circuit, 12 is a DCT mode determination circuit, 13 is a DCT circuit, 14 is a quantization circuit, 1
Reference numeral 5 is a variable length coding circuit, 16 is an inverse quantization circuit, 17 is an inverse DCT circuit, 18 is a frame buffer, and 19 is a motion compensation circuit. Further, 20 is a second image encoding unit, and 21
Reference numerals 29 to 29 are circuits having the same functions as 11 to 19, respectively, but the sizes of images that can be processed are different. Three
Reference numeral 0 is a first resolution conversion circuit, and 31 is a second resolution conversion circuit.

The operation of the conventional image coding apparatus configured as described above will be described below. The video signal is
As shown in FIG. 10, interlaced scanning is performed, and the data is divided into frames and input. The frame consists of field 1 and field 2, and frame t
And frame t + 1 is 1/30 seconds, fields 1 and 2 are 1/30
There is a 60 second interval. The input image is first converted by the first resolution conversion circuit 30 into an image having half the resolution in both horizontal and vertical directions. The first frame of encoding is intraframe encoded without taking the difference. First, the input image data is 2
The DCT mode determination circuit 12 detects the magnitude of the movement by taking a difference between lines in units of dimension blocks, determines whether to perform DCT in units of frames or fields, and outputs the result as DCT mode information. . The DCT circuit 13 inputs DCT mode information, performs DCT in frame units or field units, and converts image data into transform coefficients. The transform coefficient is quantized by the quantizer 14 and then variable-length coded by the variable-length coding circuit 15 and sent to the transmission line. The quantized transform coefficient is simultaneously returned to the real-time data through the inverse quantizer 16 and the inverse DCT transform circuit 17, and stored in the frame buffer 18.

By the way, since images generally have a high correlation, when DCT is performed, energy concentrates on the transform coefficient corresponding to a low frequency component. Therefore, it is possible to minimize image quality deterioration and reduce the amount of data by coarsely quantizing the high frequency components that are visually inconspicuous and finely quantizing the important low frequency components.
In the interlaced image, when the motion is small, the intra-frame correlation is strong, when the motion is large, the inter-frame correlation is small, and conversely, the intra-field correlation is high. By utilizing the above-mentioned interlaced scanning characteristics and switching the DCT in units of frames or fields, it is possible to efficiently encode an interlaced image.

On the other hand, for the image after the intra-coded frame, the prediction value is calculated for each frame, and the difference from the prediction value, that is, the prediction error is coded. As an encoding device, first, a motion vector used for prediction is used as a motion detection circuit 11
, Using, for example, the well-known exhaustive search method,
Calculate in units of dimensional blocks. Next, frame buffer 1
8 and the motion compensation circuit 19 use the detected motion vector to generate a motion-compensated predicted value for the next frame in units of two-dimensional blocks. The difference between the generated prediction value and the input image data is calculated to obtain the prediction error, and the prediction error is coded by the same method as the intraframe coding. The motion vector used for motion compensation, motion compensation information indicating the state of motion compensation in block units, DCT mode information, and the like are sent to the decoder together with the encoded coefficients.

According to the above coding apparatus, since the prediction error is coded optimally, the energy is reduced as compared with the case where the image data is directly coded as in the intraframe coding, and Highly efficient encoding is possible.

On the other hand, the operation of the second image coding unit 20 is as follows.
Basically, it is the same as the first image encoding unit 10 except that an image with low resolution can be used for prediction value generation. The prediction value generation is performed by the motion compensation circuit 19. At this time, the resolution conversion circuit 31 doubles the resolution of the second image of the previous frame having a low resolution stored in the frame buffer 18 to predict it. Used as one of the value candidates. The motion compensation circuit 29 calculates the difference between the predicted value read out from the frame buffer 28 and the output of the second resolution conversion circuit 31 and uses the difference from the original image, selects the smaller one, and uses it for encoding. By encoding the high-resolution image by the above method, the portion similar to the low resolution does not need to be encoded, and the encoding efficiency can be improved.

The low-resolution and high-resolution coded image data described above are multiplexed and sent to the transmission path. FIG. 11 is a block diagram of the image decoding apparatus. In FIG.
Reference numeral 40 is a first image decoding unit, and 50 is a second image decoding unit. Reference numerals 41 and 51 are variable length decoding circuits, and 42 and 45 are simple motion compensation circuits. Inverse quantization circuits 16 and 26, inverse DCT
The circuits 17, 27, the frame buffers 18, 28 and the second resolution conversion circuit 31 are the same as those in FIG. The decoding device inputs the compressed image data, performs the variable length decoding by the variable length decoding circuit 41, and then performs the same operation as the local decoding of the encoding circuit to output the images of the respective resolutions. Unlike the encoding device, the motion compensation circuit does not need to select the type of motion compensation, and only creates motion compensation data based on the type, motion vector information, etc. sent to it. Compensation circuits 42 and 52 are provided.

As described above, in the decoding device, a low-resolution image is obtained by extracting low-resolution encoded image data from one type of encoded image data and decoding it.
A high-resolution image can be obtained by extracting and decoding both low-resolution and high-resolution encoded image data. Therefore, users can switch between low-resolution and high-resolution images depending on the situation and receive them (for example, ISO / IEC
JTC1 / SC29 N659, "ISO / IEC CD 13818-2: Information t
echnology-Generic coding of moving pictures and
associated audio information-Part 2: Video ", 199
3.12).

[0012]

However, the above-mentioned conventional image coding and decoding apparatus has a problem when the input is a sequentially scanned image. The current broadcast uses interlaced scanning images, and the receiver of the receiver is dedicated to interlaced images. Therefore, when broadcasting a progressive scan image, it is desirable to simultaneously broadcast both an interlaced image and a progressive scan image. Since the above encoding device basically performs spatial resolution conversion by horizontal and vertical filtering processing, interlaced scan images having different spatial resolutions or progressive scan images are combined and encoded. When encoding by combining the progressive scan image and the interlaced scan image, the predicted image of the progressive scan image must be obtained from the interlaced scan image, and the predictive image can be obtained by the simple horizontal and vertical filter processing similar to the conventional example. It had a problem that it could not be done.

In view of the above problems, it is an object of the present invention to provide an image coding apparatus and an image decoding apparatus which are capable of efficiently performing compression coding by combining a progressive scanning image and an interlaced scanning image. Is.

[0014]

In order to achieve the above object, the image coding apparatus of the present invention is an image separation unit for inputting sequentially scanned images and separating the images into first and second interlaced scanned images. A first image encoding unit for compression encoding the first interlaced scan image separated by the image separating unit;
Differentiator for obtaining the difference between the image obtained by decoding the image coded by the image coding unit and the second interlaced scan image separated by the image separation unit, and the second image code for coding the difference And a chemical conversion unit.

The image decoding apparatus of the present invention further comprises a first image decoding section and a second image decoding section for decoding the compressed image data sent from the image coding apparatus, and first and second image decoding sections. An adder for adding the decoded image data generated by the image decoding unit, the first interlaced scanning decoded image data obtained by the first image decoding unit, and the second interlaced scanning decoded image data obtained by the second image decoding unit And an image multiplexing unit that multiplexes and generates a sequentially scanned image.

[0016]

According to the present invention, according to the above-described structure, the image of progressive scanning is first separated into a plurality of interlaced scanning images and encoded,
Since a plurality of interlaced images encoded and decoded by the decoding device are multiplexed, it is also possible to encode progressive scan images.

[0017]

【Example】

(Embodiment 1) An image coding and decoding apparatus according to a first embodiment of the present invention will be described below with reference to the drawings. 1 is a block diagram of an image coding and decoding apparatus according to a first embodiment of the present invention. In FIG. 1, 1 is an image separation unit, 2 is a first image coding unit, 3 is a difference unit, 4 is a second image coding unit, and the above is an image coding apparatus. Further, in the figure, 5 is a first image decoding unit, 6 is a second image decoding unit, 7 is an adder, 8 is an image multiplexing unit, and the above is an image decoding apparatus. FIG. 2 is a schematic diagram showing the operation of the image separation unit 1.

The operation of the image coding and decoding apparatus configured as described above will be described with reference to FIGS. 1 and 2. The progressively scanned image is input to the image separation unit 1. Figure 2
2A is the input progressive scan image. Figure 2
The bottom shows the vertical direction and the right shows the time direction. The image separation unit 1 alternately separates the progressively scanned images by vertical lines, as shown in FIGS. 2B and 2C, and separates them into two interlaced scanning images. In the figure, a11, a21,
Each of a12 and b21, b11, b22, b12 corresponds to one frame of progressive scanning. Unlike interlaced scanning, progressive scanning has pixels in a grid pattern in both the vertical and temporal directions, so there is no concept of a field, and all are composed of frames. The image separation unit 1 can be composed of a filter and a selector, or a selector only, and after applying a filter for each frame of sequential scanning, outputs alternately by a selector or the like. When using a simple configuration, the filter is omitted and only the selector is used.

FIG. 2B output from the image separation unit 1
The interlaced scan image of is compressed and encoded by the first image encoding unit 2 in the same operation as the first image encoding unit 10 in the conventional example. At that time, one frame is a11, b11, a
12, b12. On the other hand, the locally decoded image is
The other output of the image separation unit 1, that is, the interlaced scan image of FIG. When calculating the difference, a11-b21, b11-a21, a1
2-a22 or a11-b21, b11-c2
The calculation is performed for the pixels at positions 1 and a12-c22. In the former case, the second image coding unit 4 defines one frame as b21, a21, b22, a22, and in the latter case, it is composed of b21, c21, b22, c22. That is, in the former frame structure, two interlaced scanning images are composed of two frames in sequential scanning, and in the latter, three
It will consist of a frame. In the former method, the positions of pixels for which the difference is calculated are close to each other, but since the calculation reference is shifted by one frame for each vertical pixel, a difference in high frequency components may occur. On the other hand, in the latter case, although the above-mentioned disadvantages do not exist, the timing for calculating the difference is shifted by one frame in the sequential scanning, so that hardware for absorbing the delay is required. The configuration of the former or the latter is determined based on the type of image, hardware restrictions, and the like.

The image for which the difference has been calculated is compression-encoded by the second image encoding unit 4 in the same operation as that of the first encoding unit 2. The images compression-coded by the respective coding units are multiplexed into one data string and transmitted to the transmission path if necessary.

When the data received by the decoding device is multiplexed into one data string, it is separated into two data strings and input to the first and second image decoding units 5 and 6, respectively. The first image decoding unit 5 decodes the interlaced scan image by the same operation as the first image decoding unit 40 of the conventional example. The interlaced scan image decoded by the first image decoding unit 5 is added by the adder 7 to the difference image data decoded by the second image decoding unit 6 by the same operation as the first image decoding unit 40 of the conventional example. Then, the interlaced image corresponding to FIG. 2C is decoded. The image multiplexing unit 8 restores the interlaced image decoded by the first and second image decoding units 5 and 6 into a progressively scanned image by an operation reverse to that of the image separating unit 1. In the case of multiplex, it is not necessary to apply a filter unlike separation.

As described above, according to the present embodiment, it is possible to encode a progressively scanned image, and by displaying only the output of the first image decoding unit 5, it is possible to display even an image receiver dedicated to an interlaced scanned image. It will be possible.

(Embodiment 2) FIG. 3 is a block diagram of an image coding and decoding apparatus according to a second embodiment of the present invention. The difference from the first embodiment is that similar interpolation units 9A and 9B are provided in the image encoding device and the decoding device, respectively. FIG. 4 is an explanatory diagram showing the operation of the image coding apparatus according to the second embodiment. The operation of this embodiment will be described below with reference to FIGS. 3 and 4.

The image separating section 1 is similar to the first embodiment in that
The progressively scanned image, that is, FIG. 4A is separated into interlaced scanned images, FIGS. 4B and 4C. The locally decoded image of the first image encoding unit 2 is input to the interpolating unit 9A, and first, in the field (in the same frame of the sequentially scanned image), the pixel corresponding to the black circle in FIG. 4D is generated by the interpolation filter. To do. The filter may be the average of upper and lower pixels as shown in the figure, or a higher order filter. Next, only black circle pixels are thinned out from the interpolated image to obtain the interlaced image of FIG. 4 (e). The subsequent operation is similar to that of the first embodiment.

As described above, according to the present embodiment, the difference between pixels at the same space-time position is calculated, so that the difference energy is concentrated in a lower range and the compression efficiency is improved as compared with the first embodiment. You can

(Third Embodiment) FIG. 6 is an explanatory diagram showing the operation of an image coding apparatus according to the third embodiment of the present invention. The device configuration is the same as that shown in FIG. 1 or 3. FIG. 5 is an explanatory diagram showing the spatial and temporal arrangement of the luminance and color difference signals. This will be described below with reference to FIGS. 5 and 6.

In conventional interlaced image coding, generally, luminance and color difference signals having the same vertical resolution are input, and then the vertical resolution of the color difference signal is reduced to 1/2 of the luminance and then compression coding is performed. FIG. 5 is an explanatory diagram showing pixel positions after conversion of vertical resolution. White circles represent luminance, and x represents spatiotemporal positions of color differences. As shown in FIG. 5, the vertical resolution of the color difference is half the luminance, and the color difference signals are arranged at equal intervals in the same frame. Therefore, the vertical positional relationship between the luminance and the color difference is slightly different depending on the field. In order to convert the color difference signal having the same resolution as the luminance to the arrangement shown in FIG. 5, it is necessary to perform filtering with odd taps in the first field and even taps for shifting the pixel position in the second field, and then thinning out.

FIG. 6 shows a color difference signal separating method of the image separating section 1 in the first or second embodiment. When separating the color difference signals, the resolution in the vertical direction is converted in advance (1/2 in the example of the figure) as shown in FIG. 6B, and then the same separation as the luminance is performed. According to the present embodiment, the filter for resolution conversion may be a filter with the same odd number of taps in every frame. However, unlike the present embodiment, if the spatial resolution conversion is performed after the separation is performed first, it is necessary to apply different filters in field units as in the conventional case, and the hardware becomes complicated, and at the same time,
Since a new pixel is created by the interpolation operation, it may cause deterioration of image quality.

(Embodiment 4) FIGS. 7 and 8 are explanatory views showing the operation of an image coding apparatus according to the fourth embodiment of the present invention, and the apparatus configuration is the same as that in FIG. 1 or FIG.

Depending on the image of the progressive scanning, the luminance may be progressively scanned and the color difference may be interlaced. In this embodiment, a color difference signal encoding method in the above-described case is provided. In FIG. 7, in the image separation unit 1 in the first or second embodiment, the color difference signals are inter-field interpolated and converted into progressive scan, and then encoded by the same operation as in the above embodiments. By performing the intra-field interpolation, it is possible to obtain a color difference signal close to progressive scanning.

Further, in FIG. 8, the color difference signals are separated in units of fields, and one field is regarded as one frame and encoded as it is in sequential scanning. By handling the scanning as it is, the filtering operation in the spatial direction becomes unnecessary. Further, since the difference between the progressively scanned images is obtained and encoded, the compression efficiency is also improved.

In each of the above embodiments, the difference image between the locally decoded output image of the first image encoding unit 2 and the interlaced image is calculated and input to the second image encoding unit 4. However, like the conventional example, the first image encoding unit 2
It is also possible to use the locally decoded output image or the locally decoded output image, which is interpolated and decimated as in the second embodiment, as a prediction value for motion compensation of the second image encoding unit 4.

Further, although only the case of applying a filter in the spatial direction at the time of image separation has been described, the present invention is not limited to this, and may be used in combination with a filter in the time direction.

Furthermore, in each of the above embodiments, the MPEG system has been described as an example of the image coding units 2 and 4, but
Not limited to this, DPCM, vector quantization,
It is also possible to combine with orthogonal transform coding.

[0035]

As described above, according to the present invention, since a progressive scan image is separated into interlaced scan images and then encoded, it is possible to receive progressive scan images and at the same time receive only interlaced scan images. Therefore, even a receiver having a receiver dedicated to the interlaced scanning image can receive the service.

[Brief description of drawings]

FIG. 1 is a block diagram of an image encoding / decoding device according to a first embodiment of the present invention.

FIG. 2 is a schematic diagram showing the operation of the image separation unit 1 in the first embodiment of the present invention.

FIG. 3 is a block diagram of an image encoding / decoding device according to a second embodiment of the present invention.

FIG. 4 is a schematic diagram showing the operation of the image coding apparatus according to the second embodiment of the present invention.

FIG. 5 is a schematic diagram showing a spatial and temporal arrangement of luminance and color difference signals in the third embodiment of the present invention.

FIG. 6 is a schematic diagram showing an operation of an image coding apparatus according to a third embodiment of the present invention.

FIG. 7 is a schematic diagram showing the operation of the image coding apparatus according to the fourth embodiment of the present invention.

FIG. 8 is a schematic diagram showing the operation of the image coding apparatus according to the fourth embodiment of the present invention.

FIG. 9 is a block diagram of a conventional image encoding device.

FIG. 10 is a schematic diagram showing a conventional space-time arrangement of image signals.

FIG. 11 is a block diagram of a conventional image decoding device.

[Explanation of symbols]

 1 Image Separation Unit 2 First Image Encoding Unit 3 Difference Unit 4 Second Image Encoding Unit 5 First Image Decoding Unit 6 Second Image Decoding Unit 7 Adder 8 Image Multiplexing Unit 9A, 9B Interpolating Unit 10 First image coding unit 20 Second image coding unit 40 First image decoding unit 50 Second image decoding unit

Claims (13)

[Claims] 1. An image separation unit for inputting a sequentially scanned image and separating the image into first and second interlaced scanned images,
A first image encoding unit that compresses and encodes the first interlaced scan image separated by the image separating unit, an image obtained by decoding the image encoded by the first image encoding unit, and the image An image encoding device comprising: a difference device that obtains a difference from the second interlaced scan image separated by the separation unit; and a second image encoding unit that encodes the difference. 2. An image separation unit for inputting a sequentially scanned image and separating the image into first and second interlaced scanned images,
A first image encoding unit for compression encoding the first interlaced scan image separated by the image separating unit, and an image obtained by decoding the image encoded by the first encoding unit,
An interpolator for extracting a part of the interpolated image to generate an interpolated image at the same spatial position as the second interlaced scan image; a differencer for obtaining a difference between the interpolated image and the second interlaced scan image; An image encoding device comprising a second image encoding unit that encodes a difference. 3. The image coding apparatus according to claim 2, wherein the interpolation section performs interpolation of the decoded image within the same frame during sequential scanning. 4. The image encoding apparatus according to claim 1, wherein the first and second interlaced scanning images separated by the image separation unit are two frames of sequentially scanned digital images. 5. The image coding apparatus according to claim 1, wherein the first and second interlaced scanning images separated by the image separation unit extend over three frames of sequentially scanned digital images. 6. The image separation unit, when separating the first and second interlaced scan images, converts the progressive scan image into a progressive scan image of a predetermined spatial resolution, and then converts the interlaced scan image. The image encoding device according to claim 1 or 2. 7. The image separation unit includes a first progressive scan image,
When two different images, that is, the interlaced scan image obtained by separating the first interlaced scan image and the first interlaced scan image at the same spatial position, are input, the first interlaced scan image is converted into the first interlaced scan image. Is converted into a second progressive scan image having a spatial resolution similar to the above, and then the second progressive scan image is converted into a third progressive scan image having a predetermined spatial resolution, and then separated into a second interlaced scan image. The image coding apparatus according to claim 1, wherein 8. The image separating unit, when converting an image of progressive scanning into a predetermined spatial resolution, performs filtering using pixels in the same frame of progressive scanning. 7. The image encoding device according to any one of 7. 9. An image separation unit for inputting an interlaced scan image and separating the interlaced scan image into first and second progressive scan images composed only of images of the same time; and an image separation unit separated by the image separation unit. A first image coding unit for compressing and coding one progressive scan image; an image obtained by decoding the image coded by the first image coding unit; and a second image separated by the image separating unit. An image encoding device comprising a differencer for obtaining a difference from a progressively scanned image, and a second image encoding unit for encoding the difference. 10. A first image decoding unit for decoding the compressed image data according to claim 1, a second image decoding unit for decoding the compressed image data according to claim 1, and the first and second image decoding units. Adder for adding the decoded image data generated by the second image decoding unit, the first interlaced scan decoded image data obtained by the first image decoding unit, and the second interlaced scanning decoded image data obtained by the second image decoding unit
And an image multiplexing unit that multiplexes the interlaced scanning-decoded image data to generate a sequentially scanned image. 11. A first image decoding unit for decoding the compressed image data according to claim 2, a second image decoding unit for decoding the compressed image data according to claim 1, and the first decoding unit. An interpolating unit that extracts an image obtained by interpolating the decoded image generated by the image decoding unit, and generates an interpolating image at the same spatial position as the second interlaced scanning image; and a decoded image generated by the interpolating image and the second image decoding unit. And an adder for obtaining a second interlaced image, a first interlaced scanning decoded image obtained by the first image decoding unit and the second interlaced scanning decoded image data are multiplexed to obtain a progressive scanning image. An image decoding apparatus having a multiplexing unit for generating. 12. The image decoding apparatus according to claim 11, wherein the interpolation unit interpolates the decoded image within the same frame during sequential scanning. 13. A first image decoding unit for decoding the compressed image data of the first progressive scan according to claim 9, and a first image decoding unit for decoding the compressed image data of the second progressive scan according to claim 9. A second image decoding unit, an adder for adding the decoded image data generated by the first and second image decoding units, and the first
And a multiplex unit that multiplexes the first progressive scan decoded image data obtained by the image decode unit and the second progressive scan decoded image data obtained by the second image decode unit to generate an interlaced scan image. Image decoding device.