JPS63234787A - Decoder for highly efficiently encoded picture signal - Google Patents

Abstract

PURPOSE:To reduce a difference in a restored picture between a static block and a movement block and to constantly obtain the restored picture of good quality by interpolating picture data removed by a sub-sampling according to the output signal of a medium filter. CONSTITUTION:Receiving data from an input terminal 21 is separated to a decision code SJ, addition data except it and a code signal in a decision code separating circuit 22 and fed to a switch circuit 23. The average information of the static and a quasi-static blocks, and the code signal and the addition data of the movement block are respectively inputted to frame decomposition circuits 24, 27, these signals and the code are separated and an error is corrected. In the respective signals, original picture element data is restored in a two dimensional and a three-dimensional decoders 25, 28 and inputted to an adaptive interpolating circuit 31 via block decomposition circuits 26, 29 and an OR gate 30. The circuit 31 receives an interpolation control signal SI from a decoder 34 and the medium filter is used in the movement block to execute an interpolation in a filed. The restored data is outputted from a terminal 32.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to a decoding device for highly efficiently encoded image signals, which is used to compress the amount of data to be transmitted compared to the amount of original data. .

[Summary of the invention]

In this invention, image data is compressed by subsampling in which the sampling phase is sequentially shifted according to the subsampling rate, image data is compressed by a high-efficiency code, and frame drop processing adapted to motion is used. In an efficiency code decoding device, when interpolating pixels thinned out by subsampling in a moving block, by using a median filter, even when moving blocks and stationary blocks repeat in a short time, A high-quality restored image can be obtained.

[Conventional technology]

When transmitting a digital video signal, a known method for compressing the amount of data to be transmitted compared to the original amount of data is to thin out pixels by subsampling and lower the sampling frequency. In subsampling, for example, image data is thinned out by 2, and the subsampling point and the position of the subsampling point used during interpolation are indicated (i.e., the data of the subsampling point above, below, or to the left or right of the interpolation point is used). A method has been proposed that transmits flag 6 (indicating whether the

However, in the case of simple subsampling, when the compression rate is increased, the deterioration of the image quality of the restored image becomes noticeable. Therefore, an encoding device that combines subsampling and a high-efficiency code adapted to the dynamic range has been proposed.

That is, the applicant of the present application determined the dynamic range defined by the maximum and minimum values of a plurality of pixels included in a two-dimensional pro-image, as described in Japanese Patent Application No. 59-266407, We are proposing a high-efficiency encoding device that performs encoding adapted to this dynamic range. In addition, as described in the specification of Toku+91 Sho 60-232789, a high-efficiency encoding device performs encoding adapted to the dynamic range of a three-dimensional block formed from pixels in areas included in each of a plurality of frames. is proposed.

Furthermore, as described in Japanese Patent Application No. 60-268817, there is a variable length encoding method in which the number of bits changes depending on the dynamic range so that the maximum distortion caused when quantization is constant. Proposed.

The encoding method adapted to these dynamic ranges is
All pixel data within a block was always encoded, regardless of the movement of the block's image.

However, when there is no movement in the image,
As described in the '840 specification, the compression ratio can be further increased by so-called frame drop processing in which only pixel data of one area within a block is encoded.

Furthermore, by using the above-mentioned encoding method that adapts to the dynamic range of the three-dimensional block and combining it with a high-efficiency encoding device that drops frames depending on the presence or absence of motion, the compression ratio can be further increased. A patent application filed in 1986-17 was developed as a highly efficient encoding device that can restore images well.
What is described in the specification of No. 9483 has been proposed. 'In the specification of this document, in the case of an encoding method that uses subsampling to reduce the sampling frequency, the drawback that deterioration of interpolated pixel data is noticeable in still parts where visually high image quality is required is improved. A technique for doing so has been disclosed. That is, in order to properly interpolate the thinned out pixel data in the still block on the receiving side, the phase of subsampling is sequentially shifted. For example, the subsampling phases of one and the other of two three-dimensional blocks occupying the same position are complementary. On the receiving side, if the current block is a static block and the previous block is a static block, the pixel data thinned out by subsampling is replaced with the pixel data that actually exists in the previous block. Ru. Therefore, there is almost no deterioration in image quality in the still portion.

Furthermore, in the case of a motion block, since data from the previous block cannot be used, intra-field spacing is performed using pixel data within the same field. This field inner sleeve space consists of two pixels located on the left and right of the pixel to be interpolated.
The average value of 4 pixels or the average value of 4 pixels located above, below, left and right is used.

[Problem that the invention seeks to solve]

In the case of a still block, the above-mentioned high-efficiency encoding device
While interpolation can be performed with almost no deterioration in image quality, motion blocks use conventional interpolation methods.
When a still block and a moving block are repeated within a short period of time, there is a problem in which the deterioration in image quality (reduction in resolution) that occurs in the moving block is noticeable.

SUMMARY OF THE INVENTION Accordingly, an object of the present invention is to provide an image signal encoding device that can suppress the deterioration in image quality that occurs between fields within a moving block and make the deterioration in image quality less noticeable even when a still block and a moving block are repeated. There is something for every thing you do.

[Means for solving problems]

In this invention, subsampling, encoding processing performed in units of blocks formed for each two-dimensional region included in at least two temporally consecutive frames, and motion detection for each block image are performed. , in the case of a still image block, in a decoding device for an image signal that has undergone high-efficiency encoding that consists of a frame-drop process that represents multiple blocks into one block, it is possible to detect pixels within the same field as the pixels targeted for interpolation. , the median is supplied with multiple pixel data existing around the pixel, the value or average value of the multiple pixel data, and pixel data at the same spatial position included in the temporally previous program. A filter is provided, and the pixel data removed by subsampling is interpolated by the output signal of the median filter.

[Effect]

Encoding is done as follows.

For example, the sum of the absolute values of the frame differences of one block of the digital video signal is compared with a threshold value, and when the sum of the absolute values is smaller than the threshold value, the block is determined to be a still block. Further, the number of pixels in each block is subsampled to, for example, two. The phase of the subsampling is shifted sequentially for the two blocks. In other words, the subsampling phase for the i-th block and the subsampling phase for the (++1)th block have a difference of one pixel. Judgment code SJ from motion judgment circuit 3
and the output data of the sub-sampling circuit 6 are converted into transmission data in the framing circuits 15 and 16. In the still block indicated by the determination code SJ, the average value of pixels in n areas within the block is transmitted instead of the original data. When this frame drop processing and encoding using a dynamic range adaptive code are combined with subsampling, the compression ratio is extremely high.

On the receiving side, when interpolating thinned out pixels, when two temporally consecutive blocks are both stationary blocks, by composing them, the thinned out pixels in the block are interpolated by the pixels of the previous block. Interpolated. Therefore, unlike interpolation using surrounding pixels, the quality of the restored image of the still area is significantly higher.

Furthermore, in the case of a motion block, a median filter is used to interpolate the thinned out pixel data. That is, the median filter is supplied with the average value of pixel data existing around the interpolation pixel or pixel data located above, below, left and right of the interpolation pixel, and also with the average value of pixel data located above, below, left and right of the interpolation pixel. pixel data is supplied. As described above, since the phase of subsampling is shifted, pixel data at the same position as the interpolated pixel exists in the previous block. The value or average value of the pixel data supplied to these median filters has a correlation with the interpolated pixel, so by interpolating the interpolated pixel with the intermediate value of the input data output from the median filter, the original Interpolated data with less error from pixel data is formed.

Therefore, the difference in the restored image between the still block and the moving block becomes small, and even if the still block and the moving block are repeated in a short period of time, a high-quality restored image can be obtained.

〔Example〕

Embodiments of the present invention will be described below with reference to the drawings. This description is given in the following order.

a. Configuration of the transmitting side; Configuration of the receiving side; C; Open circuit in the field; d; Modification a; Configuration of the transmitting side. FIG. This shows the configuration as a whole.

For example, a digital video signal (luminance signal) in which l samples are quantized to 8 bits is input to the input terminal indicated by (2). This digital video signal is supplied to a professional conversion circuit 2.

The blocking circuit 2 converts the input digital video signal into a signal in which blocks, which are units of encoding, are continuous in the time direction. In this example, as shown in FIG. 3, temporally consecutive frames Fl.

F2. F3. F4...2 frames F1 and F2, F
3 and F4. ...A block is formed in units.

Each frame is divided into two-dimensional areas (6 lines x 6 pixels), and two temporally consecutive frames, for example, Fl and F
Two-dimensional areas A1 and A occupying the same spatial position
2 forms one block. Therefore, one block includes (6x6x2=72) pixels. In FIG. 3, O indicates a pixel to be subsampled, which will be described later, and x indicates a pixel to be thinned out by subsampling.

The output signal of the blocking circuit 2 is supplied to the motion determining circuit 3. The motion determination circuit 3 is a circuit that generates a 2-bit determination code SJ for distinguishing between a still block, a semi-still block, and a moving block from data of pixels at the same position between regions of each frame of a three-dimensional block. The determination code SJ is (00) for a stationary block with no movement, the determination code SJ is (01) for a quasi-stationary block with very small movement, and the determination code SJ is (11) for a block with movement.

The motion determination circuit 3 aggregates the absolute values of the level difference (C frame difference) of the image data of the areas An and An+l belonging to two temporally consecutive frames of one block, respectively, for one block, and uses this aggregate value as the first frame difference. It is compared with threshold data THa and second threshold data THb. A determination code SJ is formed corresponding to each level relationship between the total value of the absolute value of the frame difference and the threshold data THa, THb. That is, when the total value of the absolute values of frame differences is less than or equal to the threshold value THa, it is determined that the block is a stationary block. Also,
When the total value of the absolute values of the frame differences exceeds the threshold data THa and is less than or equal to the threshold data THb, it is determined that the block is a quasi-stationary block.

Furthermore, when the total value is larger than the threshold data THa and THb, it is determined that the block is a motion block.

This motion determination is made for each block.

Note that other configurations may be used as the motion determination circuit 3, such as one that determines whether the maximum value of the absolute value of the frame difference between two frames is less than or equal to a threshold value.

The input digital video signal from the blocking circuit 2 is supplied to one input terminal of the switch circuit 4 and to the interframe average value forming circuit 5. The output signal of the inter-frame average value forming circuit 5 is supplied to the other input terminal of the switch circuit 4. When the judgment code SJ is (00) or (01), that is, when the block is a stationary block or a semi-stationary block, the output signal of the inter-frame average value forming circuit 5 is selectively obtained at the output terminal of the switch circuit 4. When the judgment code SJ is (11), that is, when the block has movement, the blocking circuit 2
An input digital video signal from the switch circuit 4 is selectively available at the output terminal of the switch circuit 4.

The inter-frame average value forming circuit 5 has two
This circuit calculates the average value of 36 pixels located at the same position in each region, and outputs the 36 average values instead of the pixel data of the block. Therefore, the output signal of the inter-frame average value forming circuit 5 has a two-dimensional block configuration in which the average values are arranged in (6 lines x 6 pixels).

The output signal of the switch circuit 4 is supplied to the sub-sampling circuit 6.

The subsampling circuit 6 performs subsampling to make the pixels of each block . A sampling pulse having a frequency of % of the original sampling frequency is supplied to an input terminal indicated by 7, and this sampling pulse and an inverted sampling pulse passed through an inverter 10 are selected in a switch circuit 9. The switch circuit 9 is switched every n frames of the three-dimensional block (n=2 in this example) by a switch control pulse from the terminal 11. Therefore, as can be seen from FIG. 3, in the sampling pattern of two temporally consecutive blocks occupying the same position on the screen, there is a difference of one pixel between the subsampled pixels.

That is, the sampling patterns are complementary between these two blocks. Furthermore, the sampling pattern within one field is in the form of a quincunx grid, as shown in FIG.

In this example, the sub-sampling rate is 1/2, but it may be 173 or more. For example, in the case of 1/3 subsampling, the subsampling point is shifted by one pixel over three temporally consecutive blocks.

The output signal of the sub-sampling circuit 6 is sent to the switch circuit 12.
supplied to Switch circuit 1 as well as switch circuit 4
2 is controlled by the judgment code SJ, and the switch circuit 12
The data of the still block and quasi-stationary block selected by the switch circuit 12 are supplied to the two-dimensional encoder 13, and the data of the block with motion selected by the switch circuit 12 is supplied to the two-dimensional encoder 13.
It is supplied to the dimensional encoder 14.

In the two-dimensional encoder 13 and three-dimensional encoder 14,
Encoding is performed with a variable number of bits adapted to the dynamic range of each block. These encoders 13.1
4, the block dynamic range DR, minimum level MIN, and 0 to 5 bit code signal DT are obtained.

The output signals of the two-dimensional encoder 13 and the three-dimensional encoder 14 are sent to a frame forming circuit 15. '16 respectively.

The determination code SJ is supplied to the framing circuits 15 and 16 via a delay circuit 17 for phase adjustment. In this embodiment, the determination code SJ, code signal DT, dynamic range DR, and minimum value MIN are transmitted.

These data are processed in the framing circuits 15 and 16.
Converted to transmission data. The format of the transmitted data is as follows:
Judgment code SJ, dynamic range DR, minimum value MI
It is possible to use a method in which each data portion consisting of the code signal DT is encoded with an independent error correction code, and the parity of each error correction code is added for transmission. Also,
The determination code SJ other than the code signal DT, the dynamic range DR, and the minimum value MIN may each be encoded with an independent error correction code. Furthermore, the judgment codes S, J
, Guinamiso clean range DR, and minimum value MIN may be encoded with a common error correction code and their parity may be added. The output signals of the framing circuits 15 and 16 are supplied to an OR gate 18, and the transmission data is taken out at an output terminal 19 of the OR gate 18. Although not shown, this transmission data is transmitted as serial data (or recorded on a recording medium).

b. Receiving side configuration FIG. 2 shows the configuration of the receiving (or reproducing) side.

The received data from the input terminal 21 is supplied to the judgment code separation circuit 22, and the judgment code SJ is separated. Also,
Additional data other than the determination code SJ and the code signal are supplied to the switch circuit 23.

The determination code SJ is supplied to the switch circuit 23.

The switch circuit 23 separates the code signal and additional data of the average value information of the stationary block and the quasi-stationary block, and supplies this average value information to the frame decomposition circuit 24. Further, the code signal and additional data of the block with motion separated by the switch circuit 23 are supplied to the frame decomposition circuit 27. Frame decomposition circuits 24 and 27 separate the code signal DT and additional codes DR and tIN, and perform error correction processing. These code signals DT and additional codes are supplied to a two-dimensional decoder 25 and a three-dimensional decoder 28, respectively.

These decoders 25, 28 are connected to the transmitting encoder 1
3. Perform the reverse process of step 14. That is, the data DT+ after removing the 8-bit minimum level is restored as a representative level, and this data and the 8-bit minimum value MIN are added to restore the original pixel data.

The two-dimensional decoder 25 forms a decoded output of the protected stationary block or quasi-stationary block. The output signal of this two-dimensional decoder 25 is supplied to a block decomposition circuit 26. In the three-dimensional decoder 28, a decoded output of a block with motion is formed. The output signal of this three-dimensional decoder 28 is supplied to a block decomposition circuit 29. The output signals of block decomposition circuits 26 and 29 are supplied to an OR gate 30. The output signal of OR gate 30 is supplied to adaptive interpolation circuit 31. The output signal of this adaptive interpolation circuit 31 is taken out to an output terminal 32.

The block decomposition circuits 26 and 29 are circuits for converting the decoded data in the order of the blocks into the same order as the scanning of the television signal, contrary to the blocking circuit 2 on the transmission side.

The adaptive interpolation circuit 31 interpolates pixel data that is not transmitted because it is not a sub-sampling point.

In the adaptive interpolation circuit 31, one of three types of interpolation methods uses the judgment code SJ of the temporally previous block;
-, and the judgment code SJ of the current block. An interpolation control signal SI for selecting an interpolation method is generated in the decoder 34. Decoder 3
4 contains the judgment code SJ from the judgment code separation circuit 22.
(ie, SJ,) and the temporally previous block (7) IJ constant code S J i-+ from the memory 33 are supplied. The interpolation control signal sr from the decoder 34 is passed through the block decomposition circuit 36 via the delay circuit 35 for phase matching.
supplied to

The interpolation control signal SI is a 2-bit code that changes every six temporally consecutive pixels. The adaptive interpolation circuit 31 adaptively selects three interpolation methods: temporal interpolation, intra-frame interpolation, and intra-field interpolation. The temporal direction interpolation is opposed to the intra-frame spacing and the intra-field spacing in the spatial direction, and is an interpolation that combines the pixels of two temporally consecutive blocks. In this temporal interpolation, the fact that the sampling patterns are shifted by one pixel between two blocks is effectively utilized. Intraframe interpolation is interpolation using data within the same frame, for example, interpolation is performed using the average value of four pixel data on the upper, lower, left, and right sides of the interpolated pixel. Further, intra-field interpolation is interpolation using data within the same field and pixel data at the same position as the interpolated pixel in a temporally previous block. This field spacing is done adaptively using a median filter, as will be described later. It is also possible to use a median filter to perform the inter-frame spacing.

The interpolation method is selected as follows by the interpolation control signal S1.

(SI=00), time direction interpolation (SI=10)
At the time of , when there is an interval within the frame (SI = 11), there is an interval within the field.As described above, the determination code SJ is defined as follows.

(Stationary block: 5J=00) (Semi-stationary block: 5
J=01) (Block with movement: 5J=decoder 34
Then, the interpolation control signal sr is generated from the judgment code SJ, - of the previous block and the judgment code SJ of the current block as follows.

For example, if the current block is a stationary block and the previous block is a stationary block or a semi-stationary block,
(SI=00), time direction interpolation is performed, and if the current block is a block with movement, it is set (SI=11) regardless of the motion determination of the previous block, and intra-field interpolation is performed. .

C. Inner field open circuit The adaptive interpolation circuit 31 provided on the receiving side is provided with a median filter, and with this median filter,
Interpolation of interpolated pixels of the motion block is performed.

FIG. 5 shows an example of a median filter.

The example shown in FIG. 5 is configured to selectively output input data of an intermediate level among three pieces of input data.

In FIG. 5, decoded data is supplied to an input terminal indicated by 41. An IH (one horizontal period) delay circuit 42 and a one sample delay circuit 43 are connected to the input terminal 41 . A 1-sample delay circuit 44 and an IH delay circuit 46 are connected to the IH delay circuit 42 .

A 1-sample delay circuit 45 is connected to the output of the 1-sample delay circuit 44, and a 1-sample delay circuit 47 is connected to the output of the IH delay circuit 46. These IH delay circuits 42, 46 and 1 sample delay circuits 43, 44, 4
5.47, four pixel data located above, below, and to the left and right of the interpolation pixel are extracted.

FIG. 6 is a diagram used to explain intra-field interpolation, where Fl-F4 indicates four temporally consecutive frames, and B1 and B2 indicate two temporally consecutive frames. Blocks are shown. Pixel data a, b, c included in the same field as the interpolation pixel X of block B2 and surrounding the interpolation pixel X.

d is IH delay circuit 42, 46 and 1 sample delay circuit 43
．． Retrieved from 44.45.47.

Pixel data a and b are supplied to an adder circuit 48, and pixel data C and d are supplied to an adder circuit 49.

The output signals of these adder circuits 48 and 49 are sent to the doubling circuit 5.
0 and 51, respectively, and the output of the doubling circuit 50 is
'The average value of A (a+b) is generated, and the average value of V2 (C+d) is generated at the output of the doubling circuit 51.

These average values are input to the input terminals 53a and 5 of the selector 53.
3b respectively. Further, the input terminal 53c of the selector 53 is supplied with pixel data i at the same position as the interpolated pixel X of the previous block B1 from the input terminal 52. This pixel data i is extracted by a two-frame delay circuit (not shown).

Furthermore, three pieces of data to be processed by the median filter (% (a + b), 'A (c + d)
.

The comparison circuit 54 determines the magnitude relationship between the two data items in i).
, 55.56, it is beaten 8 times. These comparison circuits 5
The output signals of 4, 55, and 56 are supplied to the judgment circuit 57. This judgment circuit 57 examines the intermediate level data among the three pieces of data, and generates a control signal J'l2 for the selector 53 to select the intermediate level data. The output of the selector 53 is obtained as the output signal of the median filter at the output terminal 58. Also, the next frame F4 of block B2 in FIG.
The interpolation of the interpolation pixel y is performed in the same manner. That is, f%
Among the three pixel data (e+f), 'A (g+h), and it, intermediate level data is selected and used as interpolation data.

Since the pixel data fed to the median filter often has a correlation with the real pixel data,
By selecting an intermediate level, it is possible to prevent interpolation using data that is significantly different from the actual pixel data.

FIG. 7 shows another example of the median filter.

In the median filter shown in FIG.
2.66 and 1 sample delay circuit 63,64°65.67
Data is supplied from the input terminal 61, and pixel data a, b, c, and d around the interpolation pixel, for example, X are extracted. Further, pixel data, for example i, at the same position of the previous block is supplied from the input terminal 68. The comparison circuit 70 determines the magnitude relationship between two of these five pixel data.
, 71.72, 73.74, 75.76.77.78°79, and the determination circuit 80 examines data having an intermediate level.

The selector 69 is controlled by the output signal of the judgment circuit 80, and the five input terminals 69a, 69b, 6 of the selector 69 are
Data having an intermediate level among the five pixel data (a, b, c, d, i) supplied to 9c, 69d, and 69e, respectively, is transferred from the output terminal 69f of the selector 69 to the output terminal 8.
It is taken out at 1. The median filter shown in FIG. 7, like the median filter shown in FIG. 6, can perform interpolation using data at a level close to the actual pixel data.

d. Modifications The present invention is not limited to variable length encoding systems, but can also be applied to fixed length encoding systems. In the identification length encoding method, the dynamic range DR of each block is divided into a number of level ranges determined by the number of quantization bits, and a code signal with a predetermined number of focuses corresponding to the level range to which the data after minimum value removal belongs is formed. be done.

Further, the three-dimensional block is limited to two frames, but may be composed of data of three or more n frames.

〔Effect of the invention〕

According to this invention, when interpolating pixels thinned out by subsampling, a median filter is used in the motion block, so the average value of pixel data around the top, bottom, left and right of the interpolated pixel is used. This method can prevent deterioration of image quality compared to performing interpolation using the method described above, and can prevent deterioration of image quality from becoming noticeable when a still block and a moving block are repeated in a short period of time.

[Brief explanation of drawings]

Fig. 1 is a block diagram of an embodiment of the present invention, Fig. 2 is a block diagram showing the configuration of the receiving side, and Figs. 3 and 4 are explanations of blocks and subsampling, which are units of encoding processing. The route map to be used, FIG. 5 is a block diagram showing the configuration of an example of the median filter, FIG. 6 is a route map for explaining the interpolation operation by the median filter, and FIG. 7 is a block diagram showing another example of the median filter. It is. Explanation of main symbols in the drawings 21: Received signal input terminal, 25: Two-dimensional decoder, 2
8: Three-dimensional decoder, 26.29, 36: Block decomposition circuit, 31: Adaptive interpolation circuit, 32: Output terminal of restored data.

Claims (1)

[Claims] Subsampling, encoding processing performed in units of blocks formed for each two-dimensional region included in at least two temporally consecutive frames, and motion processing for each image of the block. is detected, and in the case of a still image block, a pixel to be interpolated is detected in an image signal decoding device that performs high-efficiency encoding that includes frame drop processing to represent a plurality of blocks as one block. In the same field, a plurality of pixel data existing around the pixel, a value or an average value of the plurality of pixel data, and pixel data at the same spatial position included in a temporally previous block. A decoding device for a highly efficiently encoded image signal, comprising a supplied median filter, and interpolating pixel data removed by subsampling using an output signal of the median filter.