JPH05103313A - Method and device for processing picture information - Google Patents

Abstract

PURPOSE:To sharply reduce the processing time of a picture information processing device by detecting the motion of a picture from the DC component value of each block. CONSTITUTION:Each frame is divided into plural blocks each of which is composed of (8X8) picture elements by means of a DCT converter 14 and the value of the DC component of each block obtained by the DCT conversion is extracted through a switch 25 and DPCM decoder 28 before the DC component passes through another DCT converter 38. Then the difference between the extracted value and the value of another DC component in the same block in another picture is fetched by using a subtracter 31 and 1-frame period delay circuit 30 and an accumulator 32 performs arithmetic operation on the difference value for the sum value of 1-frame components and selects intraframe interpolation or interframe interpolation by switching a switch 33 depending upon the sum value, whether it is larger or smaller than a threshold.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image information processing method and apparatus, and more particularly to a method and apparatus for processing image information in which each screen is divided into a plurality of blocks each having a plurality of pixels.

[0002]

2. Description of the Related Art Conventionally, a method has been known in which each screen of a video signal is divided into a plurality of blocks each having a plurality of pixels, and an image is compression-encoded in units of the divided blocks. In such a technique, the amplitude value of each pixel in each block is converted into a two-dimensional frequency component by using the discrete cosine transform (hereinafter, DCT) in recent years, and the two-dimensional frequency component is subjected to variable length coding. The method of doing is attracting attention.

By the way, in a device which performs coding on a block-by-block basis as described above, when a code error occurs, the error correction circuit corrects the code error as much as possible. Must perform an error correction (interpolation) process for replacing the image of the portion where the code error has occurred with the image of another portion.

As such interpolation processing, a so-called field or frame is used in which the pixel (block) in which the error has occurred is replaced with another pixel (block) in the same screen as the pixel (block) in which the error has occurred. Internal interpolation processing,
There is known a so-called field or interframe interpolation process of replacing a pixel (block) in which an error has occurred with another pixel (block) on the screen before and after the pixel (block) in which an error has occurred.

A conventionally known process for adaptively using the field or interframe interpolation process and the field or intraframe interpolation process.

[0006]

By the way, in the case where coding is performed on a block-by-block basis as described above, if an error-correctable code occurs in each block, the entire block cannot be restored. Here, generally, a block to be compression-encoded has a two-dimensional spread, so that the correlation of images between adjacent blocks tends to be weak, and deterioration of image quality to some extent cannot be avoided only by interpolation within a field or a frame. ..

On the other hand, conversely, for example, in the field or inter-frame interpolation in which the same block on the previous screen is replaced, a certain degree of image quality can be expected, but when there is a large motion in the image, particularly in the scene change part, the large image quality is obtained. Deterioration occurs.

Therefore, it is conceivable to use adaptive interpolation which mainly performs inter-field or inter-frame interpolation and performs inter-field or intra-frame interpolation only when there is a large motion in the image.

However, when compression encoding is performed in block units as described above, in order to detect the motion of an image in each block, all pixels in the block must be compared between screens. , The calculation time or hardware must be increased.

In particular, when the variable length compression using the DCT transform as described above is performed and the image information is transmitted, a lot of processing steps are required for the decoding, and the DCT inverse transform is not performed after that. The motion of the image cannot be detected. Therefore, it takes a very long time to detect whether or not there is a large motion in the image, which is an obstacle in performing the adaptive interpolation as described above.

Under the background as described above, the present invention significantly reduces the processing time when detecting the motion of an image when processing image information in which each screen is divided into a plurality of blocks each having a plurality of pixels. It is an object of the present invention to provide an image information processing method and apparatus that can be shortened.

[0012]

[Means for Solving the Problems] Under such a purpose,
According to the image information processing method of the present invention, in the method of processing image information in which each screen is divided into a plurality of blocks each including a plurality of pixels, the value of the DC component of each block is extracted and the other screens are displayed. The values of the DC component in the same block are compared with each other, and each block of the image information is processed based on the comparison result.

Further, as one apparatus implementing this method, for each of a plurality of blocks dividing each screen, a converting means for converting the amplitude components of a plurality of pixels into a plurality of frequency components, and for each block. Encoding means for performing compression encoding processing on the plurality of frequency components, extraction means for extracting a direct current component which is one of the plurality of frequency components for each block, and direct current extracted by the extraction means. Comparing means for comparing the value of the component with the value of the direct current component in the same block on another screen; and motion detecting means for deriving motion information relating to the motion of the image information according to the output of the comparing means,
An image information processing apparatus including: a transmission unit that adds the motion information to the image information compressed by the encoding unit and transmits the image information.

Similarly, as another apparatus that implements this method, a plurality of blocks each of which is composed of a plurality of pixels and which divides each screen is compressed by encoding a plurality of frequency components. An apparatus for processing image information, a decoding means for decoding the compressed image information for each block to reproduce the plurality of frequency components, and a DC component that is one of the plurality of frequency components for each block. Extracting means for comparing the value of the DC component extracted by the extracting means with a value of the DC component in the same block on another screen, and error detection for detecting a code error in the compressed image information. Means, and an interpolation means for interpolating a block determined based on the output of the error detection means by a block of the same screen or another screen selected by the comparison means, Image information processing apparatus is presented with.

[0015]

In the image information processing method as described above, since the motion of the image can be recognized only from the value (average value) of the DC component of each block, the amount of hardware or processing time involved in motion detection for each block is reduced. it can. In addition, particularly in the processing for performing the conversion in which the DC component is extracted during the processing, almost no hardware is added, and the processing time can be greatly shortened.

[0016]

EXAMPLES Examples of the present invention will be described in detail below.

FIG. 1 is a block diagram showing the structure of a moving picture transmitting / receiving apparatus to which the present invention is applied. In the figure, 11 is a video camera for generating moving image information, 12 is an input terminal on the transmission side, 1
3 is an analog-digital (AD) converter, 14 is a DCT
A converter, 15 converts the DCT-converted data into a DC component and A
A switch for separating into C components, 26 is a run length encoder, 17 is a Huffman encoder, 18 is a DPCM encoder, 1
9 is a switch for selectively outputting the AC component and the DC component,
20 is an error correction code encoder for forming and adding an error correction code, and 21 is an output terminal on the transmission side.

Reference numeral 22 is a transmission line such as a satellite or a telephone line, and data via this transmission line 22 is input to an input terminal 23 on the receiving side.

On the receiving side, 24 is an error correction code decoder for correcting a code error using the above error correction code, 25 is a switch for separating transmitted data into a DC component and an AC component, and 26 is Huffman decoder, 27 is run length decoder, 28 is DPCM decoder, 29 is A
A switch that selectively outputs the C component and the DC component, 30 is a delay circuit for one frame period, 31 is a subtraction circuit, 32 is an accumulation circuit, 33 is a switch that can be switched according to the output of the accumulation circuit 33, and 34 is A delay circuit for a period of 1 DCT block (8 horizontal scanning lines) in the vertical direction of the screen, 35 is a switch which is switched by an error flag from the error correction code decoder 24, 36 is a delay circuit of 1 frame period, 3
8 is a DCT decoder, 39 is a digital-analog (DA)
A converter, 40 is an output terminal on the receiving side, and 41 is a monitor as a terminal on the receiving side.

The operation of the apparatus shown in FIG. 1 will be described below.

The moving image signal from the video camera 11 is input to the transmitter via the input terminal 12 on the transmitter side, converted into a digital signal by the AD converter 13, and then input to the DCT converter 14. ..

In the DCT converter 14, each screen of the digital moving picture information from the AD converter 13 is divided into blocks of (8 × 8) pixels, and DCT conversion is applied to each divided block. DCT conversion is well known (8 × 8)
For the block consisting of pixels, the amplitude value of each pixel is converted into a (8 × 8) two-dimensional frequency component,
It is a reversible transformation.

In the thus DCT-converted data, one frequency component is a so-called direct current (DC) component, which is an average value of the pixel amplitude of each block. When the frequency components other than the DC component are referred to as AC components, the switch 15 supplies the AC component to the run length encoder 16 and the DC component to the DPCM encoder 18 via the A side terminal.

The AC components are array-transformed by the run-length encoder 16 by zigzag scanning so that the 0-run length becomes long as is well known, and appropriately quantized and then run-length encoded. Then, in the Huffman encoder 17, Huffman coding is performed and the result is led to the A terminal of the switch 19.

On the other hand, for the DC component, the DPCM encoder 18 encodes the difference value between adjacent blocks, and the switch 1
9 to the D terminal.

The operation of the switches 15 and 19 will be described with reference to FIGS. 2 and 3.

FIG. 2 is a schematic diagram showing the data sequence supplied from the DCT converter 14 in FIG. 1 to the switch 15 in a time series manner. The component is supplied to the DPCM encoder 18, and the AC component is supplied to the run length encoder 16 via the A side terminal.

On the other hand, FIG. 3 is a schematic diagram showing the data sequence supplied from the switch 19 in FIG. 1 to the error correction code encoder 20 in time series. Invert. 2 and 3, A for the DC component
The ratio of the C component is different because the AC component has a higher compression rate.

The data time-serialized by the switch 19 is formed with an error correction code by an error correction code encoder 20 so that the data can be restored even if a code error occurs due to a disturbance on a long transmission line. It is added, guided to the output terminal 21 in units of this error correction code, and sent to the transmission line 22 as a transmission output.

Signals are transmitted and received from the output terminal 21 of the transmitter to the input terminal 23 of the receiver through a transmission line. As the transmission line, various things such as satellites, optical fiber cables, telephone lines, etc. can be used. Can be considered.

The transmission data string input through the input terminal 23 of the receiver is corrected in the error correction code decoder 24 by using the error correction code formed and added in the error correction code encoder 20. Is done. When an error such as a correction code occurs in the error correction code decoder 24, the error flag EF is set to "1". The error flag EF is "0" when there is no error and when the error is corrected, and is used as a control signal for the switch 35 described later.

The data string output from the error correction code decoder 24 is separated into a DC component and an AC component by the switch 25. The timing for switching the switch 25 corresponds to the arrow 56 in FIG. 3, and corresponds to the operation of the switch 19.

The AC component extracted by the switch 25 is Huffman-decoded by the Huffman decoder 26, and further run-length decoded by the run-length decoder 27.
The signal is supplied to the A terminal of the switch 29 after being subjected to the inverse quantization, the inverse conversion of the array corresponding to the above-mentioned zigzag scan and the like.

On the other hand, the DC component extracted by the switch 25 is subjected to DPCM decoding by the DPCM decoder 28 and supplied to the B terminal of the switch 29. The switch 29 is shown in FIG.
Switching is performed at the timing of arrow 55, and the data sequence is returned to the data sequence as shown in FIG.

The DC component output from the DPCM decoder 28 is the subtractor 31 and the delay line (1FD) of one frame period.
L) 30 and the difference value of the DC component of the same block of the temporally adjacent frames is obtained from the subtractor 30. Then, this difference value is supplied to the accumulator 32 and summed up for one frame.

Since the DC component obtained by the DCT conversion is the average value of the amplitude values of all the pixels in each block, the outline of the image in each frame can be known by this DC component. Therefore, when the difference value of the DC component output from the subtractor 30 is large, it is considered that either a scene change has been made between the two frames or that the motion of the image is large. When the movement of the inner image is large in these two cases, the background does not mainly change and only the main subject changes in most cases.
There are few blocks in which the difference in components is large. In contrast,
In the case of a scene change, the entire screen changes to another screen unrelated to the previous screen, so that the difference in DC component between frames is large for most blocks.

Therefore, when the total value of all blocks on the screen is larger than the predetermined threshold value, it can be determined that a scene change has occurred. If you use this principle,
Occurrence of a scene change can be detected depending on whether the total value calculated by the accumulator 32 is less than or equal to a predetermined threshold value.

Here, the 1FDL 30, the subtractor 31,
When the calculation performed by the accumulator 32 is expressed by a mathematical expression, Y =
Σ {D (X) -X}. Where X is the DC component of the DCT block, D is the delay of one frame,
Σ indicates that the total for one frame is taken.

The accumulator 32 outputs a binary signal which is the result of comparison with the above threshold value and uses it as a control signal for the switch 33. That is, if the output of the accumulator 32 is a binary value indicating a scene change, the switch 33
Causes the intra-frame interpolation signal using the 8-horizontal scanning period delay line (8HDL) 34 to be supplied to the switch 35. If the output of the accumulator 32 is a binary value indicating that it is not a scene change, the switch 33 causes the 1FDL 36 The inter-frame interpolation signal using is supplied to the switch 35.

Although various methods are conceivable as an interpolation method within a frame, in this embodiment, as shown in FIG. 4, a block in which an uncorrectable error has occurred due to blocks adjacent in the vertical direction of the screen (in the figure). (Shown with diagonal lines). FIG. 4 schematically shows the arrangement of DCT blocks on one screen. The size of the DCT block is (8
X8) Since it is for pixels, the period for transmitting a block for one horizontal row on the screen is 8 horizontal scanning periods.
In contrast to the block directly supplied from the output of the switch 29 to the block 5, the block supplied via the 8HDL 34 is the block located one level higher on the screen. Therefore,
The switch 33 is connected to the 8HDL 34 side, and the switch 3
By connecting 5 to the switch 33 side, intra-frame interpolation is performed.

As an interpolating method between frames, in the present embodiment, as shown in FIG. 3, a block in which an uncorrectable error has occurred (hatched in the figure) is interpolated by a block at the same position on the immediately preceding screen. FIG. 5 schematically shows the arrangement of DCT blocks on a plurality of temporally consecutive screens. Here, with respect to the block directly supplied to the switch 35 from the output of the switch 29, the block supplied via the 1FDL 36 is the block at the same position on the immediately previous screen. Therefore, the switch 33 is 1FDL3.
4 is connected, and the switch 35 is connected to the switch 33 side, so that inter-frame interpolation is performed.

In this way, when a block in which an uncorrectable error occurs occurs, intra-frame interpolation is performed when a scene change is made on the screen, and inter-frame interpolation is performed otherwise. ..

The data string thus adaptively interpolated is supplied from the switch 35 to the DCT inverse converter 38, converted from the frequency component value to the amplitude value by the DCT inverse conversion, and then to the DA converter 39. Converted into an analog signal,
It is output to the output terminal 40 on the receiving side and supplied to the monitor 41 which is a terminal on the receiving side.

As described above, in the present embodiment, it is determined whether or not there is a scene change only from the value of the DC component of each block, and the interframe interpolation and the intraframe interpolation are adaptively used. Therefore, the scene change can be surely determined by a small amount of data before performing the DCT inverse transform, the time required for the arithmetic processing can be shortened, the hardware scale can be reduced, and good interpolation can be performed.
Further, since this DC component is obtained at the time of encoding, no additional circuit or processing time is spent for calculating this DC component. Particularly, in the present embodiment, run length coding or Huffman coding with a long processing time is performed for the AC component, and relatively simple DPCM coding is performed for the DC component, so that the difference between these processing times is effectively used. Therefore, the processing time required for the scene change can be substantially zero.

Next, a moving picture transmitting / receiving apparatus which is another embodiment of the present invention will be described.

FIG. 6 is a block diagram showing the structure of a moving picture transmitting / receiving apparatus according to another embodiment of the present invention. In FIG. 6, the same constituents as those in FIG. 1 are designated by the same reference numerals and detailed description thereof will be omitted. ..

In the apparatus of FIG. 6, the scene change is detected on the transmitting side. That is, of the AC component and the DC component separated by the switch 15, the DC component is 1FDL5.
4 and the subtracter 52 supply the DC of the block at the same position in the adjacent frame from the subtractor 52.
Calculate the component difference. Then, the output of the subtractor 52 is summed for one frame by the accumulator 53 and compared with a predetermined threshold to detect the occurrence of a scene change,
It is output as binary data.

The binary data output from the accumulator 53 is supplied to the motion information adding circuit 50, and motion information indicating the presence / absence of a scene change is added to a predetermined position in the data string, for example, the head position of each frame. ..

A motion information extraction circuit 51 is provided on the receiving side, and the motion information is extracted from the data string output from the error correction code decoder 24 by the circuit 51. The circuit 51 supplies binary data according to this motion information to the switch 33,
Similar to the above-described embodiment, the intra-frame interpolation is performed in the scene change portion, and the inter-frame interpolation is performed in the other portions.

Also in the embodiment of FIG. 6, the moving image can be restored in the same manner as the embodiment of FIG.

Also in the embodiment shown in FIG. 6, since the motion information is formed using the DC component which does not require processing time on the transmitting side, it is advantageous in terms of processing time and hardware scale. Further, since the motion information can be formed on the transmitting side while the AC component is run-length coded or Huffman coded, the processing time does not substantially increase at all.

Furthermore, in the present embodiment, when this kind of system is popularized, the configuration of the receiver side, which is highly likely to manufacture a large number of devices, can be simplified, and in this respect, compared with the embodiment of FIG. Is advantageous.

[0053]

As described above, according to the present invention, the motion of an image can be recognized only from the value (average value) of the DC component of each block. You can save time. Further, particularly when applied to a processing device that performs conversion such that a DC component is extracted during processing, there is almost no increase in processing time or addition of hardware, and the processing time can be greatly shortened.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a moving image transmitting / receiving apparatus as an embodiment of the present invention.

FIG. 2 is a schematic diagram showing a data sequence output from the DCT converter in FIG. 1 in time series.

3 is a schematic diagram showing a data sequence supplied to the error correction code encoder in FIG. 1 in time series.

FIG. 4 is a schematic diagram for explaining an intraframe interpolation method in the apparatus of FIG.

5 is a schematic diagram for explaining an interpolating method between frames in the apparatus of FIG.

FIG. 6 is a block diagram showing a configuration of a moving image transmitting / receiving apparatus as another embodiment of the present invention.

[Explanation of symbols]

 14 DCT converter 15, 19, 25, 29, 33, 35 switch 16 run length encoder 17 Huffman encoder 18 DPCM encoder 20 error correction code encoder 24 error correction code decoder 26 Huffman decoder 27 run length decoder 28 DPCM decoder 30, 36, 54 1 frame period delay line 31, 52 Subtractor 32, 53 Accumulator 34 8 Horizontal scanning period delay line 38 DCT inverse converter

Claims (7)

[Claims] 1. A method of processing image information in which each screen is divided into a plurality of blocks each composed of a plurality of pixels, in which a value of a DC component of each block is extracted to obtain a DC component in the same block of another screen. Is compared with each other, and each block of the image information is processed based on the comparison result. 2. When a code error occurs in each block, whether to interpolate the block by a block at another position on the same screen or by a block at the same position on another screen is used as the comparison result. The image information processing method according to claim 1, wherein the switching is performed based on the image information processing method. 3. The difference value between the DC component value of each block and the DC component in the same block of another screen is summed up for one screen, and the processing of each block is changed according to this sum. The image information processing method according to claim 1. 4. The image information processing method according to claim 3, wherein motion information determined according to the total of the difference values for one screen is added to the image information of each screen and transmitted. 5. The image information is obtained by converting amplitude components of a plurality of pixels into a plurality of frequency components for each block, and a value of a DC component of each block is obtained by encoding or decoding the plurality of frequency components. The image information processing method according to claim 1, wherein the image information is extracted after conversion. 6. A conversion means for converting the amplitude components of a plurality of pixels into a plurality of frequency components for each of a plurality of blocks dividing each screen, and a compression code for each of the plurality of frequency components for each block. Coding means for performing coding processing, extraction means for extracting a DC component which is one of the plurality of frequency components for each block, and the value of the DC component extracted by the extraction means for the same block on another screen Comparing means for comparing with the value of the DC component at, motion detecting means for deriving motion information related to the motion of the image information according to the output of the comparing means, and the motion information being compressed by the encoding means. An image information processing apparatus comprising: a transmission unit that adds the image information to the image information and transmits the image information. 7. An apparatus for processing compressed image information in which a plurality of frequency components are compression-encoded for each of a plurality of blocks each of which is composed of a plurality of pixels and which divides each screen, Decoding means for decoding the compressed image information to reproduce the plurality of frequency components, extraction means for extracting a DC component which is one of the plurality of frequency components for each block, and extraction means for extracting the DC component. Comparing means for comparing the value of the direct current component with the value of the direct current component in the same block on another screen, error detecting means for detecting a code error of the compressed image information, and determination based on the output of the error detecting means. The block
An image information processing apparatus comprising: an interpolating unit that interpolates using a block of the same screen or another screen selected by the comparing unit.