JP3200108B2 - Image processing method and apparatus - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and apparatus for processing image information, and more particularly to a method and apparatus for processing image information in which each screen is divided into a plurality of blocks each having a plurality of pixels.

[0002]

2. Description of the Related Art Conventionally, there has been known a method of dividing each screen of a video signal into a plurality of blocks each including a plurality of pixels, and compressing and encoding an image in units of the divided blocks. Among such techniques, in recent years, the discrete cosine transform (hereinafter referred to as DCT) is used to convert the amplitude value of each pixel of each block into a two-dimensional frequency component, and the two-dimensional frequency component is subjected to variable-length coding. Attention is being paid to the method of doing so.

[0003] By the way, in such a device that performs encoding in units of blocks, when a code error occurs, the error is corrected as much as possible by an error correction circuit. Must perform an error correction (interpolation) process for replacing the image of the part where the code error has occurred with the image of another part.

[0004] As such interpolation processing, a so-called field or frame is used in which another pixel (block) on the same screen as the pixel (block) in which the error has occurred replaces the pixel (block) in which the error has occurred. Internal interpolation processing,
There is known a so-called field or inter-frame interpolation process in which a pixel (block) in which an error occurs is replaced with another pixel (block) in the screen before and after the pixel (block) in which the error occurs.

[0005] Processing for adaptively using these field or interframe interpolation processing and field or intraframe interpolation processing is conventionally known.

[0006]

When coding is performed in units of blocks as described above, if an error-correctable code occurs in each block, the entire block cannot be restored. Here, since a block to be subjected to compression coding generally has a two-dimensional spread, the correlation of an image tends to be weak between adjacent blocks, and a certain degree of image quality deterioration is inevitable only by interpolation within a field or a frame. .

On the other hand, in the case of field or inter-frame interpolation, for example, where the same block in the previous screen is replaced, a certain image quality can be expected. Deterioration occurs.

[0008] Therefore, it is conceivable to use adaptive interpolation which mainly performs field or inter-frame interpolation and performs field or intra-frame interpolation only when there is a large motion in an image.

However, in the case where the compression encoding is performed in units of blocks as described above, if the motion of an image in each block is to be detected, all pixels in the block must be compared between screens. However, the time required for the operation or the hardware must be increased.

In particular, when image information is transmitted by performing variable-length compression using the DCT transform as described above, a large number of processing steps are required for the decoding, and the above-described process must be performed after the DCT inverse transform is performed. The motion of the image cannot be detected. Therefore, it takes a very long time until it is detected whether or not a large motion has occurred in the image, which has been an obstacle to performing the adaptive interpolation as described above.

Under the above-described background, the present invention significantly reduces the processing time when detecting image motion when processing image information in which each screen is divided into a plurality of blocks each including a plurality of pixels. It is an object of the present invention to provide an image information processing method and apparatus which can be shortened.

[0012]

Means for Solving the Problems Under such a purpose,
According to the image processing method of the present invention, a method of processing image data in which each screen is divided into a plurality of blocks each including a plurality of pixels, wherein an error correction process for correcting an error in the image data, The difference value between the DC component value of each block in the image data subjected to the error correction processing and the DC component of the block at the same position on the previous screen is accumulated for one screen, and this accumulation result is calculated as a predetermined value. A determination process of determining the presence or absence of a scene change by comparing with a threshold value, and an error block including an uncorrectable error in the image data when another scene change is determined by the determination process, using another block of the same screen. In addition to the interpolation, when it is determined that there is no scene change in the determination process, an interpolation process of interpolating the error block by a block on the previous screen is performed.

According to the image processing apparatus of the present invention, there is provided an apparatus for processing image data in which each screen is divided into a plurality of blocks each including a plurality of pixels, and an input means for inputting the image data. Error correction means for correcting an error in the input data; and a difference value between a DC component value of each block in the image data output from the error correction means and a DC component of a block at the same position on the previous screen. For one screen, and comparing the accumulation result with a predetermined threshold value to determine the presence or absence of a scene change;
An error block containing an uncorrectable error is interpolated by another block on the same screen when the scene change is judged by the judgment unit and another scene change is judged by the judgment unit; And interpolating means for interpolating the erroneous block with the block of the previous screen.

[0014]

[0015]

In the image information processing method as described above, since the motion of an image can be recognized only from the DC component value (average value) of each block, the amount of hardware or processing time involved in motion detection in block units is reduced. it can. In particular, in a process for performing a conversion such that a DC component is extracted during the process, hardware is hardly added, and the processing time can be greatly reduced.

[0016]

Embodiments of the present invention will be described below in detail.

FIG. 1 is a block diagram showing a configuration of a moving picture transmitting / receiving apparatus to which the present invention is applied. In the figure, reference numeral 11 denotes a video camera for generating moving image information, 12 denotes an input terminal on the transmission side, 1
3 is an analog-digital (AD) converter, 14 is a DCT
A converter 15 converts the DCT-converted data into a DC component and A
A switch for separating into C components, 26 is a run-length encoder, 17 is a Huffman encoder, 18 is a DPCM encoder, 1
9 is a switch for selectively outputting an AC component and a DC component,
Reference numeral 20 denotes an error correction code encoder for forming and adding an error correction code, and reference numeral 21 denotes an output terminal on the transmission side.

Reference numeral 22 denotes a transmission line such as a satellite or a telephone line. Data transmitted through the transmission line 22 is input to an input terminal 23 on the receiving side.

On the receiving side, reference numeral 24 denotes an error correction code decoder for correcting a code error using the above-described error correction code; 25, a switch for separating transmitted data into a DC component and an AC component; Huffman decoder, 27 is a run-length decoder, 28 is a DPCM decoder, 29 is A
A switch for selectively outputting the C component and the DC component, 30 is a delay circuit for one frame period, 31 is a subtraction circuit, 32 is an accumulation circuit, 33 is a switch that can be switched according to the output of the accumulation circuit 33, and 34 is A delay circuit for one DCT block (eight horizontal scanning lines) in the vertical direction of the screen; 35, a switch that can be switched by an error flag from the error correction code decoder 24;
8 is a DCT decoder, 39 is a digital-analog (DA)
A converter, 40 is an output terminal on the receiving side, and 41 is a monitor as a receiving terminal.

The operation of the apparatus shown in FIG. 1 will be described below.

A moving image signal from the video camera 11 is input to the transmitter via the input terminal 12 on the transmitter side, is converted into a digital signal by the AD converter 13, and then is input to the DCT converter 14. .

The DCT converter 14 divides each screen of the digital moving image information from the AD converter 13 into blocks of (8 × 8) pixels, and performs DCT conversion on each of the divided blocks. DCT transform is known (8 × 8)
Is to convert the amplitude value of each pixel into a (8 × 8) two-dimensional frequency component for a block of pixels
It is a reversible transformation.

In the DCT-transformed data, one frequency component is a so-called direct current (DC) component, which is an average value of the amplitude of the pixels in each block. If the frequency components other than the DC component are referred to as AC components, the switch 15 supplies the AC component to the run-length encoder 16 and the DC component to the DPCM encoder 18 via the A-side terminal.

As is well known, the AC component is subjected to zigzag scan by the run-length encoder 16 to perform array conversion so that the 0-run length becomes longer, quantized as appropriate, and then run-length encoded. Then, in the Huffman encoder 17, Huffman encoding is performed, and the signal is guided to the A terminal of the switch 19.

On the other hand, for the DC component, the difference value between adjacent blocks is encoded by the DPCM encoder 18 and the switch 1
9 to the D terminal.

Here, the operation of the switches 15 and 19 will be described with reference to FIGS.

FIG. 2 is a schematic diagram showing, in chronological order, a data sequence supplied from the DCT converter 14 to the switch 15 in FIG. 1. The switch 15 is inverted at the arrow 55 in FIG. The component is supplied to the DPCM encoder 18 and the AC component is supplied to the run-length encoder 16 via the A-side terminal.

On the other hand, FIG. 3 is a schematic diagram showing the data sequence supplied from the switch 19 in FIG. 1 to the error correction encoder 20 in a time-series manner. Invert. FIGS. 2 and 3 show that A
The ratio of the C component is different because the AC component has a higher compression ratio.

An error correction code is formed by an error correction code encoder 20 so that the data serialized by the switch 19 can be restored even if a code error occurs due to disturbance in a long transmission path. The signal is added to the output terminal 21 in error correction code units, and sent to the transmission line 22 as a transmission output.

Signals are transmitted and received from the output terminal 21 of the transmitter to the input terminal 23 of the receiver via a transmission line. The transmission line may be any of various types such as a satellite, an optical fiber cable, and a telephone line. Can be considered.

The transmission data string input through the input terminal 23 of the receiver is corrected by an error correction code decoder 24 using an error correction code formed by the error correction code encoder 20 to correct a code error. Is performed. When a correction code error occurs in the error correction code decoder 24, the error flag EF is set to "1". The error flag EF is “0” when there is no error or when the error is corrected, and is used as a control signal of the switch 35 described later.

The data string output from the error correction code decoder 24 is separated by a switch 25 into a DC component and an AC component. The timing of switching the switch 25 corresponds to the arrow 56 in FIG. 3 and corresponds to the operation of the switch 19.

The AC component extracted by the switch 25 is subjected to Huffman decoding by a Huffman decoder 26, and is further run-length decoded by a run-length decoder 27.
After being subjected to inverse quantization, inverse transformation of an array corresponding to the above-described zigzag scan, and the like, the resultant signal is supplied to the A terminal of the switch 29.

On the other hand, the DC component extracted by the switch 25 is subjected to DPCM decoding by the DPCM decoder 28 and supplied to the B terminal of the switch 29. The switch 29 is shown in FIG.
Are switched at the timing indicated by the arrow 55, and are chronologically returned to a data string as shown in FIG.

The DC component output from the DPCM decoder 28 is supplied to a subtracter 31 and a delay line (1FD) for one frame period.
L) 30 and the difference value of the DC component of the same block in the temporally adjacent frame is obtained from the subtractor 30. Then, this difference value is supplied to the accumulator 32 and is summed up for one frame.

Since the DC component obtained by the DCT conversion is an average value of the amplitude values of all the pixels in each block, an outline of an image in each frame can be known from the DC component. Therefore, when the difference value of the DC component output from the subtracter 30 is large, it is considered that either a scene change has been made between two frames or the motion of the image is large. When the movement of the inner image in these two cases is large, the background does not change mainly and only the main subject changes in most cases.
There are few blocks in which the difference between components is large. In contrast,
In the case of a scene change, the entire screen changes to another screen irrelevant to the immediately preceding screen, so that the DC component difference between frames becomes large for most blocks.

Therefore, when the total value of all blocks on the screen is larger than the predetermined threshold value, it can be determined that a scene change has occurred. If you use this principle,
The occurrence of a scene change can be detected based on whether or not the total value calculated by the accumulator 32 is equal to or less than a predetermined threshold.

Here, the 1FDL 30, the subtractor 31,
When the operation performed by the accumulator 32 is expressed by a mathematical expression, Y =
{D (X) -X}. Here, X is the DC component of the DCT block, D is the delay of one frame,
Σ indicates that a total of one frame is taken.

The accumulator 32 outputs a binary signal, which is a result of comparison with the threshold, and serves as a control signal for the switch 33. That is, if the output of the accumulator 32 is a binary value indicating a scene change, the switch 33
, An intra-frame interpolation signal using an 8 horizontal scanning period delay line (8HDL) 34 is supplied to a switch 35, and if the output of the accumulator 32 is a binary value indicating that it is not a scene change, the switch 33 outputs a 1FDL 36. Is supplied to the switch 35.

Various methods are conceivable as an interpolation method in a frame. In this embodiment, as shown in FIG. 4, a block in which an uncorrectable error has occurred due to an adjacent block in the vertical direction of the screen (see FIG. 4). (Indicated by diagonal lines). FIG. 4 schematically shows the arrangement of DCT blocks on one screen. The size of the DCT block is (8
× 8) Since pixels are used, the period during which one horizontal row of blocks is transmitted on the screen is eight horizontal scanning periods.
The block supplied via 8HDL 34 to the block directly supplied from the output of the switch 29 to the block 5 is the block located one level higher on the screen. Therefore,
The switch 33 is connected to the 8HDL 34 side, and the switch 3
5 is connected to the switch 33 side, so that intra-frame interpolation is performed.

As a method of inter-frame interpolation, in this embodiment, as shown in FIG. 3, a block in which an uncorrectable error has occurred (shown by hatching in the figure) is interpolated by a block at the same position on the immediately preceding screen. FIG. 5 schematically shows the arrangement of DCT blocks on a plurality of temporally continuous screens. Here, the block supplied via the 1FDL 36 to the block directly supplied from the output of the switch 29 to the switch 35 is the block located at the same position on the immediately preceding screen. Therefore, the switch 33 is set to 1FDL3
4 and the switch 35 is connected to the switch 33, so that inter-frame interpolation is performed.

As described above, when a block in which an uncorrectable error occurs occurs, intra-frame interpolation is performed when a scene change is made on the screen, and inter-frame interpolation is performed otherwise. .

The data string adaptively interpolated in this way is supplied from the switch 35 to the DCT inverse converter 38, and is converted from the frequency component value to the amplitude value by the DCT inverse transform. Is converted to an analog signal
The signal is output to the output terminal 40 on the receiving side and supplied to the monitor 41 which is the terminal on the receiver side.

As described above, in the present embodiment, it is determined whether or not a scene change has occurred from only the DC component value of each block, and inter-frame interpolation and intra-frame interpolation are adaptively used. Therefore, a scene change can be reliably determined with a small amount of data before performing the DCT inverse transform, the time required for the arithmetic processing can be reduced, the hardware scale can be reduced, and good interpolation can be performed.
Further, since this DC component is obtained at the time of encoding, no additional circuit or processing time is required for calculating this DC component. In particular, in the present embodiment, run-length coding or Huffman coding with a long processing time is performed for the AC component, and relatively simple DPCM coding is performed for the DC component. Therefore, the difference between these processing times is effectively used. The processing time required for the scene change can be substantially reduced to zero.

Next, a moving image transmitting / receiving apparatus according to another embodiment of the present invention will be described.

FIG. 6 is a block diagram showing the configuration of a moving picture transmitting / receiving apparatus according to another embodiment of the present invention. In the drawing, the same components as those in FIG. 1 are denoted by the same reference numerals, and detailed description is omitted. .

In the apparatus shown in FIG. 6, a scene change is detected on the transmitting side. That is, of the AC component and the DC component separated by the switch 15, the DC component is
4 and the subtractor 52, the DC of the block located at the same position in the adjacent frame is supplied from the subtractor 52.
Calculate the difference between the components. Then, the output of the subtracter 52 is summed for one frame by the accumulator 53, and is compared with a predetermined threshold to detect the presence or absence of a scene change.
It is output as binary data.

The binary data output from the accumulator 53 is supplied to a motion information adding circuit 50, and motion information indicating the presence or absence of a scene change is added to a predetermined position in the data string, for example, a head position of each frame. .

On the receiving side, a motion information extraction circuit 51 is provided. The circuit 51 extracts the motion information from the data string output from the error correction code decoder 24. The circuit 51 supplies binary data according to the motion information to the switch 33,
As in the above-described embodiment, the configuration is such that intra-frame interpolation is performed in a scene change portion, and inter-frame interpolation is performed in other portions.

In the embodiment shown in FIG. 6, it is possible to restore a moving image as in the embodiment shown in FIG.

Also, in the embodiment of FIG. 6, since the motion information is formed by using the DC component which does not require the processing time on the transmitting side, it is advantageous in terms of the processing time and the hardware scale. Furthermore, since the formation of motion information on the transmission side can be performed during run-length coding or Huffman coding of the AC component, the processing time does not substantially increase at all.

Further, in this embodiment, when this type of system is popularized, the configuration of the receiver side where a large number of devices are likely to be manufactured can be simplified, and in this respect, compared with the embodiment of FIG. It is advantageous.

[0053]

As described above, according to the present invention,
The difference value between the DC component screens of each block is accumulated for one screen, and the presence / absence of a scene change is determined based on the result of this accumulation, reducing the amount of birds or processing time involved in determining a scene change. it can.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a moving image transmitting / receiving apparatus as one embodiment of the present invention.

FIG. 2 is a schematic diagram showing a data sequence output and supplied from a DCT converter in FIG. 1 in a time-series manner.

FIG. 3 is a schematic diagram showing a data sequence supplied to the error correction code encoder in FIG. 1 in time series.

FIG. 4 is a schematic diagram for explaining an interpolation method in a frame in the apparatus of FIG. 1;

FIG. 5 is a schematic diagram for explaining an inter-frame interpolation method in the apparatus of FIG. 1;

FIG. 6 is a block diagram showing a configuration of a moving image transmitting / receiving apparatus as another embodiment of the present invention.

[Explanation of symbols]

 14 DCT Transformer 15, 19, 25, 29, 33, 35 Switch 16 Run Length Encoder 17 Huffman Encoder 18 DPCM Encoder 20 Error Correction Encoder 24 Error Correction Encoder Decoder 26 Huffman Decoder 27 Run Length Decoder 28 DPCM decoder 30, 36, 54 One frame period delay line 31, 52 Subtractor 32, 53 Accumulator 34 8 Horizontal scanning period delay line 38 DCT inverse transformer

Continuation of the front page (58) Field surveyed (Int.Cl. ⁷ , DB name) H04N ^7/ 24-7/68 H04N 1/41-1/419 G06T 9/00

Claims (2)

(57) [Claims] 1. A method for processing image data in which each screen is divided into a plurality of blocks each including a plurality of pixels, wherein: an error correction process for correcting an error in the image data; The difference value between the DC component value of each block in the applied image data and the DC component of the block at the same position on the previous screen is accumulated for one screen, and the accumulation result is compared with a predetermined threshold value. A discriminating process for discriminating the presence or absence of a scene change, and interpolating an error block including an uncorrectable error in the image data with another block on the same screen when it is determined that a scene change is detected by the discriminating process, An interpolation process for interpolating the error block with a block on the previous screen when it is determined by the determination process that there is no scene change. 2. An apparatus for processing image data in which each screen is divided into a plurality of blocks each including a plurality of pixels, comprising: input means for inputting the image data; and correcting an error in the input data. Error correction means, and accumulates a difference value between a DC component value of each block in the image data output from the error correction means and a DC component of a block at the same position on the previous screen for one screen, and A determination unit that determines the presence or absence of a scene change by comparing the calculation result with a predetermined threshold; and an error block including an error that cannot be corrected by the error correction unit when the determination unit determines that a scene change has occurred. Interpolation is performed by other blocks on the screen, and when the determination unit determines that there is no scene change, the error block is replaced by the block on the previous screen. Image processing apparatus comprising an interpolation means for between.