JPH04324782A - Picture processing method and device - Google Patents

Abstract

PURPOSE:To detect a motion of an object in pictures over two picture data in a short time with high accuracy while reducing the effect of a noise component. CONSTITUTION:Two fields of a video signal of one frame converted into a digital signal at an A/D converter 102 are stored respectively to an odd number field memory 104 and an even number field memory 105. A transformation device 107 applies DCT conversion to a reference block segmented from a data of the odd field memory 104 and a specific frequency component is extracted from the output and stores it to a reference block register 109. DCT conversion is applied to a referenced block segmented from a data of the even field memory 105 and inputs its output to a comparator 111. Simultaneously, a data of the reference block register 109 is inputted to the comparator, in which the both are compared.

Description

[Detailed description of the invention]

0001

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and apparatus for detecting the movement of an object in a moving image signal, for example.

0002

BACKGROUND OF THE INVENTION There has been a demand for printing a specific instantaneous image of a television image as a still image, but in the conventional television signal, the amount of information per field or frame is small, and it can be viewed as a still image. However, the images were grainy and there were few opportunities to use them.

However, in recent years, high-definition television signals, so-called high-vision, have been proposed, and this
Because it has more than twice the amount of information, it is still worth viewing even when viewed as a still image. Furthermore, with advances in computer technology, the use of outputting computer-processed images to printers as still images is expanding.

[0004] Therefore, a moving image/still image converting device that outputs a video of a specified field or frame of a moving image signal as a still image has been commercialized. Conventional television systems, including high-definition, have adopted an interlaced system, in which one frame image is formed by two field images formed by interlaced scanning in units of 1/60 seconds. ing. Therefore,
There is a time lag of 1/60 second between fields, and if the subject is moving, simply combining two adjacent field images will not yield a still image of the desired quality. .

[0005] Therefore, by using a motion vector detection method such as the matching method or the density gradient method, the amount and direction of movement of the moving object between fields, that is, the motion vector, is determined, and the image is corrected based on this to obtain a clear image without any deviation. A conversion device for forming a still image has been proposed.

[0006]

However, the conventional matching method requires a huge amount of calculation time and amount to detect a motion vector, and has the disadvantage that it is susceptible to noise components in an image in principle. In addition, the concentration gradient method has the disadvantage that although the amount of calculation is small, the accuracy is insufficient, and it is not possible to obtain sufficient detection accuracy especially for parts with large amounts of movement.
On the other hand, it had the disadvantage that it resulted in unnatural images.

Furthermore, both methods were developed for encoding moving images, and were not suitable for use in constructing clear still images. Additionally, there was a problem in terms of processing speed. The present invention has been made in view of the above conventional example, and an object of the present invention is to detect the movement of an object in an image between two images in a short time, with high precision, and while reducing the influence of noise components.

[0008]

Means for Solving the Problems In order to achieve the above object, the image processing method and apparatus of the present invention have the following configuration. In this method, given image data is subjected to DCT transformation, and movement in the target image is detected from the transformed time-series image data for a plurality of screens. The apparatus also includes a storage means for storing image data for a plurality of screens, a transformation means for performing orthogonal transformation on the image data stored by the storage means, and a transformation means for performing orthogonal transformation on each of the plurality of image data by the transformation means. and detection means for detecting movement in the image stored in the storage means based on the obtained converted data.

[0009]

[Operation] With the above configuration, any two blocks in image data can be once converted into data representing the characteristics of the image, and then they can be compared.

0010

DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 is a diagram showing the configuration of an image processing apparatus according to a first embodiment of the present invention. In the figure, 101 is a high-definition television signal input, 102 is an A/D converter,
104 is an odd field memory for temporarily storing odd field image data, 105 is an even field memory for temporarily storing even field image data, and 103 is for transmitting odd field data to the odd field memory 104 in response to a digitized high-definition television input signal. , is a switch that allocates even field data to 105.

107 is a two-dimensional discrete cosine transform (hereinafter D
This is a DCT converter that performs a CT (denoted as CT) on block data cut out from the odd field memory 104 or the even field memory 105. 10
6 is a switch for selecting field data to be subjected to DCT, and 109 is a reference block (this is the starting point for motion vector detection and is cut out from the odd field).
A reference block register 110 temporarily holds data (frequency components) after DCT has been applied to the image. The frequency component limiter 108 to be excluded is a switch that switches to the reference block register 109 side when applying DCT to the reference block, and to the comparator 111 side during motion vector search (applying DCT to the reference block). 111 is a comparator that determines the degree of similarity between the reference block and the reference block and whether they match or do not match; 112 is the output result thereof;

FIGS. 3(a) and 3(b) illustrate the principle of two-dimensional discrete cosine transform (DCT). FIG. 3A shows an image data block having a size of 8 pixels x 8 pixels, and is, for example, a reference block. Figure 3(b) shows the result of applying two-dimensional discrete cosine transformation to the previous image data block, and each element of the matrix is, for example, F11 represents the DC component of the image data block, and as the subscript increases, Represents higher-order frequency components.

The shaded area in FIG. 3(b) is the frequency component limiter 1.
10 shows an example of frequency components removed in step 10. Next, the principle of two-dimensional discrete cosine transformation will be described. When performing two-dimensional discrete cosine transformation on image data with a block size of M×N, the input is f(k1, k2) k1=0,1,...,M-1
k2=0,1,...,N-1 The output (two-dimensional discrete cosine transformation result) is F(m1, m2
) m1= 0,1,...,M-1m2= 0,1
,...,N-1, the transformation and inverse transformation are as shown in Equation 1 and Equation 2 below, respectively.

<Conversion>

[0015]

[Math 1]

<Inverse transformation>

[0017]

[Math 2]

However,

[0019]

FIG. 4 shows the state of motion vector search, and the basic operation is to search for an arbitrary block 401 in an odd field.
This is to detect which position within the even field the object has moved to after 1/60 second. At this time, in order to reduce the amount of calculation and remove the influence of noise in the image, only the frequency components that best represent the features in the image block are focused on and searched. In the figure, 403 is the block searched the first time around the position corresponding to the reference block position, and the block 403 searched the second time around the most similar block among them is 403.
04, and 405 is the block that is searched for the third time centering on the most similar block among them.

[0021] A motion vector is specified by repeating such operations. The operation will be explained below with reference to FIG. When an analog video signal is input to an input 101, the signal is digitized by an A/D converter 102 and is temporarily stored in an odd field memory 104 and an even field memory 105 for each field. Next, a reference block of 8 pixels x 8 pixels is cut out from the odd field memory 104 to be used as a base point for motion vector search, and a DCT transformer 107 applies DCT to it. The converted data is filtered by a frequency component limiter 110 to remove high frequency components that are considered to include many noise components, and then the switch 10
8 and stored in the reference block register 109.

Next, the switch 103 is switched, and reference blocks around the reference block position are sequentially extracted from the even field memory 105, subjected to DCT by the DCT converter 107, and then processed by the frequency component limiter 11.
0, the same processing as the reference block is performed, and switch 10
8 to the comparator 111. A comparator 111 compares each frequency component of the converted data of the standard block and the converted data of the reference block, and calculates the degree of similarity between the two blocks. A simple method for calculating the degree of similarity is to calculate the number of matching frequency components.

[0023] This completes the comparison of the pair of standard block and reference block. By repeating this process while changing the reference block, the motion vector of the image is detected. Figure 2
2 shows an example of a system using an embodiment of the present invention, and 201, 202, and 203 are a TV camera, a VTR, and a TV, respectively, which are image supply sources. 204 is a motion vector detector of this embodiment, 205 is an interpolator, and 206 is a printer.

[0024] TV camera 201, VTR 202, TV 2
A signal of a specific instantaneous video in the video supplied from 03 is input to the motion vector detector 204 in units of frames. One frame of input data is temporarily stored in the odd field memory 104 and even field memory 105 in FIG. 1, and a motion vector between the fields is detected. Interpolator 205
In the step, one frame of data is corrected based on the detected motion vector to form a clear still image, and the result is converted into a hard copy by the printer 206.

In this embodiment, two-dimensional discrete cosine transform has been described as the orthogonal transform, but the present invention is not limited to this, and orthogonal transforms such as Hadermarl transform, Haar transform, and Fourier transform can be used. Further, in this embodiment, the size of the cutout block is shown as 8 pixels x 8 pixels, but it is not limited to this size. However, usually the amount of calculation,
It is decided after considering the effects etc. (power of 2) x (
A power of 2) size is often selected.

Furthermore, in this embodiment, the motion is detected between two fields constituting one frame, but it is also possible to detect motion between any two fields or between frames. However, there will be some differences in implementation costs. In addition, in this embodiment, image data is first converted into a digital signal using an A/D converter, but if an already digitized signal is input, the A/D converter is used to convert the image data into a digital signal.
No converter is required.

Furthermore, although a frequency component limiter was used in this embodiment, as can be seen from the above formula (Equation 1), it is also possible to calculate only the necessary frequency components, and in this case, the amount of calculation can be reduced. can be reduced. In addition, although this embodiment is used for printing out television images, it is not limited to this, and if accuracy and processing time (depending on the hardware used) are taken into account, image encoding, feature extraction, image processing, etc. Needless to say, it can be applied to searches, etc.

[Other Embodiments] FIG. 5 is a drawing showing a second embodiment. In the figure, 101 is a high-definition television signal input, and 102 is an A/D converter. 104 is an odd field memory for temporarily storing odd field image data, 105 is an even field memory for temporarily storing even field image data, and 103 is for transmitting odd field data to the odd field memory 104 in response to a digitized high-definition television input signal. , the even field data is 105
This is a switch to allocate to.

Reference numeral 107 denotes a DCT transformer that performs discrete cosine transformation on block data cut out from the odd field memory 104 or the even field memory 105. 106 is a switch for selecting field data to be subjected to DCT; 109 is a reference block (
A reference block register 113 temporarily holds data after DCT has been applied to the block (which is the starting point for motion vector detection and is cut out from the odd field); reference block register 113 selectively extracts an arbitrary frequency component from the reference block register 109; The frequency component selector 108 is a switch that switches to the reference block register 109 side when applying DCT to the reference block, and switches to the comparator 111 side during motion vector search (applying DCT to the reference block). Reference numeral 111 is a comparator for determining the degree of similarity and coincidence/mismatch between the standard block and the reference block, and 112 is an output of the comparison result.

The motion vector search can be illustrated using FIG. 4 of the first embodiment. The basic operation is to detect which position in the even field an arbitrary block 401 in the odd field has moved to after 1/60 second. At this time, in order to reduce the amount of calculation and remove the influence of noise in the image, we first perform a rough search focusing on the low frequency components within the block, and then gradually move on to a search that also includes high frequency components. In the figure, 403 is a block searched focusing only on low frequency components, and 40 blocks are searched focusing on the most similar block and also including middle frequency components.
4, and 405 is the block that is searched centering on the most similar block and including high frequency components.

FIG. 6 shows a block of 8 pixels x 8 pixels subjected to DCT.
This is a converted diagram. From the block in Figure 6(a) to Figure 6(b)
), but first, the match of the frequency components is checked for the shaded area in FIG. 6(b), and the frequencies to be compared are increased using matching or similar blocks as candidates. A motion vector is identified by repeating such operations.

The operation will be explained below with reference to FIG. When an analog video signal is input to the input 101, the signal digitized by the A/D converter 102 is sent to the switch 10.
3, the data is temporarily stored in the odd field memory 104 and the even field memory 105 for each field. A reference block with a size of 8 pixels x 8 pixels is cut out from the accumulated image data in the odd field memory 104 to serve as a base point for motion vector search, and a DC is applied to the cut out reference block.
After DCT is applied by T-transformer 107, the generated spectrum is stored in reference block register 109 via switch 108. After that, even field memory 10
From the image data No. 5, the surroundings of the position corresponding to the reference block are sequentially cut out as a reference block, and the DCT transformer 10
7, DCT is applied to selectively obtain only the low frequency component rather than the entire spectrum, and the output thereof is input to the comparator 111. On the other hand, the frequency component selector 110 takes out only the same frequency components as those obtained from the reference block from the reference block register 109 and inputs them to the comparator 111. The comparator 111 compares the two input frequency components and calculates the degree of similarity between the standard block and the reference block. A simple method for calculating the degree of similarity is to calculate the number of matching frequency components.

With the above operations, the comparison of the low frequency components of the pair of blocks is completed. This process is repeated by changing the reference block, and several blocks that are found to be similar to the reference block are selected. Furthermore, the comparison is repeated by using these as candidates and changing the frequency components to be compared, and finally the reference block is narrowed down to one reference block and a motion vector is detected. In this example, motion is detected using DCT transformation, so motion detection can be performed at high speed and with high precision.

[0034]

Effects of the Invention As explained above, the image processing method and apparatus according to the present invention can detect the movement of an object in an image between two images in a short period of time with high precision, while reducing the influence of noise components. This has the effect of detecting

[Brief explanation of drawings]

FIG. 1 is a block diagram illustrating a first embodiment of the present invention.

FIG. 2 is a diagram of an example system utilizing the first embodiment of the present invention.

[Figure 3] Block size is 8x8 for the first example
FIG. 2 is a diagram illustrating two-dimensional discrete cosine transformation of FIG.

FIG. 4 is a diagram illustrating how a motion vector search is performed.

FIG. 5 is a block diagram illustrating a second embodiment of the present invention.

[Fig. 6] Regarding the second embodiment, the block size is 8 x 8.
FIG. 2 is a diagram illustrating two-dimensional discrete cosine transformation of FIG.

[Explanation of symbols]

104 Field memory 105 Field memory 107 Two-dimensional discrete cosine transformer 109
Reference block register 110 Frequency component limiter 111 Comparator

Claims (6)

[Claims] 1. An image processing method characterized by subjecting given image data to DCT transformation and detecting motion in a target image from the transformed time-series image data for a plurality of screens. 2. A storage means for storing image data for a plurality of screens, a transformation means for performing orthogonal transformation on the image data stored by the storage means, and an orthogonal transformation performed by the transformation means on each of the plurality of image data. 1. An image processing apparatus comprising: a detection means for detecting movement in an image stored in the first storage means based on the converted data obtained by performing the above conversion. 3. The image processing apparatus according to claim 2, wherein the comparison means focuses on and compares a predetermined frequency component of the converted data. 4. The image processing apparatus according to claim 2, wherein the comparison means selects frequency components to be compared at an appropriate time. 5. The image data stored by the storage means includes two images constituting one frame in an interlaced system.
3. The image processing apparatus according to claim 2, wherein the image processing apparatus is one field data. 6. The image processing apparatus according to claim 2, wherein the converting means selectively obtains a specific frequency component by conversion.