JP3804745B2 - Frame structure / field structure switching type image coding apparatus - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a frame structure / field structure switching type image coding apparatus, and more particularly to an image coding apparatus using motion compensation prediction capable of coding a digital moving image signal in either a frame structure or a field structure. .
[0002]
[Prior art]
Digital moving image information is expressed by a sequence of sampled still images. There are two types of image representation methods, one is called a non-interlaced image or progressive image, and the other is called an interlaced image. Next, the structure of a non-interlaced image and an interlaced image is shown.
[0003]
FIG. 2 shows an image of a circular object moving from left to right as a non-interlaced image. The moving image is sampled every t1 time and expressed as a continuous still image every t1 time. The resolution of the moving image is equal to the resolution of the sampled still image.
[0004]
On the other hand, FIG. 3 is a video representation of the same video as FIG. 2 as an interlaced image. The video is expressed by alternating an image including only odd-numbered scanning lines and an image including only even-numbered scanning lines every t2. There are two methods for expressing this interlaced image.
[0005]
One is a field structure in which only the odd-numbered scanning lines or only the even-numbered scanning lines are expressed as one image. The resolution of the field image is ½ in the direction perpendicular to the frame image. The other is that one image is created from two consecutive fields of images. This is the frame structure. In this case, the images in different fields are alternately arranged for each horizontal line of the pixel column.
[0006]
FIG. 12 shows a block diagram of a conventional video encoding apparatus. Conventionally, as shown in the figure, a moving image encoding apparatus is configured to encode with a picture structure (field structure or frame structure) designated in advance.
[0007]
FIG. 13 is a block diagram showing the configuration and operation of the moving picture encoding apparatus of FIG. In FIG. 13, when the input image signal 1 of the first screen is input, each switch is connected to the (1) side by the prediction mode control unit 12, and the input signal is used to obtain high coding efficiency. Directly input to the orthogonal transformer 3, the orthogonal transformer 3 performs orthogonal transformation using DCT (discrete cosine transformation) or the like, and the quantizer 4 quantizes the orthogonal transformation coefficient. This quantized coefficient is converted into a variable length code such as a Huffman code by the first variable length encoder 5 and input to the video multiplexer 15.
[0008]
On the other hand, the quantization coefficient input to the inverse quantizer 6 is inversely quantized, and the image data is restored by the inverse orthogonal transformer 7. The restored image data is stored in the frame memory 9. Also, the video multiplexer 15 multiplexes the encoded data from the first variable length encoder 5 and the quantization information 18 from the quantizer 4 and outputs the result as an encoded video data output 16.
[0009]
When the input image signal 1 of the next screen is input, the encoding mode control unit 12 connects each switch to the contact on the (2) side, and the input image signal 1 is input to the prediction signal subtracter 2 and the motion. Input to the detector 10. The motion detector 10 detects a motion vector from the input image signal 1 and the reference image input from the frame memory 9, and the motion vector is input to the position shifter 11 and the second variable length encoder 14. In the second variable length encoder 14, the motion vector information is converted into a variable length code such as a Huffman code and input to the video multiplexer 15.
[0010]
In the position shifter 11, an image signal specified by a motion vector is extracted from the frame memory 9 and output as a motion compensation prediction signal to the prediction signal subtracter 2 and the local decoding adder 8. The motion compensation prediction signal is subtracted from the input image signal 1 by the prediction signal subtracter 2, and the prediction error is encoded. The prediction error signal is orthogonally transformed using DCT (Discrete Cosine Transform) or the like in the orthogonal transformer 3 to obtain high coding efficiency, and the signal quantized by the quantizer 4 is the first variable length encoder 5. Is converted into a variable length code such as a Huffman code. Further, in order to use the same prediction signal as that on the decoding side, the quantization coefficient obtained by the quantizer 4 is inversely quantized by the inverse quantizer 6 and the prediction error signal is locally decoded by the inverse orthogonal transformer 7. . Further, the motion compensated prediction signal is added to the prediction error signal restored by the local decoding adder 8 and accumulated in the frame memory 9.
[0011]
In moving picture compression coding using motion compensated prediction coding, generally, the higher the correlation between images for motion compensated prediction, the higher the coding efficiency. Therefore, for video that is close to a still image, the encoding efficiency is improved by increasing the interval between the encoded image and the reference image for which motion compensation predictive encoding is performed. Then, the encoding efficiency is lowered. This is because the correlation between the encoded image and the reference image becomes low, and motion compensation prediction is not effective.
[0012]
The difference at the time of encoding due to the difference in picture structure between the frame structure and the field structure is the minimum distance between this encoded image and the reference image used for prediction. When the frame structure is used, the minimum value of the interval from the reference image is time t1 in FIG. 3, but in the case of the field structure, it can be set to time t2 (half of time t1) in FIG. is there. Therefore, in a video with intense motion, by adopting a field structure, motion compensated prediction coding can be effectively operated, and as a result, coding efficiency can be improved.
[0013]
In a moving image compression method capable of encoding with either a frame structure or a field structure, encoding with a “field structure” in which one picture image to be encoded is encoded corresponding to one field image; It is possible to use either “frame structure” encoding in which one image to be encoded is encoded corresponding to one interlaced frame image. However, conventionally, before the moving image is encoded, either the frame structure or the field structure is designated from the outside in advance, and the designated structure is fixedly used for the inputted moving image. Then, encoding is performed and encoded moving picture information is output.
[0014]
[Problems to be solved by the invention]
In the image coding by the above-described conventional method, as shown in FIG. 12, coding is performed with a fixed picture structure regardless of the feature of the image. Therefore, for example, when coding a picture material with a high motion that improves the coding efficiency by adopting a field structure, if the picture structure of the coding is designated as a frame structure in advance, the frame structure Therefore, the encoding efficiency is continued, resulting in a decrease in encoding efficiency. Conversely, when encoding with the field structure is specified, even if the encoding efficiency with the frame structure is improved, the encoding efficiency is improved because the field structure is fixedly used. do not do.
[0015]
If the input video is an interlaced image or a non-interlaced image, it is determined whether the input image is an interlaced image by some other method in advance. Based on the information thus obtained, a two-stage method is realized in which the picture structure is switched from the outside during encoding. Such a two-stage scheme is not possible when encoding in real time is assumed.
[0016]
An object of the present invention is to solve the above-mentioned problems of the prior art, automatically perform interlace / non-interlace determination of an input image for an input image having no information about the image characteristics and structure, and Images that improve the coding efficiency and improve the quality of the coded image by analyzing the characteristics of the input image and adaptively changing the picture structure in the video compression coding to the frame structure / field structure. It is to provide an encoding device.
[0017]
[Means for Solving the Problems]
In order to achieve the above object, the present invention comprises means for determining whether a continuously input image is an interlaced image or a non-interlaced image, and whether the input image is an interlaced image. In order to determine whether the image is a non-interlaced image, the spatial correlation of pixels in the vertical direction at an arbitrary position in the image is measured, and the pixels satisfying the following conditions (1) and (2) If the number is equal to or greater than a certain percentage of the number of pixels that satisfy equation (1), the image is determined to be an interlaced image, and if it is determined to be an interlaced image, field structure encoding is selected. The first feature is that the encoding in the structure is selected.
Max (d (0, -2), d (0,2), d (-1,1)) <threshold… (1)
(Max (d (0, -2), d (0,2), d (-1,1)) + offset) < Min (d (0, -1), d (0,1)) … (2 )
Here, d (a, b) represents the absolute difference value of the pixels in the vertical columns a and b .
In addition, the present invention obtains a change amount between two images only for an image determined to be an interlaced image, and selects a frame structure when the change amount is smaller than a predetermined value and a field structure when the change amount is larger. The second feature is that an image that is not determined to be an interlaced image has a frame structure selected .
[0018]
According to the above-described feature, it is possible to eliminate a decrease in encoding efficiency due to a change in the feature of the input image that could not be avoided when selecting a conventional fixed frame structure / field structure. Since the interlace / non-interlace distinction that needs to be recognized is also automatically detected during encoding, efficient encoding can be performed regardless of the characteristics and structure of the input image. It becomes possible.
[0019]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing a configuration of an embodiment of the present invention. In the following description, the encoding apparatus shown in FIG. 13 is used as the moving image encoding method, but the present invention is not limited to this.
[0020]
In this embodiment, it is determined whether or not the image is an interlaced image from continuously input image signals (still image signals). If the image is an interlaced image, a reduced feature plane is created. The encoding in the frame structure / field structure is determined based on the result of the simple motion search process using the reduced feature plane.
[0021]
In FIG. 1, an interlace / non-interlace image discriminating unit 2 discriminates continuously input image signals 1 whether the image is an interlace image or a non-interlace image. The result is output to the reduced feature plane creation unit 4 as interlace / non-interlace discrimination information 3. The reduced feature plane creation unit 4 creates reduced feature plane information 5 reflecting image features for the image determined as an interlaced image by the interlace / non-interlaced image determination unit 2, and the reduced feature plane information 5 is output to the simple motion search unit 6. The simple motion search unit 6 performs a simple motion search process between two reduced feature planes, and outputs the motion compensation prediction error amount at that time to the frame structure / field structure determination unit 8 as image change amount information 7.
[0022]
The frame structure / field structure determination unit 8 uses the image change amount information 7 obtained from the simple motion search unit 6 to encode in the frame structure when the change amount is small, and in the field structure when the change amount is large. The encoding is determined and output to the moving image encoding unit 10 as the picture structure control signal 9. The moving image encoding unit 10 performs moving image encoding on the input image signal 1 in accordance with the picture structure control signal 9 instructed from the frame structure / field structure determining unit 8 and outputs encoded moving image information 11. Here, the moving picture encoding unit 10 follows the designation of the frame structure / field structure by the picture structure control signal 9, for example, the motion compensator 10, the first variable length encoder 5 and the second variable variable in FIG. The operation of the long encoder 14 is switched to a scheme adapted to encoding in the frame structure / field structure.
[0023]
Next, an example of the configuration and operation of each unit in FIG. 1 will be described in detail. First, the interlace / non-interlace image discrimination unit 2 will be described. Whether or not the image is an interlaced image is determined by calculation using several adjacent pixels from the input image information. FIG. 4 shows the structure of the input frame image information. Image information is formed by an array of pixels arranged spatially and uniformly. From this image information, as shown in FIG. 5, the value of five pixels continuous in an arbitrary position in the vertical direction is taken out, and the absolute difference value between the two pixels is calculated therein.
[0024]
The absolute difference value to be calculated is 0 and −2, 0 and 2, −1 and 1 in five pixels from p (−2) to p (2) where p (0) is a pixel located at the center in the vertical direction. The absolute difference value between the pixels belonging to the same field and the absolute difference value between the pixels belonging to different fields of 0 and −1 and 0 and 1 are obtained. First, it is verified whether or not the following condition (1) is satisfied.
[0025]
Max (d (0, -2), d (0,2), d (-1,1)) <threshold value ... (1)
Next, when the condition of the expression (1) is satisfied, it is verified whether or not the condition of the following expression (2) is satisfied.
[0026]
(Max (d (0, -2), d (0,2), d (-1,1)) + offset) <Min (d (0, -1), d (0,1)) ... (2)
Where d (a, b) represents the absolute difference value of a, b, Max (a, b, c) is the maximum value of a, b, c, and Min (a, b, c) is a, b , c represents the minimum value. That is, when the pixel values in the same field are similar and the maximum absolute difference value is less than the threshold value (a constant value), the minimum absolute difference value in the different field is the maximum absolute difference value in the same field. It is verified whether or not the value obtained by adding offset (a constant value) to is exceeded. If this processing is performed at all pixels in the image, or at any number of positions, and the points satisfying the expressions (1) and (2) exceed a certain percentage of the points satisfying the expression (1), The image is determined to be an interlaced image, and the result is output to the reduced feature plane creation unit 4 for each frame as interlace / non-interlace determination information.
[0027]
In this example, the interlace / non-interlace determination is performed using five vertical pixels, but the number of pixels is compared between adjacent pixels in the same field and pixels in a different field. If the number of pixels is 3 or more, verification is possible with an arbitrary number of pixels. Furthermore, the pixel positions for the above verification can be performed for all pixel positions in the image. For example, assuming that 5 pixels are in the vertical direction and n pixels are in the horizontal direction as one block, a specific position in the block is used. Alternatively, it is also possible to perform a sample point survey for performing the above-described verification at an arbitrary position. Furthermore, it is also possible to randomly extract arbitrary points.
[0028]
Next, the process of the reduced feature plane creating unit 4 in FIG. 1, that is, the process of creating a reduced plane reflecting the image features from the original image will be described with reference to FIG. First, the original image is divided into small blocks, and each small block is represented by a representative value. In the present invention, the standard deviation of the pixel value for each small block is used as the representative value. In addition, an average value and a median value can be used as the representative value. In addition, as the pixel value in this calculation, the luminance component of the pixel can be used, and other components or their average can be used. The size of the small block can also be set arbitrarily. The size of the reduced feature plane is horizontal: H / ph, vertical: V with respect to the size of the original image (horizontal: H pixels, vertical: V pixels), where the small block size is horizontal: ph pixels and vertical: pv pixels. / pv and the number of samples is 1 / (ph × pv) with respect to the number of pixels of the original image. A reduced plane having the standard deviation of the small block as a representative value is referred to as reduced feature plane information 5.
[0029]
Next, the process of the simple motion search unit 6 in FIG. 1 will be described. The simple motion search unit 6 performs a motion search process between two reduced feature planes from the reduced feature plane information 5 created by the reduced feature plane creation unit 4. The temporal distance between the reference plane on which the simple motion search is performed and the target plane is a fixed arbitrary value. As a motion search method, a motion search by block matching can be used. In this case, the block can take any natural number in both the horizontal and vertical sizes on the reduced feature plane. Therefore, it is possible to perform a block unit motion search with a minimum of one sample unit and a maximum of one entire reduced feature plane as one block.
[0030]
Referring to FIG. 7, the upper left coordinate of the set block is (k, l), the element on the reduced feature plane 1 is c (k, l), and the element on the reduced feature plane 2 is r (k l), the horizontal size of the block is N, the vertical size is M, and the search range is ± sh in the horizontal direction and ± sv in the vertical direction. In this simple motion search, the average motion compensation prediction error amount E (k, l) of one element is obtained from the minimum error amount within the search range as shown in equations (3) and (4) in FIG.
[0031]
The prediction error amount E (k, l) can be subjected to a square root process after obtaining a square error amount, or an absolute difference value can be used. The prediction error amount E (k, l) obtained by this simple motion search process is performed for all the blocks on the reduced feature plane 1, and the total Esum in the reduced feature plane 1 is obtained. The motion compensation prediction error amount Esum is an index representing the magnitude of change between two images. Then, the value Esum is output as image change amount information to the frame structure / field structure determining unit. Next, the frame structure / field structure determination unit 8 in FIG. 1 performs the threshold processing of this value from the input image change amount information for each frame, and if the value is greater than or equal to the threshold, the field structure is less than the threshold. In this case, it is determined to take a frame structure, and the determination result is output as a picture structure control signal 9 to the moving picture encoding unit 10.
[0032]
1 performs compression coding of an input image signal using a picture structure specified by a picture structure control signal 9 output from a frame structure / field structure determination unit 8. The encoded moving picture information 11 is output. Specifically, the moving image encoder 10 performs operations of the motion compensator 10, the first variable length encoder 5, and the second variable length encoder 14 of FIG. 13, for example, according to the picture structure. / Switch to a method that is suitable for encoding with a field structure.
[0033]
FIG. 8 is a block diagram showing the configuration of the second exemplary embodiment of the present invention. The same reference numerals in FIG. 1 as those in FIG. This embodiment includes a reduced feature plane creation unit 4, a simple motion search unit 6, and a frame structure / field structure determination unit 8, and the frame structure / field structure determination unit 8 performs simple motion search in the simple motion search unit 6. It is characterized in that the coding with the field structure is selected when the image change amount information 7 obtained by the processing is large, and the coding with the frame structure is selected when it is small.
[0034]
FIG. 9 is a block diagram showing the configuration of the third exemplary embodiment of the present invention. The same reference numerals in FIG. 1 as those in FIG. This embodiment is composed of an interlace / non-interlace image discriminating unit 2 and a frame structure / field structure determining unit 8. The interlace / non-interlace image discriminating unit 2 determines whether or not the input image is an interlaced image. If it is determined that the image is an interlaced image, the frame structure / field structure determination unit 8 selects a field structure. If it is determined that the image is a non-interlaced image, the encoding in the frame structure is selected. There is a feature in the point.
[0035]
FIG. 10 is a block diagram showing the configuration of the fourth exemplary embodiment of the present invention. The same reference numerals in FIG. 1 as those in FIG. This embodiment includes an interlace / non-interlace image discrimination unit 2, an interlace / non-interlace switching unit 21, a reduced feature plane creation unit 4, a simple motion search unit 6, and a frame structure / field structure determination unit 8. Thus, the interlace / non-interlace image discriminating unit 2 determines the interlace / non-interlace image of the input image based on one or more images that are input first, and based on this determination, the interlace / non-interlace image is discriminated. The race / non-interlace switching unit 21 sets the switch to “0” or “1”, and for subsequent input images, the interlace / non-interlace image discriminating unit 2 performs the interlace / non-interlace discrimination. It is characterized in that it is not performed.
[0036]
FIG. 11 is a block diagram showing the configuration of the fifth embodiment of the present invention. The same reference numerals in FIG. 1 as those in FIG. This embodiment includes an interlace / non-interlace image discriminating unit 2, an interlace / non-interlace switching unit 21, and a frame structure / field structure determining unit 8. In the interlace / non-interlace image discriminating unit 2, The determination of the interlaced / non-interlaced image of the input image is made based on one or a plurality of images inputted first, and based on this determination, the interlace / non-interlaced switching unit 21 switches the switch to “ A feature is that the interlace / non-interlace image discriminating unit 2 is set to 0 ”or“ 1 ”so that the interlace / non-interlace image discriminator 2 does not discriminate interlace / non-interlace.
[0037]
【The invention's effect】
As is clear from the above description, in the conventional fixed picture structure encoding, the video with high encoding efficiency was limited. However, according to the present invention, the video according to the characteristics and changes of the input image is limited. Since encoding is selected in accordance with the picture structure, high encoding efficiency can be maintained regardless of what feature image is input or even if the image feature changes midway.
[0038]
In addition, as a moving picture coding method using motion compensated predictive coding that can be coded by either the frame structure / field structure, a video coding simulation experiment using the MPEG-2 video coding method is performed, As a result, when compression coding is performed at a coding rate of 4 Mbit / s, the present invention can improve the image quality by about 0.4 dB to 1.0 dB in PSNR as compared with the case where the frame structure is fixed. did it.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of a first embodiment of the present invention.
FIG. 2 is a diagram illustrating a configuration of an image in a non-interlace structure.
FIG. 3 is a diagram illustrating a configuration of an image in an interlace structure.
FIG. 4 is a diagram illustrating a configuration of a frame image.
FIG. 5 is a diagram for explaining a pixel for calculating an absolute difference amount;
FIG. 6 is a diagram illustrating creation of a reduced feature plane.
FIG. 7 is a diagram illustrating a simple motion search process.
FIG. 8 is a block diagram showing a configuration of a second exemplary embodiment of the present invention.
FIG. 9 is a block diagram showing a configuration of a third exemplary embodiment of the present invention.
FIG. 10 is a block diagram showing a configuration of a fourth exemplary embodiment of the present invention.
FIG. 11 is a block diagram showing a configuration of a fifth exemplary embodiment of the present invention.
FIG. 12 is a block diagram illustrating a conventional moving image encoding method.
FIG. 13 is a block diagram of a video encoding apparatus using motion compensation.
FIG. 14 is a diagram illustrating mathematical expressions.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 ... Image signal, 2 ... Interlace / non-interlace image discriminating unit, 4 ... Reduced feature plane creating unit, 6 ... Simple motion search unit, 8 ... Frame structure / field structure determining unit, 10 ... Moving picture encoding unit, 21: Interlace / non-interlace switching unit.

Claims (3)

In a frame structure / field structure switching type image encoding apparatus capable of encoding an interlaced image with either a field structure or a frame structure,
Means for discriminating whether continuously input images are interlaced images or non-interlaced images;
In order to determine whether the input image is an interlaced image or a non-interlaced image, the spatial correlation of pixels consecutive in the vertical direction at an arbitrary position in the image is measured, and the following (1) , ( When the number of pixels satisfying the condition shown in equation ( 2) is greater than or equal to a certain ratio of the number of pixels satisfying equation (1) , it is determined as an interlaced image,
Frame structure / field structure switching type image characterized in that encoding with field structure is selected when discriminated as an interlaced image, and encoding with frame structure is selected in other cases Encoding device.
Max (d (0, -2), d (0,2), d (-1,1)) <threshold… (1)
(Max (d (0, -2), d (0,2), d (-1,1)) + offset) < Min (d (0, -1), d (0,1)) … (2 )
Here, d (a, b) represents the absolute difference value of the pixels in the vertical columns a and b . In a frame structure / field structure switching type image encoding apparatus capable of encoding an interlaced image with either a field structure or a frame structure,
Means for discriminating whether continuously input images are interlaced images or non-interlaced images;
In order to determine whether the input image is an interlaced image or a non-interlaced image, the spatial correlation of pixels consecutive in the vertical direction at an arbitrary position in the image is measured, and the following (1) , ( When the number of pixels satisfying the condition shown in equation ( 2) is greater than or equal to a certain ratio of the number of pixels satisfying equation (1) , it is determined as an interlaced image,
Only for images determined to be interlaced images, the amount of change between the two images is obtained, and when the amount of change is smaller than a predetermined value, the frame structure is selected. A frame structure / field structure switching type image coding apparatus characterized in that a frame structure is selected for an image that has not been processed .
Max (d (0, -2), d (0,2), d (-1,1)) <threshold… (1)
(Max (d (0, -2), d (0,2), d (-1,1)) + offset) < Min (d (0, -1), d (0,1)) … (2 )
Here, d (a, b) represents the absolute difference value of the pixels in the vertical columns a and b . The frame structure / field structure switching type image encoding device according to claim 2,
Frame structure / field structure switching type image coding characterized by determining whether an input image is an interlaced image or a non-interlaced image based on one or a plurality of images input at the beginning of the input image apparatus.

Images