JP2001061149A - Image coder of switching frame structure/field structure - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a dynamic image coder than adaptively selects a frame structure or a field structure, in response to the features of an input image. SOLUTION: A discrimination result of an interlaced/noninterlaced image discriminating section 2 is outputted to a reduction feature planar generating section 4. The generating section 4 generates a reduction feature plan reflecting the feature of an image that is discriminated to be an interlace image. A simple motion retrieval section 6 applies a simple motion retrieval processing between two reduction feature planes at constant time intervals and outputs a motion compensation prediction error amount in this case as image change amount information. A frame structure/field structure decision section 8 decides coding by a frame structure, when the image change amount is small and decides coding by a field structure, when image change amount is large on the basis of the image change amount information and provides the output of image structure control signal. A dynamic image coding section 10 applies dynamic image codings to a received image signal, in accordance to the image structure control signal and provides the output of a compression coded dynamic image.

Description

Description: BACKGROUND OF THE INVENTION [0001] 1. Field of the Invention [0002] The present invention relates to a frame structure / field structure switching type image coding apparatus, and more particularly to a method for coding a digital moving image signal in either a frame structure or a field structure. The present invention relates to an image coding apparatus using motion compensated prediction capable of performing the following. 2. Description of the Related Art Digital moving image information is represented by a sequence of sampled still images. There are two types of image representation methods, one of which is called a non-interlaced image or a progressive image, and the other of which is called an interlaced image. Next, the structure of a non-interlaced image and an interlaced image will be described. FIG. 2 shows an image in which a circular object is moving from left to right as a non-interlaced image. The moving image is sampled every t1 time and is represented by a continuous still image every t1 time. The resolution of the moving image is equal to the resolution of the sampled still image. [0004] On the other hand, FIG. 3 shows a video image similar to that of FIG. 2 expressed by an interlaced image. The video is represented by an image containing only the odd-numbered scanning lines and an image containing only the even-numbered scanning lines at every t2 time. There are two types of expression methods for this interlaced image. One is a field structure in which only the odd-numbered scanning lines or only the even-numbered scanning lines are expressed as one image. The resolution of the field image is １ ／ in the vertical direction with respect to the frame image. The other is one in which one image is created from continuous two-field images. This is the frame structure. The image in this case is
Images of different fields are alternately arranged for each horizontal line of the pixel column. FIG. 12 shows a block diagram of a conventional moving picture coding apparatus. Conventionally, as shown in the figure, a moving picture coding apparatus is configured to perform coding with a pre-designated picture structure (field structure or frame structure). FIG. 13 is a block diagram showing the configuration and operation of the moving picture coding apparatus shown in FIG. In FIG. 13, when the input image signal 1 of the first screen is input, each switch is connected to each side by the prediction mode control unit 12, and the input signal is converted to an orthogonal transform signal to obtain high coding efficiency. 3 is input directly to the orthogonal transformer 3,
The orthogonal transform is performed using CT (discrete cosine transform) or the like,
The quantizer 4 quantizes the orthogonal transform coefficients. The quantized coefficients are converted into a variable length code such as a Huffman code by the first variable length encoder 5 and input to the video multiplexer 15. On the other hand, the quantized coefficients input to the inverse quantizer 6 are inversely quantized, and the image data is restored by the inverse orthogonal transformer 7. The restored image data is stored in the frame memory 9
Is accumulated in The video multiplexer 15 multiplexes the coded data from the first variable length coder 5 and the quantized information 18 from the quantizer 4 to output a coded video data output 1.
Output as 6. When the input image signal 1 for the next screen is input, each switch is connected to the side contact by the encoding mode control unit 12, and the input image signal 1 It is input to the detector 10. In the motion detector 10, a motion vector is detected from the input image signal 1 and the reference image input from the frame memory 9, and the motion vector is detected by the position shifter 11 and the second variable length encoder 1
4 is input. In the second variable length encoder 14, the motion vector information is converted into a variable length code such as a Huffman code and input to the video multiplexer 15. The position shifter 11 extracts an image signal specified by a motion vector from the frame memory 9 and
It is output to the prediction signal subtractor 2 and the local decoding adder 8 as a motion compensation prediction signal. The motion compensation prediction signal is subtracted from the input image signal 1 by the prediction signal subtractor 2, and the prediction error is encoded. The prediction error signal is orthogonally transformed using a DCT (Discrete Cosine Transform) in the orthogonal transformer 3 in order to obtain a high coding efficiency, and the signal quantized by the quantizer 4 is converted into a first variable-length encoder 5. Is converted into a variable length code such as a Huffman code. Further, in order to use the same prediction signal as that on the decoding side, the quantization coefficient obtained by the quantizer 4 is inversely quantized by the inverse quantizer 6, and the prediction error signal is locally decoded by the inverse orthogonal transformer 7. . Further, the motion compensation prediction signal is added to the prediction error signal restored by the local decoding adder 8,
It is stored in the frame memory 9. In compression coding of a moving image using motion compensation prediction coding, generally, the higher the correlation between the images to be subjected to motion compensation prediction, the higher the coding efficiency. Therefore, in a video close to a still image, the coding efficiency is improved by increasing the interval between the encoded image and the reference image to be subjected to motion compensation prediction encoding. If this happens, the coding efficiency will decrease. This is because the correlation between the encoded image and the reference image becomes low, and the motion compensation prediction becomes ineffective. The difference in encoding due to the difference in picture structure between the frame structure and the field structure lies in the minimum distance between the encoded image and a reference image used for prediction. When the frame structure is used, the minimum value of the interval from the reference image is time t1 in FIG. 3, but in the case of the field structure, it can be time t2 (half of time t1) in FIG. is there. Therefore, in a moving image,
By adopting the field structure, motion-compensated prediction coding can be effectively operated, and as a result, coding efficiency can be improved. In a moving image compression method capable of coding in either a frame structure or a field structure, one picture image to be coded is encoded in a “field structure” in which one picture image is coded corresponding to one field image. It is possible to use either of encoding and encoding in a “frame structure” in which one image to be encoded corresponds to one interlaced frame image. However, conventionally, before coding a moving image, a frame structure or a field structure is designated from the outside in advance, and the designated structure is fixedly used for an input moving image. Then, encoding is performed, and encoded moving image information is output. [0014] In the image coding according to the above-mentioned conventional method, as shown in FIG. 12, a fixed picture structure is applied irrespective of the characteristics of the image. . Therefore, for example, even when encoding a rapidly moving image material whose encoding efficiency is improved by adopting a field structure, if the encoding picture structure is specified in advance as a frame structure, the frame structure Will be continued, and as a result, the coding efficiency will be reduced. Conversely, when coding in the field structure is specified, even if the coding efficiency in the frame structure is improved, the coding efficiency is improved because the field structure is fixedly used. do not do. If it is not known whether the input video is an interlaced image or a non-interlaced image, it is determined in advance by another method whether or not the input image is an interlaced image. , The picture structure is switched from outside at the time of encoding based on the determined information. Such a two-stage method is not possible when coding in real time is assumed. An object of the present invention is to solve the above-mentioned problems of the prior art, and to automatically determine whether an input image is interlaced or non-interlaced for an input image having no information on the characteristics and structure of the image. In addition, the characteristics of the input image are analyzed, and the picture structure in the moving image compression coding is adaptively changed to the frame structure / field structure, thereby improving the coding efficiency and improving the coded image quality. It is an object of the present invention to provide an improved image encoding device. In order to achieve the above object, the present invention provides a means for discriminating whether a continuously input image is an interlaced image or a non-interlaced image. The first feature is that, when an image is determined to be an interlaced image, encoding in a field structure is selected, and in other cases, encoding in a frame structure is selected. Further, the present invention has a second feature in that the correlation of pixel information of a sampled still image is used when the interlaced image is detected. A third feature of the present invention is that a change amount of the input interlaced image is calculated from the input interlaced image, and the frame structure / field structure is switched based on the calculated value. A fourth feature of the present invention is that a simple motion search using a deviation of a pixel in units of small blocks is used to calculate a change amount of an input image. According to the above-mentioned feature, it is possible to eliminate a decrease in coding efficiency due to a characteristic change of an input image which cannot be avoided when a conventional fixed frame structure / field structure is selected. In order to automatically detect the distinction between interlaced and non-interlaced images that had to be grasped beforehand at the time of encoding, efficient encoding is performed regardless of the characteristics and structure of the input image. It is possible to do. Hereinafter, the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing the configuration of one embodiment of the present invention. In the following description, the encoding device shown in FIG. 13 is used as a moving image encoding method, but the present invention is not limited to this. In this embodiment, it is determined whether or not an image is an interlaced image based on a continuously input image signal (still image signal). Is created, and the coding in the frame structure / field structure is determined based on the result of the simple motion search processing using the reduced feature plane. In FIG. 1, a continuously input image signal 1 is discriminated by an interlaced / non-interlaced image discriminating section 2 as to whether the image is an interlaced image or a non-interlaced image. The discrimination result is output to the reduced feature plane creating unit 4 as interlace / non-interlace discrimination information 3. The reduced feature plane creating unit 4 creates reduced feature plane information 5 reflecting the image feature for the image determined as an interlaced image by the interlaced / non-interlaced image determining unit 2, and generates the reduced feature plane information. 5 is output to the simple motion search unit 6. The simple motion search unit 6 performs a simple motion search process between the two reduced feature planes, and outputs the motion compensation prediction error amount at that time to the frame structure / field structure determination unit 8 as image change amount information 7. The frame structure / field structure determining section 8 uses the image change amount information 7 obtained from the simple motion search section 6 to perform coding in the frame structure when the change amount is small.
If the amount of change is large, the encoding in the field structure is determined and output to the video encoding unit 10 as the picture structure control signal 9. The moving picture coding unit 10 performs moving picture coding on the input image signal 1 according to the picture structure control signal 9 instructed by the frame structure / field structure determining unit 8 and outputs coded moving picture information 11. Here, the moving image encoding unit 10 performs, for example, the motion compensator 10, the first variable length encoder 5, and the second variable length encoder 5 shown in FIG. 13 according to the specification of the frame structure / field structure by the picture structure control signal 9. The operation of the long encoder 14 is switched to a method adapted to encoding in a frame structure / field structure. Next, an example of the configuration and operation of each unit in FIG. 1 will be described in detail. First, the interlaced / non-interlaced image discriminating section 2 will be described. Whether an image is an interlaced image or not is determined by calculation using some adjacent pixels from input image information.
FIG. 4 shows the structure of the input frame image information. Image information is formed by an array of pixels that are spatially uniformly arranged. From this image information, as shown in FIG. 5, the values of five consecutive pixels in an arbitrary position in the vertical direction are extracted, and the absolute difference value between two pixels is calculated. The absolute difference values to be calculated are 0 and −2, 0 and 2, and −5 in p (−2) to p (2), where p (0) is the pixel located at the center in the vertical direction. An absolute difference value between pixels belonging to 1 and 1 and the same field and an absolute difference value between pixels belonging to 0 and −1 and pixels belonging to different fields of 0 and 1 are calculated. Then, first, it is verified whether or not the condition of the following equation (1) is satisfied. Max (d (0, −2), d (0,2), d (−1,1)) <threshold (1) Next, when the condition of the above equation (1) is satisfied, And the next
(2) Verify whether the condition of expression is satisfied. (Max (d (0, -2), d (0,2), d (-1,1)) + offset) <Min (d (0, -1), d (0,1)) (2) where d (a, b) represents the absolute difference between a and b, and Max (a, b, c)
Represents the maximum value of a, b, c, and Min (a, b, c) represents the minimum value of a, b, c. That is, when the pixel values of the same field are similar and the maximum absolute difference value is less than the threshold value (constant value), the minimum value of the absolute difference value in the different field becomes the maximum value of the absolute difference in the same field. At offset
Verifies whether the value exceeds the sum of (constant value). This processing is performed at all pixels in the image or at an arbitrary number of positions.If the points satisfying the above equations (1) and (2) exceed a certain percentage of the points satisfying the equation (1), The image is discriminated as an interlaced image, and the result is output to the reduced feature plane creating unit 4 for each frame as interlaced / non-interlaced discriminating information. Further, in this example, an example in which the interlaced / non-interlaced discrimination is performed using five pixels in the vertical direction has been described. However, the number of pixels between adjacent pixels in the same field and pixels in different fields is determined. Verification can be performed with an arbitrary number of pixels as long as three or more pixels can be compared. Further, the pixel positions to be verified can be determined for all pixel positions in the image. For example, 5 pixels in the vertical direction and n pixels in the horizontal direction are defined as one block, and a specific position in the block is determined. , Or a sample point survey that performs the above verification at any position. further,
It is also possible to extract completely arbitrary points at random. Next, the processing of the reduced feature plane creating unit 4 in FIG. 1, that is, the processing of creating a reduced plane reflecting the features of the image from the original image will be described with reference to FIG.
First, the original image is divided into small blocks, and each small block is represented by a representative value. In the present invention, the standard value of the pixel value of each small block is used as the representative value. In addition, an average value and a median value can be used as the representative value. Further, as the pixel value at the time of this calculation, a luminance component of the pixel can be used, or another component or an average thereof can be used. Also, the size of the small block can be set arbitrarily. Assuming that the size of the reduced feature plane is small: horizontal: ph pixels, vertical: pv pixels, the size of the original image (horizontal: H pixels, vertical: V pixels) is horizontal: H / ph, vertical: V / pv, and the number of samples is 1 / (ph × pv) with respect to the number of pixels of the original image. A reduced plane having the standard deviation of the small block as a representative value is defined as reduced characteristic plane information 5. Next, the processing of the simple motion search section 6 in FIG. 1 will be described. The simplified motion search unit 6 obtains 2 from the reduced feature plane information 5 created by the reduced feature plane creation unit 4.
A motion search process is performed between the reduced feature planes. The temporal distance between the reference plane on which the simple motion search is performed and the target plane is an arbitrary fixed value. As a motion search method, a motion search by block matching or the like can be used.
In this case, the block can take any natural number in both the horizontal and vertical sizes on the reduced feature plane. Therefore, it is possible to perform a block-by-block motion search with a minimum of one sample unit and a maximum of one reduced feature plane as one block. Referring to FIG. 7, the upper left coordinate of the set block is (k, l), the element on the reduced feature plane 1 is c (k, l), and the element on the reduced feature plane 2 is r (k, l),
The horizontal size of the block is N, the vertical size is M, and the search range is ± sh in the horizontal direction and ± sv in the vertical direction. In this simple motion search, the average motion compensation prediction error amount E (k, l) of one element is calculated as shown in equations (3) and (4) in FIG.
It is obtained from the minimum error amount within the search range. After calculating the square error amount, the prediction error amount E (k, l) can be subjected to a square root process, or an absolute difference value can be used. The prediction error amount E (k, l) obtained by the simple motion search processing is performed for all blocks on the reduced feature plane 1, and the sum Esum in the reduced feature plane 1 is obtained. The motion compensation prediction error amount Esum is an index indicating the magnitude of the change between the two images. Then, the value Esum is output to the frame structure / field structure determination unit as image change amount information. Next, the frame structure / field structure determination unit 8 in FIG. 1 performs threshold processing of this value from the input image change amount information for each frame, and when the value is equal to or more than the threshold, the field structure and the value less than the threshold are used. In this case, it is determined that a frame structure is to be adopted, and the result of the determination is output to the video encoding unit 10 as a picture structure control signal 9. The moving picture coding section 10 in FIG. 1 uses the picture structure specified by the picture structure control signal 9 output from the frame structure / field structure determining section 8 to compress the input image signal. And outputs encoded video information 11. Specifically, the video encoding unit 10 performs, for example, the operations of the motion compensator 10, the first variable length encoder 5, and the second variable length encoder 14 of FIG. / Switch to a method adapted to encoding in the field structure. FIG. 8 is a block diagram showing the configuration of the second embodiment of the present invention. 1 that are the same as or equivalent to those in FIG. This embodiment includes a reduced feature plane creating unit 4, a simple motion search unit 6, and a frame structure / field structure determination unit 8. The frame structure / field structure determination unit 8 performs a simple motion search in the simple motion search unit 6. It is characterized in that when the image change amount information 7 obtained by the processing is large, encoding with a field structure is selected, and when it is small, encoding with a frame structure is selected. FIG. 9 is a block diagram showing the configuration of the third embodiment of the present invention. 1 that are the same as or equivalent to those in FIG. This embodiment includes an interlaced / non-interlaced image discriminating unit 2 and a frame structure / field structure deciding unit 8, and determines whether or not an input image is an interlaced image in the interlaced / non-interlaced image discriminating unit 2. The frame structure / field structure determination unit 8 selects the field structure when the image data is determined to be an interlaced image, and selects the encoding using the frame structure when the image structure is determined to be a non-interlaced image. There is a characteristic in that FIG. 10 is a block diagram showing the configuration of the fourth embodiment of the present invention. 1 that are the same as or equivalent to those in FIG. This embodiment uses interlacing /
It is composed of a non-interlaced image discriminating unit 2, an interlaced / non-interlaced switching unit 21, a reduced feature plane creating unit 4, a simple motion search unit 6, and a frame structure / field structure determining unit 8. The discrimination of the interlaced / non-interlaced image of the input image in the image discriminating unit 2 is determined based on one or a plurality of images input first, and based on this determination, the interlaced / non-interlaced switching unit 21 is determined. Is characterized in that the switch is set to "0" or "1", and the interlaced / non-interlaced image discriminating section 2 does not discriminate between interlaced and non-interlaced for the subsequent input images. There is. FIG. 11 is a block diagram showing the configuration of the fifth embodiment of the present invention. 1 that are the same as or equivalent to those in FIG. This embodiment uses interlacing /
The interlaced / non-interlaced image discriminating unit 2 includes an interlaced / non-interlaced switching unit 21 and a frame structure / field structure determining unit 8. The discrimination of a race image is determined based on one or a plurality of images input first, and based on this determination, the interlace / non-interlace switching unit 21 sets the switch to “0” or “1”. However, the subsequent input image is characterized in that the interlaced / non-interlaced image discriminating section 2 does not discriminate between interlaced and non-interlaced images. As is clear from the above description, in the conventional coding of a fixed picture structure, the video with high coding efficiency is limited. Encoding is selected according to the picture structure according to the characteristics and changes of the image, so that high encoding efficiency is maintained regardless of what kind of image is input or the image characteristics change in the middle. be able to. As a moving picture coding method using the motion compensation prediction coding which can be coded in both the frame structure and the field structure, a moving picture coding simulation experiment using the MPEG-2 video coding method. Do
As a result, in the present invention, when compression encoding is performed at an encoding rate of 4 Mbit / s, the image quality can be improved by about 0.4 dB to 1.0 dB in PSNR as compared with the case where the frame rate is fixed. did it.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram showing a configuration of a first embodiment of the present invention. FIG. 2 is a diagram illustrating a configuration of an image in a non-interlaced structure. FIG. 3 is a diagram showing a configuration of an image in an interlaced structure. FIG. 4 is a diagram showing a configuration of a frame image. FIG. 5 is a diagram illustrating pixels for calculating an absolute difference amount. FIG. 6 is a diagram illustrating creation of a reduced feature plane. FIG. 7 is a diagram illustrating a simple motion search process. FIG. 8 is a block diagram illustrating a configuration of a second exemplary embodiment of the present invention. FIG. 9 is a block diagram showing a configuration of a third embodiment of the present invention. FIG. 10 is a block diagram showing a configuration of a fourth embodiment of the present invention. FIG. 11 is a block diagram showing a configuration of a fifth embodiment of the present invention. FIG. 12 is a block diagram showing a conventional moving picture coding method. FIG. 13 is a block diagram of a video encoding device using motion compensation. FIG. 14 is a diagram showing a mathematical expression. [Explanation of Codes] 1 ... image signal, 2 ... interlaced / non-interlaced image discriminating section, 4 ... reduced feature plane creating section, 6 ... simple motion search section, 8 ... frame structure / field structure determining section, 10 ...
Video encoding unit, 21... Interlace / non-interlace switching unit.

   ────────────────────────────────────────────────── ─── Continuation of front page    (72) Inventor Hiromasa Yanagihara             2-1-15 Ohara, Kamifukuoka City, Saitama Prefecture             Kaedi Institute (72) Inventor Masaru Kanno             2-1-15 Ohara, Kamifukuoka City, Saitama Prefecture             Kaedi Institute F term (reference) 5C059 MA00 MA05 MA23 MC11 ME01                       NN01 NN28 PP04 RB02 TA24                       TA46 TB05 TB08 TC01 TC12                       TC15 TD05 TD11 UA02 UA33

Claims (1)

Claims: 1. An image coding apparatus for switching between a frame structure and a field structure, which is capable of coding an interlaced image in either a field structure or a frame structure. Has a means for determining whether the image is an interlaced image or a non-interlaced image. If it is determined that the image is an interlaced image, the coding in the field structure is selected. A frame structure / field structure switching type image encoding apparatus characterized in that encoding of the frame structure is selected. 2. The image coding apparatus according to claim 1, wherein the input image is an interlaced image or a non-interlaced image. A frame characterized by measuring the spatial correlation of pixels consecutive in the vertical direction at the position of, and determining that the image is an interlaced image when the correlation between the same fields is higher than the correlation between different fields. Structure / field structure switching type image encoding device. 3. A frame structure / field structure switching type image coding apparatus according to claim 2, wherein the following method is used for measuring a spatial correlation of pixels consecutive in the vertical direction.
A frame characterized in that, when the number of pixels satisfying the conditions shown in the expressions (1) and (2) is equal to or more than a certain percentage of the number of pixels satisfying the expression (1), encoding using the field structure is selected. Structure / field structure switching type image encoding device. Max (d (0, -2), d (0,2), d (-1,1)) <threshold ... (1) (Max (d (0, -2), d (0,2), d (-1,1)) + offset) <Min (d (0, -1), d (0,1)) ... (2) where d (a, b) represents the absolute difference value of a, b . 4. A frame structure / field structure switching type image coding apparatus capable of coding an interlaced image in either a field structure or a frame structure, wherein a continuously input image has a time interval of Some 2
Means for calculating a correlation of an image, and means for determining whether to perform encoding in a field structure or a frame structure based on the correlation, and performing encoding in the frame structure when the correlation is high. A frame structure / field structure switching type image coding apparatus, wherein coding is performed in a field structure when the correlation is low. 5. The image coding apparatus according to claim 4, wherein the means for calculating the correlation between the two images includes a reduced plane reflecting a feature of a continuously input image. And means for performing a simple motion search process on the reduced plane, and when the amount of motion compensation prediction error obtained in the simple motion search process is large, select encoding with a field structure. A frame-structure / field-structure switching type image encoding apparatus characterized in that it is configured as described above. 6. A frame structure / field structure switching type image encoding apparatus according to claim 5, wherein the means for creating a reduced plane reflecting the characteristics of the image comprises:
A frame / field structure switching type image encoding apparatus, wherein an image is divided into small blocks, a deviation amount of each divided small block is calculated, and the deviation amount is used as an element of a reduced plane. 7. The image coding apparatus according to claim 4, further comprising means for determining whether an input image is an interlaced image or a non-interlaced image. Only for an image determined as an interlaced image, the correlation between the two images is detected to analyze the amount of change in the image and to select a field structure / frame structure. A frame structure / field structure switching type image encoding apparatus, wherein 8. The image coding apparatus according to claim 7, further comprising: an interlaced / non-interlaced picture switching setting means, wherein one or more of the input pictures are input first. It is characterized by determining whether an input image is an interlaced image or a non-interlaced image from a plurality of images, and setting interlaced / non-interlaced image switching setting means based on the determination result. Frame / field structure switching type image coding apparatus.