JP2002027465A - Image information converting device and method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To suppress fluctuations in a reference quantization scale. SOLUTION: The device for converting MPEG2 image compression information (bit stream) of interlaced scanning to MPEG4 compression information (bit stream) of progressive scanning includes a pseudo slice complexity calculation part 25 for calculating a pseudo slice complexity a VOP complexity calculation part 24 for calculating a VOP complexity, a calculation part 26 for calculating a target code quantity of VOP, and a calculation part 27 for calculating a target code quantity of pseudo slice. An MPEG4 image information encoder (I/P-VOP) 20 controls a code quantity according to the target code quantity at the time of encoding image information into MPEG4 image compression information (bit stream).

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image information conversion apparatus and method for converting image information, and more particularly, to image information (bit stream) such as MPEG compressed by orthogonal transform such as discrete cosine transform and motion compensation. And a method for converting image information used when receiving an image via a network medium such as satellite broadcasting, cable TV, or the Internet, or when processing the same on a storage medium such as an optical disk or a magnetic disk. .

[0002]

2. Description of the Related Art In recent years, image information is handled as digital data, and for the purpose of transmitting and storing information with high efficiency, compression is performed by orthogonal transform such as discrete cosine transform and motion compensation using redundancy inherent in image information. An image information compression system such as MPEG is provided. Then, an apparatus conforming to such an image information compression method is used for information distribution of a broadcasting station or the like,
It is becoming widespread in both information reception in general households.

In particular, MPEG2 (ISO / IEC 13
818-2) is defined as a general-purpose image encoding method that covers both interlaced scan images and progressive scan images, as well as standard resolution images and high definition images.

That is, according to the MPEG2 encoding and compression system, for example, a code amount (bit rate) of 4 to 8 Mbps is assigned to a standard resolution interlaced scan image having 720 × 480 pixels, and a high resolution having 1920 × 1088 pixels is assigned. By assigning a code amount (bit rate) of 18 to 22 Mbps to the interlaced scan image, a high compression rate and good image quality can be realized.

[0005] For these reasons, it is expected that MPEG2 will be used in a wide range of applications for professional use and consumer use. However, MPEG2 is mainly intended for high-quality coding suitable for broadcasting, and does not support, for example, a coding amount (bit rate) lower than that of MPEG1, that is, a coding method with a higher compression rate.

[0006] On the other hand, with the spread of portable terminals in recent years, it is expected that the need for an encoding system with a high compression ratio will increase in the future, and in response to this, standardization of the MPEG4 encoding system with a high compression ratio has been carried out. Have been done. Regarding this image coding method, ISO / IEC 14
International standard was approved as 496-2.

By the way, MPEG2 image compression information (bit stream) once encoded for digital broadcasting
There is a need to convert image compression information (bit stream) having a lower code amount (bit rate), which is more suitable for processing on a mobile terminal or the like.

In order to achieve such an object, “Field-to
-Frame Transcoding with Spatial and Temporal
 Downsampling ”(Susie L Wee, John G. Aposto
lopoulos, and Nick Feamster, ICIP 99, below
This is referred to as Document 1).
Scoder) is provided.

As shown in FIG. 6, an image information conversion device (transcoder) provided in Document 1 includes a picture type discriminating unit 1, an MPEG2 image information decoding unit (I / P picture) 2, and a thinning-out unit. It comprises a unit 3, an MPEG4 image information encoding unit (I / P-VOP) 4, a motion vector synthesizing unit 5, and a motion vector detecting unit 6.

[0010] The image information conversion apparatus includes an intra-coded image (I picture; I) encoded in a frame,
A forward predictive coded image (P picture; P) predictively coded by referring to the forward direction in the display order, and a bidirectional predictive coded image predictively coded by referring to the forward and reverse directions in the display order MPEG-2 image compression information (bit stream) of interlaced scanning composed of (B picture; B) is input.

[0011] The MPEG2 image compression information (bit stream) is supplied to an I / P
It is determined whether the picture is related to a picture or a B picture.
Output to the image information decoding unit (I / P picture) 2
The picture is discarded.

The processing in the MPEG2 image information decoding section (I / P picture) 2 is to decode the MPEG2 image compression information (bit stream) into an image signal, as in a normal MPEG2 image information decoding device.

A pixel value output from the MPEG2 image information decoding unit (I / P picture) 2 is input to a thinning unit 3. The thinning section 3 performs a 1/2 thinning process in the horizontal direction, and leaves only one of the data of the first field and the second field in the vertical direction, and discards the other. By such thinning, a progressively scanned image having a size of ４ of the input image information is generated.

The progressively scanned image generated by the thinning section 3 is converted to an MPEG4 image information encoding section (I / P-VOP) 4
Is encoded into an I-VOP encoded in the frame and a P-VOP predicted and encoded with reference to the forward direction in the display order, and output as MPEG4 image compression information (bit stream). VOP is Video objectPla
ne, which corresponds to a frame in MPEG2.

At this time, the motion vector information in the input MPEG2 image compression information (bit stream) is mapped to a motion vector for the decimated image information in the motion vector synthesizing unit 5 and the motion vector detecting unit 6
In, a highly accurate motion vector is detected based on the motion vector value synthesized by the motion vector synthesizing unit 5.

Reference 1 discloses an image information conversion apparatus for generating MPEG4 image compression information (bit stream) of a progressively scanned image having a size of 1/2 × 1/2 of the input MPEG 2 image compression information (bit stream). Is described. That is, for example, the input MPEG2 image compression information (bit stream) is transmitted by NTSC (National Television).
vision system committee), the output MPEG4 image compression information is SI
It means F size (352 × 240 pixels).

In the image information conversion apparatus shown in FIG. 6, an MPEG4 image information encoding unit (I / P-V
OP) 4 is to control the amount of code to output MPEG4
This is a major factor in determining the image quality of the image compression information (bit stream). ISO / IEC 14496-
In No. 2, the code amount control method is not specified, and each vendor has
It is possible to use a method that is considered optimal in terms of the amount of calculation and the output image quality. In the following, a typical code amount control method is MPEG2 Test Model 5 (I
SO / IECJTC1 / SC29 / WG11 N040
The method described in 0) will be described.

The flow of this code amount control will be described with reference to the flow shown in FIG. In the first step S11,
The image information encoding unit (I / P-VOP) 4 includes a target code amount (target bit rate) and a GOP (group).
of pictures) configuration is used as an input variable, and bits are allocated to each picture. Here, a GOP is a set of pictures that can be randomly accessed.

That is, in step S11, the image information encoding unit (I / P-VOP) 4 determines the amount of bits allocated to each picture in the GOP, including the picture to be allocated, of the picture not yet decoded in the GOP. Are allocated based on the amount of bits allocated to (hereinafter referred to as R). This distribution is repeated in the order of the coded pictures in the GOP. At this time, the code amount is assigned to each picture using the following two assumptions.

First, it is assumed that the product of the average quantization scale code used when encoding each picture and the generated code amount becomes a constant value for each picture type unless the picture changes. Therefore, after encoding each picture, a variable X indicating the complexity of the screen is set for each picture type.
_i, X _p, updated by X _b (grobal complelxity measure) the following equation (1).

[0021]

(Equation 4)

Where S_i, S_p, S_bIs for picture encoding
Is the amount of generated code bits, and Q_i, Q _p, Q_bIs a picture
This is an average quantization scale code at the time of key coding. Also,
The initial value is the target code amount (target bit rate) bi
Using t_rate [bits / sec], equation (2)
The value is indicated by

[0023]

(Equation 5)

Second, when the ratios K _p and K _b of the quantized scale codes of the P and B pictures with respect to the quantized scale code of the I picture are equal to the values defined in the equation (3), the whole is always obtained. Assume that the image quality is optimized.

[0025]

(Equation 6)

That is, the quantization scale code of the B picture is always 1.4 times the quantization scale code of the I and P pictures. This is because the picture quality of the I and P pictures is improved by adding the code amount that can be saved in the B picture to the I and P pictures by coding the B picture somewhat coarsely compared to the I and P pictures. It is assumed that the image quality of the B picture that refers to this is also improved.

Based on the above two assumptions, the bit amount (T _i , T _p , T _b ) allocated to each picture of the GOP is a value shown in equation (4).

[0028]

(Equation 7)

Here, N _p and N _b are the numbers of P and B pictures which have not been encoded in the GOP.

Each time each picture is coded according to steps S11 and S12 based on the allocated code amount obtained in this manner, the bit amount R allocated to the uncoded picture in the GOP is calculated by the equation (5). ) To update.

[0031]

(Equation 8)

When encoding the first picture of the GOP, R is updated by equation (6).

[0033]

(Equation 9)

N is the number of pictures in the GOP. Also,
The initial value of R at the beginning of the sequence is 0.

Next, in step S12, the image information encoding device (I / P-VOP) 4 performs rate control using the virtual buffer. That is, in step S12, the image information encoding device (I / P-VOP) 4 allocates bits (T _i , T _p , T _b ) for each picture obtained by equation (4) in step S11, In order to match the actual generated code amounts, the quantization scale code is obtained by macroblock-based feedback control based on the capacity of three types of virtual buffers independently set for each picture.

First, prior to encoding the j-th macroblock, the occupancy of the virtual buffer is determined by equation (7).

[0037]

(Equation 10)

Here, d ₀ ⁱ , d ₀ ^p , and d ₀ ^b are the initial occupancy of each virtual buffer, B _j is the amount of generated bits from the head of the picture to the j-th macroblock, and MB_cnt is the number of bits in one picture. This is the number of macro blocks. Virtual buffer occupancy at the end of each picture encoding (d _{MB_cnt} ⁱ , d _{MB_cnt} ^p ,
d _{MB_cnt} ^b ) is used as an initial value (d ₀ ⁱ , d ₀ ^p , d ₀ ^b ) of the virtual buffer occupancy for the next picture of the same picture type.

Next, the quantization scale code for the j-th macroblock is calculated by equation (8).

[0040]

[Equation 11]

Here, r is a variable called a reaction parameter that controls the response of the feedback loop, and is given by equation (9).

[0042]

(Equation 12)

The initial value of the virtual buffer at the start of encoding is given by equation (10).

[0044]

(Equation 13)

Finally, in step S13, the image information encoding device (I / P-VOP) 4 performs adaptive quantization for each macroblock in consideration of visual characteristics. That is,
In step S13, the image information encoding unit (I / P-
VOP) 4 is to quantize the quantized scale code obtained in step S12 more finely in a flat portion where the deterioration is visually conspicuous, and coarsely quantize the complicated portion of the pattern in which the deterioration is relatively inconspicuous. , And is changed by a variable called an activity for each macroblock.

The activity is calculated by using the pixel value of the luminance signal of the original picture and the pixel values of a total of 8 blocks of 4 blocks in the frame discrete cosine transform mode and 4 blocks in the field discrete cosine transform mode. Given by (11).

[0047]

[Equation 14]

Here, P _k is the pixel value in the luminance signal block of the original image. The minimum value in the equation (11) is
This is because the quantization is made fine when there is a flat portion even in only a part of the macro block.

Further, according to equation (12), the value is 0.5 to
A normalization activity Nact _j having a range of 2 is obtained.

[0050]

(Equation 15)

Here, avg_act is the average value of act _j in the picture coded immediately before.

The quantized scale code mquant _j taking into account the visual characteristics is given by equation (13) based on the quantized scale code Q _j obtained in step S12.

[0053]

(Equation 16)

It is known that the above-mentioned code amount control method defined in the MPEG2 Test Model 5 has the following restrictions. When actual control is performed,
Countermeasures against these restrictions are required. That is, the first limitation is that the first step S11 cannot respond to a scene change, and the third step S11 after the scene change.
This means that the parameter avg_act used in step 13 has an incorrect value. The second limitation is that MPEG2 and M
VBV (Video Buffe specified in PEG4)
r Verifier) is not guaranteed.

By the way, the document "Theoretical analysis of MPEG compression efficiency and its application to code amount control" (IEICE Technical Report, IE-9).
5, DSP95-10, May 1995, hereinafter referred to as Reference 2).
The code amount control method defined in Model 5 is M
In a PEG-2 image encoding device, good image quality is not always provided.

This document 2 proposes the following method as a method for giving an optimal code amount distribution for each frame in a GOP, particularly for giving a good image quality.
That is, let N _I , N _P , and N _B be the numbers of I, P, and B pictures that have not been encoded in the GOP, and let R _I , R _P , and R _B be the code amounts assigned to these.
Also, under the fixed rate condition given by equation (14),
Q _I , Q _P , Q
_B, and m is an order relating the quantization step size and the reproduction error variance (that is, the quantization step size is m
It is assumed that minimizing the average value of the powers will minimize the reproduction error variance). Then, consider minimizing equation (15).

[0057]

[Equation 17]

[0058]

(Equation 18)

The average quantization scale Q and the code amount R in each frame are represented by Test Model.
5 is related to the complexity X of each frame, which is also a medium variable used in Expression 5, as shown in Expression (16).

[0060]

[Equation 19]

The formula (1) is also taken into consideration while considering the relationship of the formula (16).
R _I , which minimizes equation (15) under the constraint of 4)
When R _P and R _B are calculated using the Lagrange's undetermined multiplier method, the following values are obtained as optimal R _I , R _P and R _B.

[0062]

(Equation 20)

When α = 1, equation (17) and MPEG2
It can be said that the relationship with Expression (4) in the code amount control method defined in Test Model 5 is as follows. That is, equation (17) uses the parameters K _p and K _b that are the code amount control parameters as the complexity X _I ,
X _P, according to X _B, nothing but that they are adaptively calculated as in Equation (18).

[0064]

(Equation 21)

In Reference 2, as the value of 1 / (1 + m),
It is shown that good image quality can be obtained by setting the value to about 0.6 to 1.2.

In the image information conversion apparatus shown in FIG.
When the EG4 image information encoding device (I / P-VOP) 4 controls the code amount by using the same method as defined in the MPEG2 Test Model 5, the GOP in the GOP due to a scene change or the like is generated. It is impossible to respond to changes in the complexity of
It is conceivable that stable code amount control becomes difficult and image quality deteriorates. MPEG2 image information decoding unit (I
/ P picture) 2, which is an input MP extracted in
The information in the EG2 image compression information (bit stream) is represented by M
It is expected that this problem can be avoided by using it in the PEG4 image compression information encoding unit (I / P-VOP) 4.

In order to solve such a problem, the present applicant has previously proposed an image information conversion apparatus as shown in FIG.

This image information conversion apparatus includes a picture type discrimination section 7, a compression information analysis section 8, an MPEG2 image information decoding section (I / P picture) 9, a thinning section 10,
A PEG4 image information encoding unit (I / P-VOP) 11;
Motion vector synthesis unit 12 and motion vector detection unit 13
, Information buffer 14 and complexity calculating unit 15
It is composed of

This image information conversion device includes a compression information analysis unit 8, an information buffer 14, a complexity calculation unit 15, and an MPEG4 image information encoding unit (I / P-VOP) 11.
The operation principle other than the code amount control in is the same as that of the image information conversion apparatus shown in FIG.
The operation principle of the compression information analysis unit 8, the information buffer 14, and the complexity calculation unit 15 and the code amount control of the MPEG4 image information encoding unit (I / P-VOP) 11 will be described.

In the compression information analysis section 8, the average value Q of the quantization scale used in the decoding process over the entire frame, and the total MPEG2 image compression information (bit stream) assigned to the frame in the input MPEG2 image compression information (bit stream). The code amount (number of bits) B is stored in the information buffer 14.

In the complexity calculating section 15,
From the information Q and B for each frame stored in the information buffer 14, the complexity X for the frame is calculated by equation (19).

[0072]

(Equation 22)

The complexity X for the frame calculated by equation (19) is 1 GOV (group
After being buffered for the amount of VOPs), it is transmitted to the MPEG4 image information encoding unit (I / P-VOP) 11 as a parameter for controlling the code amount. Therefore, 1 GOV
A minute delay is required. This delay is realized using a delay buffer (not shown). Here, GOV is a set of VOPs that can be randomly accessed.

In the following, the complexity X for each frame in the GOV calculated in equation (19) will be described.
Is an MPEG4 image information encoding unit (I / P-VOP) 1
1 will be described.
In the following, a case where the picture type determination unit 7 does not exist in the image information conversion apparatus and does not perform the frame rate conversion will be considered.

The meaning of K _P and K _B obtained by equation (18) means that P-VOP / B-VO for the ideal average quantization scale Q _{i_ideal} for I-VOP.
Ideal average quantization scale Q _{p_ideal} , Q for P
_That is, the ratio of _{b_ideal} is given by equation (20).

[0076]

(Equation 23)

The MPEG2 Test Model 5 does not adaptively calculate K _p and K _b as in the equation (18), but uses a fixed value as shown in the equation (3).

From equations (18) and (20), a certain VO
P1 and the complexity for a certain VOP2 are X ₁ and X ₂ , respectively, and the ideal quantization scale is Q
_Assuming that _{1_ideal} and Q _{2_ideal} , equation (21) is obtained.

[0079]

(Equation 24)

Alternatively, MPEG2 Test Mo
When it is desired to use the fixed value shown in Expression (3) as in del 5, Expression (22) may be used instead of Expression (21).

[0081]

(Equation 25)

Now, let R be the total code amount (the number of bits) allocated to the uncoded VOPs in the GOV.
Are assigned to each VOP as R ₁ , R ₂ ,... R _n , the image quality for the GOV is optimized. Here, a relational expression such as Expression (23) is established between R and R ₁ , R ₂ ,... R _n .

[0083]

(Equation 26)

[0084] Some VOP average quantization scale for _k Q _k, assigned code amount R _k, between the complexity X _k Note also that there is a relation of equation (24), formula (23)
Equation (25) is obtained by transforming

[0085]

[Equation 27]

[0086]

[Equation 28]

In equation (25), regarding K (X ₁ , X ₂ ), even if the value shown in equation (21) is used, equation (22)
May be used, but the former can realize more optimal code amount distribution according to the image. At this time, by setting the value of 1 / (1 + m) to 1.0, it is not necessary to perform an exponential operation, and high-speed execution is possible. Even when the value of 1 / (1 + m) is set to a value other than 1.0, high-speed execution can be performed by holding a table in advance and performing an exponential operation with reference to the table.

The complexity X _k for each VOP in the equation (25) is based on the MPEG4 image coding. The complexity for each frame based on the MPEG2 image coding and the complexity for each frame based on the MPEG4 image coding are shown. Assuming that the cities are equal, it is possible to calculate the target code amount for the VOP by using equation (25) by using X _k stored in the complexity calculating unit 15.

FIG. 9 shows a flow of calculating the target code amount. In the first step S21, the compression information analysis unit 8
Extracts an average quantization scale Q and an allocated code amount (number of bits) B for each frame in a GOP used for decoding processing in the MPEG2 image information decoding unit 9.

In step S22, the complexity calculating section 15 calculates a complexity X given by a product of the average quantization scale Q and the allocated code amount (number of bits) B.

In step S23, the MPEG4 image encoding unit (I / P-VOP) 11 calculates a target code amount (target bit) according to the complexity X.

In MPEG2 Test Model 5, it is assumed that the complexities X _i , X _p , and X _b for the I, P, and B pictures in the GOP are constant, but a scene change is actually included in the GOP. Or
This assumption does not hold when the background changes significantly in the GOP, hinders stable code amount control, and causes image quality deterioration. Even in such a case, the image information conversion apparatus shown in FIG. 8 performs code amount control based on the complexity of each frame in the input MPEG2 image compression information (bit stream). , It is possible to perform stable code amount control.

[0093]

By the way, in the code amount control method shown in FIG. 7, in order for the adaptive quantization in step S13 to work effectively, the quantization scale code for the j-th macroblock, that is, the expression (8)
It is desirable that the value of Q _J at the time T 1 takes a uniform value over the entire frame. Therefore, CCIR (Comite Consultant
if Internationale des Radio Communications) "Flower Gard"
MPEG2 image compression information (bit stream) compressed to 4 Mbps under the condition of n = 15; m = 3
By using the image information conversion device shown in FIG. 8, n = 5;
MPEG4 image compression information of m = 1 (bit stream)
FIG. 10 shows what value Q _J for a certain VOP takes when converting to VOP.

As described above, it is ideally desirable that Q _J take a uniform value over the entire VOP.
Virtual buffer occupancy of the formula ^{_{(7) (d j i,}} d j p, d
_{Since j} ^b ) changes for each macroblock, it does not become a uniform value over the entire VOP.

The present invention has been proposed in view of the above situation, and has as its object to provide an image information conversion apparatus and method for controlling the code amount by suppressing the fluctuation of Q _j. .

[0096]

SUMMARY OF THE INVENTION In order to solve the above-mentioned problems, the present invention provides an interlaced scan input image compression information compressed by a first compression encoding system by a second compression encoding system. The input image compression information and the encoded image constituting the output image compression information are converted into compressed progressively scanned output image compression information, and each of the encoded images constituting the output image compression information is configured by a pixel block including a plurality of pixels. The image information is converted by giving a target code amount for the pseudo image block sequence based on the complexity information for the pseudo image block sequence composed of pixel blocks in the encoded image of the output image compression information.

According to the present invention, MPEG2 image compression information (bit stream) of interlaced scanning is used as input image compression information, and MPEG4 image compression information (bit stream) of sequential operation is used as output image compression information. The MPEG2 image compression information (bit stream) and the MPEG4 image compression information are composed of a pixel block composed of a plurality of pixels, that is, a macroblock, and use a pseudo pixel block sequence, that is, a pseudo slice.

The MPEG2 image compression information (bit stream) and the MPEG4 image compression information (bit stream) include a group of pictures, that is, a GOP (group of picture).
s) and GOV (group of VOPs). GOP and GOV, which are image groups,
A plurality of coded images, ie, pictures and V
Each is composed of an OP (video object plane).

That is, the present invention provides an interlaced scanning MP
EG2 image information compression information (bit stream) is input, picture type discrimination unit, compression information analysis unit, MPEG
2 image information decoding unit (I / P picture), thinning unit, M
PEG4 image information encoder (I / P-VOP), motion vector synthesizer, motion vector detector, information buffer, V
An OP complexity calculating unit, a pseudo slice complexity calculating unit, a VOP target code amount calculating unit, and a pseudo slice target code amount calculating unit also serve as a complex for each slice in the input MPEG2 image compression information (bit stream). By using the city information to give a target code amount (target bit) in units of pseudo slices at the time of MPEG4 image coding, step 2 of code amount control is performed.
To provide a means for outputting progressively scanned MPEG4 image compression information (bit stream) in a state where the variation of the reference quantization scale accompanying the above is minimized and the code amount allocation to each macroblock is optimized for the image. Is what you do.

In the above configuration, the picture type discriminating section stores only the data relating to the I / P picture in the input MPEG2 image compression information (bit stream).
Discard the pictures. The compression information analysis unit
A delay of one GOP is realized, and at the same time, the input MP
In the EG2 image compression information (bit stream), the code amount (number of bits) allocated to each frame and the average quantization scale in each frame, and in each frame, each pseudo-code is stored in the subsequent MPEG4 image compression information. The average quantization scale and the generated code amount (the number of bits) over the entire macroblock constituting the slice are extracted. MPEG2 image information decoding unit (I / P picture)
Uses the compression information (bit stream) relating to the I / P picture, which is the output of the picture type discriminator, in the horizontal and vertical directions, using all the 8th-order discrete cosine coefficients, or using only the low-frequency components thereof. Perform decryption processing. The thinning unit is an MPEG2 image information decoding unit (I / P
Only the first field or the second field of the image information, which is the output of the picture, is taken out and converted to a progressively scanned image, and at the same time, downsampling for converting to a desired image frame size is performed. The MPEG4 image information encoding unit (I / P-VOP) encodes image information to be output from the thinning unit according to the MPEG4 encoding method. The motion vector synthesizing unit includes an MPEG2 image information decoding unit (I / P
Based on the motion vector value in the input image compression information (bit stream) detected in the picture, the mapping is performed to the motion vector value for the scan-converted image data. The motion vector detection unit performs highly accurate motion vector detection based on the motion vector value output from the motion vector synthesis unit. The information buffer includes a code amount (the number of bits) allocated to each frame in the input MPEG2 image compression information (bit stream) extracted by the compression information analysis unit, an average quantization scale for each frame, and each frame. In the latter MPE
The G4 image compression information stores the average quantization scale and the generated code amount (the number of bits) over the entire macroblock constituting each pseudo slice. The VOP complexity calculator calculates the input M stored in the information buffer.
In the PEG2 image compression information (bit stream),
From the code amount (number of bits) allocated to each frame and the average quantization scale for each frame, an estimated value of the complexity for each VOP in the output MPEG4 image compression information (bit stream) is calculated. In the pseudo slice complexity calculating section,
From the code amount (number of bits) assigned to each pseudo slice and the average quantization scale for each pseudo slice in the input MPEG2 image compression information (bit stream) stored in the information buffer, the output M
An estimated value of complexity for each pseudo slice in the PEG4 image compression information (bit stream) is calculated. The VOP target code amount calculation unit calculates a target code amount (target bit) for each VOP based on the complexity for each VOP calculated by the VOP complexity calculation unit, and calculates a pseudo slice target code amount. In the unit, each pseudo slice is calculated from the target code amount (target bit) for each VOP calculated by the VOP target code amount calculation unit and the complexity for each pseudo slice calculated by the pseudo slice complexity calculation unit. Is calculated, and the information is transmitted to the MPEG4 image information encoding device (I / P-VOP).

It is to be noted that an apparatus configuration that does not have a picture type discriminating unit and does not perform frame rate conversion is also possible.
Further, in the VOP target code amount calculation unit, the input M
Without using the complexity for each frame in the PEG2 image compression information (bit stream), MPEG
An apparatus configuration for calculating a target code amount for each VOP by a method similar to that specified in 2 Test Model 5 is also conceivable.

[0102]

Embodiments of the present invention will be described below with reference to the drawings.

First, an image information conversion apparatus according to a first embodiment of the present invention will be described with reference to FIG.

This image information processing apparatus includes a picture type discriminating section 16, a compression information analyzing section 17, an MPEG2 image information decoding section (I / P picture) 18, and a thinning section 19
And an MPEG4 image information encoding unit (I / P-VOP) 2
0, a motion vector synthesis unit 21, a motion vector detection unit 22, an information buffer 23, a VOP complexity calculation unit 24, and a pseudo slice complexity calculation unit 2.
5, a VOP target code amount calculation unit 26, and a pseudo slice target code amount calculation unit 27.

This image information conversion apparatus includes an intra-coded image (I picture; I) encoded in a frame,
A forward predictive coded image (P picture; P) predictively coded by referring to the forward direction in the display order, and a bidirectional predictive coded image predictively coded by referring to the forward and reverse directions in the display order MPEG-2 image compression information (bit stream) of interlaced scanning composed of (B picture; B) is input.

The MPEG2 image compression information (bit stream) is supplied to the
It is determined whether it is related to a P picture or a B picture, and only the I / P picture is output to the subsequent compression information analysis unit 17, and the B picture is discarded.

In the compression information analysis unit 17, MPEG2
In the image information decoding device (I / P picture) 18, the average value Q of the quantization scale used in the decoding process over the entire frame and the MPEG2 image compression information (bit stream) to be input are assigned to the frame. The total code amount (number of bits) B is stored in the information buffer 23.
Is stored in

The processing in the MPEG2 image information decoding section (I / P picture) 18 decodes the MPEG2 image compression information (bit stream) into an image signal as in the case of a normal MPEG2 image information decoding device. here,
The data relating to the B picture is stored in the picture type
Therefore, the MPEG2 image information decoding unit (I / P picture) 18 only needs to have a function of decoding only the I / P picture.

The pixel value output from the MPEG2 image information decoding unit (I / P picture) 18 is input to the thinning unit 19. The thinning section 19 performs 1/2 thinning processing in the horizontal direction, and leaves only data of one of the first field and the second field in the vertical direction, and discards the other. By such thinning, a progressively scanned image having a size of ４ of the input image information is generated.

Incidentally, the image output from the thinning section 19 is converted to an MPEG4 image information encoding section (I / P-VOP) 2.
In order to perform coding in units of macroblocks composed of 16 × 16 pixels at 0, the number of pixels must be a multiple of 16 in both the horizontal and vertical directions. In the thinning section 19, the pixels are supplemented or discarded at the same time as the thinning.

For example, if the input MPEG2 image compression information (bit stream) is NTSC (National Televi
In the case of an image conforming to the standards of the Vision System Committee, that is, a 720 × 480 pixel, 30 Hz interlaced scan image, the image frame after thinning is SIF (360 × 24
0 pixel) size. For this image,
In the thinning unit 19, for example, the eight lines at the right end or the left end in the horizontal direction are discarded to obtain 352 × 240 pixels.

By changing the operation of the thinning section 19, other image frames, for example, about 1 in the above example, can be obtained.
It is also possible to convert the image into an image of QSIF (176 × 112 pixels) which is an image frame of ××.

Further, in the above-mentioned Document 1, as processing in the MPEG2 image information decoding unit (I / P picture) 18, the MPEG2 image compression information (bit stream) to be input in the horizontal direction and the vertical direction is used. 8
An image information conversion apparatus that performs decoding processing using all of the following discrete cosine transform coefficients is described. However, the description is not limited to the apparatus illustrated in FIG. 1. Only the horizontal direction, or both the horizontal and vertical directions, By performing decoding using only low-frequency components of the eighth-order discrete cosine transform coefficients, it is possible to reduce the amount of computation and video memory capacity involved in decoding while minimizing image quality degradation.

The progressive scan image generated by the thinning unit 19 is an MPEG4 image information encoding unit (I / P-VOP)
20 and an I-VOP coded in a frame and a P-VO coded predictively with reference to the forward direction in the display order.
P is encoded and output as MPEG4 image compression information (bit stream).

Note that VOP means Video Object Plane and corresponds to a frame in MPEG2. Further, I-VOP is an intra-coded VOP corresponding to an I picture, P-VOP is a forward predictive coded VOP corresponding to a P picture, and B-VOP is a bidirectional predicted coded VOP corresponding to a B picture.

An MPEG4 image information encoding unit (I / P-V
OP) 20, the input MPE
The motion vector information in the G2 image compression information (bit stream) is mapped to a motion vector for the decimated image information in the motion vector synthesizing section 21, and the motion vector synthesizing section 2 in the motion vector detecting section 22.
A high-precision motion vector is detected based on the motion vector value synthesized in 1.

Here, the concept of a pseudo slice used in the image information conversion apparatus according to the present embodiment will be described.

The MPEG2 image compression information (bit stream) has a slice layer as shown in FIG. That is, in the slice layer, a horizontally long band-like area is shown in the screen as shown in FIG. 2, and the area (areas a, b, c... In FIG. 2) is composed of a plurality of slices. Even if an error occurs, the start of the next slice layer (sl
Error recovery is possible by synchronization from (ice_start_code). The slice layer is composed of one or more macroblocks, and is in raster scan order, from left to right,
They are arranged from top to bottom, and their length and start position are free and can be changed for each screen. However, for the purpose of parallel processing and effective error tolerance, one slice extends only to the right,
It does not extend down.

In the MPEG4 image compression information (bit stream), the slice layer as shown in FIG. 2 is not defined syntactically in consideration of the coding efficiency at a low bit rate, but in the present embodiment, , MPEG4
For encoding processing in the image information encoding unit (I / P-VOP) 20, a pseudo slice as shown in FIG. 3 is defined. That is, for example, the interlaced MPEG2 image compression information (bit stream), which is an input, is converted by the image information conversion apparatus shown in FIG. 1 into a progressively scanned MPEG4 image compression having a 1/2 × 1/2 picture frame. Consider a case where the information is converted into information (bit stream). At this time, four macroblocks MB ₀ , MB ₁ ,... In the MPEG2 image compression information (bit stream) to be input in FIG.
MPEs for which MB _2m and MB _{2m + 1} are the outputs shown in FIG.
It will correspond to one macroblock Mb ₀ in G4 compressed image information (bit stream). At this time, the macro blocks M _B0 , M _{B1 ,.}
Corresponding to the region formed a from B _4m-1, in the B MPEG4 image compression information to be output as shown in FIG. 3 (a bit stream), the macro block _{Mb 0, Mb 1, ...,} Mb
_A macroblock group a composed of _m-1 is defined as a pseudo slice 0. The same applies to pseudo slice 1 and subsequent slices.

[0120] The compression information analysis unit 17 stores the MPEG2 image compression information (bit stream) as input.
Information on the assigned code amount (number of bits) B _k for each frame and information on the average quantization scale Q _k in each frame are extracted and stored in the information buffer 23. At the same time, in each frame, in the MPEG4 image compression information (bit stream) to be output, the allocated code amount (number of bits) B for the macroblock group constituting the pseudo slice 1
_{pseudo_slice1} and average quantization scale Q _{pse
udo_slice1} is stored in the information buffer 23. In A of FIG. 3, where n is an integer, an input become compressed image information (bit stream) generated code amount for the macroblock MB _n with the and the quantization scale of each B _MBn, Q
Assuming _MBn , the following expression (2)
6) holds.

[0121]

(Equation 29)

The same applies to pseudo slice 1 and subsequent slices. However, if m is an odd number, for example, if the image frame of the interlaced MPEG2 image compression information (bit stream) to be input is 720 × 480 pixels, 1/2 × thereof
Although 1/2 means 360 × 240 pixels, the following MPEG4 image information encoding device (I / P-VOP) 2
In order to perform processing on a macroblock basis at 0,
In the thinning unit 19, for example, it is necessary to discard the right four pixels with respect to the image frame to make 352 × 240 pixels. In this case, the value shown in equation (26) is used as B
_They may be used as _{pseudo_slice0} and Q _{pseudo_slice0} , or _{may be expressed} by the following equation (27).

[0123]

[Equation 30]

In the VOP complexity calculating section 24, the input MP stored in the information buffer 23
From the information on the assigned code amount (number of bits) B _{k for} each frame and the average quantization scale Q _k for each frame in the EG2 image compression information (bit stream),
The estimated value X _k of the complexity for each VOP is calculated for one GOV as in the following equation (28).

[0125]

[Equation 31]

In the VOP target code amount calculating section 26,
Using the VOP complexity calculator 24, the equation (2)
According to 5), the target code amount (target bit) for each VOP is calculated. In the following, the target code amount (R _{1 in the} formula) for each VOP obtained by the formula (25) is represented as T _vop .

Pseudo slice complexity calculating section 25
In (2), the allocated code amount (number of bits) B _{ps eudo_slice1} and the average quantization scale Q _{pseudo_slice1} for the macroblock group forming the pseudo slice 1 in the output MPEG4 image compression information (bit stream) stored in the information buffer 23 From the information on the complexity X for each pseudo slice
_{Pseudo_slice1} is _calculated as in the following equation (29).

[0128]

(Equation 32)

If the VOP is composed of pseudo slice 0, pseudo slice 1,... Pseudo slice N-1, the pseudo slice target amount calculation section 27
Target code amount T for pseudo slice 1
_{pseudo_slice1} is calculated as in the following equation (30),
It is transmitted to the MPEG4 image information encoding unit (I / P-VOP) 20.

[0130]

[Equation 33]

T _{pseudo_slice1} for the I-VOP, P-VOP, and B-VOP is _defined as T _{i _pseudo_slice1} , T
_{If p_pseudo_slice1} and T _{b_pseudo_slice1} , then MPE
In G4 image information encoding unit (I / P-VOP) 20 , a virtual buffer ^{_{(d j i, d j p}} , d j b) the occupancy of, corresponding to formula (27), the following equation (31 ).

[0132]

[Equation 34]

Here, d ₀ ⁱ , d ₀ ^p , and d ₀ ^b are the occupation amounts of the respective virtual buffers at the head of the pseudo slice.
_{pseudo_slice_j} is the amount of code generated in the j-th macroblock from the head of the pseudo slice. P_SLICE_C
NT is the number of macroblocks included in one pseudo slice, and the virtual buffer occupancy at the end of each pseudo slice encoding (d _{P_SLICE_CNT} ⁱ , d _{P_SLICE_CNT} ^p , d
_{P_SLICE_CNT} ^ib ) is used as an initial value (d ₀ ⁱ , d ₀ ^p , d ₀ ^b ) of the occupation amount of the virtual buffer for the next pseudo slice.

The operation principle of a series of code amount control in the MPEG4 image information encoding unit (I / P-VOP) 20 will be described with reference to FIG.

In the first step S31, the compression information analysis unit 17 analyzes the syntax of the MPEG2 image compression information (bit stream) input via the picture type discrimination unit 16, and assigns the code amount (number of bits) to each frame. ) Extract information about B _k and the average quantization scale Q _k in each frame, and store them in the information buffer 23. At the same time, in each frame, the output MPEG
4 stores the assigned code amount for macroblocks constituting the pseudo slice 1 in the image compression information (bit stream) (number of _bits) B pseudo_slice1, and an average quantization scale _Q pseudo_slic _e1 information buffer 23.

In step S 32, the VOP complexity calculating section 24 allocates the code amount (number of bits) B _k for each frame in the input MPEG2 image compression information (bit stream) stored in the information buffer 23. , And information on the average quantization scale Q _k in each frame, an estimated value X _k of the complexity for each VOP is calculated for one GOV.

In step S33, the VOP target code amount calculator 26 calculates a target code amount (target bit) for each VOP.

In step S34, the pseudo slice complexity calculating section 25 allocates the code amount to the macroblock group constituting the pseudo slice 1 in the output MPEG4 image compression information (bit stream) stored in the information buffer 23. (Number of bits) B
_{pseudo_slice1} and average quantization scale Q
_{From the} information on _{pseudo_slice1} , the _complexity X _{pseudo_slice1} for each pseudo slice is calculated.

In step S35, the pseudo slice target code amount calculation unit 27 calculates a target code amount (target bit) corresponding to each pseudo slice, and transmits the target code amount to the MPEG4 image information encoding unit (I / P-VOP) 20. I do.

In step S36, the MPEG4 image information encoding unit (I / P-VOP) 20 performs rate control using a virtual buffer. In step S37,
MPEG4 image information encoding unit (I / P-VOP) 20
Performs adaptive quantization for each macroblock in consideration of viewing angle characteristics.

Next, an image information conversion apparatus according to a second embodiment of the present invention will be described with reference to FIG.

This image information apparatus includes a picture type discrimination section 28, a compression information analysis section 29, an MPEG2 image information decoding section (I / P picture) 30, a thinning section 31,
MPEG4 image information encoding unit (I / P-VOP) 32
, A motion vector synthesizing unit 33 and a motion vector detecting unit 3
4, an information buffer 35, a pseudo slice complexity calculating unit 36, a VOP target code amount calculating unit 37, and a pseudo slice target code amount calculating unit 38.

The difference between the image information conversion apparatus shown in FIG. 1 and the image information conversion apparatus shown in FIG. 5 is that the image information conversion apparatus shown in FIG. 1 uses an MPEG4 image information encoding unit (I / P- While the target code amount (target bit) for each VOP in the (VOP) 20 is calculated by Expression (25), the image information conversion apparatus shown in FIG. 5 uses the MPEG4 image information encoding unit (I / P- VO
P) 32 is that the target code amount (target bit) for each VOP is calculated by equation (4). That is, in the image information conversion device shown in FIG. 5, the compression information analysis unit 29 extracts the GOP structure in the input MPEG2 image compression information (bit stream) and stores it in the information buffer 35. , V
In the OP target code amount calculation unit 37, the output MP
GO in EG4 image compression information (bit stream)
The V structure is determined, and the target code amount (target bit) for each VOP is calculated based on equation (4).

As described above, the MPEG2 image compression information (bit stream) is used as the input and the MPEG4 image compression information (bit stream) is used as the output. However, the input and output are not limited to this. For example, MPEG-1 or H.264.
263 may be image compression information (bit stream).

[0145]

As described above, the present invention provides MPEG2 image compression information (bit stream) for interlaced scanning.
, And using the complexity information for each slice in the input MPEG2 image compression information (bit stream) to give a target code amount (target bit) in pseudo slice units at the time of MPEG4 image encoding. MPEG4 image compression of progressive scanning in a state in which the variation of the reference quantization scale accompanying the step of calculating the complexity in the code amount control is minimized, and the code amount allocation for each macroblock is optimized for the image. It provides means for converting the information into a bit stream.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of an image information conversion device according to a first embodiment.

FIG. 2 MPEG2 image compression information (bit stream)
FIG. 3 is a view for explaining the concept of a slice layer in FIG.

FIG. 3 shows MPEG4 image compression information (bit stream).
FIG. 3 is a diagram showing a concept of a pseudo slice in FIG.

FIG. 4 is a diagram showing an operation flow for performing code amount control using complexity.

FIG. 5 is a block diagram illustrating a configuration of an image information conversion device according to a second embodiment.

FIG. 6 is a block diagram illustrating a configuration of a conventional image information conversion device.

FIG. 7 shows MPEG2 Test Mode 1 5 (IS
O / IEC JTC1 / SC29 / WG11 N040
3 is a flowchart showing the operation principle of the code amount control method described in (0).

FIG. 8 is a diagram showing a configuration of an image information conversion device proposed by the present applicant.

9 is a flowchart illustrating an operation of code amount control in the image information conversion device of FIG. 8;

FIG. 10 shows one of the CCIR test sequences “F
Under the condition that lower Garden "is n = 15; m = 3, the MPEG2 image compression information (bit stream) compressed to 4 Mbps is converted to n = 5; m = 3 using the image information converter shown in FIG. FIG. 3 is a diagram showing what value Q _j takes for a certain VOP when converting into one MPEG4 image compression information (bit stream).

[Explanation of symbols]

16 picture type discrimination unit, 17 compression information analysis unit,
18 MPEG2 image information decoding unit (I / P picture), 19 decimation unit, 20 MPEG4 image information encoding unit (I / P-VOP), 21 motion vector synthesis unit,
22 motion vector detecting section, 23 information buffer, 24
VOP complexity calculator, 25 pseudo slice complexity calculator, 26 VOP target code amount calculator, 27 pseudo slice target code amount calculator

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Kuniaki Takahashi, 6-7-35 Kita-Shinagawa, Shinagawa-ku, Tokyo Inside Sony Corporation (72) Inventor Teruhiko Suzuki 6-35, Kita-Shinagawa, Shinagawa-ku, Tokyo Inside Sony Corporation (72) Inventor Yoichi Yagasaki 6-7-35 Kita-Shinagawa, Shinagawa-ku, Tokyo F-term inside Sony Corporation (reference) 5C059 LA07 MA00 MA05 MB00 MC11 MC38 PP05 PP06 PP07 SS01 SS11 TA46 TA57 TB03 TB08 TC10 TC38 TD03 UA38 5J064 AA01 BA09 BB03 BB05 BC01 BC14 BC26 BD02 BD03

Claims (14)

[Claims] An image information conversion apparatus for converting input image compression information of interlaced scanning compressed by a first compression encoding method into output image compression information of progressive scanning compressed by a second compression encoding method. In the above, the coded images constituting the input image compression information and the output image compression information each include a pixel block including a plurality of pixels, and a pseudo image block including a pixel block in the coded image of the output image compression information. Using a target code amount calculating means for calculating a target code amount for the column, using the target code amount calculated by the target code amount calculating means,
A coding unit for coding the image information into the output image compression information. 2. The image information conversion apparatus according to claim 1, wherein the pseudo pixel block sequence is a pixel block group for one horizontal line in the encoded image of the output image compression information. 3. An analysis for extracting, from the input image compression information, an average quantization scale and an assigned code amount of a pixel block group in an encoded image of the input image compression information corresponding to pixel blocks constituting the pseudo pixel block sequence. 3. The image information conversion apparatus according to claim 2, further comprising: an information buffer configured to store the average quantization scale and the allocated code amount detected by the analysis unit. 4. The analysis means calculates the complexity of the output image compression information and the k-th pseudo pixel block sequence in the encoded image by using the average quantization scale and the allocated code amount stored in the information buffer. The image information conversion device according to claim 3, wherein the image information is calculated by the following equation. (Equation 1) Here, the average quantization scale for the k-th pseudo pixel block sequence is Q _{pseudo _slicek,} and the assigned code amount is B
_{pseudo_slicek} and the _complexity is X _{pseudo_sli cek}
And 5. The coded image forming the output image compression information by using a target code amount for a coded image forming the output image compression information and a complexity for each pseudo pixel block sequence. 5. The image information conversion apparatus according to claim 4, wherein the complexity of the k-th pseudo pixel block sequence forming the following is calculated by the following equation. (Equation 2) Here, the target code amount for the coded image is T
_vop , the _complexity for the l-th pseudo pixel block sequence is X _{pseudo_slicel,} and the target code amount is T _{pseud
o_slicek} . 6. The input image compression information is composed of an image group consisting of a plurality of coded images, and the analysis means analyzes the structure of the image group constituting the input image compression information, thereby obtaining the output image 6. The image information conversion apparatus according to claim 5, wherein a target code amount for the encoded image of the compressed information is calculated. 7. The analyzing means extracts a complexity for each frame constituting the input image compression information, and calculates a target code amount for the encoded image of the output image compression information using the complexity. 6. The image information conversion device according to claim 5, wherein the calculation is performed. 8. The output image compression information is included in a frame.
Encoded intra-coded image, display in the forward direction
Forward prediction coded image coded by reference, display order
Prediction encoded with reference to the forward and backward directions in
The conversion means comprises a coded image,
Virtual in the j-th pixel block forming the decoded image
Give the buffer occupancy by the following formulas
6. The image information conversion device according to claim 5, wherein: (Equation 3)However, intra coded images, forward prediction coded images,
L-th target code amount T for bidirectional predictive coded images
_{pseudo_slicel}To T_{i_pseudo_slicel}, T
_{p_pseudo_slicel}, T_{b_pseudo_slicel}And the j-th pixel
The occupancy of the virtual buffer for the block is d_j ⁱ, D_j ^p,
d_j ^bAnd the initial value of the virtual buffer occupancy is d₀ ⁱ, D₀ ^p,
d₀ ^bAnd Also, j-th from the top of the pseudo pixel block column
Code amount (number of bits) generated up to the first macroblock
To B_{pseudo_slicej}And form a pseudo pixel block column
Let the number of pixel blocks be P_SLICE_CNT. 9. The method according to claim 8, wherein the conversion unit sets the occupation amount of the virtual buffer at the end of processing of each pseudo pixel block sequence as an initial value of the occupation amount of the virtual buffer for the next pseudo pixel block sequence. 9. The image information conversion device according to claim 8, wherein: 10. The input image compression information includes an intra-coded image coded in a frame, a forward prediction coded image coded by referring to a forward direction in a display order, a forward prediction coded image in a display order, Discriminating means which is composed of a bidirectional predictive coded image coded with reference to the backward direction, passes the intra coded image and the forward predictive coded image, but discards the bidirectional predictive coded image The image information conversion device according to claim 1, further comprising: 11. The first compression encoding method is an MPEG encoding method.
2. The image information conversion apparatus according to claim 1, wherein the second compression encoding method is MPEG4. 12. An image information conversion method for converting input image compression information of interlaced scanning compressed by a first compression encoding method into output image compression information of progressive scanning compressed by a second compression encoding method. In the above, the coded images constituting the input image compression information and the output image compression information each include a pixel block including a plurality of pixels, and a pseudo image block including a pixel block in the coded image of the output image compression information. An image information conversion method, comprising calculating a target code amount for a column, and encoding image information into the output image compression information using the target code amount. 13. The image information conversion method according to claim 12, wherein the pseudo pixel block sequence is a pixel block group for one column in the horizontal direction in the encoded image of the output image compression information. 14. Extracting an average quantization scale and an assigned code amount of a pixel block group in an encoded image of the input image compression information corresponding to a pixel block forming the pseudo pixel block sequence from the input image compression information; 14. The image information conversion method according to claim 13, wherein the image information is stored in an information buffer.