JP2002152752A - Image information converter and method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily obtain motion vector information of a frame representing an I picture in MPEG2 image compression information that is to be converted into a P-VOP(Video Object Plane) in MPEG4 image compression information with high accuracy and easily discriminates a mode of the frame as to whether it is an inter mode or an inter 4V mode with a small arithmetic quantity. SOLUTION: A motion vector synthesis section 14 and a motion vector detection section 15 generate a motive vector used for MPEG4 image coding. A motion vector buffer 17 stores the motion vector. A mode discrimination section 16 discriminates a mode of the frame, representing an I picture in the MPEG2 image compression information that is to be converted into the P-VOP in the MPEG4 image compression information, as to whether it is an inter mode or an inter 4V mode on the basis of the motion vector of a P-VOP just before the concerned P-VOP.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an MPEG (Moving
Image information (bit stream) compressed by orthogonal transform such as discrete cosine transform and motion compensation, such as Picture image coding Experts Group, is received via a network such as satellite broadcasting, cable television, or the Internet. The present invention relates to an image information conversion apparatus and method suitable for processing on a storage medium such as an optical or magnetic disk.

[0002]

2. Description of the Related Art In recent years, image information is handled as digital information. At this time, orthogonal transformation such as discrete cosine transformation or the like is performed by utilizing the redundancy inherent in the image information for the purpose of efficiently transmitting and storing information. A device that complies with a method such as MPEG that compresses the image information by motion compensation is becoming popular in both information distribution at broadcast stations and the like and information reception in ordinary households.

In particular, MPEG2 (ISO / IEC 13
818-2) is defined as a general-purpose image coding method, and is used as a standard covering both interlaced scan images and progressive scan images, and standard resolution images and high-definition images. It is expected to be used in the future. By using this MPEG2 compression method, for example, a standard resolution interlaced scan image having 720 × 480 pixels is 4M to 8Mbps, 1920 × 1
In the case of a high-resolution interlaced scan image having 088 pixels, a high compression rate and good image quality can be realized by assigning a code amount (bit rate) of 18 to 22 Mbps.

Here, the above-mentioned MPEG2 is mainly intended for high-quality coding suitable for broadcasting,
It does not correspond to a coding amount (bit rate) lower than EG1, that is, a coding method with a higher compression ratio. on the other hand,
With the spread of portable terminals in recent years, it is expected that the need for an encoding system with such a high compression rate will increase in the future, and in response to this, the MPEG4 encoding system has been standardized. MPE
Regarding the G4 image coding method, the standard was approved as an international standard in December 1998 as ISO / IEC 14496-2.

[0005] By the way, MPEG2 image compression information (bit stream) once encoded for digital broadcasting.
There is a need to convert MPEG4 image compression information (bit stream) with a lower code amount (bit rate), which is more suitable for processing on a mobile terminal or the like.

As an image information conversion device (transcoder) that achieves the above object, "Field-to-Frame Transco
ding with Spatial and Temporal Downsampling ”(Sus
ie J. Wee, John G. Apostolopoulos, and Nick Feamste
r, ICIP '99, hereinafter referred to as Document 1) proposes a device as shown in FIG.

In FIG. 6, data of each frame in MPEG2 image compression information (bit stream) of interlaced scanning supplied to an input terminal 110 is first input to a picture type discrimination unit 111.

In the picture type discriminating section 111, input data of each frame is related to an I picture (intra-coded picture) / P picture (forward predicted coded picture) or a B picture (bidirectional predicted coded picture). Then, only in the former case, information on the I / P picture is output to the subsequent MPEG2 image information decoding unit (I / P picture) 112.

The processing in the MPEG2 image information decoding section (I / P picture) 112 is the same as that of a normal MPEG2 image information decoding apparatus. However, since the data relating to the B picture is discarded by the picture type discrimination unit 111, the MPEG2 image information decoding unit (I / P picture)
The function at 112 is only required to be able to decode only I / P pictures. MPEG2 image information decoding unit (I
/ P picture) 112 is input to the thinning unit 113.

The thinning section 113 performs a 1/2 thinning process in the horizontal direction, leaves only one of the first field and the second field in the vertical direction, and discards the other. , A progressively scanned image having a size of １ ／ of the input image information is generated. The progressive scan image generated by the thinning unit 113 is temporarily stored in the video memory 114 and then read out, and is read by the MPEG4 image information encoding unit (I / P-VO).
P) 115.

Here, for example, the input MPEG2 image compression information (bit stream) is transmitted by NTSC (National T
elevision System Committee),
In other words, in the case of an interlaced scanning image of 720 × 480 pixels and 30 Hz, the image frame after the thinning is 360 × 240.
Although it is a pixel, when encoding is performed in the subsequent MPEG4 image information encoding unit (I / P-VOP) 115, processing in units of macroblocks is performed in the horizontal direction.
The number of pixels must be a multiple of 16 both in the vertical direction. Therefore, the pixels are supplemented or discarded in the thinning unit 113 at the same time. That is, at this time, the thinning section 113 discards the eight pixels at the right end or left end in the horizontal direction, for example, as 352 × 240 pixels as a supplement or discard of the pixels.

The MPEG4 image information encoding unit (I / P
In (-VOP) 115, the signal of the input progressively scanned image is encoded to generate MPEG4 image compression information (bit stream), and the MPEG4 image compression information is output from the output terminal 118 to the subsequent stage. At that time, MP
The motion vector information in the EG2 image compression information (bit stream) is mapped to a motion vector for the thinned image information in the motion vector synthesis unit 116,
Further, the motion vector detecting section 117 detects a highly accurate motion vector based on the motion vector value synthesized by the motion vector synthesizing section 116. Note that in MPEG4, a VOP (Video Object Plane) represents an area composed of one or a plurality of macroblocks surrounding an object, and corresponds to a frame in MPEG2. The VOP area is classified into one of an I picture, a P picture, and a B picture according to a coding scheme. An I-VOP (a VOP of an I picture) is one in which an image (area) itself is encoded (intra-encoded) without performing motion compensation. Basically, a P-VOP (a VOP of a P picture) is basically a picture (I or P-
VOP) and forward prediction coding. B-VO
P (VOP of B picture) is basically composed of two images (I or PV) located temporally before and after itself.
OP) and bidirectional predictive coding.

[0013] As described above, the document 1 describes that the input M
× of PEG2 image compression information (bit stream)
A technique relating to a device for generating MPEG4 image compression information (bit stream) of a progressively scanned image having a half size is described. That is, the input MPEG2
If the image compression information (bit stream) complies with, for example, the NTSC standard, the output MPEG4
Although the image compression information has the SIF size (352 × 240), according to the configuration of FIG. 6, by changing the operation of the thinning unit 113, other image frames, for example, in the above example, It is also possible to convert an image into a QSIF (176 × 112 pixel) size image which is an image frame of about ４ × ４.

[0014] Further, in Document 1, as processing in the MPEG2 image information decoding unit (I / P picture) 112,
Input MP for horizontal and vertical directions
An apparatus for performing a decoding process using all the 8th-order discrete cosine transform coefficients in the EG2 image compression information (bit stream) is described. However, the apparatus shown in FIG. Alternatively, in both the horizontal and vertical directions, the decoding process using only the low-frequency component of the eighth-order discrete cosine transform coefficients is performed, and the amount of calculation and video memory capacity involved in the decoding process are minimized while minimizing image quality deterioration. Has been made possible.

In the image information conversion apparatus shown in FIG. 6, an MPEG4 image information encoding unit (I / P-VOP)
When the P-VOP is encoded at 115, each macroblock is assigned to an intra (INTR) defined by MPEG4.
A) Encoding as a macro block, encoding as a 16 × 16 pixel inter (INTER) macro block, or 8 × 8 pixel inter 4V (INTER4V)
It is necessary to determine the type of coding mode for coding as a macroblock.

Here, as a general method of mode determination, MPEG-4 Video Verification is used.
It is conceivable to use the method defined in on Model (ISO / IEC JTC1 / SC29 / WG11 N2932, hereinafter referred to as Document 2).

Hereinafter, a method of mode determination (mode determination performed by the MPEG4 image information encoding unit (I / P-VOP) 115 in FIG. 6) described in Reference 2 will be described with reference to FIG.

First, in step S101, an inter motion vector and an inter 4
Find the V motion vector.

Next, in step S102, predicted images generated by the inter motion vector and the inter 4V motion vector are respectively referred to as Ref _INTER and Ref _INTER .
_{It is assumed} that the _macroblock is represented by _{IN TER4V} and the original picture is represented by Org. Further, in step S103, a prediction error ER when the macroblock is encoded as an inter macroblock and an inter 4V macroblock.
R _INTER and ERR _{INTER4 V} are calculated by equations (1) and (2), respectively, and the average value of the pixels included in the macro block is defined as Mean_MB, and the macro block is coded as an intra macro block. The prediction error ERR _INTRA is defined as in equation (3).

ERR _INTER = SAD (Org-Ref _INTER ) (1) ERR _INTER4v = SAD (Org-Ref _INTER4v ) (2) ERR _INTRA = SAD (Org-Mean_MB) (3) In the equation, SAD is an absolute value error. Sum of Absolute Di
fference).

Next, in step S104, the prediction error ERR obtained by the above equations (1) and (2)
_{From INTER} , ERR _INTER4V , it is determined which of the coding efficiency is better, that is, coding as an inter macro block or coding as an inter 4V macro block. That is, if Equation (4) is satisfied, it is determined that coding as an inter macroblock has better coding efficiency, and if not, it is determined that coding as an inter 4V macroblock has better coding efficiency.

ERR _INTER− Offset <ERR _INTER4v (4) In this equation (4), Offset is a parameter for making it easier to select an inter macroblock, and is set to 129 in Reference 2.

Next, when an inter macroblock is selected by the equation (4), the parameter ERR is defined as in the equation (5). On the other hand, when an inter 4V macroblock is selected, the parameter ERR is defined by the equation (5). Defined as 6).

ERR = ERR _INTER (5) ERR = ERR _INTER4V (6) Next, the macro block is encoded as an intra macro block from the parameter ERR and the prediction error ERR _INTRA defined by the equation (3). And encoding in the macroblock mode selected by equation (4), it is determined which encoding efficiency is higher. That is, if equation (7) is satisfied, it is assumed that coding as an intra macroblock is more efficient, and if not, coding in the macroblock mode selected by equation (3) is more efficient. .

ERR _INTRA <ERR (7) That is, in step S105 of FIG.
And it is efficient to encode the intra macroblock mode in step S108 when _{the INTER <ERR INT RA} is not satisfied, when a condition is satisfied better be encoded in inter-macroblock mode in step S107 is the efficiency. Step S10
6, ERR _INTER4V <ERR _INTRA
If this does not hold, it is assumed that the coding in the intra macroblock mode is more efficient as step S108, and that if it holds, the coding is more efficient in the inter 4V macroblock mode as step S109.

As described above, the MPEG4 image information encoding unit (I / P-VOP) 115 of FIG. 6 uses the mode determination method as shown in FIG. Is selected, and encoding processing of image information is performed using the macroblock mode of the selected type.

Although various methods can be considered for the operation principle of the motion vector synthesizing section 116 and the motion vector detecting section 117 described above, the signals are supplied to the input terminal 110 as in the image information conversion apparatus shown in FIG. MPEG
×× の of two-image compression information (bit stream)
In the case of outputting MPEG4 image compression information (bit stream) having an image frame of, the synthesis and detection of motion vectors may be performed, for example, according to the flow shown in FIG.

That is, in the motion vector synthesizing section 116, first, in step S111, the input MPEG
The motion vector information is extracted from the two-image compression information (bit stream), and then, in step S112, the motion vector information is scaled and time-corrected with respect to the extracted MPEG-2 image compression information, so that the output MPEG-4 image is obtained. Inter 4V motion vector information for the compressed information (bit stream) is synthesized. Further, in the motion vector synthesizing section 116, step S113
The average value or the representative value of the generated inter 4V motion vectors is defined as the inter motion vector.

Next, in step S114, the motion vector detecting section 117 executes
In the inter motion vector and the inter 4V motion vector generated in the above, several pixels around the inter motion vector and the inter 4V motion vector are searched, and the accuracy of the inter motion vector and the inter 4V motion vector is improved. The motion vector whose accuracy has been improved in this way is sent to the MPEG4 image information encoding unit (I / P-VOP) 115 in FIG.

[0030]

By the way, MPEG2
GOP (Group Of) of image compression information (bit stream)
Picture) When the structure is, for example, n = 15; m = 3,
Using the image information conversion device shown in FIG.
When the image compression information is converted into MPEG4 image compression information (bit stream), the GOV structure is n = 5;
That is one.

On the other hand, a lower code amount (bit rate)
In order to minimize image quality degradation when output of MPEG4 image compression information is desired, it is necessary to reduce the frequency of I-VOPs. To achieve this,
It is necessary to convert a frame that was an I picture in the input MPEG2 image compression information (bit stream) into a P-VOP in the output MPEG4 image compression information (bit stream).

However, this method has the following problems.

That is, since motion vector information does not exist in an image that was originally an I picture, motion vector information for a frame converted from an I picture into a P-VOP must be obtained by some method.

In the case of a normal P-VOP, for example, an inter motion vector and an inter 4V motion vector synthesized and refined by the motion vector synthesizing section 116 and the motion vector detecting section 117 shown in FIG. Although it is possible to determine the mode of / inter 4V, in the case of a frame converted from an I picture to a P-VOP as described above, it is not easy to determine the mode of inter / inter 4V.

Accordingly, the present invention has been made in view of such circumstances, and has been developed in accordance with the present invention.
When converting to EG4 image compression information, motion vector information for a frame converted from an I picture to a P-VOP can be obtained easily and with high accuracy.
It is an object of the present invention to provide an image information conversion apparatus and method capable of easily and inter-inter 4V mode determination for a frame converted from a picture into a P-VOP with a small amount of calculation.

[0036]

According to the present invention, there is provided an image information conversion apparatus which converts at least first image encoded information comprising an intra-coded image and an inter-picture predicted encoded image into second image encoded information. In the image information conversion device for conversion, the second
Motion vector generating means for generating motion vector information corresponding to each coding unit consisting of a plurality of pixels constituting the image coding information, motion vector storing means for storing the generated motion vector information, For determining which of the intra-picture coding mode, the first inter-picture prediction coding mode and the second inter-picture prediction coding mode to use for each coding unit constituting the picture coding information of Mode determination means for determining a coding mode, wherein the first image coding information is converted from an intra-coded image to the second image coding information, and is converted into an inter-picture predicted coded image. In this case, the mode determining means uses the first inter-picture predictive coding mode based on the motion vector information of the second picture coding information which has already been generated and stored in the motion vector storing means. By determining whether to use either de and second inter-picture predictive coding mode, solving the above problems.

Further, the image information conversion method of the present invention provides an image information conversion method for converting at least first image coding information including at least an intra-coded image and an inter-picture predicted coded image into second image coded information. In the conversion method, motion vector information corresponding to each coding unit composed of a plurality of pixels constituting the second image coding information is generated, the generated motion vector information is stored, and the second image code is generated. Mode determination for determining which of the intra-picture coding mode, the first inter-picture prediction coding mode, and the second inter-picture prediction coding mode to use for each coding unit constituting the coding information When the first image coding information is converted to an intra-coded image in the second image coding information, it is already generated and converted to an inter-picture predicted coded image in the second image coding information. No. stored above The above-mentioned problem is solved by determining which of the first inter-picture prediction coding mode and the second inter-picture prediction coding mode to use based on the motion vector information of the picture coding information of I do.

That is, according to the present invention, for example, the input MPEG2 image compression information (I picture in the bit stream, but the output MPEG4 image compression information (bit stream) is converted to P-VOP. As the motion vector information for the frame, for example, high-precision motion vector information is obtained based on the inter motion vector and the inter 4V motion vector used when encoding the P-VOP immediately before the P-VOP, Further, the motion vector information is used to easily perform an inter / inter 4 V mode determination for a VOP converted from an I picture to a P-VOP, thereby reducing the amount of calculation involved in the mode determination and reducing the MPEG2 image. The conversion from the compression information to the MPEG4 image compression information can be realized.

[0039]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 shows a schematic configuration of an image information conversion apparatus according to an embodiment of the present invention.

In FIG. 1, the MPEG2 image compression information (bit stream) of the interlaced scanning supplied to the input terminal 1 is first input to the picture type discriminating section 8.

The picture type discriminating section 8 outputs information on I / P pictures and sends it to the compressed information analyzing section 9, but discards information on B pictures. As a result, the frame rate is converted.

The compression information analysis section 9 analyzes the syntax of the image compression information sent from the picture type determination section 8 to extract information related to the encoding of the MPEG2 image compression information, Is transmitted to the information buffer 18, the MPEG2 motion vector information is transmitted to the motion vector synthesizing unit 14, and the image compression information is transmitted to the MPEG2 image information decoding unit (I / P picture) 10. The details of the information extracted by the compression information analysis unit 9 will be described later.

The MPEG2 image information decoding section (I / P picture) 10 is equivalent to that of the apparatus shown in FIG.
Since the information relating to the B picture has already been discarded in the preceding picture type discriminating unit 8, the function of the MPEG2 image information decoding unit 10 is that if it can perform decoding processing of only the information relating to the I / P picture. good. MPEG2 image information decoding unit (I / P picture) 1
The pixel value that is output as 0 is input to the thinning unit 11.

The thinning section 11 performs a 1/2 thinning process in the horizontal direction, leaves only one of the first and second fields in the vertical direction, and discards the other. , A progressively scanned image having a size of １ ／ of the input image information is generated. Here, for example, the MPE supplied to the input terminal 1
G2 image compression information (bit stream) is NTSC (Nati
onal Television System Committee), that is, if the image is a 720 x 480 pixel, 30 Hz interlaced scan image, the image frame after thinning out is 360 x
That is, 240 pixels. However, the subsequent MPEG4
When encoding is performed in the image information encoding unit (I / P-VOP) 13, the number of pixels in the horizontal and vertical directions must be a multiple of 16 in order to perform processing in units of macroblocks. Therefore, the thinning unit 11 simultaneously performs the pixel thinning or the pixel supplement to make the number of pixels a multiple of 16 simultaneously with the thinning. That is, the thinning unit 11 in this example is, for example, 360 × 240
For the pixel, for example, 8 at the right end or left end in the horizontal direction
By discarding the line, 352 which is a multiple of 16 above is used.
An image frame of × 240 pixels is configured. The progressive scan image generated by the thinning unit 11 is temporarily stored in the video memory 1.
After that, the data is read out in response to a request from an MPEG4 image information coding unit (I / P-VOP) 13 at the subsequent stage, and is read out in response to a request from a mode determination unit 16 described later.

In the motion vector synthesizing section 14, the MP
The MPEG2 motion vector information extracted from the EG2 image compression information (bit stream) is mapped to a motion vector corresponding to the thinned image information, and is further synthesized by the motion vector synthesis unit 14 in the next stage motion vector detection unit 15. Based on the obtained motion vector value and the image information stored in the video memory 12, a highly accurate motion vector is detected. The detected motion vector is stored in the motion vector buffer
It is sent to a four-image information encoding unit (I / P-VOP) 13 and a mode determination unit 16.

The MPEG4 image information encoding unit (I / P
-VOP) 13, information relating to the encoding of the MPEG2 image compression information held in the information buffer 18;
Using the motion vector information from the motion vector detection unit 15 and the inter / inter 4V mode information obtained by the determination process in the mode determination unit 16 described later, the progressive scan image supplied from the video memory 12 is used. The signal is encoded to generate MPEG4 image compression information (bit stream). The MPEG4 image compression information is output from the output terminal 2 to a subsequent stage.

In the image information conversion apparatus of the present embodiment shown in FIG. 1, the components other than the motion vector buffer 17 and the inter / inter 4V mode determination in the mode determination section 16 are shown in FIG. This is almost the same as the image information conversion device.

Hereinafter, the operation of the motion vector buffer 17 and the mode determination unit 16 of the image information conversion apparatus according to the embodiment of the present invention will be described with reference to FIGS.

First, the P-VO converted from the I picture
The operating principle of mode determination for a normal P-VOP other than P will be described with reference to the flowchart of FIG. 2 and FIG.

The mode judging section 16 first sets the intra / inter mode based on the information related to the encoding extracted from the input MPEG2 image compression information (bit stream) stored in the information buffer 18. Make a decision.

Here, in the case of outputting the progressively scanned MPEG4 image compression information (bit stream) having an image frame of about 1/2 × 1/2 of the interlaced scanning MPEG2 image compression information (bit stream), for example, As shown in FIG. 3, four macroblocks MB _{MPEG2, i} (i = 1,2,3,4) included in the image STR2 constituting the MPEG2 image compression information supplied to the input terminal 1 are:
It is _assumed that the image corresponds to the macro block MB _MPEG4,1 in the image STR4 constituting the MPEG4 image compression information.

In this example, the mode determination section 16
In MB _{MPEG2, i} of the image STR2 constituting the input MPEG2 image compression information, the number of macroblocks whose macroblock type is an intra macroblock is N
_INTRA , when the number of inter macroblocks is N _INTER and the following equation (8) is satisfied, it is determined in step S2 that the macro block is an intra macro block.
In step S6, the macro block of the image STR4 constituting the output MPEG4 image compression information is determined as an intra macro block.

N _INTRA > N _INTER (8) On the other hand, when the following equation (9) is satisfied, the mode determination unit 16
Determines in step S2 that the block is an inter macroblock or an inter 4V macroblock, and passes the process to the next step S3.

N _INTRA <N _INTER (9) When the following expression (10) is satisfied, an intra macro block, an inter macro block, or an inter 4 V macro block may be used.

N_INTRA= N_INTER (10) Also, as in the above equation (7), the mode
May be set. Or the macroblock MB
_{MPEG2, i}Is the quantization scale for each
_{MPEG2, i}(I = 1, 2, 3, 4), and
The allocated code amount (bit number) is B_{MPEG2, i}
(I = 1, 2, 3, 4), the following equation (11)
So that those macroblocks MB_{MPEG2, i}To
Complexity X_{MPEG2, i}(I = 1, 2,
3, 4) is calculated by equation (10), and this complex
Use the city to select a mode that is considered
You may make it select.

X _{MPEG2, i} = Q _{MPEG2, i} · B _{MPEG2, i} (11) Next, in step S2 of the intra / inter determination, a macroblock determined to be an inter macroblock or an inter 4V macroblock will be described. Is the VO stored in the motion vector detection unit 15.
Using the inter motion vector and the inter 4V motion vector for P, the inter / inter 4V is also determined. That is, the mode determination unit 16 _sets the x direction component and the y direction component of the inter motion vector of the macroblock to mv _{16X16 —} x and mv _{16X16 —} y, respectively, and the x direction and the y direction component of the inter 4V motion vector to mv _{8X8 —} x _{, i} mv _{8X8_y. i} (i
= 1, 2, 3, 4), and the variance Dist of the motion vector information is calculated by the following equation (12) or (13) as in step S3.

[0058]

(Equation 1)

[0059]

(Equation 2)

Further, the mode determination section 16 has a predetermined first threshold value θ ₁ and a predetermined second threshold value θ ₂ (θ ₁ <θ ₂ ).
Is stored, and as step S4,
Variance value Dist calculated by the above formula (12) or formula (13), using a first threshold value theta ₁ which said predetermined, it is determined whether the following equation (14) holds.

Dist <θ ₁ (14) When the equation (14) holds, the mode determination unit 16
In step S7, the macro block is determined to be an inter macro block.

When the equation (14) does not hold, the mode determining unit 16 determines in step S5 that the equation (1)
2) or by using the second threshold value theta ₂ which are predetermined computed variance value Dist and the formula (13), it is determined whether the following equation (15) holds.

Dist> θ ₂ (15) When the expression (15) is satisfied, the mode determination unit 16
In step S7, the macro block is determined to be an inter macro block.

Further, when the equations (14) and (15) do not hold, that is, when the relationship between the dispersed land Dist and the first and second thresholds θ ₁ and θ ₂ is the equation (16), the mode The determining unit 16 determines that the macroblock is an inter 4V macroblock in Step S8.

Θ ₁ ≦ Dist ≦ θ ₂ (16) The macroblock mode determination result obtained by the above series of mode determination processes is sent to the MPEG4 image information encoding unit (I / P-VOP) 13. The MPEG4 image information encoding unit 13 encodes the progressively scanned image signal supplied from the video memory 12 according to the mode determination result to generate MPEG4 image compression information (bit stream).

For the above-described ordinary P-VOP, it is possible to make a mode decision based on the residual using Equation (4) and / or Equation (7).

Next, a description will be given of the operation principle of the mode determination for a P-VOP which is an I picture in the input MPEG2 image compression information but is converted into a P-VOP in the output MPEG4 image compression information.

As described above, a frame converted from an I picture to a P-VOP is input to an MPEG-2
Since all macroblocks included in the I picture in the image compression information (bit stream) are intra macroblocks, it is not possible to determine the mode based on the above equations (8) to (10).

Therefore, the mode determination section 16 of the embodiment of the present invention performs a mode determination based on the residual as shown in the above equations (4) and (7) when determining the intra / inter. When determining the inter / inter 4 V, the mode is determined based on the equations (14) to (16).

That is, according to the present embodiment, the P-V immediately before the VOP converted from the I-picture to the P-VOP is used.
The motion vector information relating to the OP is stored in the motion vector buffer 17, and the motion vector information relating to the P-VOP stored in the motion vector buffer 17 is made highly accurate by the motion vector detection unit 15. The information is input to the MPEG4 image information encoding unit (I / P-VOP) 13 and also to the mode determination unit 16, and the mode determination unit 16 converts the IOP into a P-VOP. The inter / inter 4V mode determination is performed using the motion vector information with high precision related to the immediately preceding P-VOP.

FIG. 4 shows a flowchart of a first processing operation example of mode determination in the mode determination section 16 for a frame converted from an I picture into a P-VOP.

In FIG. 4, the mode determination section 16 first uses the image information stored in the video memory 12 to calculate the prediction error ERR _INTER and the prediction error ERR _INTER using the image information stored in the video memory 12 in step S10. ERR
Calculate _INTRA .

Next, the mode determination section 16 determines in step S1
In the processing of 1, it is determined whether or not the following equation (17) holds.

ERR _INTRA <ERR _INTER (17) If the equation (17) is satisfied in the process of step S11, the mode determination unit 16 performs step S15.
It is assumed that the macro block is an intra macro block.

On the other hand, if the equation (17) does not hold in the processing of step S11, the mode determination unit 16 calculates the variance Dist using the above equation (12) or (13) as the processing of step S12. Further, in steps S13 and S14, using the variance value Dist and the predetermined first thresholds θ ₁ and θ ₂ , the inter / inter 4 is calculated based on the above equations (14) to (16).
The mode of V is determined.

That is, in step S13, equation (1)
If 4) is satisfied, the mode determination unit 16 determines in step S
Assume that the macroblock is 16 as an inter macroblock.

On the other hand, in step S13, equation (14)
Does not hold, the mode determination unit 16 determines in step S1
In the process of No. 4, it is determined whether or not the expression (15) is satisfied.

If the equation (15) is satisfied in step S14, the mode determination section 16 determines that the macroblock is an inter macroblock in step S16.

On the other hand, in step S14, equation (15)
Does not hold, the mode determination unit 16 determines in step S1
7, the macro block is an inter 4V macro block.

FIG. 5 is a flowchart showing a second processing operation example of the mode determination in the mode determination section 16 for a frame converted from an I picture to a P-VOP.
Shown in

In FIG. 5, the mode determining section 16 first calculates a variance value Dist using the above equation (12) or (13) in the process of step S20, and further, in steps S21 and S22, the variance value Dist is calculated. Value Di
The inter / inter 4 V mode determination is performed based on the equations (14) to (16) using st and the predetermined first thresholds θ ₁ and θ ₂ .

That is, in step S21, the expression (1)
If 4) is satisfied, the mode determination unit 16 determines that the macroblock is an inter macroblock, and advances the process to step S23.

On the other hand, in step S21, equation (14)
Does not hold, the mode determination unit 16 determines in step S2
In the process of 2, it is determined whether or not the expression (15) is satisfied.

If the equation (15) is satisfied in step S22, the mode determining section 16 determines that the macroblock is an inter macroblock, and advances the processing to step S23.

On the other hand, in step S22, equation (15)
Does not hold, the mode determination unit 16 determines that the macroblock is an inter 4V macroblock, and advances the process to step S24.

At step S23, the mode determining section 16 uses the image information stored in the video memory 12 to calculate the prediction error ER according to the equations (1) and (3).
Calculating the R _{I NTER} and _{ERR INTRA.}

Next, the mode determination section 16 determines in step S2
In the process of 5, it is determined whether or not the expression (17) is satisfied.

In the process of step S25, the expression (1)
If 7) is satisfied, the mode determination unit 16 determines in step S
Assume that the macro block is an intra macro block.

On the other hand, if the equation (17) is not satisfied in the processing in step S25, the mode determining section 16 determines that the macro block is an inter macro block in step S27.

[0090] When the process proceeds to step S24, the mode determining unit 16 uses the image information stored in the video memory 12, the prediction by the formula (2) and (3) error _{ERR INTER4V} and _{ERR INTRA} Is calculated.

Next, the mode determination section 16 determines in step S2
In the process of 6, it is determined whether or not the following equation (18) holds.

ERR _INTRA <ERR _INTER4V (18) If the equation (18) is satisfied in the process of step S26, the mode determining unit 16 determines that
It is assumed that the macro block is an intra macro block.

On the other hand, if the equation (18) does not hold in the processing of step S26, the mode determining section 16 determines that the macro block is
Let it be a macro block.

As described above, according to the embodiment of the present invention, in an image information conversion apparatus which receives MPEG2 image compression information (bit stream) as input and outputs MPEG4 image compression information (bit stream), The MPEG-2 image compression information (bit stream) is an I-picture, but the output MPEG-4 image compression information (bit stream) is a P-VOP.
Inter / Inter 4V for frames converted to
The mode determination can be easily realized using an inter motion vector and an inter 4V motion vector related to the P-VOP immediately before the VOP converted from the I picture into the P-VOP.

In the above description, MPE is used as an input.
Using G2 image compression information (bit stream) as an example, output MPEG4 image compression information (bit stream)
However, the input and output are not limited to this, and may be, for example, MPEG1 or H.264. H.263 or other image compression information (bit stream).

[0096]

As is apparent from the above description, according to the present invention, motion vector information corresponding to each coding unit constituting the second picture coding information is generated and the first picture coding information is generated. In the information, the one that was an intra-coded image
When the image encoding information is converted into an inter-image prediction encoded image, the first inter-image prediction encoding mode and the second image encoding mode are converted based on the motion vector information of the already generated second image encoding information. By determining which of the two inter-picture prediction coding modes to use, for example, in a case where MPEG2 image compression information is converted to MPEG4 image compression information, a frame to be converted from an I picture to a P-VOP , Motion vector information can be obtained easily and with high accuracy, and the inter / inter 4V mode determination for a frame converted from an I picture to a P-VOP can be realized easily and with a small amount of calculation.

[Brief description of the drawings]

FIG. 1 is a block circuit diagram illustrating a schematic configuration of an image information conversion device according to a first embodiment of the present invention.

FIG. 2 is a flowchart illustrating a flow of a mode determination process in the image information conversion device according to the embodiment of the present invention.

FIG. 3 shows macroblocks and M of MPEG2 image compression information.
It is a figure showing the example of correspondence of the macroblock of PEG4 picture compression information.

FIG. 4 is a flowchart illustrating a first processing operation example of a mode determination unit in the image information conversion device according to the embodiment of the present invention.

FIG. 5 is a flowchart illustrating a second processing operation example of the mode determination unit in the image information conversion device according to the embodiment of the present invention.

FIG. 6 is a block circuit diagram showing a schematic configuration of a conventional image information conversion device.

FIG. 7 is a flowchart showing a flow of a mode determination process in a conventional image information conversion device.

FIG. 8 is a flowchart showing a flow of synthesizing a motion vector and improving accuracy.

[Explanation of symbols]

1 picture type discriminator, 9 image information analyzer, 10
MPEG2 image information decoding unit (I / P picture), 1
1 thinning section, 12 video memory, 13 MPEG4
Image information encoding unit (I / P-VOP), 14, 116
Motion vector synthesis unit, 15, 117 motion vector detection unit, 16 mode determination unit, 17 motion vector buffer, 18 information buffer

 ──────────────────────────────────────────────────続 き Continued on the front page (72) Inventor Teruhiko Suzuki 6-7-35 Kita-Shinagawa, Shinagawa-ku, Tokyo Inside Sony Corporation (72) Inventor Yoichi Yagasaki 6-35, Kita-Shinagawa, Shinagawa-ku, Tokyo Sony Corporation F term (reference) 5C059 KK41 MA00 MA04 MA05 MA23 NN01 NN11 NN21 PP04 SS02 SS06 SS12 TA23 TB04 TC12 TC36 TD04 TD11 UA02 UA05 UA32 UA37

Claims (12)

[Claims] 1. An image information conversion apparatus for converting at least first encoded image information composed of at least an intra-coded image and an inter-predictive encoded image into second encoded image information, wherein the second image Motion vector generating means for generating motion vector information corresponding to each coding unit consisting of a plurality of pixels constituting the coding information; motion vector storing means for storing the generated motion vector information; and the second image An encoding mode for determining which of the intra-picture encoding mode, the first inter-picture prediction encoding mode, and the second inter-picture prediction encoding mode to use for each encoding unit constituting the encoding information A mode determining unit for performing a determination. When the first encoded image information is an intra-coded image, the second encoded image information is converted to an inter-picture predictive encoded image. To In the mode determining means, the first inter-picture predictive coding mode and the second inter-picture predictive coding mode are set based on the motion vector information of the second image coding information which has already been generated and stored in the motion vector storing means. An image information conversion apparatus, which determines which of the inter-picture prediction coding modes to use. 2. The method according to claim 1, wherein the mode determining unit determines a motion vector of the coding unit of the first inter-picture predictive coding mode, which has been generated and stored in the motion vector storage unit, and the second inter-picture predictive coding mode. A variance value of a motion vector of a coding unit in a predictive coding mode is obtained, and based on a comparison between the variance value and preset first and second thresholds, the second image coding information is formed. 2. The image information conversion apparatus according to claim 1, wherein a coding mode for the coding unit is determined. 3. The mode determination means, when the second threshold value is larger than the first threshold value, when the first threshold value is larger than the variance value, or when the variance value is larger than the second threshold value. If the condition holds, the encoding mode of the encoding unit constituting the second image encoding information is set to the first encoding mode.
If the condition that the variance value is equal to or greater than the first threshold value and equal to or less than the second threshold value is satisfied, the encoding that constitutes the second image encoding information is performed. 3. The image information conversion apparatus according to claim 2, wherein the coding mode of the unit is determined to be a second inter-picture prediction coding mode. 4. The method according to claim 1, wherein the mode determining unit determines whether to use the intra-picture coding mode or the inter-picture predictive coding mode in coding mode. 2. The image information conversion device according to claim 1, wherein the image information conversion is performed based on a predetermined residual obtained from the image data. 5. The method according to claim 1, wherein the mode determining means is configured to determine which of the intra-picture coding mode and the inter-picture prediction coding mode based on a predetermined residual obtained from the picture information obtained by decoding the first picture coding information. To determine whether to use the encoding mode,
A first inter-picture predictive coding mode based on a comparison between the variance value and first and second threshold values only for the coding unit determined not to be the intra-picture coding mode by the coding mode determination. 4. The image information conversion apparatus according to claim 3, wherein a determination is made as to which of the first and second inter-picture prediction coding modes is to be used. 6. The mode determination means for the coding unit determined to be not the first inter-picture prediction coding mode by comparing the variance value with first and second thresholds. And performing a coding mode determination of which of the intra-picture coding mode and the second inter-picture prediction coding mode to use based on a predetermined residual obtained from image information obtained by decoding the first image coding information, Image information obtained by decoding the first image coding information for the coding unit determined to be not in the second inter-picture prediction coding mode by comparing the variance value with the first and second thresholds 4. The image information according to claim 3, wherein a coding mode determination is made as to which of the intra-picture coding mode and the first inter-picture prediction coding mode is to be used based on the predetermined residual obtained from the above. Conversion device. 7. An image information conversion method for converting at least first image coding information including at least an intra-coded image and an inter-picture prediction-coded image into second image coding information, wherein the second image Generating motion vector information corresponding to each coding unit composed of a plurality of pixels constituting the coding information; storing the generated motion vector information; each coding unit forming the second image coding information; For each time, a coding mode determination is made to determine which of the intra-picture coding mode, the first inter-picture prediction coding mode, and the second inter-picture prediction coding mode to use, and the first picture code When the encoded information is an intra-coded image, when the second encoded image information is converted to an inter-picture predicted encoded image, the second encoded image is generated and stored. Motion vector of encoded information Based on the information, between the first image predictive coding mode and a second
Determining which of the inter-picture prediction coding modes to use. 8. A motion vector of a coding unit of the first inter-picture prediction coding mode and a motion vector of a coding unit of the second inter-picture prediction coding mode which are already generated and stored. Is determined, and an encoding mode for the encoding unit constituting the second image encoding information is determined based on a comparison between the variance value and first and second preset threshold values. 8. The image information conversion method according to claim 7, wherein: 9. When the second threshold value is larger than the first threshold value, if a condition that the first threshold value is larger than the variance value or a condition that the variance value is larger than the second threshold value is satisfied, The encoding mode of the encoding unit constituting the second image encoding information is determined to be a first inter-picture prediction encoding mode, and the variance value is equal to or greater than the first threshold and equal to or less than a second threshold. 9. The method according to claim 8, wherein if the condition is satisfied, the coding mode of the coding unit constituting the second image coding information is determined to be the second inter-picture prediction coding mode. Image information conversion method. 10. A coding mode determination as to which of the intra-picture coding mode and the inter-picture prediction coding mode to use is performed by a predetermined residual value obtained from image information obtained by decoding the first image coding information. 8. The image information conversion method according to claim 7, wherein the method is performed based on the difference. 11. A code indicating which of the intra-picture coding mode and the inter-picture prediction coding mode to use based on a predetermined residual obtained from picture information obtained by decoding the first picture coding information. A coding mode determination is performed, and only for the coding unit determined to be not the intra-picture coding mode by the coding mode determination, a first value based on a comparison between the variance value and first and second threshold values is set. 10. The image information conversion method according to claim 9, wherein a determination is made as to which of the inter-picture prediction coding mode and the second inter-picture prediction coding mode to use. 12. The first image coding information for the coding unit determined to be not in the first inter-picture prediction coding mode by comparing the variance value with first and second thresholds. Is determined based on a predetermined residual obtained from image information obtained by decoding the encoding mode, and an encoding mode determination is made as to which of the intra-image encoding mode and the second inter-image prediction encoding mode is to be used. , For a coding unit determined to be not in the second inter-picture predictive coding mode by comparison with a second threshold, a predetermined residual value obtained from image information obtained by decoding the first image coding information. 10. The image information conversion method according to claim 9, wherein a coding mode determination is made as to which of the intra-picture coding mode and the first inter-picture prediction coding mode to use based on the difference.