WO2000062554A1

WO2000062554A1 - Image processing method and image processing device

Info

Publication number: WO2000062554A1
Application number: PCT/JP2000/002308
Authority: WO
Inventors: Satoshi Kondo
Original assignee: Matsushita Electric Industrial Co., Ltd.
Priority date: 1999-04-08
Filing date: 2000-04-07
Publication date: 2000-10-19
Also published as: AU3674900A

Abstract

An image decoder (10) decodes a code sequence (CS) and generates an original image signal (SHR) with a 1st resolution and, further, extracts encoding parameters (PAR) such as a motion vector and an orthogonal transformation type from the code sequence (CS). A resolution converter (20) judges the characteristics of the original image signal (SHR) in accordance with the encoding parameters (PAR) and converts the original image signal (SHR) into a new image signal (SLR) with a 2nd resolution by a resolution conversion method corresponding to the characteristics.

Description

Satsuki Itoda β

Image processing method and image processing apparatus

The present invention relates to an image processing technique involving resolution conversion of a video signal encoded by a high-efficiency compression encoding scheme such as the MPEG2 scheme. Background art

In recent years, the Moving Picture Experts Group 2 (MPEG2) method has been widely used as a high-efficiency compression encoding method for communication and recording of video signals. In the MPE G2 system, video frames (fields) are classified into three types of pictures, i-picture, p-picture, and b-picture, and are encoded. In the I-picture, intra-frame encoding is performed. In the P-picture, forward prediction inter-frame coding is performed using the I-picture or the P-picture that is temporally forward as a reference frame. In the B victim, bidirectional prediction interframe coding is performed using the I-picture or P-picture temporally forward and backward as a reference frame.

Motion compensation is used in predictive coding for P- and B-pictures. In MPEG2, motion compensation is performed in units called macroblocks of (16 x 16) pixels. Generally, in motion vector detection in motion compensation, a block matching method for finding a block having the highest correlation in a reference frame is used. Here, a conventional image processing technique involving resolution conversion will be described.

FIG. 22 is a block diagram of a conventional image processing apparatus that performs MPEG2 decoding, resolution conversion, and MPEG2 encoding. In FIG. 22, the code string C SA is decoded by the MPEG 2 decoder 5110 to become a high-resolution original video signal S HR. The original video signal SHR is converted by the resolution converter 520 into a new video signal SLR of low resolution. The new video signal SLR is converted to MP EG by MP EG 2 encoder 530. 2 Encoded and output as code string CSB.

When the original video signal S HR is an interlace signal, the resolution converter 520 generally performs conversion on a field basis. This is because if the resolution conversion is performed with the frame structure, the resolution of the still image part will be higher than when the resolution conversion is performed with the field structure, but the moving image part will not be converted correctly.

FIG. 23 is a diagram showing resolution conversion in the field structure. As shown in Figure 23, the first field of the new video signal is generated from the first field of the original video signal, and the second field of the new video signal is generated from the second field of the original video signal Is done. In addition, motion detection is performed prior to resolution conversion, thereby detecting a still image portion and a moving image portion, and performing resolution conversion using a frame structure for the still image portion and resolution conversion using a field structure for the moving image portion. A way to do this has also been proposed.

FIG. 24 is a diagram showing two types of resolution conversion. In the figure, (a) is a screen of the original video signal with high resolution, (b) is the result of converting the original video signal of (a) into a low-resolution letterbox image, and (c) is (a) FIG. 7 is a diagram showing a result of converting the original video signal of FIG. The aspect ratio of the original video signal in Fig. 24 (a) is 16: 9, and the aspect ratio of the new video signal is 4: 3. In the letter-box image shown in Fig. 24 (b), the image is reduced in the same ratio both vertically and horizontally, and black bands are added above and below. On the other hand, in the squeeze image shown in Fig. 24 (c), the image is reduced at different ratios in the vertical and horizontal directions, resulting in an aspect ratio of 4: 3. Solution issues

However, the above-mentioned conventional technology has the following problems.

First, when motion detection is performed prior to resolution conversion, enormous processing is required for motion detection, and as a result, the amount of hardware and software increases.

In the configuration shown in FIG. 22, the MPEG2 encoder 530 uses the code sequence CSA and In order to generate a CSB with only a different resolution, all normal MPEG 2 coding processes must be performed. For this reason, the amount of processing is increased, and as a result, the amount of hardware divided by the amount of software is increased.

Also, whether the new video signal SLR is a letterbox image or a squeezed image, the encoding method in the MPEG2 encoder 530 is the same, and the features of the resolution conversion method are not utilized. Disclosure of the invention

An object of the present invention is to enable image processing involving resolution conversion to be realized with a small processing amount.

Specifically, the present invention provides, as an image processing method, a decoding step of decoding a code string in which an original video signal having a first resolution is encoded, and extracting an encoding parameter of the code string. Resolution conversion for determining the characteristics of the original video signal from the encoding parameters and converting the decoded original video signal into a new video signal having a second resolution by a resolution conversion method according to the characteristics. And steps.

In the resolution converting step, it is preferable to determine a motion characteristic of an image in the original video signal as a characteristic of the original video signal. Further, the encoding parameters include a motion vector indicating a motion amount of a video constituent unit, an orthogonal transform type indicating whether the orthogonal transform is performed in a frame structure or a field structure, and a motion compensation in a frame structure. Alternatively, it is preferable to include at least one of the motion compensation modes indicating which of the field structures is used.

In the resolution conversion step, the image of the decoded original video signal is divided into a still area and a moving area using the encoding parameter, and the original video signal is mutually divided in the still area and the moving area. It is preferable to convert to the new video signal using a different resolution conversion method. The original video signal is an interlaced signal. In the still area, it is preferable to perform resolution conversion in units of frames, while in the moving area, it is preferable to perform resolution conversion in units of fields. Further, the encoding parameter is a motion vector indicating a motion amount of a video constituent unit, and the area division is preferably performed based on a comparison result between an absolute value of the motion vector and a predetermined value. .

In addition, the present invention provides an image processing apparatus, comprising: decoding a code string in which an original video signal having a first resolution is encoded; and extracting video encoding parameters of the code string. An original video signal output from the video decoder and an encoding parameter are input, a characteristic of the original video signal is determined from the encoding parameters, and the resolution of the original video signal according to the characteristic is determined. And a resolution converter for converting to a new video signal having the second resolution by a conversion method.

Then, it is preferable that the resolution converter determines a motion characteristic of an image in the original video signal as a characteristic of the original video signal. Further, the coding parameter includes a motion vector indicating a motion amount of a video constituent unit, an orthogonal transform parameter indicating whether the orthogonal transform is performed in a frame structure or a field structure, and a motion compensation. It is preferable to include at least one of the motion compensation modes indicating whether to perform the frame structure or the field structure.

The resolution converter may further include: a region dividing unit that divides an image of the input original video signal into a still region and a moving region using the encoding parameter; and the still image output from the region dividing unit. A static area resolution conversion unit that converts a video signal of a region into the video signal of the second resolution, and converts the video signal of the moving region output from the region divider into a video signal of the second resolution It is preferable to include a moving area resolution converter. Further, the original video signal is an interlaced signal, the static area resolution conversion section performs resolution conversion in units of frames, and the moving area resolution conversion section performs resolution conversion in units of fields. It is preferred that In addition, the encoding parameter is a motion vector indicating a motion amount of a video constituent unit. It is preferable that the region dividing unit divides the region based on a comparison result between the absolute value of the motion vector and a predetermined value. Further, the present invention provides, as an image processing method, a decoding step of decoding a code string obtained by encoding an original video signal having a first resolution, and extracting a motion vector from the code string. And a resolution conversion step of converting the decoded original video signal into a new video signal having the second resolution using the extracted motion vector.

The resolution converting step includes an area dividing step of dividing the decoded image of the original video signal into a quasi-stationary area and a moving area using the extracted motion vector. It is preferable that the resolution conversion is performed using the extracted motion vector in the quasi-stationary region, but not using the extracted motion vector in the moving region.

Then, the area dividing step detects, from the extracted motion vector, a motion vector in a pixel unit having a direction similar to the motion vector, and determines an area in which the motion vector in the pixel unit is detected. It is preferable that the region is a stationary region and an undetected region is a moving region. Further, it is preferable that the resolution conversion to the new video signal in the quasi-stationary region is performed using the detected motion vector in pixel units.

In the region dividing step, a region where the absolute value of the extracted motion vector is smaller than a predetermined threshold is set as a quasi-stationary region, and a region larger than the predetermined threshold is set as a moving region. Is preferred. Further, the present invention provides, as an image processing device, a video decoding device that decodes a code sequence obtained by encoding an original video signal having a first resolution and extracts a motion vector from the code sequence. An original video signal and a video output from the video decoder. A resolution converter that receives a vector as an input and converts the original video signal into a new video signal having a second resolution using the motion vector.

The resolution converter receives the original video signal and the motion vector as input, and divides the image of the original video signal into a quasi-stationary region and a moving region using the motion vector. A quasi-stationary region resolution conversion unit that converts the video signal of the quasi-stationary region output from the region division unit to a video signal having a second resolution using the motion vector, and an output from the region division unit. It is preferable that the image processing apparatus further includes a moving area resolution conversion unit that converts the video signal of the moving area into a video signal having a second resolution without using the motion vector.

The region dividing unit further includes a motion vector detecting unit that detects a motion vector in a pixel unit having a similar direction to the motion vector from the motion vector, and the motion vector detector detects the motion vector in the pixel unit. It is preferable that an area where the motion vector is detected is set as a quasi-static area, and an area where no motion vector is detected is set as a moving area. Further, it is preferable that the quasi-static region resolution conversion unit performs resolution conversion using the pixel-by-pixel motion vector detected by the motion vector detection unit.

Further, the region dividing unit sets a region where the absolute value of the motion vector is smaller than a predetermined threshold as a quasi-stationary region, and sets a region larger than the predetermined threshold as a moving region. preferable. Further, the present invention provides, as an image processing method, a first code string obtained by encoding an original video signal having a first resolution, and a first code string is decoded from the first code string. Extracting an encoded parameter sequence; converting the decoded original video signal to a new video signal having a second resolution; and converting the first encoded parameter sequence to the new video signal. Converting to a second encoding parameter used for encoding, and encoding the new video signal using the second encoding parameter to generate a second code sequence. . Then, the encoding parameter conversion step includes: converting a first encoding parameter used for encoding a first area of an image of an original video signal into an image of a new video signal; It is preferable to convert a second region including the same video as the second region into a second encoding parameter for encoding.

The first and second encoding parameters are preferably motion vectors. Further, in the encoding parameter conversion step, a value obtained by performing a predetermined operation on the motion vector of the first area, for example, a weighted average value is set as the motion vector of the second area. Is preferred.

The first and second encoding parameters, preferably orthogonal transformation was a frame structure or a is an orthogonal transform type indicating whether to perform any of Fi one field structure _c The present invention relates to an image processing The apparatus decodes a first code string obtained by coding an original video signal having a first resolution, and outputs a first coding parameter from the first code string. A resolution converter for converting the original video signal output from the video decoder into a new video signal having a second resolution; and the first encoding output from the video decoder. An encoding parameter converter for converting a parameter into a second encoding parameter used for encoding the new video signal, and a new video signal output from the resolution converter from the encoding parameter converter. Output It encoded using the second encoding parameter, in which the second code string and a video encoder that generates.

The coding parameter converter converts the first coding parameter used for coding the first region of the image of the original video signal into the first coding parameter of the image of the new video signal. It is preferable to convert the second region including the same video as the first region into second encoding parameters for encoding.

Preferably, the first and second coding parameters are motion vectors. Further, the coding parameter converter may calculate a value obtained by performing a predetermined operation on the motion vector of the first area, for example, a weighted average value, in the second area. Preferably, it is a motion vector of the area.

Further, the first and second encoding parameters are preferably orthogonal transform types indicating whether the orthogonal transform is performed by using a frame structure or a field structure. Decoding a first code string obtained by encoding an original video signal having a first resolution, extracting a first motion vector from the first code string, and decoding the decoded original video signal Converting the signal into a new video signal having a second resolution; determining setting information for obtaining a second motion vector used for encoding the new video signal from the first motion vector; The second motion vector is obtained by using the obtained setting information, and the new video signal is encoded by using the obtained second motion vector to generate a second code sequence.

Then, it is preferable to determine an initial value of the second motion vector as the setting information. Alternatively, it is preferable that a search range for obtaining a second motion vector be determined as the setting information.

Further, the present invention provides an image processing device, which decodes a first code sequence obtained by encoding an original video signal having a first resolution, and performs a first motion from the first code sequence. A video decoder for extracting a vector, a resolution converter for converting an original video signal output from the video decoder to a new video signal having a second resolution, and a video decoder output from the video decoder A motion compensation setting unit that generates setting information for obtaining a second motion vector used for encoding the new video signal from the first motion vector; and a setting information generated by the motion compensation setting device. Then, the second motion vector is obtained, and a new video signal output from the resolution converter is encoded using the obtained second motion vector, thereby generating a second code sequence. With a gasifier

It is preferable that the motion compensation setting device determines an initial value of a second motion vector as the setting information. Alternatively, the motion compensation setting device includes the setting information It is preferable to determine a search range for obtaining the second motion vector. The present invention also provides an image processing method comprising: converting an original video signal having a first resolution into a new video signal having a second resolution and having a black level region in a part of an image; Among the signals, a first code sequence is generated by encoding a video signal in a region excluding the black level region, and a second code sequence in which the video signal in the black level region is encoded is a first code sequence. And generating a code sequence of the new video signal. Alternatively, as an image processing method, an original video signal having a first resolution is converted into an area excluding a black level area of a new video signal having a second resolution and having a black level area in a part of an image. Converting to a video signal, encoding the video signal to generate a first code sequence, connecting the second code sequence obtained by encoding the video signal in the black level region to the first code sequence, This is to generate a code sequence of a new video signal.

Further, the present invention provides, as an image processing apparatus, a resolution converter for converting an original video signal having a first resolution into a new video signal having a second resolution and having a black level region in a part of an image. And a second code sequence that encodes a video signal in an area of the new video signal excluding the black level area to generate a first code string, and encodes the video signal in the black level area. And a video encoder that generates a code sequence of the new video signal by concatenating the first code sequence with the first code sequence.

Alternatively, as an image processing device, the original video signal having the first resolution is excluded from the black level region of the new video signal having the second resolution and having a black level region in a part of the image. A resolution converter that converts the image signal into an area video signal, generates a first code sequence by encoding the video signal, and converts the second code sequence obtained by encoding the video signal in the black level area into the first code And a video encoder for generating a code sequence of the new video signal. BRIEF DESCRIPTION OF THE FIGURES FIG. 1 is a block diagram showing the overall configuration of the image processing apparatus according to the first embodiment of the present invention.

FIG. 2 is a block diagram showing the configuration of the video decoder in FIG.

FIG. 3 is a block diagram showing a configuration of the resolution converter in FIG.

FIGS. 4A and 4B are schematic diagrams illustrating an example of motion determination according to the first embodiment of the present invention, and FIG. 4A illustrates a case where a motion vector is used as a coding parameter.

FIG. 4 (b) is a diagram illustrating a case where DCT Eve is used as an encoding parameter. FIGS. 5A to 5C are schematic diagrams showing an example of region division according to the first embodiment of the present invention. FIG. 5A shows a motion determination result, FIG. Fig. 5 (c) shows the moving region.

6 (a) and 6 (b) are schematic diagrams showing resolution conversion according to the first embodiment of the present invention. FIG. 6 (a) is resolution conversion with the frame structure unchanged, and FIG. This is a resolution conversion with a single-level structure.

FIGS. 7A to 7E are schematic diagrams showing the area synthesis according to the first embodiment of the present invention.

FIG. 8 is a block diagram showing the overall configuration of the image processing device according to the second embodiment of the present invention.

FIG. 9 is a block diagram showing the configuration of the resolution converter in FIG.

FIG. 10 is a schematic diagram for explaining the operation of the motion vector detection unit and the quasi-static region resolution conversion unit in FIG.

FIG. 11 is a schematic diagram for explaining the operation of the moving area resolution conversion unit in FIG.

FIG. 12 is a block diagram showing another configuration example of the resolution converter in FIG. FIG. 13 is a block diagram showing the overall configuration of the image processing apparatus according to the third embodiment of the present invention.

Figures 14 (a) to 14 (c) explain the operation of the coding parameter converter in Figure 13 FIG.

FIG. 15 is a block diagram showing the configuration of the video encoder in FIG.

FIG. 16 is a block diagram showing an overall configuration of an image processing apparatus according to the fourth embodiment of the present invention.

FIG. 17 is a block diagram showing the configuration of the video encoder in FIG.

FIGS. 18 (a :) to (c) are schematic diagrams for explaining the operation of the motion vector calculator in FIG.

FIG. 19 is a block diagram showing the overall configuration of the image processing apparatus according to the fifth embodiment of the present invention.

FIGS. 20 (a) and (b) are diagrams showing an example of resolution conversion according to the fifth embodiment of the present invention.

FIG. 21 is a block diagram showing the configuration of the video encoder in FIG.

FIG. 22 is a block diagram showing a configuration of a conventional image processing apparatus.

FIGS. 23 (a) and 23 (b) are schematic diagrams showing resolution conversion in a field structure. FIGS. 24A to 24C are diagrams showing two types of resolution conversion. BEST MODE FOR CARRYING OUT THE INVENTION

(First Embodiment)

FIG. 1 is a block diagram showing the overall configuration of the image processing apparatus according to the first embodiment of the present invention. As shown in FIG. 1, the image processing apparatus according to the present embodiment includes a video decoder 10 and a resolution converter 20. A code sequence CS of an original video signal having a high resolution as a first resolution is input, and the video decoder 10 decodes the input code sequence CS into an original video signal SHR. At the same time as decoding, the coding parameter PAR of the code string CS is extracted. The resolution converter 20 converts the decoded original video signal SHR into a new video signal SLR having a low resolution as the second resolution by using the encoding parameter PAR and outputs it. Here, it is assumed that the code string CS is encoded by the MPEG2 method. Therefore, the video decoder 10 is a decoder of the Moving Picture Expert Group 2 (MPEG2) system. Here, it is assumed that both the original video signal S HR and the new video signal S LR are interlaced signals (interlaced scanning signals). FIG. 2 is a diagram showing the internal configuration of the video decoder 10. As shown in FIG. 2, the video decoder 10 includes a variable length decoding unit 11, an inverse quantization unit 12, an inverse DCT (Discrete Cosine Transform) unit 13, a frame memory 14, and a system control unit. 15, adder 16, and switch 17. The variable length decoding unit 11 also extracts the encoding parameter PAR.

The operation of the video decoder 10 shown in FIG. 2 will be described. The code sequence CS for one frame is coded in units of (16 × 16) pixel macroblocks, and is input to the video decoder 10 in macroblock order.

The input code string CS is subjected to variable-length decoding by the variable-length decoding unit 11. Among the outputs of the variable length decoding unit 11, those related to image data are input to the inverse quantization unit 12, and data other than image data is sent to the system control unit 15. The data sent from the variable length decoding unit 11 to the system control unit 15 includes a picture type, a DCT type, and a motion vector, which are coding parameters of the code string CS. Here, the “victure type” indicates whether the frame is intra-frame encoded or inter-frame encoded, and the “DCT type” indicates whether the block has a field structure or a frame structure. This indicates whether DCT processing has been performed using the structure shown in FIG.

The data input to the inverse quantization unit 12 is subjected to inverse quantization and input to the inverse DCT unit 13. In the inverse DCT section 13, an inverse DCT operation is performed. Here, in the MPEG2 system, DCT processing is performed in units of (8 × 8) pixel blocks. When intra-frame coding is performed, switch 1 符号 is connected to a, while when inter-frame coding is performed, switch 17 is connected to b. Now Assuming that the code sequence CS of the frame subjected to the interframe coding is being processed, the output of the inverse DCT unit 12 is sent to the adder 16 because the switch 17 is connected to b. Is output.

A reference image is extracted from the frame memory 14 using the motion vector sent from the variable-length decoding unit 11 to the system control unit 15. The frame serving as the reference image has already been decoded and is stored in the frame memory 14. The data subjected to the inverse DCT operation, which is the output of the inverse DCT unit 13, and the reference image, which is the data read from the frame memory 14, are added by the adder 16 to obtain a decoded image, and Stored in 14 In this way, the macroblocks are sequentially decoded and stored in the frame memory 14.

Decoding is similarly performed for the subsequent frames, and is stored in the frame memory 14. However, when intra-frame encoding is performed, switch 17 is connected to a, and the output of inverse DCT section 13 is stored in frame memory 14 as it is. When inter-frame coding is performed, switch 17 is connected to b, the output of inverse DCT section 13 and the reference image are added by adder 16 and stored in frame memory 14. You.

The decoded images stored in the frame memory 14 in this manner are output from the frame memory 14 in time order as the original video signal SHR. Further, the system controller 15 outputs an encoding parameter P AR.

FIG. 3 is a diagram showing the internal configuration of the resolution converter 20. As shown in FIG. 3, the resolution converter 20 includes an area dividing section 21, a still area resolution converting section 23, a moving area resolution converting section 24, and an area synthesizing section 25. The area dividing section 21 has an image dividing section 21a and a motion determining section 21b.

The operation of the resolution converter 20 shown in FIG. 3 will be described. First, the motion determining unit 21b determines the motion characteristics of the image in the original video signal SHR using the input encoding parameter PAR. Then, a determination is made between the stationary region and the moving region. FIG. 4 is a diagram illustrating a motion determination in the motion determination unit 21b. In the same figure, (a) is an example of motion judgment when a motion vector is used as a coding parameter overnight PAR, and (b) is a motion judgment when a DCT evening is used as a coding parameter PAR. It is a figure showing an example.

First, the motion determination when a motion vector is used as the encoding parameter PAR will be described using FIG. 4 (a). In FIG. 4A, the frame screen is divided into macroblock units as video constituent units of (16 × 16) pixels. In the MPEG2 method, the motion vector represents the horizontal and vertical displacement of a macroblock in 0.5 pixel units. Here, it is assumed that the motion vector is compared with a predetermined value, and the motion is determined based on the comparison result. For example, when the absolute value of the motion vector is smaller than a predetermined value, it is determined that the macro block belongs to the stationary region, and when it is larger than the predetermined value, it is determined that the macro block belongs to the moving region. I do. In Fig. 4 (a), the macro blocks painted in black belong to the moving area, and the macro blocks shown in white belong to the stationary area. Next, with reference to FIG. 4 (b), a description will be given of the motion determination in a case where the DCT parameter is used as the encoding parameter PAR. In Fig. 4 (b), the frame screen is divided into (8 x 8) pixel blocks. In the MPEG2 system, each block is subjected to a DCT operation by a frame structure or a field structure, which has a smaller difference sum with the adjacent pixel in the vertical direction. The DCT type refers to a structure in which a DCT operation has been performed. Here, when the DCT type has a frame structure, it is determined that the block belongs to a still area, and when the DCT type has a field structure, it is determined that the block belongs to a moving area. In Fig. 4 (b), the blocks painted in black belong to the moving area, and the blocks shown in white belong to the stationary area.

The motion determining unit 21b performs the above-described motion determination, and sends the result to the image dividing unit 21a. The image dividing unit 21a is based on the motion determination result obtained from the motion determining unit 21b. To divide the frame image into a still area and a moving area.

FIG. 5 is a schematic diagram showing an example of area division. In the figure, (a) is the motion determination result shown in FIG. 4 (a), (b) is the still region extracted by the region dividing unit 21 and (c) is the moving region extracted by the region dividing unit 21. It is.

In the original video signal SHR, the video data divided into the still area is input to the still area resolution converter 23, and the video data divided into the moving area is input to the moving area resolution converter 24. In still region resolution converter 2 3 and the moving region resolution converter 2 4, thereby it by a method corresponding to the input video data, _c 6 the resolution conversion is performed still region resolution converter 2 3 and dynamic regions FIG. 9 is a diagram showing an example of resolution conversion from an original video signal SHR to a new video signal SLR performed in a resolution conversion section 24. FIG. 6 shows the state of pixels arranged in the vertical direction, where “〇” is a pixel belonging to the first field, and “△” is a pixel belonging to the second field. Fig. 6 shows the case where the vertical pixels are converted to 1/2. In Fig. 6, (a) is the resolution conversion with the frame structure, and (b) is the resolution conversion with the field structure. It is. The still area resolution conversion section 23 performs resolution conversion while maintaining the frame structure. That is, as shown in Fig. 6 (a), the video data of the first field of the new video signal SLR is generated using both the pixels of the first and second fields of the original video signal SHR, and The video data of the second field of the new video signal SLR is also generated using the pixels of both the first field and the second field of the original video signal SHR. On the other hand, the moving area resolution conversion section 24 performs resolution conversion without changing the field structure. That is, as shown in FIG. 6 (b), the video data of the first field of the new video signal SLR is generated only from the pixels of the first field of the original video signal SHR, and the second field of the new video signal SLR is generated. The field video data is generated using only the pixels in the second field of the original video signal SHR.

The new video signals of the still area and the moving area generated by the still area resolution converting section 23 and the moving area resolution converting section 24 are output to the area synthesizing section 25. Area synthesis The unit 25 combines the new video signals in the still area and the moving area, converts them into frame images, and outputs them as new video signals SLR.

FIG. 7 is a diagram showing the operation of the area combining section 25. In the figure, (a) and (b) are input images to the still area resolution converter 23 and the moving area resolution converter 24, respectively, and correspond to the images in FIGS. 5 (b) and (c). I do. FIGS. 7 (c) and 7 (d) show the output images of the still area resolution converter 23 and the moving area resolution converter 24, respectively, and the number of pixels is reduced as a result of the resolution conversion. The area combining section 25 generates a combined image, that is, a new video signal SLR as shown in FIG. 7 (e) from the images of FIGS. 7 (c) and 7 (d).

As described above, according to the present embodiment, the code string of the original video signal is decoded, and the coding parameter of the code string is extracted. Then, the characteristics of the original video signal are determined from the encoding parameters, and the original video signal is converted into a new video signal by a resolution conversion method according to the characteristics. Thereby, processing such as motion vector calculation for resolution conversion can be significantly reduced. Also, since the still and moving areas of the image of the original video signal are discriminated from the encoding parameters and the resolution is converted by a different method between the still and moving areas, the processing amount is extremely small. A new video signal with high image quality can be obtained.

In this embodiment, in order to determine the motion characteristics of an image, the motion vector and the DCT parameter are used as the coding parameter parameter PAR, but other coding parameter parameters may be used. It is possible. Another encoding parameter is, for example, a motion compensation mode. The motion compensation mode indicates whether the motion compensation is performed in the field structure or the frame structure. When the motion compensation mode is used as the encoding parameter PAR of the present embodiment, for example, for a region where the motion compensation mode has the field structure, the resolution conversion is performed in the field structure, and when the motion compensation mode is For a region having a frame structure, resolution conversion may be performed using the frame structure. Further, in the present embodiment, resolution conversion is performed with a frame structure in a still region and resolution conversion is performed with a field structure in a moving region. However, another resolution conversion method may be used.

Also, in the present embodiment, when performing the resolution conversion in the frame structure or the field structure, as shown in FIGS. Although the case where the pixel of the signal SLR is generated has been described, other pixels may be used.

In the present embodiment, the case where the resolution is set to 1/2 is described, but this may be another value.

Further, in the present embodiment, a case has been described in which a high-resolution original video signal is converted to a low-resolution new video signal. Conversely, a low-resolution original video signal is converted to a high-resolution new video signal. Even in the case of converting to a signal, it is possible to use the encoding parameters as in the present embodiment.

(Second embodiment)

FIG. 8 is a block diagram showing the overall structure of the image processing device according to the second embodiment of the present invention. As shown in FIG. 8, the image processing device according to the present embodiment includes a video decoder 110 and a resolution converter 120. The code sequence CS of the original video signal having the low resolution as the first resolution is input, and the video decoder 110 decodes the input code sequence CS into the original video signal SLR. At the same time as decoding, the motion vector MV is extracted from the code sequence C S. The resolution converter 120 converts the decoded original video signal SLR into a new video signal SHR having a high resolution as the second resolution using the motion vector MV, and outputs the new video signal SHR.

Here, the original video signal SLR is an interlace signal (interlaced scanning signal), and the new video signal SHR is a progressive signal (sequential scanning signal). The video decoder 110 is basically the same as the video decoder 110 according to the first embodiment. Since it has the same configuration and differs only in that it outputs the motion vector MV, a detailed description is omitted here.

FIG. 9 is a block diagram showing the internal configuration of the resolution converter 120. As shown in FIG. 9, the resolution converter 120 is composed of an area dividing section 121, a quasi-static area resolution converting section 123, a moving area resolution converting section 124, an area synthesizing section 125, and a frame memory. 1 2 7 is provided. Further, the region dividing unit 121 has a motion vector detecting unit 121 a, a motion determining unit 121 b, and an image dividing unit 121 c.

The frame memory 127 stores the input original video signal SLR. The motion vector detection unit 12 21 a includes a motion vector MV output from the video decoder 110, a current field of the original video signal SLR, and a past field stored in the frame memory 127. Using this field, a motion vector MV in pixel units having a direction similar to that of the motion vector MV between the current field and a temporally neighboring field is detected. The operation of the motion vector detection unit 121a will be described with reference to FIG. FIG. 10 is a diagram schematically showing pixels of three frames n, n + 1, and n + 2 of the original video signal SLR. In FIG. 10, “〇” denotes a pixel belonging to the first field of each frame, and “X” denotes a pixel belonging to the second field of each frame.

Now, suppose that the first field of frame n + 2 is to be converted into a progressive image having twice the resolution. Then, a case where the pixel D (“△”) shown in FIG. 10 is generated will be described.

In this case, among the motion vectors M V output from the video decoder 110, the motion vector A of the area G including the pixel D to be generated is used. Now, the motion vector A is a motion vector starting from the area F of the first field of the frame n. Then, for the pixel D, the motion vector E having the same direction as the motion vector A is obtained from the motion vectors from the neighboring field, that is, the second field of the frame n + 1.

Then, the pixel near the pixel D in the first field of the frame n + 2 and the pixel Using the pixels in the second field of frame n + 1, among the motion vectors whose directions are similar to the motion vector E, the correct motion vector B of the pixel near the pixel D is calculated. Ask. That is, pixel D can be considered to be pixel C in the second field of frame n + 1, which has been moved by motion vector B.

The motion vector detection unit 121 a outputs the motion vector B obtained in this way to the motion determination unit 122 b as a motion vector M V in pixel units.

The motion determining unit 122b uses the motion vector MVa input from the motion vector detecting unit 122a to determine the quasi-stationary region and the moving region. In this case, a pixel for which no motion vector MV is found in the pixel unit in the motion vector detection unit 122a is determined to belong to the motion region, and the motion vector MV line in the pixel unit is found. The determined pixel belongs to the quasi-static region. The motion judging unit 121b performs the motion judgment in this way, and sends the result to the image dividing unit 122c.

Here, the term “quasi-static region” refers to a region that has motion as an image, but can be regarded as a still image by shifting the image by the amount of motion using the pixel-based motion vector MV. That means.

The image dividing unit 122c divides the frame image into a quasi-stationary region and a moving region based on the result of the motion determination obtained from the motion determining unit 122b. The video data segmented into the quasi-stationary region is input to the quasi-stationary region resolution converter 123, and the video data segmented into the moving region is input to the moving region resolution converter 124. The quasi-stationary area resolution conversion section 123 and the moving area resolution conversion section 124 perform resolution conversion on the input video data by the respective methods.

The resolution conversion in the quasi-static region resolution conversion unit 123 will be described with reference to FIG. The quasi-stationary area resolution conversion unit 123 converts the original video signal SLR, the moving region image output from the region division unit 121, and the pixel unit output from the motion vector detection unit 122a. The resolution conversion is performed using the motion vector MV.

For example, when performing resolution conversion for the first field of frame n + 2, Since the motion vector B is detected as the motion vector MV in pixel units near the element D, the pixel C of the second field of the frame n + 1 is obtained from the frame memory 127. The pixel value is used as the pixel value of the pixel D. Similarly, pixels at other interpolation positions are also generated by using the motion vector MV per pixel detected by the motion vector detection unit 121a, thereby obtaining a high-resolution video signal. Generate

The resolution conversion in the moving area resolution conversion unit 124 will be described with reference to FIG. In FIG. 11, “〇” indicates a pixel in a low-resolution video signal, and “X” indicates a pixel generated by interpolation in a high-resolution video signal. As shown in FIG. 11, the moving area resolution conversion unit 124 generates a high-resolution video signal from pixels in a field of the low-resolution video signal by interpolation. For example, the pixel value of pixel C in the high-resolution video signal is the same value as pixel A in the low-resolution video signal, and the pixel value of pixel D in the high-resolution video signal is calculated from pixel A and pixel B in the low-resolution video signal. Generated.

The video signal generated by the quasi-static region resolution conversion unit 123 and the moving region resolution conversion unit 124 is input to the region synthesis unit 125. The area synthesizing unit 125 synthesizes the input video signals and outputs them as a new video signal SHR.

In the present embodiment, the motion vector detection unit 12 recalculates the motion vector MV for each pixel using the motion vector MV obtained from the code sequence CS of the original video signal SLR. Although 1a is provided, the segmentation may be performed using the obtained value of the motion vector MV as it is.

FIG. 12 is a diagram showing the configuration of the resolution converter 120 A in such a case. In FIG. 12, the area dividing unit 122 includes a motion determining unit 122 a that directly performs motion determination from the motion vector MV, and an image dividing unit 122 b. In this case, for example, when the absolute value of the motion vector MV is smaller than a predetermined threshold, it is determined to be a quasi-static region, and when it is larger than the predetermined threshold, it is determined to be a moving region. What is necessary is just to perform area division. Also, only when the motion vector MV is extremely large, it may be determined to be a motion region. Furthermore, all regions may be regarded as quasi-stationary regions, and resolution conversion may be performed using the value of the motion vector MV as it is.

As described above, according to the present embodiment, a code sequence of an original video signal is decoded, and a motion vector is extracted from the code sequence. Then, a new video signal is generated by resolution conversion using the extracted motion vector. As a result, processing such as motion vector calculation for resolution conversion can be significantly reduced. Also, the quasi-static region and the moving region of the image of the original video signal are determined from the motion vector, and the resolution conversion using the motion vector is performed in the quasi-static region. Therefore, the amount of processing such as motion vector calculation can be greatly reduced. Furthermore, since resolution conversion is performed by a different method between the quasi-static region and the moving region, a new video signal of high image quality can be obtained with a very small amount of processing.

In the present embodiment, a case has been described in which a low-resolution original video signal is converted to a new high-resolution video signal. Conversely, a high-resolution original video signal is converted to a low-resolution new video signal. Even in the case of converting to a signal, it is possible to use a motion vector as in the present embodiment.

Further, in the present embodiment, the motion vector unit 121 a calculates the motion vector with the past field, and the quasi-static region resolution conversion unit 123 interpolates from the pixels of the past field. The pixels are generated, but this may be such that a motion vector with a future field is obtained, and an interpolation pixel is generated from a pixel in a future field.

Further, in the present embodiment, the moving area resolution conversion unit 124 generates the interpolated pixels and performs the resolution conversion as shown in FIG. 11, but other resolution conversion methods may be used. Absent.

In the present embodiment, the quasi-static region resolution conversion unit 123 generates the interpolation pixel from the pixel of the past nearest field, but the interpolation is performed from the pixel of the field a few fields away. Pixels may be generated. (Third embodiment)

FIG. 13 is a block diagram showing the overall configuration of the image processing apparatus according to the third embodiment of the present invention. As shown in FIG. 13, the image processing device according to the present embodiment includes a video decoder 210, a resolution converter 220, a video encoder 230, and an encoding parameter converter 240. I have.

A first code sequence CSA, which is a code sequence of an original video signal having a high resolution as the first resolution, is input, and the video decoder 210 converts the input first code sequence CSA into the original video. Decode into signal SHR. At the same time as decoding, the coding parameter PARA of the first code string CSA is extracted. The resolution converter 220 converts the original video signal SHR into a new video signal SLR. The encoding parameter converter 240 converts the first encoding parameter PARA output from the video decoder 210 for low-resolution video, and outputs it as a second encoding parameter PARB. The video encoder 230 encodes the new video signal SLR using the second encoding parameter PAR B, and outputs it as a second code string CSB.

Here, it is assumed that the first code string CSA is coded by the MPEG2 method. Therefore, the video decoder 210 is a MPEG2 decoder. The configuration and operation of the video decoder 210 are the same as those of the video decoder 10 according to the first embodiment, and a detailed description thereof will be omitted.

The resolution converter 220 converts the original video signal SHR decoded by the video decoder 210 into a new video signal SLR. Here, it is assumed that the number of pixels of the resolution converter 220 is 1/2 in both the vertical and horizontal directions.

Next, the operation of the coding parameter converter 240 will be described with reference to FIG. 14.The _c coding parameter converter 240 converts the first coding parameter PAR A for the original video signal SHR into a new video. now _c for converting the second encoding parameter PARB for signal S LR, as shown in FIG. 1 4 (a), frame image of the original image signal SHR is vertical way It is converted to a frame image of the new video signal SLR with half the number of pixels in both the horizontal and horizontal directions.

First, the case where the coding parameter to be transformed is a motion vector will be described with reference to FIG. 14 (b). In FIG. 14 (b), the frame image is shown divided into macroblock units. As shown in Fig. 14 (b), the areas of macroblocks MBH1, MBH2, MBH3, and MBH4 as the first area in the original video signal SHR are converted into new areas by resolution conversion. The area is reduced to the area of the macro block MBL as the second area in the video signal SLR. Here, it is assumed that the motion vector MVL of the macroblock MBL in the new video signal SLR is obtained by a weighted average of the motion vector MVHi of each macroblock MBHi in the original video signal SHR. That is, the motion vector MVL can be obtained by equation (1).

N

M V L = {∑ (C i xMVH i)} x-(1)

i = l

Here, C i is a weight coefficient satisfying the following equation (2).

N

∑ C i = 1-(2)

i = l

N is the number of macroblocks of the original video signal SHR for obtaining the motion vector MVL of the macroblock MBL in the new video signal SLR, and is “4” here. Also, "hi" is a ratio of the number of pixels of the screen size between the original video signal and the new video signal, and generally has different values in the horizontal direction and the vertical direction.

Here, when the ratio of the number of pixels before and after the resolution conversion is 1/2, 1/4, the boundary of the macroblock after the conversion matches the boundary of the macroblock before the conversion, so that the weight coefficient C i May be equal. However, the ratio of the number of pixels is 1/3, 2/5 In such a case, since the boundaries of the converted macroblocks do not match the boundaries of the macroblock before the conversion, it is necessary to change the weights of the macroblocks before the conversion depending on the value of the weight coefficient C i.

Next, a case where the encoding parameter to be transformed is a DCT type as an orthogonal transformation type will be described with reference to FIG. 14 (c). In FIG. 14 (c), the frame image is shown divided into blocks. As shown in FIG. 14 (c), the areas of the blocks BH1, BH2, BH3, and BH4 as the first areas in the original video signal are converted into the second areas in the new video signal by the resolution conversion. Is reduced to the area of the block BL. Here, the DCT type of the block BL in the new video signal is obtained using the DCT type of the blocks BH1 to BH4 in the original video signal. For example, if all DCTs of blocks BH1 to BH4 have a frame structure, the DCT type of block BL has a frame structure, and at least one of the DCT types of blocks BHI to BH4 has a frame structure. If the field structure is used, the DCT type of block BL is used as the field structure. Alternatively, the larger one of the DCTs of the original block may be set as the DCT type of the converted block.

As described above, the encoding parameter converter 240 converts the first encoding parameter PAR A for the original video signal S HR into the second encoding parameter PAR A for the new video signal SLR. .

FIG. 15 is a block diagram showing the internal configuration of the video encoder 230. The video encoder 230 in FIG. 15 has basically the same configuration as a normal MPEG encoder, but because a motion vector is given as the second encoding parameter PARB. However, since the motion detection unit is omitted and the DCT parameter is given as the second encoding parameter PARB, the means for determining the DCT type by dispersion calculation or the like is omitted from the DCT calculation unit 23. Has been.

Here, the video encoder 230 transmits the new video signal SLR in the MPEG2 format. Encoding. It is assumed that inter-frame coding is performed.

First, the new video signal SLR input to the video encoder 230 is divided into macroblocks of (16 x 16) pixels by the blocking unit 231 and the motion compensation unit 231 is arranged in the order of the macroblocks. Entered in 2. The motion compensation unit 2 32 uses the motion vector included in the second encoded parameter obtained from the encoded parameter overnight converter 240 for the input macroblock, Perform motion compensation. That is, the motion compensating unit 232 performs motion compensation by reading a reference macroblock from the frame memory 238 using a motion vector and obtaining a difference between the reference macroblock and the input macroblock. The obtained difference macroblock is input to the DCT calculation unit 233.

The 0-th arithmetic unit 2 33 converts the difference macroblock into DCT coefficients for each (8 × 8) picture. At this time, a DCT operation is performed in accordance with the DCT parameter included in the second encoding parameter PARB obtained from the encoding parameter converter 240. The obtained DCT coefficient is output to the quantization section 234, and the quantization section 234 performs a quantization process on the DCT coefficient. The variable-length coding unit 235 performs variable-length coding on the output of the quantization unit 234, and outputs the result as a second code string CSB. The output of the quantization unit 234 is decoded by the inverse quantization unit 236 and the inverse DCT operation unit 237, and the reference macro block read from the frame memory 238 and the adder 239 are output. And is stored in the frame memory 238. The stored data is used as a reference image in subsequent frame encoding.

As described above, according to the present embodiment, the first code string of the original video signal is decoded, and the decoded original video signal is converted into a new video signal by resolution conversion. In addition, together with the decoding, the first encoding parameters such as the motion vector and the DCT type are extracted from the first code string. Then, the first encoding parameter is converted into a second encoding parameter for the new video signal, and the new video signal is converted using the second encoding parameter. Into a second code string. This eliminates the need to determine the encoding parameters when encoding a new video signal, and can greatly reduce the processing amount. In particular, when a motion vector is used as a coding parameter, the amount of reduction in the amount of processing becomes very large.

In this embodiment, the motion vector and the DCT type have been described as the encoding parameters for performing the transform. However, the same applies to other encoding parameters such as the motion compensation mode. It is feasible.

Also, the method of encoding parameter conversion according to the present embodiment is an example, and the conversion may be performed using another method. In the present embodiment, the coding parameters of a predetermined area (macroblock or block) of the new video signal are determined using only the coding parameters of the area of the original video signal corresponding to the predetermined area. For example, coding parameters may be used not only for the original video signal area corresponding to the predetermined area but also for the surrounding area.

Further, in the present embodiment, the motion vector conversion is performed using a weighted average as a predetermined operation. However, as another method, a method of selecting the median of the motion vector component, A method of ignoring the motion vector or a method of weighting based on the prediction error can be considered.

Further, in the present embodiment, when the boundaries of the macroblocks before and after the conversion match, the weighting coefficient C i is described as being equal, but this is not necessarily the case.

Further, in the present embodiment, a case has been described in which a high-resolution original video signal is converted to a low-resolution new video signal. Conversely, a low-resolution original video signal is converted to a high-resolution new video signal. Even in the case of converting to a signal, it is possible to perform conversion of the entire encoding parameter as in the present embodiment.

(Fourth embodiment) FIG. 16 is a block diagram showing an overall configuration of an image processing apparatus according to the fourth embodiment of the present invention. As shown in FIG. 16, the image processing apparatus according to the present embodiment includes a video decoder 310, a resolution converter 320, a video encoder 330, and a motion compensation setting unit 340.

A first code sequence CSA, which is a code sequence of an original video signal having a high resolution as the first resolution, is input, and the video decoder 310 outputs the input first code sequence CSA to the original video signal. Decrypt to SHR. At the same time as decoding, the first motion vector MV is extracted from the first code string CSA. The resolution converter 320 converts the original video signal S HR into a new video signal S LR having a low resolution as the second resolution. Also, the motion compensation setting unit 340 sets the motion compensation operation in the video encoder 330 using the motion vector MV of the code string C S A output from the video decoder 310. The video encoder 330 obtains a second motion vector using the setting information SET output from the motion compensation setting unit 340, encodes the new video signal SLR using the second motion vector, Is output as a code string CSB.

Since the configurations and operations of the video decoder 310 and the resolution converter 320 are the same as those of the video decoder 110 and the resolution converter 320 according to the second embodiment, a detailed description thereof will be given here. Is omitted.

The operation of the motion compensation setting device 340 will be described. The motion compensation setting unit 340 receives the motion vector MV of the original video signal SHR obtained by the video decoder 310 as input, and obtains motion compensation setting information SET in the video encoder 330. As the setting information SET, for example, the initial value of the motion compensation, that is, the initial value of the second motion vector, the range of the motion compensation, that is, the search range of the second motion vector, and the like are determined.

First, the case where the initial value of the motion compensation is obtained as the motion compensation setting information SET will be described with reference to FIG. 14 (b). In Fig. 14 (b), the frame image is shown divided into macroblock units. As shown in Fig. 14 (b), the original video signal The area of the macro blocks MB H 1, MB H 2, MB H 3, and MBH 4 in the signal is reduced to the area of the macro block MB L in the new video signal by resolution conversion. In this case, the motion compensation setting unit 340 obtains, for example, a motion vector MVL obtained from the above equation (1) as an initial value of motion compensation when the video encoder 310 encodes the macro block MBL. . The motion vector MVL is output as motion compensation setting information SET.

Next, the case where the range of motion compensation is determined as the motion compensation setting information S ET will be described with reference to FIG. 14 (b). In this case, the motion compensator 340 uses the motion vectors of the macroblocks MBH1 to MBH4 to determine the range of motion compensation when the video encoder 330 calculates the motion vector for the macroblock MBL. You. As a range of the motion compensation, for example, there is a method of using a value in which the absolute value of each component of each motion vector of the macroblocks MVH1 to MVH4 is maximum. Alternatively, there is a method using an average value and a variance value of the motion vector values of the macroblocks MVH1 to MVH4.

As described above, the motion compensation setting unit 340 uses the motion vector MV of the original video signal SHR to set the initial value and range for calculating the motion vector for the new video signal as the motion compensation setting information SET. Determined and output to video encoder 330.

FIG. 17 is a block diagram showing the internal configuration of the video encoder 33. Here, it is assumed that the video encoder 330 encodes the new video signal SLR according to the MPEG-2 system. It is assumed that inter-frame coding is performed.

First, the new video signal S LR input to the video encoder 330 is divided into macroblocks of (16 × 16) pixels by the blocking unit 331, and the motion compensation unit 332 and the motion It is input to the vector calculator 350.

The motion vector calculator 350 calculates the second motion vector MV2 for the input macroblock based on the motion compensation setting information SET input from the motion compensation setting device 340. At this time, the reference image is retrieved from the frame memory 338. Embed.

The operation of the motion vector calculator 350 will be described with reference to FIG. In the figure, (a) shows a frame to be encoded of the new video signal SLR input from the resolution converter 320, and (b) and (c) show reference frames read from the frame memory 338. I have. Now, it is assumed that the motion vector for the macro block MB 1 shown in FIG. 18 (a) is obtained.

First, the operation when the initial value of the motion compensation is input as the motion compensation setting information S E T will be described with reference to FIG. 18 (b). In this case, the motion vector calculator 350 uses the initial value S ET obtained by the motion compensation setting device 340 as the initial value of the second motion vector MV 2. Then, the vicinity of the end point of the motion vector SET when the macroblock MB1 located at the same position as the macroblock MB1 is set as the start point is set as the search range S R1 of the second motion vector MV2. In this search range SR1, calculation of the second motion vector MV2 for the macroblock MB1 is performed.

The operation when the range of motion compensation is input as the motion compensation setting information S E T will be described with reference to FIG. 18 (c). In this case, the motion vector calculator 350 uses the range S ET obtained by the motion compensation setting device 340 as the search range S R 2 for obtaining the second motion vector MV 2. Then, in the search range SR 2 (that is, the range SET), the second motion vector MV 2 for the macro block MB 1 is calculated.

The motion compensator 332 performs motion compensation on the input macroblock using the second motion vector MV2 obtained from the motion vector calculator 350. That is, the motion compensation unit 332 reads the reference macroblock from the frame memory 338 using the second motion vector MV2, and calculates the difference between the reference macroblock and the input macroblock, thereby performing motion compensation. Do. The obtained difference macro block is input to the DCT operation unit 333.

DCT calculation section 333, quantization section 334, variable length coding section 335, inverse quantization section 3 36, the operations of the inverse DCT operation unit 337 and the adder 339 are the same as the operations according to the second embodiment, and thus the description is omitted here. The second code string CSB is output from the variable length coding unit 335.

As described above, according to the present embodiment, the first code string of the original video signal is decoded, and the decoded original video signal is converted into a new video signal by resolution conversion. In addition to the decoding, the first motion vector used for encoding the original video signal is extracted. Then, setting information such as an initial value and a range of motion compensation for determining a second motion vector to be used for encoding a new video signal is determined from the first motion vector, and this setting information is determined. The second motion vector is detected using low resolution, and the new video signal is encoded. As a result, the search range is narrower than in the past, and the amount of motion vector calculation when encoding a new video signal is greatly reduced. Further, since the motion compensation setting information is obtained from the first motion vector used for encoding the original video signal, the accuracy of the motion compensation is kept high.

Note that the method for determining the motion compensation setting information according to the present embodiment is an example, and another method may be used. For example, not only the macroblock of the original video signal corresponding to the predetermined macroblock of the new video signal, but also the motion vector of the surrounding macroblock may be used. In the present embodiment, the initial value of the second motion vector is determined using the weighted average. However, as another method, a method of selecting the median value of the motion vector component, A method of ignoring the motion vector and a method of weighting based on the prediction error can be considered.

The motion compensation setting unit 340 may determine both the initial value and the search range of the second motion vector as the motion compensation setting information S ET.

Further, in the present embodiment, a case has been described in which a high-resolution original video signal is converted to a low-resolution new video signal. Conversely, a low-resolution original video signal is converted to a high-resolution new video signal. Even in the case of conversion into a signal, it is possible to determine the motion compensation setting information SET, as in the present invention. (Fifth embodiment)

FIG. 19 is a professional / soc diagram showing the entire configuration of an image processing apparatus according to the fifth embodiment of the present invention. As shown in FIG. 19, the image coding apparatus according to the present embodiment includes a video decoder 410, a resolution converter 420, a video encoder 430, and a system control unit 440.

A code sequence C S A of a video signal having a high resolution as the first resolution is input, and the video decoder 410 decodes the input code sequence C S A into an original video signal SHR. The resolution converter 420 converts the decoded original video signal SHR into a new video signal SLR having a low resolution as the second resolution. The video encoder 430 encodes the new video signal SLR and outputs it as a second code string CSB.

Here, it is assumed that the first code string CSA is coded by the MPEG2 method. Therefore, the video decoder 410 is an MPEG2 decoder. Since the configuration and operation of the video decoder 410 are the same as those of the video decoder 10 according to the first embodiment, detailed description thereof is omitted here.

As shown in FIG. 20, the resolution converter 420 converts the original video signal SHR into a video signal of a one-box image. In the example shown in FIG. 20, the original video signal S HR having an aspect ratio of 16: 9 shown in (a) is converted into a new video signal S LR having an aspect ratio of 4: 3 shown in (b). . When converted to a single box image, band-shaped black level regions 451, 452 are added above and below the frame image 450 of the new video signal SLR.

FIG. 21 is a block diagram showing the internal configuration of the video encoder 430. As shown in FIG. 21, the video encoder 430 includes a region dividing unit 431, a blocking unit 432, a DCT calculating unit 433, a quantizing unit 434, a variable length coding unit 435, and a code string generating unit 43. 6 and a code string storage section 437. Here, it is assumed that the video encoder 430 encodes the new video signal SLR according to the MPEG-2 system. Also, —Intra-system encoding shall be performed.

In response to the instruction signal SI from the system control unit 44, the region dividing unit 431 receives the instruction signal SI from the system control unit 4400 and converts the new video signal SLR, which is a box image, into an effective data region,

The area 450 excluding the black level areas 451, 52 of 20 (b) is cut out and output. The video signals in the black level areas 451, 52 are not output. Blocking part 4

Reference numeral 32 denotes a block for dividing the input video signal of the area 450 into blocks, and a DCT operation unit 43 33 converts the block-divided video signal into DCT coefficients. The DCT coefficient output from the DCT operation unit 433 is quantized by the quantization unit 4334, and then converted into the first code string CSB1 by the variable length coding unit 435. It is output to the generator 436.

In the code string storage section 437, a second code string CSB2 in which the video signals of the black level areas 451, 452 are encoded is stored in advance. As the second code string CSB2, both the code subjected to intra-frame coding and the code subjected to inter-frame coding are stored in the code string storage unit 437. Here, the second code string CSB2 that has been subjected to intra-frame coding is output from the code string storage section 437.

The code sequence generation unit 436 converts the first code sequence CSB 1 output from the variable length coding unit 4 35 and the second code sequence CSB 2 output from the code sequence storage unit 437 into connect. Here, the concatenation of the code strings is performed sequentially from the head in the frame. That is, the code string of the black level area 451 output from the code string storage section 437, the code string of the area 450 output from the variable length coding section 435, and the code string storage section 4 3 Concatenation is performed in the order of the code string of the black level area 4 52 output from 7. The code string concatenated in this way is output as a code string CSB.

Note that the resolution converter 420 may output the new video signal SLR except for the black level regions 451 and 452. In this case, the region dividing unit 431 can be omitted from the video encoder 4300.

In addition, the area dividing unit 431 converts the input new video signal SLR into a box image. It may be determined whether or not the image is an image. In this case, there is no need to externally provide the video encoder 430 with the instruction signal SI.

As described above, according to the present embodiment, when encoding a new video signal having a letter-box structure obtained by resolution conversion from an original video signal, the black level region is not actually encoded. Concatenate the code strings stored in advance. As a result, it is not necessary to perform the encoding process for the black level region, and the processing amount can be greatly reduced. For example, when the frame of the new video signal has a size of 720 pixels horizontally and 480 pixels vertically, and the size of the region other than the black level region in the vertical direction is 360 pixels, the entire image is The amount of processing can be reduced by 25% compared to the case of encoding.

In the present embodiment, the black level area is added vertically, but may be added only to the upper side or only the lower side. Further, a black level area may be added to a region other than the top and bottom, for example, only the left and right, the left, or only the right. In the present embodiment, the case where the original video signal having the aspect ratio of 16: 9 is converted to the new video signal having the aspect ratio of 4: 3 has been described. However, these aspect ratios may be different combinations.

Further, in the present embodiment, a case has been described in which a high-resolution original video signal is converted to a low-resolution new video signal. Conversely, a low-resolution original video signal is converted to a high-resolution new video signal. Even in the case of conversion into a signal, encoding of the black level region can be omitted as in the present invention.

Further, in the present embodiment, the case where the video encoder 430 performs intra-frame encoding has been described, but the same applies to the case where inter-frame encoding is performed.

In each of the above embodiments, the case where MPEG 2 is used as the encoding method has been described. However, this may be another encoding method, for example, MPEG 1 or H.261. .

Claims

The scope of the word

1. a decoding step of decoding a code string in which the original video signal having the first resolution is encoded, and extracting a coding parameter of the code string;

A resolution conversion step of determining characteristics of an original video signal from the encoding parameters and converting the decoded original video signal into a new video signal having a second resolution by a resolution conversion method according to the characteristics. With

An image processing method comprising:

2. In the image processing method according to claim 1,

The resolution conversion step includes:

Determining a motion characteristic of an image in the original video signal as a characteristic of the original video signal;

An image processing method comprising:

3. In the image processing method according to claim 2,

The encoding parameter overnight,

A motion vector that indicates the amount of motion of the video constituent unit, an orthogonal transform type that indicates whether the orthogonal transform is performed using the frame structure or the field structure, and whether the motion compensation is a frame structure or the field structure Include at least one of the motion compensation modes that indicate

An image processing method comprising:

4. The image processing method according to claim 1,

The resolution conversion step includes:

The decoded image of the original video signal is divided into a still area and a moving area using the encoding parameters, The original video signal is converted into the new video signal using different resolution conversion methods in the still region and the moving region.

An image processing method comprising:

5. The image processing method according to claim 4,

The original video signal is an interlaced signal,

In the static area, resolution conversion is performed in units of frames, while in the above-described moving area, resolution conversion is performed in units of fields.

An image processing method comprising:

6. The image processing method according to claim 4,

The encoding parameter is a motion vector indicating a motion amount of a video constituent unit, and the region division is performed based on a comparison result between an absolute value of a motion vector and a predetermined value.

An image processing method comprising:

7. The code sequence of the original video signal is coded by an M PEG (Moving Picture Expert Group) method.

2. The image processing method according to claim 1, wherein:

8. The first resolution is higher than the second resolution

2. The image processing method according to claim 1, wherein:

9. A video decoder that decodes a code string in which an original video signal having a first resolution is encoded, and that extracts an encoding parameter of the code string,

Inputting the original video signal and coding parameters output from the video decoder; A resolution converter that determines characteristics of the original video signal from the encoding parameters, and converts the original video signal into a new video signal having a second resolution by a resolution conversion method according to the characteristics. Was

An image processing apparatus characterized by the above-mentioned.

10. The image processing apparatus according to claim 9,

The resolution converter,

As the characteristics of the original video signal, the motion characteristics of the image in the original video signal are determined.

An image processing apparatus characterized by the above-mentioned.

11. The image processing apparatus according to claim 10, wherein

The encoding parameter overnight,

A motion vector that indicates the amount of motion of the video constituent unit, an orthogonal transformation parameter that indicates whether the orthogonal transformation is performed in a frame structure or a field structure, and a motion compensation that has a frame structure or a field structure. This includes at least one of the motion compensation modes that indicate

An image processing apparatus characterized by the above-mentioned.

12. The image processing apparatus according to claim 9,

The resolution converter,

An area dividing unit that divides an image of the input original video signal into a still area and a moving area by using the encoding parameters;

A still area resolution conversion unit that converts the video signal of the still area output from the area division unit into a video signal of the second resolution;

The video signal of the moving area output from the area divider is A moving area resolution conversion unit for converting the image into a video signal.

An image processing apparatus characterized by the above-mentioned.

13. The image processing apparatus according to claim 12,

The original video signal is an in-night race signal,

The still area resolution conversion unit performs resolution conversion on a frame basis.

The moving area resolution conversion section performs resolution conversion in units of fields.

An image processing apparatus characterized by the above-mentioned.

14. The image processing apparatus according to claim 12,

The encoding parameter is a motion vector indicating a motion amount of a video constituent unit;

Region segmentation is performed based on the result of comparing the absolute value of the motion vector with a predetermined value.

An image processing apparatus characterized by the above-mentioned.

15. The code sequence of the original video signal is coded according to the MPEG method.

10. The image processing device according to claim 9, wherein:

1 6. The first resolution is higher than the second resolution

10. The image processing device according to claim 9, wherein:

1 7. Decode a code string obtained by encoding the original video signal having the first resolution. And a decoding step of extracting a motion vector from the code string, and a resolution of converting the decoded original video signal into a new video signal having a second resolution using the extracted motion vector. With a conversion step

An image processing method comprising:

18. The image processing method according to claim 17,

The resolution conversion step includes:

An area division step of dividing the decoded image of the original video signal into a quasi-stationary area and a moving area using the extracted motion vector;

The resolution conversion to the new video signal is performed using the extracted motion vector in the quasi-stationary region, but is performed without using the extracted motion vector in the moving region.

An image processing method comprising:

1 9. The image processing method according to claim 18,

The area dividing step includes:

From the extracted motion vector, a motion vector in pixel units having a direction similar to the motion vector is detected.

The region where the pixel-based motion vector is detected is a quasi-stationary region, and the region where the motion vector is not detected is a moving region.

An image processing method comprising:

20. The image processing method according to claim 19,

The resolution conversion to the new video signal in the quasi-stationary region is performed using the detected motion vector in pixel units.

An image processing method comprising:

21. The image processing method according to claim 18,

The area dividing step includes:

A region where the absolute value of the extracted motion vector is smaller than a predetermined threshold is a quasi-stationary region, while a region larger than the predetermined threshold is a moving region.

An image processing method comprising:

22. The code sequence of the original video signal is coded by the MPEG method.

18. The image processing method according to claim 17, wherein:

2 3. The first resolution is lower than the second resolution.

18. The image processing method according to claim 17, wherein:

24. A video decoder that decodes a code sequence obtained by encoding an original video signal having a first resolution, and extracts a motion vector from the code sequence; and A resolution converter that receives the output original video signal and the motion vector as inputs, and converts the original video signal into a new video signal having a second resolution using the motion vector.

An image processing apparatus characterized by the above-mentioned.

25. The image processing apparatus according to claim 24,

The resolution converter,

An area dividing unit that receives the original video signal and the motion vector, and divides the image of the original video signal into a quasi-static area and a moving area using the motion vector; A quasi-stationary region resolution conversion unit that converts the video signal of the quasi-static region output from the region division unit to a video signal having a second resolution using the motion vector; A video signal having a second resolution, without using the motion vector, for converting the obtained video signal of the video region into a video signal having a second resolution.

An image processing apparatus characterized by the above-mentioned.

26. The image processing apparatus according to claim 25,

The area dividing unit includes:

A motion vector detecting unit that detects a pixel-based motion vector having a similar direction to the motion vector from the motion vector;

An image processing apparatus wherein an area where the motion vector is detected by the motion vector detector is a quasi-stationary area, and an area where the motion vector is not detected is a motion area. .

27. The image processing apparatus according to claim 26,

The quasi-static region resolution conversion unit,

The resolution conversion is performed using the pixel-by-pixel motion vector detected by the motion vector detection unit.

An image processing apparatus characterized by the above-mentioned.

28. The image processing apparatus according to claim 25,

The area dividing unit includes:

An image in which an area in which the absolute value of a motion vector is smaller than a predetermined threshold is a quasi-stationary area, while an area larger than the predetermined threshold is a moving area. Processing equipment.

29. The code sequence of the original video signal is coded according to the M PEG system.

25. The image processing device according to claim 24, wherein:

30. The first resolution is lower than the second resolution.

25. The image processing device according to claim 24, wherein:

31. decoding a first code string obtained by coding an original video signal having a first resolution, and extracting a first coding parameter from the first code string; ,

Converting the decoded original video signal into a new video signal having a second resolution;

Converting the first encoding parameter into a second encoding parameter used for encoding the new video signal;

Encoding the new video signal using the second encoding parameter to generate a second code sequence.

An image processing method comprising:

3 2. The image processing method according to claim 31,

The encoding parameter conversion step includes:

A first encoding parameter used for encoding the first region of the image of the original video signal is replaced with a second encoding parameter of the image of the new video signal that includes the same video as the first region. Convert to the second encoding parameter for encoding the region

An image processing method comprising:

33. In the image processing method according to claim 32,

The first and second coding parameters are motion vectors

An image processing method comprising:

34. In the image processing method according to claim 33,

The encoding parameter conversion step comprises:

A value obtained by performing a predetermined operation on the motion vector of the first area is defined as the motion vector of the second area.

An image processing method comprising:

35. In the image processing method according to claim 33,

The encoding parameter conversion step comprises:

The weighted average value of the motion vector of the first area is defined as the motion vector of the second area.

An image processing method comprising:

36. In the image processing method according to claim 32,

The first and second encoding parameters are orthogonal transformation types indicating whether the orthogonal transformation is performed in a frame structure or a field structure.

An image processing method comprising:

3 7. The image processing method according to claim 31,

The image processing method according to claim 1, wherein the first code string is coded according to the MPEG method.

38. In the image processing method according to claim 31, The second code sequence is encoded by an MPEG method.

An image processing method comprising:

39. Video decoding that decodes a first code string obtained by encoding an original video signal having a first resolution and outputs a first encoded parameter from the first code string. And

A resolution converter for converting the original video signal output from the video decoder to a new video signal having a second resolution;

An encoding parameter converter that converts the first encoding parameter output from the video decoder into a second encoding parameter used for encoding the new video signal;

A video encoder that encodes a new video signal output from the resolution converter using a second encoding parameter output from the encoding parameter converter, and generates a second code sequence; With

An image processing apparatus characterized by the above-mentioned.

40. The image processing apparatus according to claim 39,

The coding parameter converter,

A first encoding parameter used for encoding the first region of the image of the original video signal is replaced with a second encoding parameter of the image of the new video signal that includes the same video as the first region. Transforms into a second encoding parameter to encode the region

An image processing apparatus characterized by the above-mentioned.

41. The image processing apparatus according to claim 40,

The first and second coding parameters are motion vectors

An image processing apparatus characterized by the above-mentioned.

42. The image processing apparatus according to claim 41, wherein

The coding parameter converter,

An image processing apparatus characterized by the above-mentioned.

43. The image processing apparatus according to claim 41, wherein

The coding parameter converter,

An image processing apparatus characterized by the above-mentioned.

44. The image processing apparatus according to claim 40,

The first and second encoding parameters are an orthogonal transform type indicating whether the orthogonal transform is performed in a frame structure or a field structure.

An image processing apparatus characterized by the above-mentioned.

45. In the image processing apparatus according to claim 39,

The image processing device according to claim 1, wherein the first code sequence is coded according to the MPEG method.

46. In the image processing apparatus according to claim 39,

The second code sequence is encoded according to the MPEG method.

An image processing apparatus characterized by the above-mentioned.

47. A first code string obtained by encoding an original video signal having a first resolution is decoded, and a first motion vector is extracted from the first code string and decoded. Setting for converting the original video signal into a new video signal having a second resolution, and obtaining a second motion vector used for encoding the new video signal from the first motion vector. Determine information,

Using the obtained setting information, the second motion vector is obtained,

The new video signal is encoded using the obtained second motion vector to generate a second code sequence.

An image processing method comprising:

48. In the image processing method according to claim 47,

Determine the initial value of the second motion vector as the setting information

An image processing method comprising:

49. In the image processing method according to claim 47,

An image processing method, wherein a search range for obtaining a second motion vector is determined as the setting information.

50. The image processing method according to claim 47, wherein

51. The image processing method according to claim 47, wherein

The second code sequence is encoded according to the MPEG method.

An image processing method comprising:

52. A video decoder that decodes a first code string obtained by encoding an original video signal having a first resolution, and extracts a first motion vector from the first code string. ,

A resolution converter for converting the original video signal output from the video decoder into a new video signal having a second resolution;

A motion compensation ax iaf and i i a which generate setting information for obtaining a second motion vector used for encoding the new video signal from the first motion vector output from the video decoder.

The second motion vector is obtained based on the setting information generated by the motion compensation setting device, the new video signal output from the resolution converter is obtained, and the obtained second motion vector is obtained. An image processing apparatus, comprising: a video encoder configured to generate a second code string by encoding using the same.

53. The image processing apparatus according to claim 52,

The motion compensation setting device,

An image processing apparatus according to claim 1, wherein the setting information determines an initial value of a second motion vector.

54. The image processing apparatus according to claim 52,

The motion compensation setting device,

As the setting information, a search range for obtaining a second motion vector is determined.

An image processing apparatus characterized by the above-mentioned.

55. The image processing apparatus according to claim 52,

The first code string is coded by the MPEG method. An image processing apparatus characterized by the above-mentioned.

56. The image processing apparatus according to claim 52,

The second code sequence is encoded according to the MPEG method.

An image processing apparatus characterized by the above-mentioned.

57. Convert the original video signal having the first resolution into a new video signal having the second resolution and having a black level region in a part of the image;

Encoding a video signal in an area excluding the black level area in the new video signal to generate a first code string;

A second code sequence obtained by encoding the video signal in the black level region is connected to the first code sequence to generate a code sequence of the new video signal.

An image processing method comprising:

58. The original video signal having the first resolution is converted to the video signal of the area excluding the black level area of the new video signal having the second resolution and having a black level area in a part of the image. Converted,

Encoding the video signal to generate a first code sequence;

An image processing method comprising:

5 9. The encoding method is the MPEG method

The image processing method according to claim 57 or 58, wherein:

60. An original video signal having a first resolution is converted into an image having a second resolution and A resolution converter for converting the image signal into a new video signal having a black level area in a part thereof, and encoding a video signal in an area of the new video signal excluding the black level area to generate a first code sequence. A video code ft unit that connects a second code sequence obtained by coding the video signal in the black level region to the first code sequence, and generates a code sequence of the new video signal.

An image processing apparatus characterized by the above-mentioned.

6 1. The video signal of the area excluding the black level area of the new video signal having the original resolution having the first resolution and the second resolution and having a black level area in a part of the image. A resolution converter for converting to

Encoding the video signal to generate a first code sequence; connecting a second code sequence obtained by encoding the video signal in the black level region to the first code sequence; And a video encoder for generating a signal sequence.

An image processing apparatus characterized by the above-mentioned.

6 2. The video encoder performs encoding according to the MPEG 2 method.

The image processing apparatus according to claim 60 or 61, wherein: