JPH10336646A - High efficiency encoder of video signal - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve encoding efficiency and to reduce the quantization distortion of a block including a boundary at the time of highly efficiently encoding the video signals of a letter box type. SOLUTION: Input analog video signals are A/D converted in an A/D converter 2, separated into luminance signals Y and color difference signals Cr and Cb in a Y/C separator 3, and stored in a frame memory 4. A letter box boundary detector 5 detects the boundary of a non-image part and an image part for respective frames by a line unit. In the case that the boundary is over the image processing unit (block) of video signal high efficiency encoding, an encoding object image control part 6 shifts the entire screen, replaces the non- image part positioned on the inner side of the image processing unit (block) including the boundary with the data of the adjacent image part and replaces the non-image part with specified uniform data. The encoding object image control part 6 reads the image signals of the image part from the frame memory 4 and supplies them to an encoder 7.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video signal high-efficiency coding apparatus for high-efficiency coding of a letterbox type video signal in which the upper and lower portions of an image signal are masked.

[0002]

2. Description of the Related Art Different aspect ratios (ratio of horizontal and vertical lengths of a screen) are adopted depending on types and methods of images.
In movie images, there are known Vistavision having an aspect ratio of 1.85 to 1, cinemascope having an aspect ratio of 2.35 to 1, and the like. As the television image, an aspect ratio of 16: 9 (1.78: 1) used in the EDTV and HDTV systems, and an aspect ratio of 4: 3 (1.33: 1) used in the conventional NTSC system, etc. It has been known.

FIG. 17 is an explanatory diagram of a letter box type image. For example, a cinemascope (aspect ratio 2.
35: 1) movie image with its aspect ratio (2.35)
The aspect ratio of the screen is 4 to 3 (1.3) without changing the aspect ratio.
In the case of outputting in the NTSC system television size (3: 1), as shown in FIG. 17, a letterbox type video composed of a non-image portion and an image portion masked up and down is obtained. The number of horizontal scanning lines (number of lines) of the NTSC system is 480 lines / frame (24
0 / field).

FIG. 18 is a block diagram of a conventional video signal high-efficiency encoding apparatus for encoding a letterbox type video signal with high efficiency. A conventional video signal high-efficiency encoding device 101 includes an A / D converter 102 and a Y / C separator 10.
3 and an encoder 104. A letterbox type analog video signal, which is an analog input signal, is input to the A / D converter 102 and is converted into a digital video signal. The digital video signal converted by the A / D converter 102 or the digital video signal which is a digital input signal is input to the Y / C separator 103. The Y / C separator 103 performs digital processing of luminance signal / chrominance signal separation on the digital video signal, and outputs a digital luminance signal Y and two digital color difference signals Cr and Cb. The digital luminance signal Y and the digital color difference signals Cr and Cb are supplied to the encoder 104. The encoder 104 performs a high-efficiency encoding process on the digital luminance signal Y and each of the digital chrominance signals Cr and Cb, and generates and outputs a high-efficiency encoded video encoded signal 104a.

FIG. 19 is a block diagram showing a specific example of the encoder shown in FIG. FIG. 19 shows an encoder 104 that performs MPEG encoding. Note that MP
About EG, ISO-IEC1172-2, ITU
-TH. Since H.262 / ISO-IEC13818-2 is described in detail, only the outline will be described here. The encoder 104 for MPEG encoding shown in FIG.
Is a first internal frame memory 111 and a motion detection unit 11
2, a motion compensator 113, a subtractor 114, a discrete cosine transformer (DCT) 115, a quantizer 116, a variable length encoder 117, an inverse quantizer 118, and an inverse discrete cosine transformer. (Inverse DCT) 119 and a second internal frame memory 120.

A digital image signal (hereinafter, referred to as an image signal) composed of a digital luminance signal (hereinafter, referred to as a luminance signal) Y and two digital color difference signals (hereinafter, referred to as color difference signals) Cr, Cb is a first internal signal. The data is input to the frame memory 111 and stored. The first internal frame memory 111 stores image signals of a plurality of frames.

FIG. 20 is an explanatory diagram of an encoding processing block. As shown in FIG. 20, the first internal frame memory 11
1, the luminance signal Y is 16 × 16.
Each color difference signal Cr, Cb is processed by a block of pixels.
Are sub-sampled in the vertical direction (vertical direction) in units of 16 × 8 pixels, and are processed as blocks each having 8 × 8 pixels.

[0008] The motion detecting section 112 shown in FIG.
A motion vector by prediction is detected, and the detected motion vector 112a is output. The motion vector 112a is supplied to the motion compensation unit 113. A local decoding circuit for decoding an image encoded by the inverse quantizer 118 and the inverse discrete cosine transformer 119 is configured, and the locally decoded image signal 119a that has been locally decoded is stored in the second internal frame memory 120. Is done. The motion compensating unit 113 receives the motion vector 112a and the local decoded image 120a read from the second internal frame memory 120 as inputs, and performs a motion compensation image signal 113a on the local decoded image that has been subjected to motion compensation based on the motion vector. And output it,
The motion vector 113b and the prediction mode 113c are output. The motion compensation image signal 113a is supplied to a subtractor 114. Motion vector 113b and prediction mode 1
13c is supplied to the variable-length encoder 117.

The subtracter 114 subtracts the motion compensated image signal 113a from the image signal 111a of the image block to be encoded read from the first internal frame memory 111, and outputs a difference image signal 114a. Difference image signal 1
14a is supplied to the discrete cosine converter 115. The discrete cosine transformer 115 performs a discrete cosine transform on the difference image signal 114a, calculates a DCT coefficient 115a, and outputs the result. The DCT coefficient 115a is supplied to the quantizer 116. The quantizer 116 quantizes the DCT coefficient 115a to generate a quantized signal (quantization number) 116.
a is output. The quantized signal (quantization number) 116a is
It is supplied to the variable length encoder 117 and the inverse quantizer 118, respectively.

The variable-length encoder 117 performs variable-length coding on the quantized signal (quantization number) 116a, the motion vector 113b, and the prediction mode 113c, respectively.
A video coded signal 104a is generated by multiplexing the variable length coded quantized signal, motion vector, and prediction mode based on a predetermined data format, and outputs the coded video signal 104a.

In the MPEG system, an intra-coded I picture (intra-coded pi) is used.
) and a P picture (predictive coded picture) which has been subjected to inter-frame coding.
re) is used as a reference signal for motion compensated prediction. For this reason, a reproduced image of an I picture and a P picture is required, and the inverse quantizer 118 and the inverse discrete cosine transformer 11
9 to perform local decoding (local decoding).
The inverse quantizer 118 generates a quantized signal (quantization number) 116
a is subjected to inverse quantization to output a DCT coefficient 118a. The inverse discrete cosine transformer 119 calculates the DCT coefficient 11
8a, and generates and outputs a local decoded image signal 119a by performing an inverse discrete cosine transform. Local decoded image signal 1
19a is stored in the second internal frame memory 120.

Japanese Patent Application Laid-Open No. Hei 7-177476 discloses that a letterbox format wide-aspect television signal in which a reinforcing signal is multiplexed in an upper and lower mask portion is divided into blocks and the upper and lower mask portions and the reinforcing signal Describes a video signal processing device capable of reducing the distortion of the reinforcement signal in a block spanning.

This video signal processing apparatus is provided with replacement means for replacing a predetermined number of lines of the video signal of the main part adjacent to the reinforcement signal of the upper and lower mask parts with the level of the reinforcement signal, and the image of the main part adjacent to the reinforcement signal is provided. By setting the signal to the level of the reinforcement signal, the correlation between the signals in the block becomes larger in the boundary block extending between the video signal of the main part and the reinforcement signal of the upper and lower mask parts, and the distortion of the reinforcement signal due to high efficiency coding is reduced. I try to reduce it.

Further, the video signal processing device includes a rearrangement means for rearranging the order of the lines of the reinforcement signal in the upper and lower mask portions. When distortion due to efficiency coding occurs, the position is such that interference due to the distortion of the augmentation signal does not visually pose a problem.

Further, the video signal processing apparatus includes a level difference detecting means for detecting a level difference in a vertical direction in the block, and a quantizer having a fine quantization width for a block in which the level difference exceeds a predetermined level. And a quantization control means for controlling the quantization means so as to perform the quantization so as to reduce distortion of the reinforcement signal due to the quantization by performing fine quantization on the boundary block.

[0016]

The configuration of the conventional video signal high efficiency coding apparatus shown in FIG. 18 has the following problems. When a letter-box type video signal including a non-picture part containing random noise and the like is subjected to high-efficiency coding as it is, an increase in AC components in quantization of the no-picture part, a decrease in prediction efficiency of motion compensation prediction between frames, etc. As a result, a larger amount of code is allocated to the coding of the non-image portion, and as a result, the image quality of the image is degraded. As a countermeasure, it is conceivable to set the position uniquely and replace the non-image part with uniform data.However, in the case of movie material, etc., there are cases where the borderline differs for each scene due to editing, so simply It cannot be replaced with uniform data.

When a boundary between a non-image portion and an image portion in a letterbox type video signal straddles a block, quantization distortion occurs in the quantization of the block, and particularly, the AC component of the image portion in the block. And the non-image portion is degraded, and the boundary line becomes unclear.

The video signal processing apparatus described in Japanese Patent Application Laid-Open No. 7-177476 reduces distortion accompanying encoding of a boundary block by setting a video signal of a main part adjacent to a reinforcement signal to a level of the reinforcement signal. However, in this case, the image of the main unit may be damaged. In addition, by rearranging the order of the reinforcing signal lines in the upper and lower mask portions, it is possible to set a position where interference due to the influence of the distortion of the reinforcing signal does not visually pose a problem, but the distortion itself cannot be reduced. Furthermore, although it is possible to reduce distortion by performing quantization with a fine quantization width on the boundary block, there is a possibility that the coding efficiency is reduced.

SUMMARY OF THE INVENTION The present invention has been made to solve such a problem, and when encoding a letterbox type video signal with high efficiency, it is intended to improve the encoding efficiency and to quantize blocks including boundaries. It is an object of the present invention to provide a video signal high-efficiency encoding device capable of reducing distortion.

[0020]

SUMMARY OF THE INVENTION In order to solve the above-mentioned problems, a video signal high-efficiency encoding apparatus according to the present invention is characterized in that upper and lower boundaries between a masked non-image portion and an unmasked image portion are framed. A letterbox boundary line detector that detects each image in line units, and when the boundary between a non-image part and an image part straddles the image processing unit (block) of the video signal high-efficiency coding, the upper and lower boundary lines are An encoding target image control unit that performs processing on the image data so as to be at an optimum position in the signal high-efficiency encoding.

When the boundary line extends over the coding processing unit (block), the encoding target image control unit shifts the entire screen so that the upper and lower boundary lines are located at the optimum positions in the video signal high efficiency coding. It is characterized by having done.

The encoding target image control unit is characterized in that a non-image portion located inside an image processing unit (block) including a boundary line is replaced with data of an adjacent image portion.

[0023] The encoding target image control unit is characterized in that the non-image portion is replaced with specific uniform data for each frame.

The encoding target image control unit is characterized in that it is configured to perform a high-efficiency encoding process only on an image portion.

The video signal high-efficiency coding apparatus according to the present invention detects the position of each boundary line between the upper and lower non-image portion and the image portion masked by the letterbox boundary line detector line by line for each frame. I do. When the position of the upper and lower boundary lines straddles the encoding processing block, the encoding target image control unit shifts the entire screen so that the position of the boundary line becomes an optimal position for high-efficiency encoding.

The encoding target image control unit replenishes the data of the adjacent non-image part with respect to the free area of the upper or lower non-image part caused by shifting the entire screen.

When the position of the boundary line extends over the coding processing block, the coding target image control unit replaces the data of the non-image part in the coding processing block with the data of the adjacent image part.

The encoding target image control unit replaces the data of the non-image portion with specific uniform data.

The encoding target image control unit performs high-efficiency encoding processing only on the image portion by supplying data of only the image portion to the encoder.

Therefore, the video signal high-efficiency encoding apparatus according to the present invention, when encoding a letterbox type image signal in which the upper and lower portions of the image signal are masked, improves the prediction efficiency and improves the efficiency near the boundary line. Can be reduced, and the coding efficiency can be improved.

[0031]

Embodiments of the present invention will be described below with reference to the accompanying drawings. FIG. 1 is an overall block configuration diagram of a video signal high efficiency coding apparatus according to the present invention. The video signal high-efficiency encoding apparatus 1 according to the present invention includes an A / D converter 2, a Y / C separator 3, a frame memory 4, a letterbox boundary detector 5, and an encoding target image control unit. (Read / write address generator) 6 and an encoder 7.

The video signal high-efficiency coding apparatus 1 can receive two types of video signals, an analog video signal and a digital video signal. The analog video signal is A /
The signal is input to a D converter 2, converted into a digital video signal by the A / D converter 2, and supplied to a Y / C separator 3. The digital video signal which is a digital input is Y / C
It is supplied directly to the separator 3. The Y / C separator 3 performs digital processing of luminance signal / color difference signal separation on the digital video signal, and outputs a digital luminance signal (hereinafter, referred to as a luminance signal) Y and two digital color difference signals (hereinafter, a color difference signal). It outputs Cr and Cb.

The luminance signal Y and each digital color difference signal C
The digital video signal separated into r and Cb is supplied to the frame memory 4 and written to an address specified based on a write address supplied from the encoding target image control unit (read / write address generator) 6. The frame memory 4 stores digital video signals of a plurality of frames. The digital video signal stored in the frame memory 4 is read based on a read address supplied from an encoding target image control unit (read / write address generator) 6. The luminance signal Y and each color difference signal Cr / Cb read from the frame memory 4
Are the letterbox boundary detector 5 and the encoder 7
Supplied to

The letterbox boundary line detector 5 detects the boundary of a letterbox (line-free portion and image portion) on a line basis based on the luminance signal Y read from the frame memory 4 and each color difference signal Cr / Cb. Is detected, and a boundary detection signal 5a is output. Boundary line detection signal 5
a is supplied to the encoding target image control unit (read / write address generator) 6.

An encoding target image control unit (read / write address generator) 6 performs signal processing based on the boundary detection signal 5a, and supplies a read address and an image for supplying a digital video signal to the encoder 7. The position information 6a is output.

The encoder 7 provides image position information supplied from the encoding target image controller (read / write address generator) 6 to the digital video signal read from the frame memory 4 based on the read address. 6a, performs high-efficiency encoding, and outputs a video encoded signal 7a.

FIG. 2 is a block diagram of a letterbox boundary detector. Letterbox boundary detector 5
An activity detector 51, a level detector 52, a comparator 53, an activity reference value generator 54, and a level reference value generator 55 are provided.

The luminance signal Y and each color difference signal Cr / Cb read from the frame memory 4 are supplied to an activity detecting section 51 and a level detecting section 52. The activity detection unit 51 obtains an activity value (ACT) described later for each line based on the luminance signal Y and each color difference signal Cr / Cb. There are two types of activity values (ACT), horizontal and vertical.
H, ACTV). Each activity value (A
CTH, ACTV) are supplied to the comparison unit 53. The level detector 52 outputs the luminance signal Y and the color difference signals Cr / C
Based on b, a level value (DC) described later for each line
Ask for. The level value (DC) is supplied to the comparison unit 53.

The comparing section 53 includes an activity detecting section 51
Activity values (ACTH, AC
TV) and an activity reference value 54a supplied from the activity reference value generation unit 54, and the level value (DC) detected by the level detection unit 52 and the level supplied from the level reference value generation unit 52 are compared. By performing a magnitude comparison with a reference value, a boundary line is determined.

The comparing section 53 corrects each reference value (activity values (ACTH, ACTV) and level reference value) for performing the above-mentioned magnitude comparison so that the boundary line can be accurately discriminated. It is configured to do this.
The comparing unit 53 adjusts the activity reference value based on the activity values (ACTH, ACTV) and the level value DC detected by the detecting units 51 and 52 for the non-image portion of the letterbox image to be encoded. In addition to generating and outputting an activity reference value correction signal 53a for adaptively correcting, a level reference value correction signal 53b for adaptively correcting the level reference value is output. The activity reference value correction signal 53a is supplied to the activity reference value generator 54. The level reference value correction signal 53b is supplied to the level reference value generator 55.

The activity reference value generator 54 has a preset horizontal activity reference value BACTH,
In addition, a preset vertical activity reference value BACTV is supplied to the comparison unit 53, and when the activity reference value correction signal 53a is supplied from the comparison unit 53, the activity reference value for the preset activity reference value is supplied. Activity reference values BACTH, BAC corrected based on the value correction signal 53a
The TV is supplied to the comparison unit 53.

The level reference value generation section 55 supplies a preset level reference value BDC to the comparison section 53, and when the level reference value correction signal 53b is supplied from the comparison section 53, the preset level reference value BDC is supplied to the comparison section 53. The level reference value BDC obtained by correcting the reference value based on the level reference value correction signal 53b is supplied to the comparison unit 53.

FIG. 3 shows an encoding target image control section (read / read).
FIG. 3 is a block diagram of a write address generator). The read / write address generator 6 includes a boundary line corrector 6
1, a line counter 62, and a read / write address generator 63.

The boundary line correction section 61 outputs the boundary line detection signal 5a.
Based on the line counter value 62a, the boundary line is shifted to an appropriate position in the encoding processing block, the data of the non-image portion in the block is replaced, the entire non-image portion is replaced with uniform data, only the image portion Necessary for the extraction of the boundary line correction signal 61a
Output as The boundary line correction signal 61a is read /
It is supplied to the write address generator 63. When extracting only the image portion, the boundary line correction section 61 generates necessary position information of the image by the encoder 7 and outputs it as image position information 6a.

The line counter 62 counts the line numbers of the letterbox image signal, and outputs the counted line numbers as a line count value 62a.

Read / write address generator 63
Generates and outputs a read address corresponding to each of the above-described processes based on the boundary line correction signal 61a and the line count value 62a.

When performing the operation of detecting a letter box boundary line, the read / write address generator 63
On the basis of the boundary detection window width signal 6b and the line count value 62a, a read address for reading an image signal in the boundary detection window from the frame memory 4 is generated. Thus, an image signal in the boundary detection window is supplied to the letterbox boundary detector 5.

FIG. 4 shows an image signal of a cinemascope having an aspect ratio of 2.35 to 1 having an aspect ratio of 4 to 3.
It is explanatory drawing which shows an example of the letterbox image at the time of outputting to the television size of the current NTSC system (1.33 to 1). Hereinafter, the operation of the video signal high-efficiency encoding apparatus 1 according to the present invention using the letterbox image shown in FIG. 4 as an input image signal will be described.

FIG. 5 is an explanatory diagram showing the write timing when writing a letterbox image signal to the frame memory. FIG. 5A shows the line number of the letterbox image signal and the write address to the frame memory 4, and FIG. 5B shows the type of data to be written (the distinction between a non-image portion and an image portion).

The luminance signal Y and the color difference signals Cr and Cb output from the Y / C separator 3 shown in FIG. 1 are written to the frame memory 4 at the write timing shown in FIG. When the encoder 7 shown in FIG. 1 performs MPEG coding, the processing unit of the luminance signal Y is a block of 16 × 16 pixels as shown in FIG. 20, and the processing of each color difference signal Cr / Cb is performed. The processing unit is a block of 16 × 8 pixels. Here, as shown in FIG. 4, a group in the line direction of the encoding processing unit is defined as a slice (16 lines). When the aspect ratio of the letterbox is known in advance, a borderline detection window is provided as shown in FIG. 4, and the detection process is performed only on the lines (32 lines) in the borderline detection window, so that the letterbox is detected. The box boundary can be detected efficiently.

FIG. 6 is an explanatory diagram showing the detection timing of the boundary line. FIG. 6A shows a read address for reading an image signal in the boundary detection window (for 32 lines) from the frame memory 4 in correspondence with a line number.
FIG. 6B shows the timing at which the activity value and the level difference of each line are calculated, in association with the line numbers. In FIG. 6B, the line numbers at which the activity values are calculated are shown at the top, and the line numbers at which the level values are calculated are shown at the bottom. FIG. 6C shows the output timing of the comparison result from the comparison unit 53. FIG. 6D shows the output timing of the boundary detection signal based on the comparison result.

The read / write address generator 63 shown in FIG. 3 receives the boundary line detection window width signal 6b supplied from the outside and the line count value 62a output from the line counter 62 as inputs, and A read address for reading an image signal to the letterbox boundary line detector 5 is generated at the timing shown in FIG. As a result, an image signal in each boundary detection window (32 lines × 2) is read from the frame memory 4.
The activity detection unit 51 shown in FIG. 2 detects the activity value ACT based on the image signal (32 lines × 2) in each boundary detection window read from the frame memory 4. The level detection unit 52 shown in FIG. 2 detects the level value DC based on the image signal in each boundary detection window (32 lines × 2) read from the frame memory 4.

FIG. 7 is an explanatory diagram showing a method of detecting an activity. First, the level difference between the adjacent left and right pixels for line 0 is calculated in the horizontal direction, and the sum of the absolute values is calculated. Next, the same calculation is performed for line 1. Next, the level difference between the upper and lower pixels adjacent to the line 0 and the line 1 is calculated in the horizontal direction, and the sum of absolute values for two lines is calculated. In this way, the horizontal and vertical activities of line 0 and line 1 are obtained. Next, activities are similarly obtained for the lines 1 and 2. That is, the pixel level is set to P (x,
y), the activity value ACTH in the horizontal direction is obtained from the operation of Expression 1, and the activity value ACTV in the vertical direction is obtained from the operation of Expression 2. In Equations 1 and 2, m is the number of pixels, and n is the number of lines.

[0054]

(Equation 1)

[0055]

(Equation 2)

FIG. 8 is an explanatory diagram showing a level detection method. First, the average of the pixel levels of the line 0 is obtained. next,
The average of the pixel level of line 1 is determined. That is, assuming that the pixel level is P (x, y), the level value DC can be obtained from Expression 3. In Equation 3, m is the number of pixels, and n is the number of lines.

[0057]

(Equation 3)

The comparing unit 53 shown in FIG.
H, ACTV and activity reference values BACTH, BACTV supplied from the activity reference value generation unit 54
6 is compared with the level value DC supplied from the level detection unit 52 and the level reference value BDC supplied from the level reference value generation circuit 55 to determine a boundary line. The boundary line detection signal is output at the timing shown in FIG.

Next, the operation of determining a boundary line will be described. First, the activity will be described. In FIG. 7, it is assumed that a boundary line above a letter box exists between line 0 and line 1. In this case, since the line 0 is a non-image portion and has little motion, the horizontal activity value ACTH0 of the line 0 is a small value. On the other hand, since the movement of the line 1 is larger than that of the non-image part since the line 1 is an image part, the horizontal activity value ACTH1 of the line 1 becomes a large value. Line 0
Is the vertical activity value ACTV0 of line 0
And the activity in the vertical direction of line 1, that is, the activity between the non-image portion and the image portion, the value is large.

Next, the levels will be described. In FIG. 8, it is assumed that a boundary line above a letter box exists between line 0 and line 1. In this case, since the line 0 is a non-image portion and the line 1 is an image portion, the level value DC0 of the line 0 is equal to the level value D of the line 1.
It is closer to the black level than C1.

The activity reference values (BACTH, BACTV) serve as the reference values for these magnitude comparisons.
And the level reference value BDC. The comparison unit 53 determines that a boundary line exists between the line n and the line n + 1 when the condition shown in Expression 4 is satisfied.

[0062]

{(ACTHn <BACTH <ACTHn + 1) | (ACTHn> BACTH> ACTHn + 1)} & (ACTVn> BACTV) & [{(DCn-BLK) <BDC} & {(DCn + 1-BLK)> BDC}] | [{(DCn-BLK)> BDC} & {(DCn + 1-BLK) <BDC}]

In equation (4), the symbol |
The symbol & means AND. BLK is a black level.

The condition shown in Expression 4 is as follows.
First, the horizontal activity reference value of the line n is smaller than the horizontal activity reference value, and the line n + 1
Is larger than the horizontal activity reference value (ACTHn <BACTH).
<ACTHn + 1) or when the horizontal activity reference value of the line n is larger than the horizontal activity reference value and the horizontal activity reference value of the line n + 1 is smaller than the horizontal activity reference value (ACTHn>BACTH> ACTHn + 1). In other words, a case where the horizontal activity reference value of one of the two adjacent lines is larger than the horizontal activity reference value and the horizontal activity reference value of the other line is smaller than the horizontal activity reference value is the first. The conditions are as follows.

Next, the vertical activity value of the line n is larger than the vertical activity reference value (AC
(TVn> BACTV) is the second condition.

Further, the difference between the level value of the line n and the black level is smaller than the level reference value, and the line n + 1
Is greater than the level reference value {(DCn-BLK) <BDC} & {(DCn +
1−BLK)> BDC｝, or the difference between the level value of line n and the black level is larger than the level reference value, and
When the difference between the level value of line n + 1 and the black level is smaller than the level reference value {(DCn-BLK)> BDC} &
{(DCn + 1−BLK) <BDC}, in other words,
In the case where the difference between the level value of one line and the black level in the two adjacent lines is larger than the level reference value and the difference between the level value of the other line and the black level is smaller than the level reference value, a third case is described. It is a condition.

When both of the first, second and third conditions are satisfied, the comparing unit 53 sets the n-th line and the n-th line.
It is determined that there is a boundary between the +1 line.

Further, the comparing section 53 provides the reference value correction signals 53a and 53b for the activity reference value and the level reference value so that the adaptive reference value can be corrected for accurate detection of the boundary line. Is output. For example,
Each reference value correction signal 53 is obtained from the activity and level of the non-image portion of the letterbox image being encoded.
a, 53b are generated. Activity reference value generator 5
4 and the level reference value generator 55, based on the respective reference value correction signals 53a and 53b,
TH, BACTV, and BDC are corrected, and corrected reference values BACTH, BACTV, and BDC are output.

When the comparator 53 detects the upper boundary of the letter box, as shown in FIG. 6 (d), the comparator 53 sets the logic level of the boundary detection signal to the L level, as shown in FIG. When the lower boundary of the letter box is detected, the logic level is returned to the H level.

Next, the operation of correcting a boundary line will be described. The boundary line correction unit 61 shown in FIG. 3 receives the boundary line detection signal 5a and the line count value 62a as input, shifts the boundary line to an appropriate position in the encoding processing block, and outputs data of a non-image portion in the block. , The information necessary for the replacement of the entire non-image portion with uniform data, and the extraction of only the image portion are generated and output as a boundary line correction signal 61a. When extracting only the image portion, the encoder 7
To generate necessary image position information and output it as image position information 6a. Read / write address generator 63
Generates and outputs a read address corresponding to each of the above-described processes based on the boundary line correction signal 61a and the line count value 62a.

First, the processing for shifting the boundary line to an appropriate position in the encoding processing block will be described. The boundary line correction unit 61 shown in FIG. 3 counts slices based on the boundary line detection signal 5a and the line count value 62a, and determines where in the block (in the slice) the upper and lower boundary lines are located. .

FIG. 9 is an explanatory diagram showing an example of shifting the boundary line. The boundary line correction unit 61 determines the position of the boundary line in FIG.
The screen is shifted so that the boundary line is at the upper or lower boundary of the encoding processing block as shown in FIG. The upper and lower encoding processing blocks are selected so that the non-image portion is reduced in the encoding processing block.

FIG. 10 is an explanatory diagram showing an operation of replenishing data in the non-image portion free area. FIG. 10A shows an operation of supplementing data in an empty area of an upper end non-image portion, and FIG. 10B shows an operation of supplementing data in an empty area of a lower end non-image portion. FIG. 10 (a), FIG.
As shown in (b), the empty area at the upper end or the lower end generated by shifting the image replenishes the data of the adjacent non-image part.

The boundary line correction section 61 shown in FIG. 3 outputs the number of lines to be shifted and the shift direction (up and down) for shifting the boundary line to an appropriate position as a boundary line correction signal 61a. The read / write address generator 63 sets a read address for the frame memory 4 based on the boundary correction signal 61a and the line count value 62a so that the video signal whose entire screen is shifted is supplied to the encoder 7. Generate and output.

FIG. 11 is an explanatory diagram showing an operation of replacing data of a non-image portion. FIG. 11A shows the data replacement operation of the non-image portion when the encoding processing block extends over the upper boundary line, and FIG. 11B shows the data of the non-image portion when the encoding processing block extends over the lower boundary line. This shows the replacement operation.

The boundary line correction section 61 shown in FIG. 3 counts slices based on the boundary line detection signal 5a and the line count value 62a, and the position of the upper or lower boundary line in a block (slice) is determined. Determine if there is. Then, the data of the upper and lower non-image portions in the block are replaced with the data of the adjacent image portions as shown in FIG. The boundary correction unit 61 supplies the read / write address generation unit 63 with the line number of the non-image portion to be replaced and information indicating which block is upper or lower as a boundary correction signal 61a.

Read / write address generator 63
Generates and outputs a read address to the frame memory 4 based on the boundary line correction signal 61a and the line count value 62a such that the data in the upper and lower non-image portions are replaced with the data in the adjacent image portions. This allows
The encoder 7 is supplied with image data in which the data in the upper and lower non-image portions is replaced with the data in the adjacent image portion.

FIG. 12 is an explanatory diagram showing the shift operation of the boundary line. 12A shows line numbers of the letterbox image shown in FIG. 4, FIG. 12B shows slice numbers, FIG. 12C shows line numbers in a slice, and FIG.
12D shows a boundary line detection signal, FIG. 12E shows a read address for reading image data from the frame memory 4, and FIG. 12F shows image data read from the frame memory 4. I have. FIG. 12 shows the read address and the image read from the frame memory 4 when both the shifting of the boundary and the replacement of the non-image portion in the block are performed using the letterbox image shown in FIG. 4 as an input image signal. This shows the relationship with the data.

FIG. 12 shows an example in which the entire screen is shifted by seven lines in order to set the boundary line at an appropriate position, and the read address is generated from seven lines. The non-image portion (line 103) in the block straddling the boundary line is replaced by the data of the adjacent image portion (line 104). The empty area of the non-image portion at the lower end is supplemented with data of the adjacent non-image portion (472 to 479 lines). The read / write address generation unit 63 generates a read address for the frame memory 4 so that the image signal after the above-described data replacement and data replenishment is supplied to the encoder 7. As a result, the boundary line is corrected to an appropriate position, and the image signal in which the data is replaced and the data is supplemented is supplied to the encoder 7.

Next, a process for replacing the entire non-image portion with uniform fixed data will be described. The boundary line correction unit 61 shown in FIG. 3 determines the number of upper and lower boundary lines based on the boundary line detection signal 5a and the line count value 62a, and corrects the determined line numbers of the respective boundary lines. Signal 61
Output as a. The read / write address generator 63 generates and outputs a read address for replacing the non-image portion with uniform fixed data based on the boundary line correction signal 61a and the line counter value 62a.

FIG. 13 is an explanatory diagram showing the structure of the frame memory. In the frame memory 4, another area is provided in addition to an area for storing an image signal to be encoded, and uniform fixed data is stored in this other area. In FIG. 13, a luminance signal area for storing a luminance signal of an image to be encoded, a fixed luminance signal area for storing a preset fixed luminance signal, and each of the color difference signals Cr and Cb of the image to be encoded. Each color signal area to be stored and each fixed color difference signal Cb, C
3 shows an example of a memory configuration of the frame memory 4 including fixed color signal areas each storing r and a plurality of empty areas.

When the video signal of the non-image portion is supplied to the encoder 7, the read / write address generator 63 outputs a read address designating a fixed luminance signal area when supplying a luminance signal. When supplying each color difference signal, a read address designating each fixed color difference signal area is output. Thereby, when the video signal of the non-image portion is supplied to the encoder 7, the fixed luminance signal stored in the fixed luminance signal area and the fixed color difference signal stored in each fixed color difference signal area are read. It is.

FIG. 14 is an explanatory diagram of an operation of replacing a non-image portion with fixed data. 14A shows the line number, FIG. 14B shows the slice number, FIG. 14C shows the line number in the slice, FIG. 14D shows the boundary detection signal, and FIG. The read address is changed as shown in FIG.
Indicates read data. FIG. 14 shows the operation of shifting the boundary line, replacing the non-image portion in the block, and replacing the entire non-image portion with uniform fixed data. Here, in the frame memory 4, the fixed luminance signal and each fixed color difference signal are stored in the area corresponding to the line number 480, respectively, and the fixed luminance signal is designated by specifying the read address corresponding to the line number 480. A signal and each fixed color difference signal are read. Therefore, by specifying a read address corresponding to the line number 480 as shown in FIG. 14E, the non-image portion can be replaced with fixed data as shown in FIG.

Next, extraction of only the image portion will be described. The boundary line correction unit 61 shown in FIG. 3 determines the number of upper and lower boundary lines based on the boundary line detection signal 5a and the line count value 62a, and outputs this line number as the boundary line correction signal 61a. . Further, the boundary correction unit 61
Generates and outputs image position information 6a necessary for encoding from the number of shift lines when the processing for shifting the boundary line to an appropriate position in the encoding processing block is performed.

For example, when the encoding of the MPEG2 system is used, vertical_size_value, di specified in ISO / IEC13818-2.
spray_vertical_size, frame
By describing the position information of an image in _center_vertical_offset, encoding of only the image portion can be performed. In the decoding, when only the image portion is decoded and shifted to an appropriate position in the encoding processing block, the position can be returned to the original position for each field and reproduced.

This can be applied to the above-described shift to an appropriate position in the encoding processing block.
For example, instead of supplementing other image data to the empty area of the non-image part at the upper end or lower end due to the shift, only the part excluding the empty area is to be encoded, and the same processing can be performed. is there.

The read / write address generator 63 shown in FIG. 3 reads the read address based on the boundary correction signal 61a and the line counter value 62a so as to read the video signal of only the image portion from the frame memory 4. Generate and output. Thereby, the video signal of only the image portion is read from the frame memory 4 and the video signal of only the image portion is supplied to the encoder 7.

FIG. 15 is an explanatory diagram showing the operation of extracting only the image portion. FIG. 15A shows line numbers, and FIG.
15B shows a slice number, FIG. 15C shows a line number in a slice, FIG. 15D shows a boundary line detection signal, and FIG.
FIG. 15E shows a read address, and FIG. 15F shows read data. FIG. 15 shows reading when the letterbox image shown in FIG. 4 is used as an input image signal and three types of processing of shifting a boundary line, replacing a non-image portion in a block, and extracting only an image portion are performed. The relationship between the address and the image data read from the frame memory 4 is shown. As shown in FIG. 15E, the read address is generated so as to access only the area where the data of the image portion is stored.

FIG. 16 is a block diagram of the encoder. As the encoder 7, for example, an encoder that performs encoding according to the MPEG system is used. For MPEG, ISO-
IEC1172-2, ITU-TH262 / ISO-I
Since EC13818-2 has a detailed description,
Here, only the outline will be described.

The encoder 7 includes a first internal frame memory 7
1, a motion detecting section 72, a motion compensating section 73, and a subtractor 7
4, a discrete cosine transformer (DCT) 75, a quantizer 76, a variable length encoder 77, an inverse quantizer 78, an inverse discrete cosine transformer (inverse DCT) 79, and a second internal frame. And a memory 80. The image position information 6a is supplied to the variable length encoder 77, and the variable length encoder 77 is configured to perform variable length encoding including the image position information 6a. Is basically the same as the conventional encoder 104 shown in FIG.

The code read from the frame memory 4 shown in FIG. 1 based on the read address generated by the image control unit (read / write address generator) 6 shown in FIG. 1 and FIG. The video signals (luminance signal Y and color difference signals Cr, Cb) to be converted are
The data is input to and stored in the first internal frame memory 71 shown in FIG. The first internal frame memory 71 stores image signals of a plurality of frames.

The image signals (luminance signal Y and each color difference signal Cr, Cb) stored in the first internal frame memory 71
As shown in FIG. 20, the luminance signal Y is processed in blocks of 16 × 16 pixels, and each of the color difference signals Cr and Cb is sub-sampled in the vertical direction (vertical direction) by 16 × 8 pixels. Each is processed as a block of 8 × 8 pixels.

The motion detecting section 72 detects a motion vector by inter-frame (or inter-field) prediction for each block of image signal, and outputs the detected motion vector 72a. The motion vector 72a is supplied to the motion compensator 73. A local decoding circuit for decoding an image encoded by the inverse quantizer 78 and the inverse discrete cosine transformer 79 is configured, and the locally decoded image signal
9a is stored in the second internal frame memory 80. The motion compensating unit 73 receives the motion vector 72a and the local decoded image 80a read from the second internal frame memory 80, and performs a motion compensation image signal 73a on the local decoded image based on the motion vector. Is generated and output, and the motion vector 73b and the prediction mode 73c are output. The motion compensated image signal 73a is supplied to a subtractor 74. The motion vector 73b and the prediction mode 73c are supplied to the variable length encoder 77.

The subtractor 74 is provided in the first internal frame memory 7
The motion compensation image signal 73a is subtracted from the image signal 71a of the encoding target image block read from 1 to output a difference image signal 74a. The difference image signal 74a is supplied to the discrete cosine converter 75. Discrete cosine transformer 7
5 performs a DCT coefficient 75a by performing a discrete cosine transform on the difference image signal 74a. DCT coefficient 75a
Is supplied to the quantizer 76. The quantizer 76 has a DCT
The coefficient 75a is quantized to output a quantized signal (quantization number) 76a. Quantized signal (quantization number) 7
6a is supplied to the variable length encoder 77 and the inverse quantizer 78, respectively.

The variable length encoder 77 includes a quantized signal (quantization number) 76a, a motion vector 73b, and a prediction mode 73.
c and the image position information 6a supplied from the encoding target image control unit (read / write address generator) 6 shown in FIGS. A video coded signal 7a in which the quantized signal, the motion vector, and the prediction mode are multiplexed based on a predetermined data format is generated and output.

In the MPEG system, an intra-coded I picture (intra-coded pi) is used.
) and a P picture (predictive coded picture) which has been subjected to inter-frame coding.
In re), since a reproduced image to be used as a reference signal for motion compensation prediction is required, local decoding (local decoding) is performed by the inverse quantizer 78 and the inverse discrete cosine transformer 79. The inverse quantizer 78 outputs a quantized signal (quantization number) 76
a is subjected to inverse quantization to output a DCT coefficient 78a. The inverse discrete cosine transformer 79 performs an inverse discrete cosine transform based on the DCT coefficient 78a to generate and output a locally decoded image signal 79a. The local decoded image signal 80a is stored in the second internal frame memory 120.

[0097]

As described above, the video signal high-efficiency encoding apparatus according to the present invention comprises a letterbox boundary detector for detecting each boundary between a non-image portion and an image portion of a letterbox type image; Encoding target image control that performs processing on image data so that the upper and lower boundaries are located at the optimal positions in video signal high-efficiency coding when the boundary lines extend over image processing units (blocks) of the video signal high-efficiency coding. When encoding a letterbox type image signal in which the upper and lower portions of the image signal are masked,
The prediction efficiency can be improved, the quantization noise near the boundary can be reduced, and the coding efficiency can be improved.

[Brief description of the drawings]

FIG. 1 is an overall block configuration diagram of a video signal high efficiency coding apparatus according to the present invention.

FIG. 2 is a block diagram of a letterbox boundary detector.

FIG. 3 is a block diagram of an encoding target image control unit (read / write address generator).

FIG. 4 is a letterbox when a cinemascope image signal having an aspect ratio of 2.35 to 1 is output to a current NTSC television size having an aspect ratio of 4 to 3 (1.33 to 1). Explanatory diagram showing an example of an image

FIG. 5 is an explanatory diagram showing writing timing when writing a letterbox image signal to a frame memory.

FIG. 6 is an explanatory diagram showing detection timing of a boundary line.

FIG. 7 is an explanatory diagram showing a method of detecting an activity.

FIG. 8 is an explanatory diagram showing a level detection method.

FIG. 9 is an explanatory diagram showing an example of shifting a boundary line.

FIG. 10 is an explanatory diagram showing an operation of supplementing data to a non-image portion free area.

FIG. 11 is an explanatory diagram showing an operation of replacing data in a non-image portion.

FIG. 12 is an explanatory diagram showing a shift operation of a boundary line.

FIG. 13 is an explanatory diagram showing a configuration of a frame memory.

FIG. 14 is an explanatory diagram of an operation of replacing a non-image portion with fixed data.

FIG. 15 is an explanatory diagram showing an operation of extracting only an image portion;

FIG. 16 is a block diagram of an encoder.

FIG. 17 is an explanatory diagram of a letter box type image.

FIG. 18 is a block diagram of a conventional video signal high efficiency coding apparatus.

FIG. 19 is a block diagram showing a specific example of the encoder shown in FIG. 18;

FIG. 20 is an explanatory diagram of an encoding processing block.

[Explanation of symbols]

1. High efficiency video signal encoding device 2. A / D converter 3.
... Y / C separator, 4 ... frame memory, 5 ... letter box boundary line detector, 5a ... boundary line detection signal, 6 ... coding target image control unit (read / write address generator), 6a ... image position information , 6b ... borderline detection window signal,
7 Encoder, 51 Activity detector, 52 Level detector, 53 Comparison unit, 54 Activity reference value generator, 55 Level reference value generator, 61 Boundary correction unit, 62 Line counter , 63: read / write address generator, ACTH: horizontal activity value, ACTV: vertical activity value, Cr, Cb
... Color difference signal, DC: level value, Y: luminance signal.

Claims (5)

[Claims] 1. A video signal high-efficiency coding apparatus for high-efficiency coding of a letterbox type video signal in which the upper and lower portions of an image signal are masked. And a letterbox boundary line detector for detecting each boundary line by line for each frame, and when each boundary line extends over a coding processing unit (block) of video signal high efficiency coding, A video signal high-efficiency encoding device, comprising: an encoding target image control unit that performs processing on image data so as to be at an optimum position in video signal high-efficiency encoding. 2. The encoding target image control unit, when each of the boundary lines detected by the letterbox boundary line detector extends over the encoding processing unit (block), sets each of the boundary lines to a video signal. 2. The video signal high-efficiency encoding apparatus according to claim 1, wherein the entire screen is shifted so as to be at an optimum position in the high-efficiency encoding. 3. The encoding target image control unit removes a non-image part located inside an image processing unit (block) including the boundary detected by the letterbox boundary detector from an adjacent image part. 2. The video signal high-efficiency encoding device according to claim 1, wherein the encoding device is configured to be replaced with data. 4. The encoding target image control unit recognizes the non-image portion based on the boundary detected by the letterbox boundary detector and specifies the non-image portion for each frame. 2. The video signal high-efficiency encoding apparatus according to claim 1, wherein the encoding apparatus is configured to replace the data with uniform data. 5. The encoding target image control unit recognizes the image part based on the boundary detected by the letterbox boundary detector, and performs high-efficiency encoding on only the image part. 2. The video signal high-efficiency encoding apparatus according to claim 1, wherein the apparatus is configured to perform processing.