JP2576704B2 - Video signal encoding device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving picture signal encoding apparatus using a band compression technique.

[0002]

2. Description of the Related Art As a coding method of a moving image signal using a conventional band compression technique, for example, "I" described in 1989 of IEICE Spring National Convention, document number D-233.
An SDN-compatible color moving image videophone device and the like are known. In this encoding method, a face area is extracted and a map is created. Then, the image encoding unit performs inter-frame intra-frame adaptive prediction. At this time, if the region is a face region, the encoding is performed up to the last stage, and if the region is other than the region, the encoding is stopped at the previous stage. This reduces the code amount.

[0003]

However, in the above-described encoding method, since the background portion other than the face is roughly encoded, useless information is generated due to noise in the background portion. Also, if the background changes from a background portion to a face portion between successive screens, the coding changes from coarse coding to fine coding, so that a considerable amount of prediction error signal is generated here as well, resulting in coding of unnecessary information. turn into. As a result, the coding efficiency decreases.

[0004]

SUMMARY OF THE INVENTION A moving picture signal encoding apparatus according to the present invention divides a picture into blocks of a plurality of pixels in coding a moving picture signal using correlation between pictures.
A difference between screens is detected for each block, and when the difference value is equal to or larger than a predetermined first threshold, the block is regarded as a valid block. When the difference value is smaller than a predetermined first threshold, the block is regarded as an invalid block, Valid / invalid determining means for creating a first valid block map for each frame, first weighting means for performing first weighting on the first valid block map, and Weighting means for performing the second weighting, the first effective block map having the first weighting, and the fourth effective block map having the second weighting added. Adding means for obtaining a weighted second effective block map; and performing segmentation on the second effective block map, the effective block of the fourth effective block map. Ratio determining means for determining whether or not the number of effective blocks in the first effective block map with respect to the number of blocks is equal to or greater than a predetermined second threshold value; If the value is equal to or greater than the threshold value, a third threshold value is selected as the threshold value in the segmentation. If the output of the ratio determiner is less than the second threshold value, the value is determined as the threshold value in the segmentation. Selecting a fourth threshold value, referring to a block near each target block in the second valid block map, and when the values of the nearby blocks and the target block are equal to or greater than the selected threshold value, The block is set as an effective block, and when the block is smaller than the selected threshold value, the block is set as an invalid block and a third effective block map is obtained. Means, in the isolated invalid block in the third valid block map,
Referring to the neighboring block, if the value of the neighboring block is equal to or greater than a predetermined fifth threshold value, the invalid block is replaced with the valid block, and the value of the neighboring block is changed to the fifth threshold.
If the threshold value is less than the threshold value, the invalid block is kept as an invalid block, and an isolated invalid block removing means for obtaining a fourth valid block map, and a delay for delaying an input signal and aligning time with the fourth valid block map Means and the fourth
The area defined as an effective block in the effective block map of
Encoding means for performing encoding using either one of the correlation between screens or the correlation within the screen, or both.

[0005]

In a videophone or the like, since the background portion is fixed and the speaker moves mainly, if the speaker portion is cut out and coded, the amount of useless coded information generated by noise from the background or the like is reduced. Can be eliminated and the coding efficiency can be increased.

In the present invention, the coding efficiency is improved by cutting out the speaker portion and coding only the speaker portion.

Next, a method of extracting a speaker will be described in detail with reference to the drawings. Times t0, t1, t2 in FIG.
It is assumed that after the speaker moves as shown in (2), at time t3, the speaker is stationary and only the mouth part is moving. And time t
When the difference between the screens at time 1 and time t2 is obtained, the hatched area in FIG. 2 is obtained, and the difference between the screens at time t2 and time t3 is shown by the hatched area in FIG. 7A. It is assumed that an isolated hatched portion of the background portion is a difference signal generated by background noise. Next, the screen is divided into blocks each including a plurality of pixels of n pixels in the horizontal direction × n pixels in the vertical direction, and when the sum of absolute values of the difference signals in each block is equal to or greater than a predetermined first threshold value, A block is defined as a valid block, and when the sum of absolute values of the difference signal is less than the first threshold, the block is defined as an invalid block. FIG. 3B shows a first effective block map between times t1 and t2 obtained by the above processing, and FIG. 7 shows an effective block map between times t2 and t3.
B. The portions painted black in FIGS. 3B and 7B are effective blocks. FIG. 3A assumes a fourth effective block map obtained between the screens at time t0 and time t1. Then, the effective block map of the current screen, that is, the first
The first weighting is performed on the effective block map, and the second weighting is performed on the fourth effective block map which is the effective block map of the previous screen. An example of weighting is shown below.

For example, the valid block of the current frame is "2", and the invalid block of the current frame is "0". The valid block of the previous frame is “1”, and the invalid block is “0”. The effective block map of the previous frame weighted in this way and the effective block map of the current frame are added and synthesized to obtain a second block map. The second effective block map at time t2 obtained by the above weighting is as shown in FIG. 4A.

Next, segmentation is performed on the added and combined effective block map of FIG. 4A. At this time, the value of the number of valid blocks in the previous frame, that is, the number of valid blocks in the fourth valid block map is used as the denominator, and the number of valid blocks in the first valid block map, which is the valid block of the current frame, is used as the numerator. When the ratio is equal to or greater than a predetermined second threshold value, for example, FIG. 4 shows an example of the segmentation in which the number of effective blocks in the current frame is substantially equal to or larger than 1/2 the number of effective blocks in the previous frame. This will be described with reference to FIG. For example, if k in FIG. 5 is a target block for segmentation,
Blocks a, b, c, d, e, f, near block k
Refer to the values of g and h. That is, the value of the second effective block map in FIG. 4A is referred to. Nearby blocks a,
When the values of b, c, d, e, f, g, h, and the block k are equal to or larger than a predetermined third threshold value, the target block k is set as an effective block, and neighboring blocks a, b, and
When the values of c, d, e, f, g, h, and block k are less than a predetermined third threshold value, the target block k is regarded as an invalid block.

FIG. 4B shows a third effective block map newly obtained by segmentation. An isolated invalid block may occur in the third valid block map. For this reason, if encoding is performed only on the effective block area in the third effective block map, the isolated invalid block portion in the effective block area is not encoded, so that encoding distortion occurs in that part. As a result, the encoded image may be very unsightly. Therefore, isolated invalid blocks are removed. As a method of removing an isolated invalid block, a process similar to the segmentation is performed on an invalid block. That is, a block near the invalid block is referred to, and when the nearby block is equal to or greater than a predetermined fifth threshold value, the target invalid block is replaced with a value indicating a valid block. With the above processing, the area that was the isolated invalid block in FIG. 4B is removed, and a fourth valid block map is obtained. FIG. 6 shows a fourth effective block map from which isolated invalid blocks have been removed.

Next, as at time t3, when the ratio between the number of valid blocks in the previous frame and the number of valid blocks in the current frame is less than the second threshold, the number of valid blocks in the current frame is An example of segmentation when the number of blocks is considerably smaller than the number of blocks, for example, when the number of blocks is smaller than 1/2, will be described with reference to FIGS. 6, 7, 8, and 9. When the first and second weights are combined with the fourth effective block map obtained at time t2 in FIG. 6 and the first effective block map at time t3 in FIG. 7B, respectively, the result is shown in FIG. 8A. A second valid block map as shown is obtained. For this second valid block map,
When the segmentation is performed based on the third threshold, a third effective block map indicated by hatching in FIG. 8B is obtained. Then, for the third effective block map,
Based on the fifth threshold value, isolated invalid blocks are removed to obtain a fourth valid block map. At this time, since there is no isolated invalid block in the third valid block map obtained at time t3, the fourth valid block map is similar to the third valid block map. Then, encoding is performed only in the effective block area in the fourth effective block map. However, the effective block area of this fourth effective block map is shown in FIG.
Since the speaker's chest and upper right part of the head are missing as shown in the figure, if the coding is performed as it is, an uncoded area will occur in the chest and head, and the story of the coded image will be lost. It is conceivable that a discontinuous portion occurs in the user area, and the encoded image becomes unsightly. Therefore, when the ratio between the number of valid blocks in the previous frame and the number of valid blocks in the current frame is less than the second threshold and the number of valid blocks in the current frame is small as at time t3, the threshold in the segmentation is set to The switching prevents the loss of the speaker area. For example, in the weighted second effective block map of FIG. 8A, when performing the segmentation, the threshold value in the segmentation may be, for example, k
Is smaller than 0, the threshold value is set sufficiently low so that k, which is the target block of the segmentation, is regarded as an effective block, so that a third effective block map as shown in FIG. 9 can be obtained. Can be prevented from being lost.
The threshold at this time is set as a fourth threshold. As described above, when the ratio between the number of valid blocks in the previous frame and the number of valid blocks in the current frame is equal to or greater than the second threshold, the third threshold is selected and segmentation is performed. If the ratio of the number of valid blocks in the current frame is less than the second threshold and the number of valid blocks in the current frame is considerably smaller than the number of valid blocks in the previous frame, the fourth threshold is selected and segmentation is performed. Do. 6 or 9 is generated by noise such as a background by encoding either the correlation between the screens or the correlation within the screen, or both, in the effective block area of FIG. 9, that is, the speaker area. Useless information can be easily deleted, and coding efficiency can be improved.

For each of the above threshold values and weighting values, optimal values statistically checked in advance are used. The arrangement of the reference blocks in the segmentation and the removal of the isolated invalid block may be an arrangement other than the above and the number of blocks.

[0013]

An embodiment of the present invention will be described in detail with reference to the drawings. FIG. 10 shows an embodiment of the present invention.
The input moving image signal is sent to a valid / invalid
And delay 9. The valid / invalid determiner 1 stores the signal of the previous screen, obtains a frame difference signal from the moving image signal newly input via the line 10, and calculates the frame difference signal by n pixels in the horizontal direction × vertical direction. The image is divided into blocks each including a plurality of pixels of n pixels, and for each block, a sum of absolute values of frame difference values in the block is obtained.
If the sum of absolute values of the obtained frame differences is equal to or greater than a predetermined first threshold, the block is regarded as an effective block. If the sum of absolute values of the frame differences is less than the first threshold, the block is invalidated. As a block, a first valid block map is obtained. The first value obtained by the valid / invalid
Is given to the weighting circuit 2 and the ratio judging unit 8. The weighting circuit 2 performs a predetermined first weighting on the first valid block map provided from the validity / invalidity determiner 1. The weighting circuit 2 can be constituted by a ROM (Read Only Memory). For example, if the output of the valid / invalid determiner 1 indicates that the block is a valid block, the output is set to "2". It is good to put it. As another method, the logic can be assembled by a logic gate circuit or the like. The first effective block map weighted by the weighting circuit 2 is provided to the adder 4. The adder 4 adds the first effective block map given from the weighting circuit 2 and the fourth effective block map given from the weighting circuit 3, and weights the second effective block map. Get. The second effective block map obtained by the adder 4 is provided to a segmentation circuit 5. Next, an example of the ratio determiner 8 is shown in FIG. The ratio determiner 8 includes counters 81 and 82 and a ROM 83. The counter 81 of the ratio determiner 8 counts the number of valid blocks in the first valid block map supplied from the validity / invalidity determiner 1 via the line 120. The counter 82 of the ratio judging device 8 counts the number of valid blocks in the fourth valid block map supplied from the isolated invalid block removing circuit 6 via the line 680. The number of effective blocks output from the counters 81 and 82 is supplied to the ROM 83. ROM83
Indicates whether the ratio of the number of valid blocks in the first valid block map to the number of valid blocks in the fourth valid block map obtained by the counters 81 and 82 is equal to or greater than a predetermined second threshold value Is determined. RO
M83 indicates that the ratio of the number of valid blocks in the first valid block map to the number of valid blocks in the fourth valid block map is equal to or greater than the second threshold value in an address area equal to or greater than a predetermined second threshold value. A signal indicating the above is written in advance, and the ratio of the number of valid blocks in the first valid block map to the number of valid blocks in the fourth valid block map is less than a predetermined second threshold. In the address area, a signal indicating that the value is smaller than the second threshold value is written in advance. A signal indicating whether or not the output of the ROM 83 is equal to or greater than a second threshold is supplied to the segmentation circuit 5 via a line 85 as an output of the ratio determiner 8. The segmentation circuit 5 performs a segmentation process on all blocks in the second effective block map provided from the adder 4. For example, when the judgment signal given from the ratio judgment unit 8 indicates that the judgment signal is equal to or larger than the second threshold, a third threshold is selected as a threshold in the segmentation, and the judgment signal is
If it is less than the second threshold value, the fourth threshold value is selected as the threshold value in the segmentation, and the segmentation is performed. In the segmentation, as shown in FIG. 5, when a block to be segmented is k, k, a, b, c, and k in the vicinity of k are set.
The values of the blocks d, e, f, g, and h are referred to, and if those values are equal to or larger than the threshold value selected by the judgment signal given from the comparison judgment unit 8, the block k is regarded as an effective block, and the reference is made. If the values of the block and the block k are less than the selected threshold value, the block k is regarded as an invalid block, and a third valid block map is obtained. FIG. 12 shows an example of the segmentation circuit 5. The segmentation circuit 5 includes line memories 50 and 51, a delay 52,
53, 54, 55, 56, 57 and the ROM 58. The second valid block map of the output of adder 4 is stored in line memory 5 via line 45.
0, delay 52 and ROM 58. The line memory 50 delays the effective block map supplied via the line 45 by one block line, and
The delay 54 and the ROM 58 are supplied. The line memory 51 delays the signal supplied from the line memory 50 by one block line and supplies the signal to the delay 56 and the ROM 58. The delays 52, 53, 54, 55, 56, and 57 delay the supplied signal by one clock and supply the delayed signal to the ROM 58 and the delay of the next stage. The output signal of the delay 54 is the block k to be segmented in FIG. RO
M58, when the signal supplied from the comparison / determination unit 8 via the line 85 indicates that the value is equal to or larger than the second threshold, the determination value based on the third threshold is written in the address area. If the signal supplied from the comparison / determination unit 8 via the line 85 indicates that the value is less than the second threshold, a determination value based on the fourth threshold is stored in the address area. Write it down. For example, a value indicating a valid block is written in an address area where a signal supplied from a line memory and a delay is equal to or greater than a predetermined third threshold, and an address area less than the third threshold is written in an address area less than the third threshold. Write a value indicating an invalid block. In addition, a value indicating an effective block is written in an address area indicated by the fourth threshold or more, and a value indicating an invalid block is written in an address area less than the fourth threshold. As described above, by using the line memory, the delay, and the ROM, it is possible to refer to the value of the block k to be subjected to the segmentation and the values of the neighboring blocks.

Next, the isolated invalid block removing circuit 6 will be described with reference to FIG. Isolated invalid block removal circuit 6
Are line memories 60 and 61, delays 62, 63, 64, 65, 66 and 67, as in the segmentation circuit 5.
And the ROM 68, and removes an isolated invalid block by referring to the value of a block near the invalid block. The signal supplied from the delay 64 to the ROM 68 is the target block k for the removal of the isolated invalid block. That is, if the signal supplied from the delay 64 to the ROM 68 is an invalid block and the neighboring blocks are equal to or greater than a predetermined fifth threshold value, the ROM 68 outputs a value indicating the valid block, If the signal obtained is an invalid block and the number of neighboring blocks is less than the fifth threshold, the ROM 68
Outputs a value indicating an invalid block. If the signal supplied from the delay 64 is an effective block, the ROM 68
Outputs a value indicating an effective block to the output. By writing the above logic in the ROM 68 in advance, isolated and invalid blocks are removed and valid blocks are connected. The fourth effective block map of the output of the isolated invalid block removing circuit 6 is supplied to the weighting circuit 3 and the encoder 7 via a line 680. The weighting circuit 3 performs the second weighting on the fourth effective block map supplied from the isolated invalid block removing circuit 6. The weighting circuit 3 can be configured by a ROM, like the weighting circuit 2. At this time, a value for performing the second weighting is written in the ROM in advance. The fourth effective block map in which the output of the weighting circuit 3 is weighted is supplied to the adder 4. Next, the delay 9 performs delay time compensation for the input moving image signal from when the input moving image signal is supplied to when the fourth effective block map is supplied to the encoder 7. And the input signal are timed. The time-compensated input video signal at the output of the delay 9 is provided to the encoder 7 via line 970. FIG. 14 shows the configuration of the encoder 7. The moving image signal supplied from the delay 9 via the line 970 is supplied to the moving vector detector 71 and the subtractor 72 inside the encoder 7. The fourth effective block map of the output of the isolated invalid block removing circuit 6 is supplied via a line 680 to the quantizer 74 inside the encoder 7 as a signal indicating a region where coding is to be performed. The motion vector detector 71 stores a signal of the previous screen, detects a motion between the screen and a signal input via a new line 970, and generates a motion vector indicating the amount and direction of the motion from the frame. It is supplied to a memory 77 and a variable length encoder 78.
The subtracter 72 subtracts the input signal supplied via the line 970 from the motion-compensated prediction signal supplied from the frame memory 77 to obtain a motion-compensated prediction error. The motion compensation prediction error signal obtained by the subtractor 72 is supplied to the orthogonal transformer 73. The orthogonal transformer 73 performs orthogonal transform on the motion compensation prediction error signal supplied from the subtractor 72, and converts the spatial domain prediction error signal into a frequency domain prediction error signal. The frequency domain prediction error signal output from the orthogonal transformer 73 is supplied to the quantizer 74. The quantizer 74 quantizes the prediction error signal supplied from the orthogonal transformer 73 for a block indicating that the fourth valid block map supplied via the line 680 is a valid block, Blocks whose fourth valid block map provided via line 680 indicates that they are invalid blocks stop encoding by zeroing out the quantizer output. The output signal of the quantizer 74 is supplied to an inverse orthogonal transformer 75 and a variable length encoder 78. The inverse orthogonal transformer 75 performs an inverse orthogonal transform on the prediction error signal supplied from the quantizer 74, and returns to a spatial domain prediction error signal. The output signal of the inverse orthogonal transformer 75 is added to an adder 76.
Supplied to The adder 76 adds the spatial domain prediction error signal supplied from the inverse orthogonal transformer 75 and the motion compensation prediction signal supplied from the frame memory 77 to obtain a local decoded signal. The local decoded signal at the output of the adder 76 is
It is supplied to the frame memory 77. The frame memory 77 changes the delay amount of the local decoded signal supplied from the adder 76 according to the motion vector supplied from the motion vector detector 71 to obtain a motion compensated prediction signal. The motion compensation prediction signal output from the frame memory 77 is subtracted from the subtracter 7
2 and to the adder 76. Next, the variable length encoder 7
Reference numeral 8 denotes a variable-length predictive error signal in the frequency domain supplied from the quantizer 74 and the motion vector supplied from the motion vector detector 71, which are variable-length encoded using an efficient code such as a Huffman code. Encoding to reduce redundancy. The variable-length code with reduced redundancy is matched with the encoding speed and the speed of the transmission line, becomes the output of the variable-length encoder 78, and is output from the encoder 7 to the transmission line. As described in detail above, according to the fourth effective block map, encoding is performed only on the effective block area, that is, the part indicated as the speaker area, and encoding is stopped on the background part indicated by the invalid block. .

As for the method of stopping the encoding, the subtractor 72
It is also possible to replace the output with zero. As the above-mentioned threshold values and weighting values, optimal values statistically checked in advance are used.

[0016]

As described above in detail, the moving picture signal encoding method according to the present invention encodes only the speaker area obtained by the segmentation, so that the wasteful noise caused by the background noise is generated. Information can be deleted, and coding efficiency can be improved.

[Brief description of the drawings]

FIG. 1

FIG. 9 is a view for explaining a method for cutting out a screen according to the present invention.

FIG. 10 is a block diagram according to one embodiment of the present invention.

FIG. 11

FIG. 14 is a block diagram showing each part of the embodiment of the present invention in more detail;

[Description of Code] 1 valid / invalid determining unit 2, 3 weighting unit 4, 76 adder 5 segmentation unit 6 isolated invalid block removing unit 7 encoding unit 8 ratio determining unit 9, 52, 53, 54, 55, 56, 57, 62, 6
3, 64, 65, 66, 67 Delay 50, 51, 60, 61 Line memory 58, 68, 83 ROM 71 Motion vector detector 72 Subtractor 73 Orthogonal transformer 74 Quantizer 75 Inverse orthogonal transformer 76 Adder 77 Frame memory 78 Variable length encoder 81, 82 Counter

Claims (1)

(57) [Claims] In encoding a moving image signal using correlation between screens, a screen is divided into blocks each including a plurality of pixels, a difference between screens is detected for each block, and the difference value is determined in advance. A valid block when the difference value is equal to or greater than the first threshold, and an invalid block when the difference value is less than a predetermined first threshold. A first weighting means for performing a first weighting on one effective block map; and a second weighting means for performing a second weighting on the fourth effective block map.
A second weighting means for weighting the first effective block map, the first effective block map having the first weighting,
Adding means for adding and combining the weighted fourth effective block map to obtain a weighted second effective block map; and performing segmentation on the second effective block map. Ratio determining means for determining whether or not the number of valid blocks of the first valid block map with respect to the number of valid blocks of the fourth valid block map is equal to or greater than a predetermined second threshold value; If the output of the determination means indicates that it is equal to or greater than the second threshold, a third threshold is selected as the threshold in the segmentation, indicating that the output of the ratio determiner is less than the second threshold. If so, select the fourth threshold as the threshold in segmentation,
Referring to a block near each target block in the second effective block map, when the values of the nearby blocks and the target block are equal to or greater than the selected threshold,
A segmentation unit for setting the block as an effective block, and when the value is less than the selected threshold value, the block is regarded as an invalid block, and a third effective block map is obtained;
In the isolated invalid block in the third valid block map, a nearby block is referred to, and when the value of the nearby block is equal to or larger than a predetermined fifth threshold value, the invalid block is replaced with a valid block, When the value of the block is less than the fifth threshold value, the invalid block is kept as an invalid block, and an isolated invalid block removing means for obtaining a fourth invalid block map; And a delay means for performing time alignment with the above, and coding an area determined as an effective block in the fourth effective block map using either one of a correlation between screens or a correlation within a screen, or both. A moving picture signal coding apparatus comprising: coding means.