JPH04129491A - Coding system for moving picture signal - Google Patents

Abstract

PURPOSE:To eliminate useless information generated due to the noise of a background part and to enhance a coding efficiency by coding only the inside of a talker area obtained by segmentation. CONSTITUTION:A valid block obtained by an isolated invalid block elimination section 6 is given to a valid block decision section 8, a valid section reset section 9 and a coding section 7, and when the valid block number is a predetermined threshold level or over, the valid block set block 9 replaces the valid block entirely into the invalid block and when the valid block number is less than the threshold level, the block is given to a weight section 3 via a frame delay section 10, in which 2nd weighting is applied and the result is given to an adder 4. The coding section 7 applies coding of a moving picture signal given from the delay section 11 only to a part represented to be the valid block area, that is, a talker area in the given valid block map. Thus, useless coding information quantity generated due to noise from the background is eliminated and the coding efficiency is improved.

Description

DETAILED DESCRIPTION OF THE INVENTION (Field of Industrial Application) The present invention relates to a video signal encoding method using band compression technology.

(Prior Art) As an encoding method for a moving image signal using conventional band compression technology, for example, the r I SDN compatible color moving image television described in the 1989 Institute of Electronics, Information and Communication Engineers Spring National Conference, Material No. D-233 is used. "telephone device" is known. In this video signal encoding method, a map is created by extracting face areas on the screen. Then, the image encoding unit performs inter-frame intra-frame adaptive prediction, and at this time, if it is an egg area, encoding is performed up to the final stage, and if it is an other area, encoding is stopped at the previous stage. This reduces the amount of code.

(Problems to be Solved by the Invention) However, in the conventional video signal encoding method described above, the background portion other than the facial amount is also coarsely encoded, resulting in the generation of useless information due to noise in the background portion. Also, if the background part changes to the face part between consecutive screens,
Since coarse encoding changes to fine encoding, a large amount of prediction error signal is generated here as well, resulting in useless information being encoded. As a result, encoding efficiency decreases.

(Means for Solving the Problems) The video signal encoding method of the present invention is a video signal encoding method that uses correlation between screens, and one screen of an input video signal is divided into multiple pixels. divided into blocks consisting of
The difference from the previous screen is detected for each block, and when the difference value is greater than or equal to a predetermined first threshold, it is considered a valid block, and when the difference value is less than the first predetermined threshold, it is considered an invalid block and the frame is framed. means for creating a first effective block map for each screen, means for assigning a first weight to the first effective block map, and means for assigning a second weight to the sixth effective block map in the previous screen; The weighting is performed by adding and combining the first effective block map subjected to the first weighting and the sixth effective block map subjected to the second weighting. means for obtaining a second effective block map; and with respect to the second effective block map, referring to blocks in the vicinity of each block, the values of the neighboring blocks and the target block are greater than or equal to a predetermined second threshold; means for obtaining a third valid block map by performing segmentation in which the target block is set as a valid block when the value is less than a second threshold;
Regarding the isolated invalid block in the valid block map of
By referring to neighboring blocks, when the value of the neighboring block is equal to or greater than a predetermined third threshold, the isolated invalid block is replaced with a valid block, and when it is less than the third threshold, the isolated invalid block is replaced with an invalid block. means for obtaining a fourth valid block map as is, and if the number of valid blocks in the fourth valid block map is equal to or greater than a fourth predetermined threshold, all valid blocks in the fourth valid block map are invalidated; If the number of valid blocks in the fourth valid block map is less than a fourth predetermined threshold, the fourth valid block map is left as it is and is used as a fifth valid block map. means for generating a block map; means for delaying the fifth effective block map by one frame time to obtain a sixth effective block map; means for giving the video signal a time delay of and means for encoding using correlation within or both.

(Function) In videophones, etc., the background part is fixed and the speaker mainly moves, so if the part of the speaker is cut out and encoded, the amount of wasted encoded information generated by noise from the background etc. can be reduced. It can be removed and coding efficiency can be increased.

In the present invention, the coding efficiency is improved by cutting out the speaker's part and encoding only the speaker's part. How to extract speakers will be explained in detail with reference to the drawings.

Time to, tl in FIG.

Assume that the speaker moves as shown at t2. Then, when calculating the difference between the screens at time to and time t1, the second
The shaded area in the figure is obtained, and the isolated shaded area in the background is a differential signal generated by background noise.6 Next, the screen is divided into multiple pixels of n pixels in the horizontal direction x n pixels in the vertical direction. When the sum of the absolute values of the difference signals in each block is greater than or equal to a predetermined first threshold, that block is considered a valid block, and the sum of the absolute values of the difference signals in each block is determined as a valid block. Sometimes, the block is designated as an invalid block. The effective block map at time t1 obtained through the above processing is shown in FIG. 3(B). The black portions in FIG. 3(B) are valid blocks. It is assumed that FIG. 3(A) is the sixth effective block map obtained between the time to and the time tO-1, which is 61 screens before time 10. Then, the first weighting is applied to the effective block map of the current screen (FIG. 3(B)), that is, the first effective block map, and the sixth effective block map of the previous screen (FIG. 3(A)) is given the first weight. The second weighting is applied to the valid block map. An example of weighting is shown below.For example, the valid blocks of the previous frame are set to 1, and the invalid blocks are set to 0. The valid block of the current frame is set to 2, and the invalid block of the current frame is set to O like the invalid block of the previous frame. The effective block map of the previous frame weighted in this manner and the effective block map of the current frame are added and combined to obtain a second effective block map. The second effective block map becomes the row shown in FIG. 3(C). Next, Figure 3 (C)
For the second effective block map that has been additively synthesized,
Perform segmentation. An example of segmentation will be described with reference to FIGS. 3 and 4. For example, if k in FIG. 4 is the target block for segmentation, blocks a and b in the vicinity of the block.

Refer to the values of c, d, e, f, g, and h. That is, the value of the second effective block map shown in FIG. 3(C) is referred to. Neighboring blocks a, b, c, d, e.

A second block with predetermined values of f, g, h, and
When the threshold value is greater than or equal to the threshold value, the target block k is regarded as a valid block, and neighboring blocks a, b, and c.

When the values of d, e, f, g, h and the block are less than a predetermined second threshold, the target block k is determined to be an invalid block.

The third effective block map newly obtained by segmentation is shown in FIG. 3(D).

In some cases, isolated invalid blocks may occur in the moving portion of the third valid block map.

This is done in 1. when obtaining the first effective block map. A block whose difference between screens in a moving part is slightly lower than the first threshold value becomes an invalid block, so that an isolated invalid block occurs in a moving part. An example of an isolated invalid block is shown in FIG. If encoding is performed only on the valid block area in the third valid block map that includes isolated invalid blocks as shown in Figure 5, isolated invalid block parts within the valid block area will not be encoded and will therefore be invalid. The continuity of the encoded image is lost between the block portion and the surrounding portions, resulting in encoding distortion, which may result in a very unsightly encoded image. Therefore, as a method for removing isolated invalid blocks, processing similar to segmentation is performed to remove isolated invalid blocks. That is, by referring to blocks near the isolated invalid block, and when the neighboring blocks are equal to or higher than a predetermined third threshold value, the target isolated invalid block is replaced with a value indicating a valid block6.
Through the above processing, the area which is an isolated invalid block in FIG. 5 is removed, and a fourth valid block map is obtained. The fourth effective block map is shown in FIG. 3(D).

Next, the processing at time t2 will be explained.

Find the difference between the screens at time t1 and time t2, and calculate the difference between the screens at time t1 and time t2.
If validity/invalidity is determined according to the threshold value of Fig. 6 (A
) is obtained. A first weighting is applied to this first effective block map. Since the effective block map at time t1, which is the previous screen, is shown in FIG. 3(D), the second weighting is applied to the effective block map in FIG. 3(D), and then the first weighting is applied. When combined with the first effective block map, we get
A second effective block map shown in FIG. 6(B) is obtained. When the segmentation is performed on the second effective block map shown in FIG. 6(B), a third effective block map shown in FIG. 6(C) is obtained. Next, isolated invalid blocks are removed from the third valid block map. Since there was no isolated invalid block in the third valid block map in FIG. 6(C), the third valid block map was taken as the fourth valid block map and the story obtained by segmentation The actual speaker area at time t2, which is the speaker area, is approximately the left half of the screen, whereas the speaker area obtained by segmentation extends considerably into the background area of the right half of the screen. If the fourth effective block map in FIG. 6(C) is used as is, there is a possibility that background noise will also be encoded, which is not very desirable at time 0 t1.
, when the movement is large and the number of effective blocks obtained by segmentation is large, as in the case of t2, the effective block map of the previous screen will affect the map and bulge into the speaker area of the previous screen. be. Therefore, if there is a large movement between screens, that is, if the number of effective blocks in the fourth effective block map is greater than or equal to a predetermined fourth threshold, the fourth effective block map is reset, and the fourth effective block map is reset. All valid blocks in the fourth valid block map are replaced with invalid blocks to form a fifth valid block map. The fifth valid block map is delayed by one frame time to become the sixth valid block map, which is used for segmentation at the next time. for example,
Let FIG. 3(A) be the fourth effective block map of the previous frame, and FIG. 3(B) be the effective block map of the current frame, that is, the first effective block map. Then, if the number of valid blocks in the fourth valid block map obtained at time t1 is greater than or equal to the fourth threshold, all valid blocks in the fourth valid block map are replaced with invalid blocks and 5, the sixth effective block map at time t2 obtained by delaying the fifth effective block map by one frame time also becomes all invalid blocks. As a result, time t
The first effective block map in 2 is shown in FIG. 6(A).
If so, the weighted second effective block map becomes as shown in FIG. 6(D), and when the segmentation is performed on this second effective block map, FIG. A third effective block map as shown in ) is obtained. Since this third valid block map did not include any isolated invalid blocks, the third valid block map becomes the fourth valid block map as it is, and the background portion can be deleted.

By encoding the effective block area of the advantageous fourth effective block map, that is, the speaker area, using inter-screen correlation, intra-screen correlation, or both, it is possible to reduce noise such as background noise. Generated useless information can be easily deleted and encoding efficiency can be improved.

For each of the above threshold values and weighting values, optimal values statistically investigated in advance are used. Further, the arrangement of reference blocks in segmentation and isolated invalid block removal may be other than the above arrangement and number of blocks.

(Example) Next, an example of the present invention will be described in detail with reference to the drawings. FIG. 7 shows an embodiment of the present invention. The input moving image signal is sent to the validity determination unit 1 via a line 100.
and is supplied to the delay section 11. The validity/invalidity determination unit 1 is
The moving image signal of the previous screen is stored, the frame difference signal between the moving image signal of the previous screen and the moving image signal newly inputted via It is divided into blocks each consisting of a plurality of n pixels in the vertical direction, and the sum of absolute values of frame difference values within the block is determined for each block. If the sum of the absolute values of the calculated frame difference values is greater than or equal to a predetermined first threshold, the block is made a valid block, and if the sum of the absolute values of the frame difference values is less than the first threshold, the block is made invalid. A first effective block map is obtained as a block. The first valid block map obtained by the validity determining section 1 is provided to the weighting section 2. Weighting section 2
performs a predetermined first weighting on the first valid block map given by the validity determining unit 1. The first effective block map weighted by the weighting section 2 is provided to the adder 4. The adder 4 adds the first effective block map given from the weighting section 2 and the sixth effective block map, which is the effective block map in the previous screen given from the 4 weighting section 3, and performs weighting. A second effective block map is obtained. The second effective block map obtained by adder 4 is
It is given to the segmentation unit 5. The segmentation unit 5 performs segmentation processing on all blocks in the second effective block map given from the adder 4. For example, as shown in FIG. Then, the values of blocks a+ b, c, a, e, f, g+ h near k and k are referred to, and those values are set to a predetermined second
If the value of block k is greater than or equal to the threshold value, the block k is determined to be a valid block, and if the value of the neighboring blocks and k is less than the second threshold value, the block k is determined to be an invalid block and a third valid block map is obtained. The third valid block map obtained by the segmentation section 5 is provided to the isolated invalid block removal section 6. The isolated invalid block removing unit 6 performs a process of removing isolated invalid blocks for the invalid blocks included in the third valid block map given from the segmentation unit 5, and connects the valid blocks.

In the processing of an isolated invalid block, similar to segmentation, blocks in the vicinity of the target isolated invalid block are referred to, and if the value of the nearby block is greater than or equal to a predetermined third threshold, the isolated invalid block is processed. Make it a valid block. If the value of a neighboring block is less than a predetermined third threshold, the isolated invalid block is left as an invalid block, and the fourth valid block map from which the isolated invalid block has been removed is obtained through the above process. . The fourth valid block map obtained by the isolated invalid block removing section 6 is provided to a valid block number determining section 8, a valid block resetting section 9, and an encoding section 7. The valid block number determining unit 8 receives the fourth block number from the isolated invalid block removing unit 6.
If the number of valid blocks in the valid block map is equal to or greater than a predetermined fourth threshold, an instruction to execute a reset is given to the valid block reset unit 9.

Further, if the number of valid blocks in the fourth valid block map given from the isolated invalid block removing unit 6 is less than a predetermined fourth threshold, the valid block number determining unit 8 determines that the valid block resetting unit 9 gives the instruction to reset and stop. When the valid block reset unit 9 is given an instruction to execute a reset from the valid block number determination unit 8,
All valid blocks of the fourth valid block map given from the isolated invalid block removal unit 6 are replaced with invalid blocks to form a fifth valid block map.

Further, when the valid block reset unit 9 is given an instruction to stop the reset from the valid block number determination unit 8,
The fourth valid block map given from the isolated invalid block removing unit 6 is used as the fifth valid block map without any processing. The fifth effective block map obtained by the effective block reset section 9 is provided to the frame delay section 10. The frame delay unit 10 delays the fifth valid block map given from the valid block reset unit 9 by one frame time to obtain a sixth valid block map. The sixth effective block map obtained by the frame delay section 10 is provided to the weighting section 3. The weighting unit 3 performs second weighting on the sixth effective block map provided from the frame delay unit 10, and provides a weighted fourth effective block map to the adder 4. .

The delay section 11 compensates for the delay time for the input moving image signal from when the input moving image signal is supplied to when the fourth effective block map is given to the encoding section 7, and converts the input moving image signal into a fourth effective block map. and time alignment of the input video signal. The time-compensated moving image signal output from the delay section 11 is given to the encoding section 7. The encoding unit 7 converts the moving image given from the delay 11 only to the part indicated as the valid block area, that is, the speaker area, in the fourth valid block map given from the isolated invalid block removal unit 6. The signal is encoded, and the background portion indicated by the invalid block is not encoded.

Encoding methods include a method that uses inter-screen correlation such as motion compensation, a method that uses intra-screen correlation such as orthogonal transformation, or an encoding method that uses both inter-screen and intra-screen correlation. Use methods.

Regarding each threshold value and reference block arrangement mentioned above,
The fi3iI value statistically investigated in advance is used.

(Effects of the Invention) As explained in detail above, the video signal encoding method of the present invention encodes only within the speaker region obtained by segmentation, thereby eliminating unnecessary noise caused by background noise. Information can be deleted and encoding efficiency can be increased.

[Brief explanation of drawings]

1, 2, 3, 4, 5, and 6 are diagrams explaining the present invention in detail, and FIG. 7 is a diagram showing one embodiment of the present invention. 1... Validity/invalidity determination section, 2, 3... Weighting section, 4
...Adder, 5...Segmentation unit, 6...
- Isolated invalid block removal unit, 7... Encoding unit, 8...
- Effective block number determination unit, 9... Effective block reset unit, 10... Frame delay unit, 11... Delay unit.

Claims (1)

[Claims] In a video signal encoding method that uses correlation between screens, one screen of an input video signal is divided into blocks each consisting of multiple pixels, and the difference between each block and the previous screen is detected, and the difference value is means for creating a first effective block map for each frame; when the difference value is equal to or greater than a predetermined first threshold value, the block is determined to be a valid block; when the difference value is less than the first predetermined threshold value, the block is determined to be an invalid block; means for applying a first weight to the first effective block map; means for applying a second weight to the sixth effective block map in the previous screen; means for obtaining a weighted second effective block map by adding and combining the first effective block map and the sixth effective block map subjected to the second weighting; For the block map, refer to the blocks in the vicinity of each block, and if the values of the neighboring blocks and the target block are greater than or equal to a predetermined second threshold, the target block is regarded as a valid block, and if the value of the target block is less than the second threshold. In some cases, means for obtaining a third valid block map by performing segmentation with the target block as an invalid block, and for an isolated invalid block in the third valid block map, referring to neighboring blocks and calculating the value of the neighboring block. means for obtaining a fourth valid block map by replacing the isolated invalid block with a valid block when the value is greater than or equal to a third predetermined threshold, and by leaving the isolated invalid block as an invalid block when it is less than the third threshold; , if the number of valid blocks of the fourth valid block map is equal to or greater than a predetermined fourth threshold, all valid blocks of the fourth valid block map are replaced with invalid blocks to form a fifth valid block map; means for using the fourth effective block map as a fifth effective block map when the number of effective blocks in the fourth effective block map is less than a predetermined fourth threshold;
The effective block map of the sixth block is delayed by one frame time.
means for obtaining an effective block map of the fourth effective block map; means for providing the moving image signal with a time delay from the input of the moving image signal to the generation of the fourth effective block map; and the moving image to which the delay has been applied. A moving image signal, characterized in that the signal includes means for encoding an area determined as a valid block in the fourth valid block map using inter-screen correlation, intra-screen correlation, or both. encoding method.