JP3364233B2 - Image coding / decoding method - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image coding / decoding system, and more particularly to an image coding / decoding system in which background separation / synthesis by keying is applied to image block coding. 2. Description of the Related Art Today, with the development of a digital public network, a conference system that allows a video conference between multiple points has become widespread. In this type of system, a block coding method having a high coding efficiency is adopted, and a method is also considered in which keying (chroma key) of TV broadcasting is used together and a moving person is extracted to perform block coding. By doing so, it is not necessary to encode the flicker of the fluorescent lamp, the error of the A / D converter, the flicker of the background caused by the incomplete Y / C separation, etc., and it is expected that the encoding efficiency is improved. However, there is a problem of image quality deterioration in encoding a block including an edge between a person and a background, and improvement in image quality is desired.

[0002]

2. Description of the Related Art FIG. 11 is a diagram showing a configuration of a conventional image coding system. In the figure, 6 is a background separating section, 61 is a key extracting section, 62 is a key signal K generating section, and 63 is a switch circuit. is there. Further, 1 is a standardized H.264 standard for video conferencing / telephone coding methods. An interframe coding unit according to H.261, 11 is a video memory (VM), 12 is a subtractor, 13 is a discrete cosine transformer (DCT), 14 is a quantizer (QTZ), 15 is an encoder (ENC), and 16 is Inverse quantizer (IQTZ), 17 is an inverse discrete cosine transformer (IDCT), 18 is an adder, 1
Reference numeral 9 is a frame memory (FM), 20 is a variable delay buffer (VDB), and 21 is a motion compensation unit (MC). 3 more
Is a standardized H.264 standard. An interframe decoding unit according to H.261, 31
Is a decoder (DEC), 32 is an inverse quantizer (IQTZ),
33 is an inverse discrete cosine transformer (IDCT), 34 is an adder, 35 is a frame memory (FM), 36 is a variable delay buffer (VDB), and 37 is a motion compensation prediction unit (MC). 5 is a background composition unit, 51 is a key extraction unit, and 52 is a key extraction unit.
Is a background signal generator (BGG), and 53 is a switch circuit.

For example, when an image signal V of a person photographed against a background of a blue curtain is input, the separating section 6 extracts a signal of the blue curtain from the image signal V and uses this as a pure key signal K.
(For example, green signal B). In this way, the image signal VK having the background separated with the pure blue signal B is obtained.
Becomes a key signal K for synthesizing the background image BG on the decoding side.

In the interframe coding unit 1, the subtractor 1
2 is a difference between the current video data VK of the video memory 11 and the prediction data P, and the obtained inter-frame difference data E is (8 × 8) by the discrete cosine converter 13.
Orthogonal transformation is performed in block units of pixels, and the quantizer 14
Is quantized into prediction error data PE. Then, this prediction error data PE is further encoded by the encoder 15 and transmitted.

In this state, the inverse quantizer 16 and the inverse discrete cosine transformer 17 reproduce the interframe difference data E'from the prediction error data PE, and the adder 18 converts the interframe difference data E into the prediction data P. By adding ', the current video data VK' is locally reproduced, and thereby the contents of the frame memory 19 are updated. Frame memory 1
The frame data FD of 9 is the video data V of the next frame.
The frame data FD, which is read out for the prediction of K, undergoes a variable delay by the variable delay buffer 20 to compensate for the movement of the image between the frames at that time, and becomes the prediction data P. The motion compensation prediction unit 21 obtains the motion of the image between frames at that time, transmits the obtained optimum motion vector MV, and controls the delay amount of the variable delay buffer 20 by the vector MV.

On the other hand, the interframe decoding unit 3 reproduces the video data VK 'based on the prediction error data CPE and the optimum motion vector MV, and outputs this to the synthesizing unit 5. The synthesizing unit 5 then decodes and reproduces the image signal VK ′.
The key signal K in the inside is extracted, and that portion is replaced with the background signal BG, thus finally obtaining the image signal VB in which the background is combined.

FIG. 12 is a diagram for explaining the problems of the conventional image coding system. In the figure, P / Q is a person before / after moving,
B _K is a pixel block including only the key signal K, and B _V is a pixel block having a person's edge portion. According to the configuration of FIG. 11, the inter-frame difference E does not occur between blocks having no motion, and this block is an invalid block that does not substantially require encoding. Therefore, the more the number of such invalid blocks is, the higher the coding efficiency is, and in this respect, the inter-frame difference E does not occur between the pixel blocks B _K , so that the high coding efficiency is expected.

However, since the motion compensation prediction is not perfect in the pixel block of the person P / Q and the pixel block B _V around it, the inter-frame difference E occurs, and this block is substantially an effective block that needs to be coded. Become. Particularly, between the pixel blocks B _V including the edges of the person and the key signal, a relatively large inter-frame difference E occurs due to imperfect motion prediction between frames. Therefore, the inter-frame difference E is block-encoded by the discrete cosine transform unit 13 and the quantization unit 14, but the prediction error data PE contains a considerable amount of encoding distortion due to information compression and quantization at that time. Get lost. Then, the inverse discrete cosine transform unit 16
And when decoded by the inverse quantizer 17, the obtained inter-frame difference E ′ contains coding distortion and is no longer completely the same as the original inter-frame difference E. Therefore,
The reproduced image VK 'in which the prediction data P is added is not the same as the original image VK.

On the other hand, when the same image signal VK 'is reproduced also in the inter-frame decoding unit 3, especially when the decoded key signal K'has coding distortion,
The key signal K'may deviate from the original key signal K in some cases, and therefore, the key extraction in the synthesizing unit 5 is no longer necessary and the quality of the synthesized image VB is significantly impaired.

Moreover, the larger the movement of the person, the more the blocks that would otherwise be invalid are coded, and the improvement in coding efficiency that was originally thought could not be obtained.

[0011]

As described above, in the conventional image coding system, the coding distortion occurs in the edge block between the person and the key signal, which deteriorates the image quality at the edge portion of the output composite image. There was a problem of doing. An object of the present invention is to provide an image marks No. / decoding method no deterioration in image quality at the edge portion of the output of the synthesized image.

[0012]

The above-mentioned problems can be solved by the structure shown in FIG. That is, according to the image coding method of the present invention (1) , the image signal VK whose background is separated by the key signal K is coded.
Image to be inter-frame coded in block units by the encoding unit 1
In the encoding method, local reproduction is performed in the encoding unit 1.
Within a predetermined range including the key signal K with respect to the pixel signal VK ′
Distortion removal filter that makes color pixel signals the same as key signals
10 is provided. Further, the image decoding of the present invention (2)
The system is an image signal C encoded by the system of claim 1.
Inter-frame decoding of VK in block units by the decoding unit 3
In the image decoding method according to
For the generated pixel signal VK ′, a predetermined range including the key signal K
The distortion removal flag that makes the pixel signals of the colors in the box the same as the key signals.
The filter 30 is provided.

[0013]

[0014]

In FIG. 1, a relatively large inter-frame difference E occurs due to imperfect inter-frame prediction especially between pixel blocks that include edges between a person and a key signal. Therefore, the inter-frame difference E is block-encoded, but the prediction error data PE contains a considerable amount of encoding distortion due to information compression and quantization at that time.
When this is locally decoded, the obtained inter-frame difference E'includes coding distortion and is no longer completely the same as the original inter-frame difference E. Therefore,
The reproduced pixel block VK 'in which the prediction data P is added is not the same as the original pixel block VK.

On the other hand, also in the decoding section 3, the influence of coding distortion appears in the same manner. Particularly, when the influence of coding distortion appears on the part of the decoded and reproduced key signal K ′, the reproduced key signal K ′ may not be the key signal K in some cases, and in such a case, the keying technique is applied to the block coding. The meaning that I made is lost. Therefore, the distortion removal filter 1
0, and the pixel signal VK locally reproduced by the encoding unit 1
The coding distortion of 'is removed. Preferably, the distortion removal filter 10 makes a pixel signal VK ′ whose difference from the key signal K is within a predetermined range in luminance, hue, and saturation equal to the key signal K. Therefore, in the encoding unit 1, even when the pixel block including the edge between the person and the key signal is locally reproduced, the key signal K ′ subjected to the encoding distortion is particularly generated.
Is restored to the original key signal K, the influence of coding distortion on the key signal K is reduced.

On the other hand, the distortion removal filter 30 of the decoding unit 3
Removes the coding distortion of the reproduced pixel signal VK 'in the same manner as above. Therefore, even when the pixel block including the edges of the person and the key signal is decoded and reproduced, the key signal K ′ subjected to the coding distortion is returned to the original key signal K, and accordingly, the key signal in the synthesizing unit 5 is generated. K is extracted correctly.

[0017]

[0018]

[0019]

Embodiments of the present invention will be described in detail below with reference to the accompanying drawings. FIG. 3 is a diagram showing the configuration of the image coding system of the first embodiment. In the figure, the same reference numerals as those in FIG. 11 denote the same or corresponding portions, and reference numerals 10 and 30 denote distortion elimination filters. The distortion removal filter 10 is a filter that removes the coding distortion included in the pixel signal VK ′ locally reproduced by the coding unit 1. Preferably, the difference from the key signal K is within a predetermined range in luminance, hue, and saturation. Of the pixel signal VK 'of the same as the key signal K. Therefore, even when the pixel block including the edges of the person and the key signal is decoded and reproduced, the key signal K ′ subjected to the coding distortion is returned to the original key signal K, and the error due to the coding distortion is not accumulated. .

On the other hand, the distortion removal filter 30 similarly removes the coding distortion contained in the pixel signal VK 'reproduced by the decoding section 3. Therefore, even when the pixel block including the edges of the person and the key signal is decoded and reproduced, the key signal K ′ subjected to the coding distortion is returned to the original key signal K, and therefore the key signal K in the synthesizing unit 5 is restored. Is correctly extracted.

FIG. 4 is a diagram for explaining the operation principle of the distortion removal filter of the first embodiment. As is well known, the three primary colors R,
The NTSC color TV signal based on G and B is a luminance signal Y.
And a color signal Cr having information on hue and saturation (saturation)
It is composed of (RY) and Cb (BY). FIG. 4 shows typical saturated color signals R, G, B on the orthogonal plane of the color signal Cr-Cb. At the center of the coordinates, when the luminance signal Y is 1.0, white W is displayed. And the luminance signal Y
When is 0.0, it is black BK.

In this embodiment, a typical saturated blue signal B is used as the key signal K, and the key signal K'decoded and reproduced by the encoding unit 1 or the decoding unit 3 is affected by encoding distortion. As a result, it is dispersed around the green signal B. Therefore, the distortion removal filters 10 and 30 make the pixel signal VK ′ within the predetermined range H whose difference from the blue signal B is the same as the blue signal B in terms of luminance, hue, and saturation.

FIG. 5 is a block diagram of the distortion elimination filter of the first embodiment, in which 100 is a generator of a blue signal B, 1
01 to 103 are absolute difference value calculators, 104 to 106 are comparators (CMP), 107 is an AND gate circuit, and 108 is a switch circuit. The switch circuit 108 is normally connected to the terminal a side, and the decoded reproduction image signal VK of the input terminal IN is connected.
′ (Y _i , C _ri , C _bi ) is output to the output terminal OUT. In this state, the difference absolute value calculator 101 determines the luminance signal Y _i of the decoded reproduction image signal VK ′ and the luminance signal Y B of the blue signal _B.
The difference absolute value D _Y = | Y _i −Y _B | is output, and the comparator 104 compares the difference absolute value D _Y with a predetermined threshold value T _HY to determine if D _Y <T _HY . When satisfied,
HIGH level is output to the output terminal. The same applies to the other color signals Cr and Cb. AND gate circuit 1
07 outputs HIGH level when all the inputs are HIGH level, whereby the switch circuit 108 is connected to the terminal b side and outputs the _{blue signal} B (Y _B , C _rB , C _bB ) to the output terminal OUT.
Output to. Thus, in the decoded reproduction image signal VK ′ at the input terminal IN, the pixel signal VK ′ whose difference from the blue signal B is within the predetermined range H in luminance, hue and saturation is made the same as the blue signal B.

FIG. 6 is a diagram showing the structure of the image coding system of the second embodiment. In the figure, 8 is a key detection unit, 81 is an encoder (ENC), 100 is a block coding unit, and 82 is a decoder ( DEC), 300 is a block decoding unit, 52 is a background signal generating unit (BGG), and 7 is a combining unit. In addition, this
FIG. 2 shows a principle configuration diagram corresponding to the second embodiment. Figure 6
In the above, the key detection unit 8 is a circuit that outputs the HIGH-level key detection signal KD while detecting the key signal K, and outputs the LOW-level key detection signal KD otherwise. Such a key detection unit 8 may have, for example, a configuration substantially similar to that of the distortion removal filters 10 and 30 of FIG.
Connect the IGH level, and the threshold T _H is a predetermined range H may be configured to be narrower to form. Further, the block encoding unit 100 of the second embodiment may be anything as long as it encodes the image signal VK in block units, and does not need to be the interframe encoding method. Block decoding unit 3
The same applies to 00.

FIG. 7 is a diagram for explaining the operation of the image coding system of the second embodiment. In the timing chart of FIG. 7A, the input image signal VK is synchronized with the pixel clock signal CLK according to a predetermined scanning direction, and here, a part of the key signal K and a part of the image signal V of a person or the like are included. Are mixed. The key detection signal KD is at the HIGH level in the portion of the key signal K and at the LOW level in the other portions. The synthesizing unit 7 selects the background signal BG when the key detection signal KD is at the HIGH level, and selects the reproduced image signal VK ′ when the key detection signal KD is at the LOW level. Therefore, a composite signal VB of the image signal V of the person or the like and the background signal BG is obtained at the output of the combining unit 7.

FIG. 7B shows the above relationship based on the image signal VK.
It is a plan view of / VK '. When the scanning lines are sequentially scanned in the direction of arrow b, the key detection unit 8
Accurately preliminarily separates the portion of the key signal K from the portion of the image signal V of a person or the like based on the input image signal KV that has not been subjected to coding distortion. Therefore, even if the pixel signal Kn ′, which should originally be the key signal K, deviates from the key signal K on the decoded reproduction image signal VK ′ as a result of being subjected to the encoding distortion, the background of the synthesizing unit 7 is It has no effect on synthesis.

FIG. 8 is a diagram showing the structure of the image coding system of the third embodiment. In the figure, the same symbols as in FIG. 6 indicate the same or corresponding portions, and 9 is an edge elimination filter. Second above
According to the embodiment, the background is properly synthesized regardless of which block encoder is used.
In this state, the coding efficiency of the block encoder is further improved.

By the way, generally, in an orthogonal transformer such as a discrete cosine transformer, when a flat pixel block is transformed, the output information amount is small, but the pixel block at the edge portion where the person and the background are switched is transformed. When this is done, the amount of high-frequency components is large, so the amount of information output is large. Therefore,
In the third embodiment, the pixel block including the image signal V and the key signal K is provided in the preceding stage of the block encoding unit 100.
An edge erasing filter 9 for erasing edges by replacing the key signal K with a signal based on the image signal V is provided.

FIG. 9 is a diagram for explaining the operation principle of the edge elimination filter of the third embodiment. The pixel block of the input image signal VK is divided into three patterns. That is, a pixel block including only the image signal V of a person, the image signal V
And a key signal K, and a pixel block including only the key signal K. Edge elimination filter 9
Allows the pixel block consisting of only the image signal V or the key signal K to pass through as it is, and the pixel block consisting of the image signal V and the key signal K is filled with a signal based on the image signal V. It loses the edge.

FIG. 10 shows an edge elimination filter according to the third embodiment.
In the block diagram of the figure, 91 is a control unit, and 92 is a key.
The detection unit, 93 to 95 are switch circuits, and 96 is a video battery.
FA, 97 is a comparator, 98 is a counter, 99 is an average value times
It is a road. The block of the image signal VK is stored in the video buffer 96.
When writing the clock, the control unit 91 connects the switch circuit 93.
It is connected to the point a side. As a result, the key detection unit 92
When the key signal K in the image signal VK is detected, the HIGH level is detected.
Output signal. This HIGH level signal is
The level is inverted by the barter circuit, and the average value circuit 99 and the cow
Function to prevent operation of the input 98. That is, this
The average value circuit 99 accumulates only the value of the image signal V of a person or the like.
The product is added and the counter 98 displays the number of image signals V of a person or the like.
It works like counting only. The comparator 97 is a cow
Counter 98 count value CD and predetermined threshold value T_HCompare with
Or, if CD> T_H(For example, T_H= 2) is satisfied
When added, a HIGH level signal is output. Also, C
D> T _HWhen the condition of is not satisfied, that is, images of people, etc.
When the number of signals V does not exceed 2 in the entire block
Is considered noise and outputs a LOW level signal.
It

When the writing to the video buffer 96 is finished, the control unit 91 checks the output of the comparator 97. C
For D> T _H, a pixel block consisting of pixel blocks or image signals consisting of the image signal V only V and key signal K such as a person. Therefore, the control unit 91 controls the switch circuit 9
The connection of 3 is set to the b side and the switch circuit 94 is closed. As a result, the key detection unit 92 causes the video buffer 96 to
When the key detection unit 92 detects the key signal K, the switch circuit 95 checks the read block data of b.
Side, otherwise it is connected to side a. At this time, the count value CD is input to the average value circuit 99, so that the average value circuit 99 outputs the addition average value AD of the image signals V of a person or the like. Therefore, the output terminal OU
At T, a pixel block including only the image signal V of a person or the like is output as it is, and a pixel block including the image signal V and the key signal K is output with the portion of the key signal K replaced with the addition average value AD.

If CD> T _H , the key signal K
It is a pixel block consisting of only. Therefore, the control unit 91
Switches the switch circuit 93 to the side b and switches the switch circuit 9
Open 4 As a result, the switch circuit 95 remains connected to the a side. Therefore, the pixel block including only the key signal K is directly output to the output terminal OUT.

In this way, the edge block is flattened by the edge erasing filter 9, which improves the coding efficiency of the block coding unit 100. On the other hand, the background processing of this edge block is performed by the key detection signal K which is side information.
According to D, it is properly performed in the synthesizing unit 7. In the above embodiment, the distortion removal filters 10 and 30 have a predetermined range H.
Is rectangular, but is not limited to this. For example, the distortion removal filters 10 and 30 have the decoded reproduction image signal V at the input terminal IN.
_{K'(= Y i, C ri} , C bi) can be configured in ROM as an address input to, any address space to the _{ROM (Y i, C ri,} C bi) is output becomes HIGH level By writing the data in advance, the predetermined range H can be formed in an arbitrary shape.

[0034]

As described above, according to the present invention, the coding distortion is removed from the pixel signal VK 'locally reproduced by the interframe coding unit 1 and the pixel signal VK' reproduced by the interframe decoding unit 3. Since the distortion removing filters 10 and 30 are provided, the coding distortion can be removed particularly from the background portion of the block including the edge, and the image quality of the output image is improved.

[0035]

[Brief description of drawings]

FIG. 1 is a principle configuration diagram of the present invention.

FIG. 2 is a principle configuration diagram of the present invention.

FIG. 3 is a diagram showing a configuration of an image coding system according to a first embodiment.

FIG. 4 is a diagram for explaining the operation principle of the distortion removal filter of the first embodiment.

FIG. 5 is a block diagram of a distortion removal filter of the first embodiment.

FIG. 6 is a diagram showing a configuration of an image coding system according to a second embodiment.

FIG. 7 is a diagram for explaining the operation of the image coding system according to the second embodiment.

FIG. 8 is a diagram showing a configuration of an image coding system according to a third embodiment.

FIG. 9 is a diagram for explaining the operation principle of the edge cancellation filter of the third embodiment.

FIG. 10 is a block diagram of an edge cancellation filter according to a third embodiment.

FIG. 11 is a diagram showing a configuration of a conventional image coding method.

FIG. 12 is a diagram illustrating a problem of a conventional image coding method.

[Explanation of symbols]

1 Encoding section 3 Decoding section 5 Synthesis Department 7 Composition Department 8 key detector 10,30 distortion removal filter 100 block coding unit 300 block decoding unit

─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Osamu Kawai 1015 Kamiodanaka, Nakahara-ku, Kawasaki City, Kanagawa Prefecture, Fujitsu Limited (72) Inventor, Kiichi Matsuda, 1015 Kamedotachu, Nakahara-ku, Kawasaki City, Kanagawa Prefecture, Fujitsu Limited ( 56) References JP-A-1-101087 (JP, A) JP-A-57-103481 (JP, A) JP-A-3-46468 (JP, A) JP-A-63-199589 (JP, A) JP-A Flat 3-89792 (JP, A) (58) Fields investigated (Int.Cl. ⁷ , DB name) H04N ^7/ 24-7/68 H04N ^7/ 14-7/173 H04N 5/262-5/278

Claims (2)

(57) [Claims] 1. In an image coding method in which an image signal background-separated by a key signal is inter-frame coded in block units by a coding unit, a pixel signal locally reproduced in the coding unit is within a predetermined range including the key signal. An image coding method characterized by comprising a distortion removal filter for making color pixel signals the same as key signals. 2. An image decoding system in which an image signal encoded by the system of claim 1 is inter-frame decoded by a decoding unit in units of blocks, wherein a predetermined range including a key signal is included in a pixel signal locally reproduced in the decoding unit. An image decoding method characterized by comprising a distortion removal filter for making the pixel signals of the colors inside the same as the key signals.