JP3277815B2 - Image data encoding apparatus / method and decoding apparatus / method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image data encoding apparatus / method and decoding apparatus / method, for example, a recording / reproducing system for recording / reproducing image data via a recording medium such as a video tape or a video disk, This is suitable for application to a television broadcast or a video conference system.

[0002]

2. Description of the Related Art FIG. 7 shows a conventional general image synthesizing method. That is, when generating the composite image I, the foreground image F and the background image B are composited at a certain mixing ratio. This mixture ratio is called a contribution ratio α (Key signal, Mate value). Here, if α = 0, it is a foreground image, if α = 1, it is a background image, and the boundary between the foreground image and the background image is 0 <
α <1. When this synthesis is expressed by an equation, the following equation (1) is obtained: I = αF + (1−α) B (1)

Now, the foreground image F and the contribution ratio α are generated, and the synthesis with the background image B is not always performed immediately or on the spot. It is synthesized at the place.

[0004] For this reason, as shown in FIG. 8, foreground image F and contribution rate α generated on the encoding side are encoded by encoders 311 and 312, respectively, and further multiplexed by multiplexer 320. , Recorded on the recording medium 330,
Alternatively, transmission is performed through a transmission path.

[0005] As shown in FIG. 9, on the decoding side,
The information is reproduced from the recording medium 330, or
The foreground image F and the contribution rate α are obtained by receiving from the transmission path, demultiplexing by the demultiplexer 410 and decoding by the respective encoders 421 and 422, and combining them with the new background image B * by the combiner 430. I do. At this time, the combiner 430 uses two multipliers 431 and 432 and one adder 433 to obtain the composite image I using the above-described equation (1).
Generate

[0006]

By the way, in the above-mentioned conventional example, the foreground image F to be synthesized is obtained in advance. However, for example, when an image in which the foreground image F is not separated as shown in FIG. First estimates the contribution rate α from the input image I, uses the contribution rate α to separate the foreground image F from the background image B of the original image, and combines it with the new background image B *. When this composition is expressed by a formula, input image: I = αF + (1-α) B (2.1) Composite image: I * = αF + (1-α) B * (2.2) Equation (2) is obtained.

[0007] In such an example, a chroma key method is used to convert a blue-back image (an image with a solid blue background).
It is often seen when the contribution ratio α is calculated and then synthesized. As described above, when the foreground image F is separated and then synthesized, encoding the foreground image F and the contribution rate α as in the conventional example for transmission or recording causes waste. Because, as shown in FIG. 11, in order to obtain the foreground image F from the input image I, in addition to the arithmetic processing units 301 and 302 for calculating the background image B and the contribution rate α from the input image I,
F = (I− (1−α) B) / α (3) for finding the foreground image F However, when α = 0 and 1, F = I.

A calculation processing unit 30 for calculating the following equation (3)
3 is required.

However, as is apparent from equation (2.2),
Since the foreground image αF is required for the synthesis, there is a problem in that the foreground image αF must be calculated again from the foreground image F and the contribution rate α, which causes a problem.

An object of the present invention is to provide an image data encoding apparatus / method and a decoding apparatus / method which solve the above-mentioned problems.

[0011]

According to the present invention, in order to solve the above-mentioned problems, when the foreground image F is separated and when the foreground image F is not separated, the foreground image F is appropriately separated in each case. Conversion to image data and reduction in the amount of calculation when encoding. Further, a flag indicating how the image data is converted is encoded and multiplexed with the image data and α. On the decoding side, an appropriate combining method is selected in accordance with the flag, and the amount of calculation for combining is reduced.
Accordingly, in the pre-processing when encoding the image data and the post-processing after the decoding, unnecessary computation can be eliminated. Further, the coding efficiency of image data is also improved.

That is, the image data encoding apparatus according to the present invention comprises a means for obtaining a contribution rate α and a background image B from an input image I, and a foreground image αF obtained by correcting the influence of the background image B from the input image I. And (A) a combination of the foreground image F and the contribution rate α, and (B) a combination of the foreground correction image αF obtained by correcting the influence of the background image B from the input image I and the contribution rate α. A controller that selects from among two or more types of combinations and outputs a selection flag, an encoding unit that encodes each of the combinations selected by the controller, and a multiplex of the encoded combination and the selection flag. And multiplexing means for converting and outputting.

Further, the image data encoding method according to the present invention comprises at least (A) a combination of a foreground image F and a contribution rate α, and (B) a foreground obtained by correcting the influence of the background image B from the input image I. A selection is made from two or more combinations including a combination of the corrected image αF and the contribution ratio α, a selection flag is output, and the contribution ratio α, from the input image I according to the combination indicated by the selection flag, One of the foreground corrected image αF in which the influence of the foreground image F and the background image B is corrected is obtained, encoded, and the combination of the encoded image information and the selection flag are multiplexed and output. And

The image data decoding apparatus according to the present invention comprises at least (A) a combination of a foreground image F and a contribution rate α, and (B) a foreground correction in which the influence of the background image B from the input image I is corrected. A selection is made from two or more combinations including a combination of the image αF and the contribution ratio α, a selection flag is output, and the selected combination is encoded,
Means for demultiplexing the image data generated by multiplexing the encoded combination and the selection flag, and the combination of the demultiplexed image data based on the selection flag obtained by the demultiplexing means. And a decoding means for decoding any of the coded data of the contribution ratio α, the foreground image F, and the foreground corrected image αF obtained by the demultiplexing means. .

Further, the image data decoding method according to the present invention comprises at least (A) a combination of a foreground image F and a contribution rate α, and (B) a foreground correction in which the influence of the background image B from the input image I is corrected. A selection is made from two or more types of combinations including a combination of the image αF and the contribution rate α, a selection flag is output, the selected combinations are coded, and the coded combination and the selection flag are coded. Demultiplexes the image data generated by multiplexing, the controller determines which combination of the demultiplexed image data is based on the selection flag obtained by demultiplexing, and the controller determines the combination. It is characterized in that any one of the image data of the contribution rate α, the foreground image F, and the foreground correction image αF is decoded.

[0016]

[0017]

[0018]

[0019]

[0020]

[0021]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Some preferred embodiments of an image data encoding apparatus / method and decoding apparatus / method according to the present invention will be described below with reference to the drawings.

FIG. 1 and FIG. 2 are views showing a first embodiment of the present invention.

In the first embodiment, the background image B and the contribution ratio α are calculated from the input image I by the first and second arithmetic processing units 11 and 12, as in the encoding device 10 shown in FIG. And the third arithmetic processing unit 13 calculates the foreground image αF from the input image I, the calculated background image B, and the contribution ratio α.
Ask for.

As described above, when the foreground image F is separated from the input image I, in the operation of obtaining the corrected foreground image αF without obtaining the foreground image F, the above-described division of the equation (3) in the conventional example is performed. ΑF can be obtained without using the following equation (4).

ΑF = (I− (1−α) B) (4) However, when α = 0 and 1, αF = I.

Although the contribution rate α may be encoded, (1-α) may be encoded instead of the contribution rate α.

Then, the generated foreground image αF and contribution ratio α
Are encoded by encoders 14 and 15, respectively, and are multiplexed by a multiplexer 16 to be recorded on a recording medium 17 or transmitted through a transmission path.

On the decoding side, the multiplexed foreground image αF and the information of the contribution rate α are reproduced from the recording medium 17 or received from the transmission path, and the reverse is performed, as in the decoding device 20 shown in FIG. The foreground image αF and the contribution ratio α are obtained by the separation by the multiplexer 21 and the decoding by the respective encoders 22 and 23, and the synthesized image I is obtained by synthesizing them with the new background image B * by the synthesizer 24. Generate *.

At this time, in the synthesizer 24, a single multiplier 25 and an adder 26 are used to obtain a synthesized image using the following equation (4): I * = αF + (1−α) B * (4) Generate I.

As described above, since αF is decoded on the decoding side, there is no need to multiply the contribution rate α by the foreground image F. Also, since the contribution rate α is only used for multiplication of the new background image B * and (1−α), (1) is used instead of the contribution rate α.
If (-α) is encoded, the calculation for (1-α) can be omitted.

Next, FIG. 3 and FIG. 4 are views showing a second embodiment of the present invention.

In the second embodiment, the background image B and the contribution ratio α are calculated from the input image I by the first and second arithmetic processing units 31 and 32 as in the encoding device 30 shown in FIG. Then, the calculated background image B and contribution ratio α are encoded by the encoders 33 and 34, respectively, and the input image I is encoded by the encoder 35 as it is. Then, the background image B, the contribution rate α and the input image I encoded by the encoders 33, 34, and 35 are multiplexed by the multiplexer 36 and recorded on the recording medium 37 or transmitted through a transmission path. Perform transmission.

As described above, in the second embodiment, the input image I is encoded without obtaining the foreground image F from the input image I. Further, the background image B is separated from the input image I and encoded. Further, the contribution rate α is encoded and multiplexed and recorded on the recording medium 37, or
Transmission is performed through a transmission path.

In the second embodiment, the background image B is encoded. Therefore, when a large amount of data is not required for encoding the background image B, for example, when the input image I is a background image such as a blue background image. It is significant when B is uniform or simple.

On the decoding side, as in the decoding device 40 shown in FIG.
And the information of the input image I is reproduced from the recording medium 37 or received from the transmission path, demultiplexed by the demultiplexer 41, and decoded by the respective decoders 42, 43, 44 into the background image B,
The contribution ratio α and the input image I are decoded. The arithmetic processing unit 45 removes the influence of the background image B from the input image I and obtains the foreground image αF using the three data of the background image B, the contribution ratio α, and the input image I. The combiner 46 multiplies the result of multiplication of the new background image B * obtained by the multiplier 47 by (1−α) with the foreground image α calculated by the arithmetic processing unit 45.
By adding to F by the adder 48, a composite image I * is generated.

As described above, in the second embodiment, even when the result of synthesis based on the contribution rate α calculated by the encoding device 30 is not desirable, calculation is performed by another method from the input image I and the background image B. There is an advantage that can be fixed. In that sense, the information to be encoded is redundant, but if the background image B is nearly uniform and can be encoded with a very small amount of information compared to the input image I and the contribution rate α, the efficiency of encoding is reduced. There is no problem from.

Next, in the third embodiment of the present invention shown in FIGS. 5 and 6, the encoding apparatus 100 determines a combination of image data suitable for an input image, and uses the combination information as a flag. Encode. Specific examples are (1) foreground image F and contribution rate α (2) foreground image αF and contribution rate α (3) input image I, contribution rate α, and background image B.

As shown in FIG. 5, the encoding apparatus 100 according to the third embodiment has first and second switches 111 and 111 controlled by a controller 110 to which an input image (I or F) is supplied. 112, first and second changeover switches 121 and 122, and first to third arithmetic processing units 131, 132, 133, first to fourth
, Encoders 141, 142, 143, and 144 and a multiplexer 150.

In the encoding device 100, the controller 110 operates according to the input image and the selection of the user.
From the above three combinations (1), (2) and (3),
An optimal combination is selected, and the foreground image F (in the case of 1) or the input image I (in the case of 3) is encoded as it is by controlling the first and second changeover switches 121 and 122. And calculating the corrected foreground image αF from the input image I (in the case of 2), and controls the first and second switches 111 and 112 to encode the background image B ( 3) or not. Further, the controller 110 outputs this selection information as a flag. The selection information is assigned to a 2-bit flag, for example, as shown in Table 1 below.

[0040]

[Table 1]

The first arithmetic processing unit 131 performs the first
From the input image (I) supplied via the switch 111 of FIG. Further, the second arithmetic processing unit 132 calculates the contribution rate α from the input image (I or F). Further, the third arithmetic processing unit 133 controls the input image I supplied via the first switch 121.
The foreground image αF is calculated from the calculated background image B and the contribution ratio α using the above-described equation (4).

The first encoder 141 encodes the background image B calculated by the first arithmetic processing unit 131 and supplies it to the multiplexer 150 via the second switch 112. Further, the second encoder 142 encodes the contribution rate α calculated by the second arithmetic processing unit 132 and supplies the result to the multiplexer 150. The second encoder 142
Then, (1-α) may be encoded instead of the contribution rate α. In addition, the third encoder 143 controls the input image (I or F) supplied via the second switch 122.
Alternatively, the foreground image αF calculated by the first arithmetic processing unit 131 is encoded and supplied to the multiplexer 150.
Further, the fourth encoder 144 is provided with the controller 1
The selection information (flag) given by 10 is encoded and supplied to the multiplexer 150.

Then, the multiplexer 150 is connected to the first
, The background image B encoded by the fourth encoders 141, 142, 143, and 144, the contribution rate α, and the input image (I
Or F) or multiplexes the foreground image αF with the selection information (flag) and records it on the recording medium 160 or transmits it through a transmission path.

The decoding device 2 according to the third embodiment
6 includes a demultiplexer 210, first to fourth decoders 221, 222, 223, and 224, an arithmetic processing unit 230, and a combiner 240 as shown in FIG. First and second switches 261, 262 and first and second changeover switches 27
1,272.

The decoding apparatus 200 reproduces the multiplexed background image B, contribution ratio α, input image (I or F) or foreground image αF, and information of selection information (flag) from the recording medium 160. Alternatively, the image data is received from the transmission path, separated by the demultiplexer 210, and the first to fourth decoders 221, 222, 223 and 224 receive the background image B, the contribution rate α, the input image (I or F) or the foreground The image αF and the selection information (flag) are decoded.

The controller 250 controls the fourth decoder 22
4 to determine which image combination of the above (1) to (3) is encoded based on the selection information (flag) decoded by the first and second switches 26 and 26.
1, 262 and first and second changeover switches 271, 27
2 is controlled.

That is, the image information includes (1) a foreground image F,
(2) Foreground image αF or (3) Input image I, the controller 250 sets the first and second changeover switches 271 and 271 so as to perform appropriate processing according to the selection information (flag). 272 is controlled.

(1) For the foreground image F, the multiplier 24
The foreground image αF is obtained by multiplying by 1 with the contribution rate α.

(2) In the case of the foreground image αF, no processing is performed.

(3) In the case of the input image I, the foreground image αF is obtained by the arithmetic processing unit 230 using the contribution ratio α and the background image B.

Further, the controller 250 has (3)
In the case of (1), the first and second switches 261 and 262 are set so that the background image B necessary for the processing in the arithmetic processing unit 230 is decoded by the first decoder 221 and given to the arithmetic processing unit 230. Control.

Then, the foreground image αF and the contribution ratio α are obtained by these processes, so that the synthesizer 240 synthesizes with the new background image B *.

That is, in the combiner 240, the contribution rate α decoded by the second decoder 222 and the first
And a foreground image αF selected by the second changeover switches 271 and 272, and a multiplier 242 and an adder 24.
3, a composite image I * is generated by using the above-described equation (4).

The present invention is not limited to the above-described embodiment. For example, the present invention corresponds to a combination of the two cases (1) to (3) and sets a 1-bit flag. Various modifications such as use are conceivable.

[0055]

As described above, according to the apparatus and method for encoding image data according to the present invention, the case where the foreground image F is separated and the case where the input image and the foreground image F are separated. If there is no such combination, an appropriate combination is selected from two or more combinations, and is output while being superimposed on the selection flag, so that the amount of calculation at the time of encoding can be reduced. Further, according to the image data decoding apparatus and the decoding method according to the present invention, it is possible to select an appropriate combination method on the decoding side based on the selection flag, and it is possible to reduce the amount of calculation when combining.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of an encoding device for describing a first embodiment of an image encoding method according to the present invention.

FIG. 2 is a configuration diagram of a decoding device according to the first embodiment.

FIG. 3 is a configuration diagram of an encoding device for explaining a second embodiment of the present invention.

FIG. 4 is a configuration diagram of a decoding device according to the second embodiment.

FIG. 5 is a configuration diagram of an encoding device for explaining a third embodiment of the present invention.

FIG. 6 is a configuration diagram of a decoding device according to the third embodiment.

FIG. 7 is a diagram showing a general method of synthesizing an image.

FIG. 8 is a basic configuration diagram on the encoding side in a conventional image combining method.

FIG. 9 is a basic configuration diagram on the decoding side in a conventional image synthesis method.

FIG. 10 is a diagram illustrating a method of synthesizing an image when an image whose foreground image is not separated is handled.

FIG. 11 is a basic configuration diagram on the encoding side when an image whose foreground image is not separated is handled.

[Explanation of symbols]

10, 30, 100 encoding device, 20, 40, 200
Decoding device, 11, 12, 13, 31, 32, 45, 1
31, 132, 133, 230 arithmetic processing unit, 14, 1
5,33,34,35,141,142,143,14
4 encoder, 16, 36, 150 multiplexer, 17,
37,160 recording medium, 21,41,210 demultiplexer, 22,23,42,43,45,221,22
2,223,224 decoder, 24,46,240 synthesizer, 111,112,261,262 switch, 1
21,122,271,272 Changeover switch

Claims (6)

(57) [Claims] 1. A contribution rate α and a background image B from an input image I
Means for obtaining a foreground image αF in which the influence of the background image B is corrected from the input image I; and (A) a combination of the foreground image F and the contribution rate α; and (B) the input image I A combination of a foreground corrected image αF in which the influence of the background image B is corrected and a contribution ratio α, and a controller that selects from two or more combinations and outputs a selection flag, and a combination selected by the controller. And a multiplexing unit that multiplexes the encoded combination and the selection flag and outputs the multiplexed combination and the selection flag. 2. At least: (A) a combination of a foreground image F and a contribution rate α; and (B) a combination of a foreground correction image αF obtained by correcting an influence of a background image B from an input image I and a contribution rate α. A selection flag is output from two or more combinations including the selection flag, and a selection flag is output. From the input image I, the influence of the contribution ratio α, the foreground image F, and the background image B is corrected in accordance with the combination indicated by the selection flag. A method for encoding image data, wherein one of the foreground-corrected images αF is obtained, encoded, and a combination of the encoded image information and a selection flag are multiplexed and output. 3. At least: (A) a combination of a foreground image F and a contribution ratio α; and (B) a combination of a foreground correction image αF obtained by correcting the influence of the background image B from the input image I and a contribution ratio α. Selection from among two or more types of combinations, including outputting a selection flag, encoding each of the selected combinations, and demultiplexing the image data generated by multiplexing the encoded combination with the selection flag. Means for determining the combination of the demultiplexed image data based on the selection flag obtained by the demultiplexing means; and the contribution rate α obtained by the demultiplexing means; A decoding device for decoding any one of the encoded data of the foreground image F and the foreground corrected image αF. 4. A means for calculating the foreground corrected image αF based on the selection flag, if necessary, and a synthesizing means for synthesizing the new background image B * and the foreground corrected image αF. 4. The method according to claim 3, wherein
3. An image data decoding device according to claim 1. 5. At least: (A) a combination of a foreground image F and a contribution rate α; and (B) a combination of a foreground correction image αF obtained by correcting the influence of the background image B from the input image I and a contribution rate α. Selection is made from two or more combinations including
A selection flag is output, the selected combination is encoded, and the coded combination and the selection flag are multiplexed to demultiplex the image data generated. The controller determines which combination the demultiplexed image data is based on, and decodes any one of the contribution ratio α, the foreground image F, and the foreground corrected image αF determined by the controller. A decoding method of image data, characterized by the following. 6. The foreground corrected image αF is calculated based on the selection flag if necessary, and a new background image B * is combined with the foreground corrected image αF. 2. A method for decoding image data according to item 1.