JP2004349785A - Apparatus and method for image processing - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus and an image processing method capable of specifying color corresponding to a background more exactly and performing the transparent processing of an image thereof comparatively easily. <P>SOLUTION: When the value of a B color of a chroma key color in an inputted pixel is larger than the one having a large value of an R color and a G color as other colors by a first value or larger (step 12:Y), and when an absolute value of the difference between the R color and the G color is equal to or lower than a second value (step S13: Y); the pixel is subjected to transparent processing as one belonging to a region corresponding to the background (step S15). The others are processed as foreground images (step S14). In this way, the number of comparing processings is reduced, and further, pixels corresponding to the background can be extracted more precisely. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image processing apparatus for making a region of a specific color transparent and combining a background image with this region.
[0002]
[Prior art]
For example, when performing image processing such as video editing, a technique is often used in which the background portion of one image is replaced with another image to combine two images into one image. In such a case, the color of the portion to be the background is treated as a transparent color, a foreground object such as a person in the image is extracted, and a separately prepared background image is combined with the transparent portion.
[0003]
FIG. 5 to FIG. 7 are for explaining such processing conventionally performed. 5 is an image obtained by arranging a person 12 in front of a blue background (background) 11 of a blue cloth, paper, or the like (hereinafter simply referred to as cloth) having a complementary color relationship with the skin color. FIG. 6 shows a mountain image 13 set as a background of the image of the person 12. When the person 12 is extracted from the image shown in FIG. 5 with the portion of the blue background 11 as a transparent color and is superimposed on the mountain image 13 shown in FIG. 6, as shown in FIG. An image 14 in which a person 12 is present is synthesized.
[0004]
Image processing using such chroma processing has been conventionally proposed (for example, see Patent Document 1). In this technique, a transparent process for making a designated range transparent from a natural image is performed. Specifically, the range in which the chroma processing (transparency processing) is to be performed is set to the upper limit or the upper limit for each of the components of R (red), G (green), and B (blue) using 24-bit digital image data. By specifying a lower limit and processing the range of colors determined by this as a transparent color, this portion is replaced with a separately created background image. Here, the portion of the image to be subjected to the background processing is configured by a blue cloth blue background, but the blue background portion is not necessarily configured by an image having exactly the same color tone and brightness. Actually, even in the image of the same blue background, there are portions where light is reflected and brightened, and conversely, there are portions where shadows are darkened. All of these parts need to be treated as a transparent color in the same manner.
[0005]
Therefore, in this proposal, by specifying the upper limit or the lower limit for each of the components of R, G, and B colors, the portion for performing the transparent processing of the image with a certain width is extended, and the portion where light is reflected and It is ensured that the processing as a transparent color can be reliably performed even in a portion where a shadow occurs. On the other hand, if the proposed processing is performed, there is a risk that the transparent processing is performed up to a color having no blue feature depending on the combination of the respective ranges of the R, G, and B colors. This will be described below.
[0006]
8 to 12 show the characteristics of the blue-back image under various lighting conditions. In these figures, the horizontal axis represents the components of each of the R, G, and B colors, and the vertical axis displays the values of each color in 256 levels. FIG. 8 shows the values of R, G, and B in an ideal blue illumination state used as a blue background. In the example shown in FIG. 8, only the B color component exists, and the R color and the G color having no blue feature are not included.
[0007]
On the other hand, FIG. 9 shows the characteristics of three colors in a state where light is reflected and brightened, and the component of B color is the same as that of FIG. The values of the colors G and G are large. FIG. 10 shows the characteristics of an image in which the values of R and G colors are the same as in FIG. 9, but the value of B color is reduced.
[0008]
If the chroma processing can be performed for each of the illumination states shown in FIGS. 8 to 10, the transparent processing can be performed even for the B color in which the illumination unevenness occurs. For this purpose, it is necessary to set upper and lower limits for each of the three components of R, G, and B colors. According to this proposal, the upper and lower limits set for these three color components are given by the following equation (1).
“0” <R <r
“0” <G <r
b <B <“255” (1)
Here, the symbols r and b indicate the values shown in FIG. 9 or FIG.
[0009]
[Patent Document 1]
JP-A-6-308938 (paragraph 0013, FIG. 1)
[0010]
[Problems to be solved by the invention]
As described above, according to this proposal, not only the original B color component but also the R and G color components are included in the range of the expression (1), so that each color having the characteristic of the chroma color is transparent. Can be processed as a color. However, on the other hand, all combinations of the three colors are processed as transparent colors within the range of the expression (1). As a result, there may be a case where a color that is originally different from the chroma color is processed as a transparent color.
[0011]
FIG. 11 shows an example in which a color which is originally different from the chroma color, that is, an image portion which does not have a characteristic of blue and which should not be transparently processed is treated as a transparent color. In this example, the R color has a lower limit value of “0” or a value close thereto, while the G color has an upper limit value. In this case, as a whole, the color becomes blue which is substantially different from the chroma color shown in FIG. In the proposed method, even such a color is treated as a transparent color, and is replaced with a background color.
[0012]
Further, in the equation (1), the value of the B color needs to be larger than the value “b”. As a result, there is a case where the dark B color that is shaded is excluded from the transparent colors.
[0013]
FIG. 12 shows an example in which an image portion to be subjected to the transparent processing is excluded from the transparent color depending on the lighting condition. In the case of the example shown in FIG. 12, the color components of R and G have the value “0” as their lower limit, and the value of the component of B has the value “b” as its lower limit. "Less than. Therefore, such an image portion is not treated as a transparent color and remains as a foreground color. When an image is photographed using a chromatic color cloth to make the background transparent as described above, a part of the image that is originally processed as the foreground color is transparent depending on the lighting (lighting) state. There is an inconvenience that a part of an image treated as a color or processed as a transparent color remains as a foreground color.
[0014]
Of course, setting the upper and lower limits individually for each of the three colors of R, G, and B in this way means that two values are determined for each of three colors per pixel, and six determinations are required for each. It becomes. Therefore, there is also a problem that the load of image processing increases when the resolution of the image is high.
[0015]
SUMMARY OF THE INVENTION It is an object of the present invention to provide an image processing apparatus and an image processing method capable of specifying a color corresponding to a background more accurately and performing transparent processing of the image portion relatively easily.
[0016]
[Means for Solving the Problems]
According to the first aspect of the present invention, (a) one specific color is used as a first primary color for replacing the background, two other colors are used as a second primary color and a third primary color, respectively, and these three primary colors are further used. When the minimum value and the maximum value that can be taken are the same, the value of the first primary color is larger than the larger value of the values of the second and third primary colors for each pixel constituting the predetermined image. A first comparing means for comparing whether the value is greater than or equal to a predetermined first value smaller than (b); and (b) an absolute value of a difference between the second and third primary colors for each pixel constituting the predetermined image. (C) comparing whether the value of the first primary color is equal to or less than a predetermined positive second value, and (c) determining that the value of the first primary color is the value of the second and third primary colors. And the second comparing means determines the absolute value of the difference between the second and third primary colors. To and a transparent color processing means for replacing the pixel of the predetermined background image only pixels determined to be equal to or less than the second value as a transparent color in the image processing apparatus.
[0017]
That is, in the first aspect of the present invention, one of the three primary colors is used as the first primary color for replacing the background. Pixels displayed in the first primary color are processed as a background color. Actual pixels combine three primary colors. The first comparing means compares the values of these three primary colors. As a premise, the minimum value and the maximum value that can be taken for each primary color are the same. The first comparing means compares whether the value of the first primary color is larger than the larger value of the second and third primary colors by a first value or more. This is because the pixel that matches the background color or the pixel that is to be determined to match needs to be the same color as the first primary color or a color similar thereto. Therefore, the first condition requires that the value of the first primary color be larger than the value of the second primary color and the value of the third primary color by a certain degree or more. The second comparing means compares whether the absolute value of the difference between the second and third primary colors is equal to or less than a predetermined positive second value. This focuses on the fact that, if the three primary colors of the image corresponding to the background are different due to excessive or insufficient illumination, the values of the two colors other than the first primary color do not greatly open. This is the same principle that an achromatic color is obtained when the three primary colors have equal values. However, given that the illumination is not pure white light, the values of the second and third primary colors have a predetermined width.
[0018]
According to a second aspect of the present invention, in the image processing apparatus according to the first aspect, foreground processing means for processing a pixel which is not processed by the transparent color processing means as a transparent color as a pixel constituting a foreground positioned in front of the background. are doing.
[0019]
That is, the second aspect of the present invention shows a case in which pixels that are not processed as a transparent color are not replaced with a background, and are used as they are as a foreground image located in front of the background.
[0020]
According to a third aspect of the present invention, in the image processing apparatus of the first aspect, the first primary color is blue, and the second primary color and the third primary color are different colors of red and green, respectively. It is characterized by:
[0021]
That is, the third aspect of the present invention shows a case where the first primary color is blue. Since blue is a color close to the color of a person's face, if it is extracted and processed as a transparent color, there is little possibility that a part of the face image will be replaced by the background. Of course, it is free to set the first primary color to a primary color other than blue.
[0022]
According to the fourth aspect of the present invention, (a) one specific color is used as a first primary color for replacing a background, and the other two colors are used as a second primary color and a third primary color, respectively, and further, these three primary colors are used. When the minimum value and the maximum value that can be taken are the same, the value of the first primary color is larger than the larger value of the values of the second and third primary colors for each pixel constituting the predetermined image. A first comparing step of comparing whether or not the value is greater than or equal to a predetermined first value smaller than (b) an absolute value of a difference between the second and third primary colors for each pixel constituting the predetermined image A second comparison step of comparing whether is less than or equal to a predetermined positive second value, and (c) the value of the first primary color is the value of the second and third primary colors in the first comparison step Is greater than or equal to the first value and the second and third values in the second comparing step. It is provided with the absolute value of the transparent color processing step of replacing a pixel of predetermined background image only pixels determined to be equal to or less than the second value as a transparent color differences in the color image processing method.
[0023]
That is, in the invention described in claim 4, one of the three primary colors is used as the first primary color for replacing the background. Pixels displayed in the first primary color are processed as a background color. Actual pixels combine three primary colors. In the first comparison step, the values of these three primary colors are compared. As a premise, the minimum value and the maximum value that can be taken for each primary color are the same. In the first step, a comparison is made as to whether the value of the first primary color is greater than the greater of the values of the second and third primary colors by a first value or more. This is because the pixel that matches the background color or the pixel that is to be determined to match needs to be the same color as the first primary color or a color similar thereto. Therefore, the first condition requires that the value of the first primary color be larger than the value of the second primary color and the value of the third primary color by a certain degree or more. In the second comparison step, it is determined whether or not the absolute value of the difference between the second and third primary colors is equal to or less than a predetermined positive second value. This focuses on the fact that, if the three primary colors of the image corresponding to the background are different due to excessive or insufficient illumination, the values of the two colors other than the first primary color do not greatly open. This is the same principle that an achromatic color is obtained when the three primary colors have equal values. However, given that the illumination is not pure white light, the values of the second and third primary colors have a predetermined width.
[0024]
According to a fifth aspect of the present invention, in the image processing method of the fourth aspect, a foreground processing step is provided in which a pixel not processed as a transparent color in the transparent color processing step is processed as a pixel constituting a foreground positioned in front of the background. are doing.
[0025]
That is, the invention according to claim 5 shows a case in which, in the image processing method according to claim 4, pixels that are not processed as a transparent color are not replaced with a background, but are used as they are as a foreground image located in front of the background. .
[0026]
BEST MODE FOR CARRYING OUT THE INVENTION
FIG. 1 shows an outline of an image processing method according to an embodiment of the present invention. In this image processing method, a background color is transparently processed from a predetermined image. Here, the chromaki color processed as a transparent color can be arbitrarily selected from R (red), G (green), and B (blue). In this embodiment, The B color which is close to the complementary color relationship to the color is set to this. It is assumed that the R, G, and B colors are each represented in 256 levels from a value “0” to a value “255”. Here, the value “0” indicates a state where the color is not included, and the value “255” indicates a state where the color is the maximum value.
[0027]
First, image data of one unprocessed pixel in an image to be processed is input (step S11). Then, the respective values of R, G, and B in the image data are checked, and the value of B as the chroma color is determined by the larger of the values of R and G as the other colors. Is also determined to be greater than or equal to the first value (step S12). This is the first condition of the transparent processing.
[0028]
The reason why the first condition of the transparent processing is required will be described. FIG. 8 shows the most ideal case as a background. In FIG. 8, the values of the R and G colors are both “0” and the B color is the maximum value, so that the first condition of the transparent processing is satisfied. In FIG. 9, the B color shows the maximum value, but the R color and the G color also have a certain value “r” at the same time. However, since these values “r” are smaller than the maximum value “255”, it can be regarded as image data of a portion where the chromaki color cloth is wrinkled and becomes whitish as a reflection of illumination. it can. Such image data also needs to be transparently processed. In the present embodiment, the first value is a predetermined value smaller than a value obtained by subtracting the value “r” from the maximum value “255”. This is a value smaller than the value "b" in FIG.
[0029]
In the case of FIG. 12, the values of the R color and the G color are both “0”, but the B color is not the maximum value, and is slightly smaller than the value “b”. Here, it is assumed that the value “b” does not include an infinitely small range, but is larger than the value “r” of the other colors appearing by the reflection shown in FIG. The image data of FIG. 12 in such a state is, for example, a state in which the chromaki color cloth is wrinkled and becomes a shadow of illumination. In such a case, it is necessary to perform the transparent processing. In the case of the example of FIG. 12, the first condition of the transparent processing is satisfied.
[0030]
On the other hand, in the case of the image data of the image as shown in FIG. 10, although the values of the R color and the G color both have the value “r”, the B color is also lower than the maximum value. The value is “b”. Here, the image content is such that light irradiation and shadow generation occur simultaneously at one pixel position, and it is clear that the image content is not a target to be subjected to the transparent processing. Therefore, in order to exclude such a case, the first condition of the transparent processing is that the difference between the value of the B color and the value of the next largest color is not less than the first value. Therefore, for example, if the color B is slightly lower than the maximum value due to some situation at the time of reading an image, the larger of the values of the colors R and G is similarly larger than this decrease. , The first condition of the transparent processing is similarly satisfied.
[0031]
When the input pixel satisfies the first condition of the transparent processing shown in step S12 of FIG. 1 (Y), it is checked whether or not the second condition of the transparent processing is further satisfied. The second condition of the transparent processing is that the absolute value of the difference between the R color and the G color is equal to or less than the second value (step S13). This is because even if R and G colors, which are unrelated to the chroma color, appear in image data, they often occur due to non-uniform illumination, so that only one color appears as an extremely large value. Is based on the inconceivable reason. In FIG. 11, the color is blue, which is close to a gray color, and the second condition of the transparent processing is a condition for not performing such transparent processing on the image data.
[0032]
FIG. 2 illustrates the first condition and the second condition of the transparent processing described above. Looking at opening V ₁ of the value of the larger of the B color and the other R color and G-color as black Maki color in the first condition of the transparency processing. We will be looking at the open V ₂ of black color R except Maki color and G color at a second condition for this. The method of extracting the pixel to be subjected to the transparent processing is completely different from the above-described prior art expression (1).
[0033]
Returning to FIG. 1, the description will be continued. The transparent processing is not performed on the image data of the pixel determined not to satisfy the first or second condition of the transparent processing in step S12 or S13. That is, the pixel is processed as constituting the foreground (step S14). For example, a mask process for not performing the transparency process may be performed on the pixel.
[0034]
For a pixel that further satisfies the second condition of the transparency processing shown in step S13 (Y), the pixel is subjected to transparency processing (step S15). Also in this case, it is not necessary to perform the transparent processing at that time, and for example, a mask processing for performing the transparent processing may be performed. It is determined whether there is a pixel to be processed next (step S16). If there is a remaining pixel (Y), the process returns to step S11 to perform the same processing for the next pixel. After the process of step S14 is performed, the process similarly proceeds to step S16. If there are no remaining pixels to be processed finally in step S16 (N), the series of image processing described above ends (END).
[0035]
【Example】
Hereinafter, the present invention will be described in more detail with reference to examples.
[0036]
FIG. 3 shows an outline of the configuration of an image processing apparatus according to an embodiment of the present invention. The image processing apparatus 21 of the present embodiment has the same configuration as a normal personal computer or workstation, and includes a CPU (central processing unit) 22. The CPU 22 includes a program recording medium 24, a working memory 25, an image input unit 26, a converted image storage unit 27, and an image output unit 28 via a bus 23 such as a data bus.
[0037]
Here, the program recording medium 24 is a recording medium in which a series of image processing according to the present embodiment described later in detail is stored as a control program. As the recording medium, various data writable recording media such as a magnetic disk, an optical disk, a ROM (Read Only Memory) or a RAM (Random Access Memory), not shown, can be used. The working memory 25 is a memory area for temporarily storing various data temporarily required when executing the control program stored in the program recording medium 24. The image input unit 26 inputs an image to be subjected to the chroma processing. There are various image input modes. For example, image data may be input using an external storage medium such as a floppy disk or an optical disk, or may be connected to a communication network such as the Internet to obtain necessary image data. The image data may be obtained using a USB (Universal Serial Bus) or a dedicated cable. The converted image storage section 27 is a section for storing the converted image data, and can be constituted by various storage media such as a magnetic disk and a RAM. The image output unit 28 is a unit that outputs the image data stored in the converted image storage unit 27. The data may be output to the above-described recording medium, or may be output to a cable or a communication network.
[0038]
FIG. 4 shows a flow of image processing by the image processing apparatus of the present embodiment. The CPU 22 shown in FIG. 3 of the image processing device 21 inputs the image data of one pixel input from the image input unit 26 (Step S31). The image data for one pixel represents the three primary colors of R, G, and B in 256 gradations. In this embodiment, the B color is a color to be subjected to the chroma processing (transparency processing).
[0039]
The CPU 22 compares the R color value and the G color value in the input image data of one pixel (step S32). As a result of this the case towards the R color value is greater than the value of G color (Y), from the values of R color values values V ₂ and B color values are subtracted G color from the R color value calculates a subtracted value _{V 1} (see FIG. 2) (step S33). This is when the value of R color is less than or equal to the value of G color against (step S32: N), G color values from the color value V ₂ obtained by subtracting the value of R color B value Then, a value V ₁ (see FIG. 2) obtained by subtracting the value of the G color from is calculated (step S34).
[0040]
Then, in the next step S35, it is determined whether or not both the first and second conditions of the above-described transparent processing are satisfied. First, it is confirmed greater than the first value MIN _BR value V ₁ is predetermined as a first condition transparent process. Greater if the value V ₂ as a second condition transparent treatment is small is confirmed than the second value MAX _RG a predetermined. If any one of the first and second conditions is not satisfied (step S35: N), the values of the three primary colors of R, G, and B as the image data of the one screen Is output as the foreground color as it is (step S36).
[0041]
On the other hand, when the image data of the pixel satisfies both the first and second conditions of the transparent processing at the same time (step S35: Y), this is replaced with the image data of the background pixel (step S37). ). That change in the value R _BG image data of the background in the corresponding position for the R color, to change the value G _BG image data of the background in the corresponding position for the G color, background corresponding position for B color _Is changed to the value _BBG of the image data. Then, the image data of the pixel after the change is output (step S36). Thereafter, it is checked whether or not there is a remaining pixel to be processed (step S38). If there is (Y), the process returns to step S31. When there are no more remaining pixels to be processed (step S38: N), a series of processing ends (end).
[0042]
In the present embodiment, the first value MIN _BR used for the first condition of the transparent processing is set as a value larger than the second value MAX _RG used for the second condition of the transparent processing. It is not limited.
[0043]
Further, in the embodiments and examples described above, the chroma color is B color, but the R color or G color may be designated as the chroma color. Naturally, the present invention can be similarly applied to other three primary color concepts such as cyan, magenta, and yellow.
[0044]
【The invention's effect】
As described above, according to the first and fourth aspects of the present invention, the first and second conditions are set in consideration of the situation where the illumination actually illuminating the cloth or the like to be replaced with the background color (transparent color) is close to white light. Is set, the transparent color is specified more accurately, and this enables the chroma synthesis of a high-quality image. In addition, two comparisons need to be performed for each pixel, and the comparison processing is simplified as compared with the related art, so that high-speed processing of an image can be performed, and the resolution can be increased accordingly. As a result, the edge of the background color (transparent color) is sharpened, so that the image quality can be improved in this sense.
[Brief description of the drawings]
FIG. 1 is a flowchart showing an outline of an image processing method according to an embodiment of the present invention.
FIG. 2 is an explanatory diagram showing a first condition and a second condition in the present invention.
FIG. 3 is a block diagram illustrating an outline of an image processing apparatus according to an embodiment of the present invention.
FIG. 4 is a flowchart showing a flow of image processing by the image processing apparatus of the present embodiment.
FIG. 5 is a plan view of an image obtained by placing a person in front of a blue background of a blue cloth.
FIG. 6 is a plan view showing an image of a mountain set as a background.
7 is a plan view of an image obtained by synthesizing both FIGS. 5 and 6. FIG.
FIG. 8 is a characteristic diagram showing respective values of R, G, and B colors when an image portion to be subjected to a transparent process is in an ideal illumination state.
FIG. 9 is a characteristic diagram showing respective values of R, G, and B colors in the first example in which reflection by illumination of an image portion to be subjected to a transparent process occurs.
FIG. 10 is a characteristic diagram illustrating respective values of R, G, and B colors in a second example in which a shadow due to illumination of an image portion to be subjected to a transparent process occurs.
FIG. 11 is a characteristic diagram illustrating values of R, G, and B colors in an example of an image that should not be subjected to the transparent processing.
FIG. 12 is a characteristic diagram of R, G, and B colors illustrating an example in which an image portion to be subjected to a transparent process is excluded from a transparent color depending on an illumination state.
[Explanation of symbols]
21 image processing device 22 CPU
23 bus such as data bus 24 program recording medium 25 working memory 26 image input unit 27 converted image storage unit 28 image output unit

Claims (5)

One specific color is used as a first primary color for replacing the background, the other two colors are used as a second primary color and a third primary color, respectively, and the minimum and maximum values that can be taken by these three primary colors are the same. When it is assumed that the value of the first primary color is larger than the larger of the values of the second and third primary colors for each of the pixels constituting the predetermined image, and is equal to or larger than the predetermined first value. First comparing means for comparing whether or not it is greater;
Second comparing means for comparing whether or not the absolute value of the difference between the second and third primary colors is equal to or less than a predetermined positive second value for each pixel constituting the predetermined image;
The first comparing means determines that the value of the first primary color is greater than the values of the second and third primary colors by a first value or more, and the second comparing means determines that the value of the second and third primary colors is larger. An image processing apparatus comprising: a transparent color processing unit that replaces only pixels determined to have an absolute value of a difference of not more than a second value with pixels of a predetermined background image as transparent colors. 2. The image processing apparatus according to claim 1, further comprising foreground processing means for processing pixels which are not processed by the transparent color processing means as a transparent color as pixels constituting a foreground positioned in front of the background. 2. The image processing apparatus according to claim 1, wherein the first primary color is blue, and the second primary color and the third primary color are respectively different ones of red and green. One specific color is used as a first primary color for replacing the background, the other two colors are used as a second primary color and a third primary color, respectively, and the minimum and maximum values that can be taken by these three primary colors are the same. When it is assumed that the value of the first primary color is larger than the larger of the values of the second and third primary colors for each of the pixels constituting the predetermined image, and is equal to or larger than the predetermined first value. A first comparing step of comparing whether it is greater or not;
A second comparing step of comparing whether or not the absolute value of the difference between the second and third primary colors is equal to or less than a predetermined positive second value for each pixel constituting the predetermined image;
In the first comparing step, it is determined that the value of the first primary color is larger than the values of the second and third primary colors by a first value or more, and in the second comparing step, the second and third primary colors are determined. A transparent color processing step of replacing only pixels determined as having an absolute value of the difference of not more than a second value with pixels of a predetermined background image as transparent colors. 5. The image processing method according to claim 4, further comprising a foreground processing step of processing pixels not processed as a transparent color in the transparent color processing step as pixels constituting a foreground positioned in front of the background.

Images