JP2006025448A - Apparatus and method for generating key signal, image compositing device, and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus and method for automatically calculating the background color of a local area and removing the background color from an image. <P>SOLUTION: A background color calculation section 6 uses information on the boundary of a foreground and a background constituting the input of an image 1 and the input of the image 1 to calculate a local background color in pixels close to the boundary of the foreground and the background. Furthermore, a background color removal section 7 removes the influence of the background color from the image on the basis of the background color calculated by this background color calculation section 6. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image processing apparatus and method for removing the influence of a background color from an input image, a key signal generation apparatus and method for generating a key signal from an image composed of a foreground and a background, and a foreground and a second image of a first image. The present invention relates to an image synthesizing device that synthesizes the background and a recording medium.

  Conventionally, for example, processing for synthesizing the foreground of the first image a and the second image b consisting of the background has been performed when making a movie or creating another video file.

  For example, as shown in FIG. 12, the foreground of the first image a and the background of the second image b out of the first image a composed of the foreground and the background and the second image b composed only of the background. When combining, a mask image c for extracting the foreground from the first image a had to be generated.

  The ratio of the foreground contributed in the pixel is called a key signal, but the mask image c can be easily generated by calculating the key signal. For example, the value of the key signal is 1 in the foreground, 0 in the background, and 0 or more and 1 or less in a region where the foreground and the background are mixed. The mask image c can be easily created as a monochrome image taking a value from 0 to 255 using, for example, the value of the key signal.

  However, there is a problem when the foreground image is simply extracted from the first image a using the key signal. Since the foreground color and background color are mixed in the vicinity of the foreground boundary, if the extracted foreground image and the second image b (background) are combined using a key signal, This is because the background color of the first image a remaining in the vicinity of the boundary and the new background color of the second image b are mixed.

  For this reason, when the two images a and b are mixed, it is desirable to delete the influence of the background color of the first image from the first image a in advance.

  On the other hand, there is a chroma key method as a conventional technique for calculating a key signal from an image. The chroma key determines an isosurface of the key value k in a color space (for example, RGB space), maps the color of the pixel of interest into the color space, and sets the key value k on the isosurface that passes through the mapped point. This is a method of calculating and using this as a key signal.

  Patents using a chroma key include a “pattern mask creation method” disclosed in Patent Document 1, a “cutout mask creation method” disclosed in Patent Document 2, and a “Patent Mask Creation Method” disclosed in Patent Document 3. Digital Line Chroma Key Device ", Vlahos. P, “Composing system for recorded signal color”, US-Patent. In addition, as a paper using a chroma key, there are many application examples including Non-Patent Document 1, but here, a well-known basic method will be described.

  The chroma key is a method in which a person or a car is photographed in a room painted in blue, for example, and the blue color as a background color is deleted from the completed image in a color space as shown in FIG.

  For example, when it is desired to calculate the key signal of the target pixel of the image in the upper right diagram of FIG. 13, the color of the target pixel is mapped to the RGB space. However, it is assumed that the background color is blue. In the RGB space, the background color is a point (0, 0, 1) as shown in the left diagram of FIG. In addition, the boundary between the foreground-only area and the area where the foreground and the background are mixed is defined as four side faces of a regular quadrangular pyramid generated by specifying the intercept b and the inclination m. Here, an isosurface with a key value k between 0 and 1 is a positive surface obtained by interpolating a point with a key value k of 0 and an isosurface with a key value k of 1. The four sides of the four-sided pyramid.

  In this case, in FIG. 13, the key value is included in the area below the isosurface of 1 and inside the cube represented by 0 ≦ R ≦ 1, 0 ≦ G ≦ 1, 0 ≦ B ≦ 1. The points are all points whose key value k is 1. Then, the key value k calculated for each pixel on the screen is used as a key signal α indicating the proportion of the influence of the foreground in the pixel, and (1-α) is multiplied by the blue color of the background to obtain the color of the pixel. By drawing, an image of only the foreground color from which the influence of the background color is deleted can be obtained.

  In this method, the operator can create an appropriate isosurface of the key by adjusting the intercept b and the inclination m while viewing the image from which the influence of the background color has been deleted using the generated key, for example. Adjusting the intercept b and the slope m is to determine the boundary of the region where the key value k is 1, so that the operator determines the foreground region in the RGB space.

Japanese Patent Laid-Open No. 1-155476 JP-A-4-68763 JP-A-3-5116 Mishima, “Chromakey software using polyhedral slices,” NICOGRAPH papers Vo. 8 pp-44-52 (1992)

  However, as described with reference to FIG. 12 above, when the background of the first image a includes a plurality of colors, for example, the operator selects the background color near the boundary of the foreground to be extracted from the first image a. Since it was necessary to specify the background color for each uniform area, the amount of work was very heavy.

  Further, as described with reference to FIG. 13, there is a problem with a method represented by a chroma key, which is a conventional technique, when calculating a key signal from an image. This is because, for example, a method of specifying a background color in advance such as a chroma key cannot cope with a case where many colors are included in the background near the boundary between the foreground and the background. Also, in the key creation process, if the background color (for example, blue) specified in the foreground is mixed, the key is created by misunderstanding the background even though it is an area in the foreground.

  These problems can be solved by creating a key for each local region near the foreground and background boundary.For example, the operator can foresee the foreground (foreground color) for each local region near the foreground and background boundary. The determination of the (region) and the background color (background region) makes the operator's work enormous when there are a large number of background colors near the boundary between the foreground and the background.

  Therefore, the present invention has been made in view of the above circumstances, and an object of the present invention is to provide an image processing apparatus and method for automatically calculating a background color of a local area and removing it from an image. An object of the present invention is to provide a key signal generation apparatus and method capable of automatically calculating a local foreground color and a background color in the vicinity of a background boundary and reducing an operator's workload when reproducing the key signal. Alternatively, it is an object of the present invention to provide an image composition device capable of reducing an operator's work amount at the time of image composition by using the key signal generation device and method, and to provide a program relating to the image processing method or the key signal generation method. It is an object of the present invention to reduce the amount of work for an operator by providing a recording medium on which is recorded.

  In the image processing apparatus according to the present invention, in order to solve the above-described problem, the background color calculation unit uses the information on the boundary between the foreground and the background constituting the image and the input image to detect the vicinity of the boundary between the foreground and the background. The local background color in the pixel is calculated, and the background color removing unit removes the influence of the background color from the image based on the background color calculated by the background color calculating unit.

  Here, the background color calculation means is information about the boundary between the foreground and the background, for example, information for dividing the foreground only area, the background only area, and the foreground and background mixed area, or parameters. Using the information about the key represented by the curved surface controlled by the, the pixels included in the area where the foreground and the background are mixed, from the pixels which exist in the vicinity of the pixel of interest and are included in the background only area The background color is automatically calculated. The background color removing unit automatically deletes the influence of the background color from the image using the calculated background color and key signal.

  In addition, in order to solve the above-described problem, the image processing method according to the present invention uses the information on the boundary between the foreground and the background constituting the image and the image, and uses a local background in pixels near the boundary between the foreground and the background. The color is calculated, and the influence of the background color is removed from the image based on the background color.

  Further, in order to solve the above problem, the key signal generation device according to the present invention calculates the boundary information between the foreground and the background by the boundary information calculation unit, and calculates the boundary information calculation unit by the boundary information calculation unit by the foreground color calculation unit. A local foreground color in the vicinity of the boundary between the foreground and the background is calculated using boundary information and the image, and the boundary information and the image calculated by the boundary information calculation unit by the background color calculation unit are used. A local background color near the boundary between the foreground and the background is calculated, and a key is generated using the foreground color calculated by the foreground color calculation means and the background color calculated by the background color calculation means in the key signal generation device. Generate a signal.

  Here, the foreground color calculating means and the background color calculating means are for dividing the information into the boundary between the foreground and the background, for example, the foreground only area, the background only area, and the foreground and background mixed area. For pixels included in the area where the foreground and the background are mixed using information or information about the key represented by the curved surface controlled by the parameter, the area only exists in the vicinity of the pixel of interest and is only in the foreground The foreground color is calculated from the pixels included in the image, and similarly, the background color is calculated from the pixels included in the region including only the background. The key signal generation unit calculates a key signal using the calculated foreground color and background color.

  Further, in order to solve the above problems, the key signal generation method according to the present invention calculates boundary information between the foreground and the background, and uses the boundary information and the image to determine the vicinity of the boundary between the foreground and the background. A local foreground color and a background color are calculated at, and the key signal is generated using the foreground color and the background color.

  In order to solve the above problem, the image composition device according to the present invention uses the information on the boundary between the foreground and the background constituting the image and the image, and uses the local background in the pixels near the boundary between the foreground and the background. Background color calculating means for calculating a color, and background color removing means for removing the influence of the background color from the image based on the background color calculated by the background color calculating means.

  Here, the background color calculation means is information about the boundary between the foreground and the background, for example, information for dividing the foreground only area, the background only area, and the foreground and background mixed area, or parameters. Using the information about the key represented by the curved surface controlled by the, the pixels included in the area where the foreground and the background are mixed, from the pixels which exist in the vicinity of the pixel of interest and are included in the background only area The background color is automatically calculated. The background color removing unit automatically deletes the influence of the background color from the image using the calculated background color and key signal.

  In order to solve the above problem, the image composition device according to the present invention includes boundary information calculation means for calculating boundary information between the foreground and the background, the boundary information calculated by the boundary information calculation means, and the image. Using the foreground color calculation means for calculating a local foreground color near the boundary between the foreground and the background, and the boundary information and the image calculated by the boundary information calculation means, Using the background color calculation means for calculating a local background color in the vicinity of the background boundary, the foreground color calculated by the foreground color calculation means and the background color calculated by the background color calculation means, the key signal A key signal generating means for generating the first image, and a combining means for combining the foreground of the first image and the background of the second image using the key signal generated by the key signal generating means. .

  Here, the foreground color calculating means and the background color calculating means are for dividing the information into the boundary between the foreground and the background, for example, the foreground only area, the background only area, and the foreground and background mixed area. For pixels included in the area where the foreground and the background are mixed using information or information about the key represented by the curved surface controlled by the parameter, the area only exists in the vicinity of the pixel of interest and is only in the foreground The foreground color is calculated from the pixels included in the image, and similarly, the background color is calculated from the pixels included in the region including only the background. The key signal generation unit calculates a key signal using the calculated foreground color and background color.

  Further, the recording medium according to the present invention uses a background color calculation step for calculating a local background color in pixels near the boundary between the foreground and the background using the information on the boundary between the foreground and the background constituting the image and the input image. And a background color removing step for removing the influence of the background color from the image based on the background color calculated by the background color calculating means.

  Further, the recording medium according to the present invention uses the boundary information calculation step for calculating the boundary information between the foreground and the background, the boundary information calculated in the boundary information calculation step, and the image, and uses the foreground and the background. A foreground color calculation step for calculating a local foreground color in the vicinity of the boundary of the image, and a local background near the boundary between the foreground and the background using the boundary information and the image calculated in the boundary information calculation step. A background color calculation step for calculating a color, and a key signal generation step for generating the key signal using the foreground color calculated in the foreground color calculation step and the background color calculated in the background color calculation step. The processing procedure to be provided is recorded.

  The image processing apparatus and method according to the present invention automatically calculates the background color of the local area and removes it from the image. Therefore, even when the background color includes a plurality of colors, the background color from the image is obtained. Can be removed with high accuracy. In addition, the amount of work can be greatly reduced as compared with the case where the operator designates a background color for each local region.

  Further, the key signal generation apparatus and method according to the present invention automatically calculates a local foreground color and a background color near the boundary between the foreground and the background to generate a key signal. There is an effect that an accurate key signal can be calculated. In addition, the amount of work can be greatly reduced as compared with the case where the operator designates the foreground color and the background color for each local area.

  In addition, since the image composition apparatus according to the present invention uses the image processing apparatus and method or the key signal generation apparatus and method, it is possible to reduce an operator's workload during image composition.

  In addition, the recording medium according to the present invention can widely provide an effect of reducing an operator's work amount when removing a background color or generating a key signal.

  DESCRIPTION OF EMBODIMENTS Embodiments relating to an image processing apparatus and method, a key signal generation apparatus and method, an image composition apparatus, and a recording medium according to the present invention will be described below.

  First, a background color removing apparatus to which the image processing apparatus and method used in the embodiment of the present invention are applied will be described with reference to FIGS.

  As shown in FIG. 1, the background color removing apparatus performs a background color removing process for removing the influence of the background color from the input image from the image input terminal 1, and therefore relates to the boundary between the foreground and the background constituting the image 1 input. Based on the information and the background color calculation unit 6 that calculates the local background color in the pixels near the boundary between the foreground and the background using the image 1 input, and from the image based on the background color calculated by the background color calculation unit 6 And a background color removing unit 7 for removing the influence of the background color.

Further, the background color removing apparatus includes a foreground only region R ₁ , a background only region R _3, and a foreground according to the key signal α supplied from the input terminal 2, that is, the information on the boundary between the foreground and the background. The region dividing unit 3 that forms the region R ₂ in which the background is mixed, the target pixel determining unit 4 that determines the target pixel p from which the background color is to be removed, and the target pixel determined by the target pixel determining unit 4 closest to p, becomes also includes recent pixel calculation unit 5 for calculating the pixel q of on the region R _3.

  Then, the background color calculation unit 6 calculates some pixels that are close to the nearest pixel q calculated by the nearest pixel calculation unit 5 under a predetermined condition, and averages the colors of these some pixels. The value is calculated as the background color b.

  The background color b is supplied to the background color removal unit 7. The background color removal unit 7 calculates the color C2 of the pixel from which the influence of the background color has been removed using the color C1 of the pixel of interest p including the background color, the key signal α, and the background color b, and outputs the result Supply to terminal 8.

  The arithmetic processing in each unit shown in FIG. 1 is performed by a logical operation unit (ALU) 11 inside the central processing unit (CPU) 10 shown in FIG.

  The CPU 10 includes a register 12 and a controller 13 in addition to the ALU 11. The CPU 10 temporarily stores input data supplied via the interface (I / F) 15 in the RAM 18, and then outputs it as output data via the RAM 18 or the ROM 19 after calculation by the ALU 11.

  The register 12 temporarily stores data used for calculation in the ALU 11 and calculation results. The ALU 11 reads the data held in the register 12, performs a predetermined operation on the data, and outputs the operation result to the register 12. The controller 13 controls the register 12 and the ALU 11 in accordance with the program stored in the ROM 19 so as to perform calculations stored in the program.

  In addition to storing a program for controlling the CPU 10, the RAM 18 stores data supplied from the CPU 10 and outputs it to the I / F 15. The ROM 19 is configured to temporarily store data and programs used in the CPU 10.

  The operation of the background color removal apparatus shown in FIGS. 1 and 2 will be described below using the flowchart of FIG. 3 and FIG.

  Here, explanations of operations in the register 12 and the controller 13 are omitted, but these units operate in the above-described range as necessary in each step.

First, it is assumed that an image and a key signal are input through the I / F 15 and stored in two image storage areas secured in the ROM 19. Further, parameters representing the two curves of the boundary lines L ₀ and L ₁ of the image 1 shown in FIG.

  Variables used in each step of the flowchart of FIG. 3 are sent from the RAM 18 to the register 12 and are subjected to necessary calculations by the ALU 11 whose detailed configuration is shown in FIG. Sent. A program for performing a series of processes in the flowchart is input from the I / F 15 and stored in the RAM 18.

The key signal α input to the background color removing device can be generated by the technique of “key signal generating device and method” disclosed in Japanese Patent Laid-Open No. 9-37153 by the applicant of the present application. This technique is based on a method of calculating by approximating a curved surface of a key signal, which is a mixture ratio of foreground and background, and controlling with a plurality of parameters. As shown in FIG. 4 Using this method, it is possible to calculate the boundary L ₀ border L ₁ and the background side of the foreground side of the mixed area of the foreground and background, a region R ₁ of the entire image foreground only The foreground and the background can be divided into a region R ₂ and a background only region R ₃ .

In the background removal apparatus, the area dividing unit 3 constituting the ALU 11 calculates the boundary line L ₁ and the boundary line L ₀ based on the key signal α, and the entire image is converted into a foreground-only area R ₁ . The region is divided into a region R ₂ where the foreground and the background are mixed and a region R _{3 including} only the background.

  First, in step S1, the pixel-of-interest determination unit 4 configuring the ALU 11 sets the pixel number i as i == pixel (1), pixel (2),..., Pixel (n) in order. Initialize with 1. Specifically, 1 is substituted for i in the ALU 11.

In step S2, the pixel-of-interest determination unit 4 checks whether the pixel i is included in the foreground only region R ₁ , the foreground / background mixed region R ₂ , or the background only region R ₃ . Specifically, the area corresponding to the pixel of number i is calculated from the area determination map stored in the RAM 18. Here, the processing at the step S3~ step S6 if determined to be included in the region _{R 2.} Further, if determined to be included in the region R ₁ proceeds to step S8, if it is determined to be included in the region R ₃ proceeds to step S9.

Here, the image i in step S2 is determined to be included in the region R _2, recently pixel calculation unit 5 constituting the ALU 11, in step S3, the pixel included in the region R _3, whose distance is close to the pixel i Find the pixel q. Specifically, a pixel included in the region R ₃ is searched from the region determination map recorded in the RAM 18, the distance to the pixel i is calculated by the latest pixel calculation unit 5, and the pixel coordinates and the calculated distance are calculated. If it is stored in a temporary variable and a distance shorter than the stored distance is obtained, the coordinates of the temporary pixel and the calculated distance are updated, and finally the pixel q closest to the pixel i is calculated. The coordinates of the pixel q are stored in the RAM 18.

Next, in step S4, the background color calculating section 6 constituting the ALU 11, the pixel included in the region _{R 3,} the distance 2 or less from the pixel q, pixels _{q 1} under conditions that the distance 2 or less from the boundary line _{L 0} , Q ₂ ,... Specifically, using a region determination map secured in the RAM 18, the pixel q _i included in the region R ₃ is searched for and the distance from the pixel q is calculated. Further, the equation of the curve is calculated from the parameter of the boundary line L ₀ stored in the RAM 18, and the distance between the pixel q _i and the boundary line L ₀ is calculated. Then, pixels that are less than or equal to the distance 2 from the pixel q and less than or equal to the distance 2 from the boundary line L ₀ are calculated one after another, and the color is written in the pixel array prepared in the RAM 18.

In step S5, the background color calculator 6 calculates the average value b of the colors of the pixels q ₁ , q ₂ ,. Specifically, an average value of all the colors stored in the pixel array secured in the RAM 18 is calculated and written to the background color variable b secured in the RAM 18.

In step S6, the background color removing unit 7 constituting the ALU 11 inputs the key signal α in the pixel i, and changes the color C2 to
C2 = C1- (1-α) * b
Ask from. Specifically, referring to the variable C1, b secured in the RAM 18 and the map of the key signal stored in the image storage area secured in the ROM 19, the key signal α of the pixel i obtained therefrom is obtained. Then, the color C2 from which the influence of the background color is removed from the pixel i is calculated as in the above equation, and written in a location corresponding to the pixel i in the image storage area secured in the RAM 18.

It determines that the pixel i in step S2 is included in the area _{R 1,} the process proceeds to step S8, the background color removal section 7, and C2 = C1. That is, the value C1 of the pixel i is substituted for the variable C2 extracted from the RAM 18.

Further, it is determined that the pixel i in step S2 is included in the area R _3, In step S9, the background color removal section 7, and C2 = 0. That is, the value 0 of the pixel i is substituted into the variable C2 fetched from the RAM 18.

  In step S 7, 1 is added to the variable i stored in the RAM 18. Further, in step S10, it is checked whether the pixel i ≦ n. If yes, the process proceeds to step S2, if no, the end is set, and the color C2 of the image storage area secured in the RAM 18 is displayed on the monitor 17 via the I / F 15. The

  As described above, the background color removing apparatus shown in FIGS. 1 and 2 can automatically determine the background color locally in the vicinity of the boundary between the foreground and the background as compared with the conventional case, and a plurality of background colors. Even in the case where the image is included, the influence of the background color can be removed from the image with high accuracy. For this reason, compared with the case where an operator designates a background color for every local area | region, a work amount can be reduced significantly.

  Next, the key signal generation apparatus and method used in the embodiment of the present invention will be described with reference to FIGS.

  This key signal generation device is a key signal generation device that generates a key signal from an image composed of a foreground and a background. As shown in FIG. 5, boundary information between the foreground and background of an image input from an image input terminal 20 is obtained. A boundary information calculation unit 21 to calculate, and a foreground color calculation unit 25 to calculate a local foreground color in the vicinity of the boundary between the foreground and the background, using the boundary information calculated by the boundary information calculation unit 21 and the image. And a background color calculation unit 27 that calculates a local background color in the vicinity of the boundary between the foreground and the background using the boundary information and the image calculated by the boundary information calculation unit 21, and a foreground color calculation unit 25. And a key signal generation unit 28 for generating the key signal using the foreground color calculated in step 1 and the background color calculated by the background color calculation unit 27.

In addition, the key signal generation device divides the input image into a region R ₁ only for the foreground, a region R ₃ only for the background, and a region R ₂ where the foreground and the background are mixed based on the boundary information calculated by the boundary information calculation unit 21. On the region R ₁ that is closest to the target pixel p determined by the target pixel determination unit 23, the target pixel determination unit 23 that determines the target pixel for which the key signal is to be generated, recently pixel calculation unit 24 for calculating a pixel r of the target pixel closest to p, becomes also includes recent pixel calculation unit 26 for calculating the pixel q of on the region R _3.

  Then, the foreground color calculation unit 25 calculates some pixels that are close to the nearest pixel r calculated by the nearest pixel calculation unit 24 under a predetermined condition, and averages the colors of these some pixels. The value is calculated as the foreground color f.

  Further, the background color calculation unit 27 calculates some pixels that are close to the nearest pixel q calculated by the nearest pixel calculation unit 26 under a predetermined condition, and averages the colors of these some pixels. The value is calculated as the background color b.

  The key signal generation unit 28 generates a key signal α for the pixel p based on the foreground color f and the background color b and supplies the key signal α to the output terminal 29.

  The arithmetic processing in each part shown in FIG. 5 is also performed by the ALU 11 inside the CPU 10 shown in FIG. Here, the basic operations of the CPU 10, ALU 11, register 12, controller 13, I / F 15, RAM 18, and ROM 19 are as described above.

  The operation of the key signal generation device shown in FIGS. 5 and 2 will be described below with reference to the flowchart of FIG. 6 and FIGS.

  Again, although explanation of the operation of the register 12 and the controller 13 is omitted, these units operate within the above-described range as necessary in each step.

  First, it is assumed that the input image is stored in the image storage area secured in the ROM 19. Further, it is assumed that a program for performing a series of processes in the flowchart of FIG. 6 is input from the I / F 15 and stored in the RAM 18.

  6 are sent from the RAM 18 to the register 12 and are subjected to necessary calculations by the ALU 11, and then sent to the RAM 18 through the register 12 again.

Note that the boundary information calculation unit 21 uses the technique of “key signal generation device and method” disclosed in Japanese Patent Application Laid-Open No. 9-37153 by the applicant, as shown in FIG. As intermediate data, a boundary line L ₀ with a key value of ₀ and a boundary line L ₁ with a key value of ₁ are calculated.

Then, as shown in FIG. 7 by the boundary lines L ₀ and L ₁ , the area dividing unit 22 divides the entire image into the foreground only area R ₁ , the foreground / background mixed area R _2, and the background only area R Divide into _three .

  First, in step S11, the pixel-of-interest determination unit 23 configuring the ALU 11 sets pixels in the image as pixels (1), pixels (2),. Initialize with 1. Specifically, 1 is substituted for i in the ALU 11.

In step S12, the pixel-of-interest determination unit 23 checks whether the pixel i is included in the foreground only region R ₁ , the foreground / background mixed region R ₂ , or the background only region R ₃ . Specifically, the area corresponding to the pixel of number i is calculated from the area determination map stored in the RAM 18. Here, the processing at the step S13~ step S16 if determined to be included in the region _{R 2.} Further, if determined to be included in the region R ₁ proceeds to step S18, if determined to be included in the region R ₃ proceeds to step S19.

  If it is determined in step S12 that the image i is included in the region R2, the nearest pixel calculation unit 24 configuring the ALU 11 determines in step S13 that the pixel r is the pixel included in the region R1 and closest to the pixel i. Ask for. Specifically, the pixel included in the region R1 is searched from the region determination map recorded in the RAM 18 to calculate the distance to the pixel i, and the pixel coordinates and the calculated distance are stored in a temporary variable. When a distance shorter than the stored distance is obtained, the coordinates of the temporary pixel and the calculated distance are updated, the pixel r closest to the pixel i is finally calculated, and the coordinates of the pixel r are stored in the RAM 18. To remember. Similarly, in step S13, the nearest pixel calculation unit 26 configuring the ALU 11 obtains a pixel q that is the pixel closest to the pixel i among the pixels included in the region R3.

Next, in step S14, the foreground color calculation unit 25 configuring the ALU 11 includes pixels r ₁ , r ₂ , pixels that are included in the region R _{1 and} have a distance of 2 or less from the pixel r and a distance of 2 or less from the boundary line L ₁ . Calculate ... Specifically, using a region determination map secured in the RAM 18, the pixel r _i included in the region R ₁ is searched for and the distance to the pixel r is calculated. Further, an equation of a curve is calculated from the parameters of the boundary line L ₁ stored in the RAM 18, and the distance between the pixel r _i and the boundary line L ₁ is calculated. Then, pixels that are less than or equal to the distance 2 from the pixel r and less than or equal to the distance 2 from the boundary line L ₁ are calculated one after another, and the color is written in the pixel array prepared in the RAM 18. Similarly, the background color calculation unit 27 calculates pixels q ₁ , q ₂ ,... That are pixels included in the region R _{3 and} have a distance of 2 or less from the pixel q and a distance of 2 or less from the boundary line L ₀ .

In step S15, the foreground color calculation unit 25 calculates the average value f of the colors of the pixels r ₁ , r ₂ ,. Specifically, the average value of all the colors stored in the pixel array secured in the RAM 18 is calculated and written to the background color variable f secured in the RAM 18. Similarly, the background color calculation unit 27 calculates the average value b of the colors of the pixels q ₁ , q ₂ ,.

  In step S16, the key signal generation unit 28 constituting the ALU 11 calculates the key signal α in the pixel i.

As shown in FIG. 8, the key signal generation unit 28 maps the color C1, the foreground color f, and the background color b of the pixel p to the RGB space, and points C1 with respect to the half line connecting the points f and b. The point g which is the foot of the perpendicular of
α = | g−b | / | f−b |
To calculate the key signal α of the pixel p.

Determines that the pixel i in step S12 is included in the area _{R 1,} the process proceeds to step S18, the key signal generating section 28, and alpha = 1.

Also, pixel i in step S12 is judged to be included in the region _{R 3,} the process proceeds to step S19, the key signal generating section 28, and alpha = 0.

  In step S17, 1 is added to the variable i stored in the RAM 18. In step S20, it is checked whether the pixel i ≦ n. If yes, the process proceeds to step S12. If no, end is set, and the key signal α secured in the RAM 18 is displayed on the monitor 17 via the I / F 15.

  As described above, the key signal generation device shown in FIGS. 5 and 2 can automatically determine the foreground color and the background color locally in the vicinity of the boundary between the foreground and the background, as compared with the prior art. An accurate key signal can be calculated for each correct area. For this reason, compared with the case where an operator designates a foreground color and a background color for each local area, the amount of work can be greatly reduced.

  Next, an image processing apparatus and method used in another embodiment of the present invention will be described with reference to FIGS.

  This image processing apparatus is a background color removal apparatus that automatically calculates a local background color and removes it from the image in order to delete the influence of the background color from the image.

  The configuration of this background color removing device is basically the same as the configuration shown in FIGS. The difference is the calculation process of the nearest pixel B in the nearest pixel calculation unit 5, and the process of step S3 is different in the flowchart shown in FIG.

  First, in step S1, the pixel-of-interest determination unit 4 configuring the ALU 11 initializes the pixel number i with i = 1 as described above.

In step S2, the pixel-of-interest determination unit 4 checks whether the pixel i is included in the foreground only region R ₁ , the foreground / background mixed region R ₂ , or the background only region R ₃ . Specifically, using the technique of “key signal generation apparatus and method” disclosed by the present applicant in Japanese Patent Laid-Open No. 9-37153, the foreground key is a three-dimensional model, that is, a third-order Bezier. When the curved surface is generated in a shape that is continuously connected to the shape of a ring in the direction of the third-order parameter, this is projected onto the plane of xy = 0, and the foreground only region R ₁ , the foreground and background mixed The region R ₂ and the background only region R ₃ are classified, and it is examined which of the regions R ₁ , R ₂ , and R ₃ the pixel i is included in.

FIG. 10 is a diagram in which the height of a curved surface is mapped onto a plane when n foreground keys Q _k (1 ≦ k ≦ n) are continuously connected to represent a foreground key.

Here, the Bezier curved surface Q _k (1 ≦ k ≦ n) is a three-dimensional curved surface representing the key of the area where the foreground and the background are mixed, and the height direction of the foreground key (z direction in the xyz coordinate system). ) Represents a key value k, and the vertical and horizontal directions (xy directions in the xyz coordinate system) of the foreground key represent coordinates (x, y) on the image.

In the Bezier curved surface Q _k (1 ≦ k ≦ n), the height direction of the foreground key (z direction of the xyz coordinate system) is set to the parameter s (0 ≦ s ≦ 1), and the vertical and horizontal directions of the foreground key (xyz). The xy direction of the coordinate system is controlled by a parameter t (0 ≦ t ≦ 1).

If it is determined that the image i is included in the area R ₂ in step S2, recently pixel calculation unit 5, in step S3, the pixel included in the region R _3, obtaining the pixel B whose distance is close to the pixel i. Specifically, a normal line is calculated for the curve in the cubic direction of the curved surface Q _k (1 ≦ k ≦ n) including the pixel i at the center point P of the pixel i, and the pixel on this normal line is calculated. Te, the pixel included in the region R _3, obtaining the pixel B whose distance is close to the pixel i.

In step S4, the background color detection unit 6 calculates pixels B ₁ , B ₂ ,... That are pixels included in the region R _{3 and} have a distance of 2 or less from the pixel B and a distance of 2 or less from the curve L ₀ . In step S5, the average value b of the colors of the pixels B ₁ , B ₂ ,... Is calculated.

That is, in step S3 to step S5, a normal line with respect to the curve in the parameter s direction is calculated at the center point P of the pixel i of interest as shown in FIG. The pixel immediately after passing through the curve L ₀ is B, and the pixel is included within the distance Q 2 from the pixel B and outside the curved surface Q _k (1 ≦ k ≦ n) within the distance 2 from the curve L _0. Is calculated as the background color b.

In step S6, the background color removing unit 7 constituting the ALU 11 inputs the key signal α in the pixel i, and changes the color C2 to
C2 = C1- (1-α) * b
Ask from.

It determines that the pixel i in step S2 is included in the area _{R 1,} the process proceeds to step S8, the background color removal section 7, and C2 = C1.

Further, it is determined that the pixel i in step S2 is included in the area R _3, In step S9, the background color removal section 7, and C2 = 0.

  In step S 7, 1 is added to the variable i stored in the RAM 18. Further, in step S10, it is checked whether the pixel i ≦ n. If yes, the process proceeds to step S2, if no, the end is set, and the color C2 of the image storage area secured in the RAM 18 is displayed on the monitor 17 via the I / F 15. The

  As described above, the background color removal apparatus whose operation is shown in FIG. 9 can automatically and accurately determine the background color locally in the vicinity of the boundary between the foreground and the background. Even if the color is included, the influence of the background color can be removed from the image with high accuracy. For this reason, compared with the case where an operator designates a background color for every local area | region, a work amount can be reduced significantly.

  Next, the image composition apparatus used in the embodiment according to the present invention will be described with reference to FIG.

  In FIG. 11, the image providing devices 61 and 62 are configured to reproduce and output a predetermined image using, for example, a video tape player or a disk player such as a hard disk device. Here, an image T1 (first image) in which the foreground or background reproduced by the image providing device 61 is F or B, respectively, is supplied to the key signal generating device 64 and the mixer 65. This image T1 is a target for generating a key signal. The image T2 (second image) reproduced by the image providing device 62 is supplied to the mixer 65, and the foreground F of the image T1 is combined with the image T2. Yes.

  The key signal generation device 64 is configured in the same manner as the key signal generation device shown in FIG. 5, generates the key signal α as described above, and outputs it to the mixer 65. The mixer 65 uses the key signal α from the key signal generation device 64 to synthesize the images T1 and T2 input thereto.

  Here, the mixer 65 has the function of the background removing apparatus shown in FIG. 1 and eliminates the influence of the background color when extracting the foreground F from the image T1. Then, the foreground F is synthesized (inserted) into the image T2.

  Next, the operation will be described. In the image providing devices 61 and 62, the images T1 and T2 are reproduced and supplied to the mixer 65, respectively. Further, the image T1 reproduced by the image providing device 61 is also supplied to the key signal generating device 64.

  The key signal generator 64 generates the key signal α and outputs it to the mixer 65 as described above. In the mixer 65, when the foreground F is extracted from the image T1 input thereto using the key signal α by using the function of the background color removing device, the background color is deleted, and the foreground F is converted into the image T2. Synthesize. As a result, an image (synthesized image) T4 obtained by combining the foreground F with the image T2 is generated.

  As described above, the image synthesizing apparatus synthesizes the image using the key signal generating apparatus using the key signal generating method and the mixer using the image processing method at the time of image synthesis. The amount of work can be reduced.

  Here, the key signal generation method of the present invention is applied to the key signal generation device 64, and the image processing method of the present invention is applied to the mixer 65, but only one of them, that is, the key signal generation device 64 has the other. The image processing method of the present invention may be applied only to the mixer 65. Further, the key signal generation method of the present invention is applied to the key signal generation device 64, and the mixer 65 does not have to have a function of particularly removing the background color.

  In the above embodiment, the information on the boundary between the foreground and the background is information for dividing the foreground only area, the background only area, and the foreground and background mixed area. For information, for example, the contour of an image created by an operator using a paint tool or the like is a region where the foreground and the background are mixed, the inside of this contour is the region of the foreground only, and the outside of this contour is the region of the background only You can also.

  Further, in the above embodiment, when calculating the foreground color and the background color in the local area of the image, from the pixels included in the mixed area of the foreground and the background, only the area of the foreground or only the background is calculated. Calculate the pixel closest to the pixel of interest from among the pixels included in the region, and use the boundary between the region where the foreground and background are mixed and the region where only the foreground is present and within a certain distance from the calculated pixel. Calculate the average value of the pixels that exist within a certain distance range as the foreground color of the pixel of interest, and the pixels that exist within a certain distance from the boundary line between the area where the foreground and the background are mixed and the area only with the background Although the method of calculating the average value of the pixel and setting it as the background color of the pixel of interest has been shown, when the foreground color and the background color in the local region are calculated by other methods, that is, in the vicinity of the pixel of interest Pixels into areas with only foreground Uses another method to calculate from the background-only region, or uses another method to calculate the foreground color and background color of the pixel of interest from the pixels in the vicinity of the calculated pixel of interest Of course, the method of the present invention is applicable.

  Further, the image processing method used in the embodiment of the present invention, that is, the local background color in the pixels near the boundary between the foreground and the background is obtained by using the information on the boundary between the foreground and the background constituting the image and the image. The processing procedure (program) using the method of calculating and removing the influence of the background color from the image based on the background color is recorded on the recording medium, and this program is read out as appropriate, so that the pixel in the target pixel can be easily read. The influence of the background color can be eliminated.

  In detail, this processing procedure uses a background color calculation step of calculating a local background color in pixels near the boundary between the foreground and the background using the information on the boundary between the foreground and the background constituting the image and the input image. And a background color removing step of removing the influence of the background color from the image based on the background color calculated by the background color calculating means.

  In addition, the key signal generation method used in the embodiment of the present invention, that is, the boundary information between the foreground and the background is calculated and the local information near the boundary between the foreground and the background is calculated using the boundary information and the image. The foreground and background colors are calculated, and the foreground and background colors are used to read the program from a recording medium that records the processing procedure (program) using the method of generating the key signal. By executing this, it is possible to easily generate the key signal of the pixel of interest.

  More specifically, the processing procedure includes a boundary information calculation step for calculating boundary information between the foreground and the background, the boundary information calculated in the boundary information calculation step, and the image. A foreground color calculation step for calculating a local foreground color in the vicinity of the boundary of the image, and a local background near the boundary between the foreground and the background using the boundary information and the image calculated in the boundary information calculation step. A background color calculation step for calculating a color, and a key signal generation step for generating the key signal using the foreground color calculated in the foreground color calculation step and the background color calculated in the background color calculation step. Become.

  According to the image processing apparatus and method used in the embodiment of the present invention as described above, the background color of the local area is automatically calculated and removed from the image, so the background color includes a plurality of colors. Even in such a case, the influence of the background color can be removed from the image with high accuracy. In addition, the amount of work can be greatly reduced as compared with the case where the operator designates a background color for each local region.

  Further, the key signal generation apparatus and method according to the embodiment of the present invention automatically calculates a local foreground color and a background color in the vicinity of the boundary between the foreground and the background to generate a key signal. There is an effect that an accurate key signal can be calculated for each region. In addition, the amount of work can be greatly reduced as compared with the case where the operator designates the foreground color and the background color for each local area.

  In addition, since the image composition apparatus used in the embodiment of the present invention uses the image processing apparatus and method or the key signal generation apparatus and method, it is possible to reduce an operator's workload during image composition.

  Further, the recording medium used in the embodiment of the present invention can widely provide an effect of reducing the amount of work for the operator when removing the background color or generating the key signal.

  It should be noted that the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present invention.

It is a functional block diagram of a background color removing apparatus that is an image processing apparatus and method used in an embodiment according to the present invention. FIG. 2 is a block diagram of the background color removal apparatus showing an overall configuration including a logical operation unit (ALU) that operates the functional block diagram of FIG. 1. It is a flowchart for demonstrating operation | movement of the said background color removal apparatus. It is a figure for demonstrating operation | movement of the said background color removal apparatus. It is a functional block diagram of the key signal generation device used as the embodiment of the key signal generation device and method concerning the present invention. It is a flowchart for demonstrating operation | movement of the said key signal generation apparatus. It is a figure for demonstrating operation | movement of the said key signal generation apparatus. It is a characteristic view for demonstrating operation | movement of the principal part of the said key signal generation apparatus. It is a flowchart for demonstrating operation | movement of the background removal apparatus and method used for other embodiment of this invention. It is a figure for demonstrating operation | movement of the background removal apparatus and method shown in the said FIG. It is a block diagram which shows the structure of the image synthesizing | combining apparatus used for embodiment which concerns on this invention. It is a figure for demonstrating an image composition process. It is a figure for demonstrating a chroma key production | generation process.

Explanation of symbols

  5 Recent (closest) pixel calculation unit, 6 Background color calculation unit, 7 Background color removal unit, 24 Recent pixel calculation unit, 25 Foreground color calculation unit, 26 Recent pixel calculation unit, 27 Background color calculation unit, 28 Key signal generation Part

Claims (21)

In an image processing apparatus that performs background color removal processing to remove the influence of background color from an input image,
Information on the boundary between the foreground and the background constituting the image, and a background color calculation means for calculating a local background color in pixels near the boundary between the foreground and the background using the input image;
An image processing apparatus comprising: a background color removing unit that removes the influence of a background color from an image based on the background color calculated by the background color calculating unit.   2. The image processing apparatus according to claim 1, wherein the information relating to the boundary is information for forming a region including only the foreground, a region including only the background, and a region where the foreground and the background are mixed.   The image processing apparatus according to claim 1, wherein the boundary information is information relating to a key represented by a curved surface controlled by a parameter.   A direction perpendicular to the outline of the foreground is calculated for the pixels near the boundary between the foreground and the background using the boundary information, and the background color calculating unit calculates the background color along this direction. The image processing apparatus according to claim 3. In the image processing method for removing the influence of the background color from the input image,
Using the information on the boundary between the foreground and the background constituting the image and the image, the local background color in the pixels near the boundary between the foreground and the background is calculated, and the influence of the background color is calculated from the image based on the background color. The image processing method characterized by removing.   6. The image processing method according to claim 5, wherein the information relating to the boundary is information for forming a region including only the foreground, a region including only the background, and a region where the foreground and the background are mixed.   6. The image processing method according to claim 5, wherein the boundary information is information relating to a key represented by a curved surface controlled by a parameter.   8. The background color is calculated along a direction perpendicular to the outline of the foreground for the pixels near the boundary between the foreground and the background using the boundary information. Image processing method. In a key signal generation device that generates a key signal from an image composed of a foreground and a background,
Boundary information calculating means for calculating boundary information between the foreground and the background;
Foreground color calculating means for calculating a local foreground color near the boundary between the foreground and the background using the boundary information and the image calculated by the boundary information calculating means;
Background color calculation means for calculating a local background color near the boundary between the foreground and the background using the boundary information calculated by the boundary information calculation means and the image;
A key signal generation device comprising: key signal generation means for generating the key signal using the foreground color calculated by the foreground color calculation means and the background color calculated by the background color calculation means.   10. The key according to claim 9, wherein the boundary information calculated by the boundary information calculating means is information for forming a foreground-only area, a background-only area, and a foreground-background mixed area. Signal generator.   10. The key signal generation apparatus according to claim 9, wherein the boundary information calculated by the boundary information calculation means is information relating to a key represented by a curved surface controlled by a parameter.   A direction perpendicular to the outline of the foreground is calculated for the pixels near the boundary between the foreground and the background using the boundary information, and the foreground color calculation unit and the background color calculation unit perform the foreground color and the background color along the direction. The key signal generation device according to claim 11, wherein a color is calculated. In a key signal generation method for generating a foreground key signal from an image composed of a foreground and a background,
The boundary information between the foreground and the background is calculated, and using the boundary information and the image, a local foreground color and a background color near the boundary between the foreground and the background are calculated. A key signal generation method characterized by generating the key signal using a color.   14. The key signal generation method according to claim 13, wherein the boundary information is information for forming a foreground only region, a background only region, and a region in which the foreground and the background are mixed.   14. The key signal generation method according to claim 13, wherein the boundary information is information relating to a key represented by a curved surface controlled by a parameter.   The foreground color and the background color are calculated along a direction perpendicular to the outline of the foreground for the pixels near the boundary between the foreground and the background using the boundary information. 15. The key signal generation method according to 15. In an image composition device for receiving a first image composed of at least a foreground and a background and a second image composed of at least a background and combining the foreground of the first image and the background of the second image,
A background color calculation means for calculating a local background color in a pixel near the boundary between the foreground and the background using the information on the boundary between the foreground and the background constituting the image and the image;
An image composition apparatus comprising: a background color removing unit that removes the influence of a background color from an image based on the background color calculated by the background color calculating unit. In an image composition device for receiving a first image composed of at least a foreground and a background and a second image composed of at least a background and combining the foreground of the first image and the background of the second image,
Boundary information calculating means for calculating boundary information between the foreground and the background;
Foreground color calculation means for calculating a local foreground color near the boundary between the foreground and the background using the boundary information calculated by the boundary information calculation means and the image;
Using the boundary information calculated by the boundary information calculating means and the image, a background color calculating means for calculating a local background color in the vicinity of the boundary between the foreground and the background;
Key signal generating means for generating the key signal using the foreground color calculated by the foreground color calculating means and the background color calculated by the background color calculating means;
An image synthesizing apparatus comprising: synthesizing means for synthesizing the foreground of the first image and the background of the second image using the key signal generated by the key signal generating means.   Based on the key signal generated by the key signal generation means and the background color calculated by the background color calculation means, the image processing apparatus includes background color removal means for removing the influence of the background color from the image. 19. The image synthesizing apparatus according to claim 18, wherein the foreground of the first image from which the influence of the background color is removed by the color removing unit and the background of the second image are synthesized. In a recording medium recording a processing procedure for removing the influence of the background color from the input image,
A background color calculation step of calculating a local background color in pixels near the boundary between the foreground and the background using the information about the boundary between the foreground and the background constituting the image and the input image;
A recording medium comprising: a processing procedure including: a background color removing step of removing an influence of a background color from an image based on the background color calculated by the background color calculating unit. In a recording medium recording a processing procedure for generating a foreground key signal from an image composed of a foreground and a background,
A boundary information calculating step for calculating boundary information between the foreground and the background;
A foreground color calculation step of calculating a local foreground color near the boundary between the foreground and the background using the boundary information and the image calculated in the boundary information calculation step;
A background color calculation step for calculating a local background color near the boundary between the foreground and the background using the boundary information and the image calculated in the boundary information calculation step;
Using the foreground color calculated in the foreground color calculation step and the background color calculated in the background color calculation step to record a processing procedure including a key signal generation step for generating the key signal. A characteristic recording medium.

Images