JP5182590B2 - Image composition apparatus and image composition method - Google Patents

Description

  The present invention relates to an image composition technique, and more particularly to an image composition technique using a chroma key technique.

  Conventionally, various studies on image composition using chroma key technology have been performed. In particular, when a subject has a transmission region or a reflection (reflection) region, a technique for natural synthesis is being studied. As such a technique, for example, in a chroma key device that generates a composite image in which a foreground subject included in a foreground image is inserted into a part of a background image, a specific color generated from the video signal of the foreground image is extracted. When the foreground subject is extracted, a color cancel key signal for canceling the specific color remaining in the foreground subject, a matte signal related to the specific color corresponding to the background portion of the foreground image, and A background image video signal to be fitted with a foreground subject and a reflection correction signal generated from the background image video signal are supplied to an image synthesis unit. In this image synthesis unit, the foreground subject is added to the background image. An embedded composite image is obtained, and the composite process is performed with a part of the background image reflected in the foreground subject. There is shall (see Patent Document 1).

  In the technique of this patent document 1, the background of the reflection or transmission part in the subject is canceled by using the color cancel key signal, but the reflection is expressed by adding the correction signal to the foreground image.

  Further, in the technique of Patent Document 1, the foreground is embedded in the background image by combining the edge signal of the foreground subject with the low-frequency component of the background image video signal superimposed on the composite video signal. The color change near the edge of the subject is slowed down.

JP 2002-95007 (paragraph numbers 0010-0012, 0056-0059)

  However, the technique of Patent Document 1 requires a correction signal for realizing reflection in addition to the foreground image and the background image. Since there are an infinite number of background image colors, it is impossible to prepare correction signals that can be used for these background images in advance. Also, the sense of incongruity is reduced by blurring the background of the edge portion during synthesis. However, for that purpose, an edge must be detected, which increases the amount of calculation, which is not preferable.

  An object of the present invention is to provide an image composition technique for realizing natural chroma key composition even for an image including a reflection portion.

  In order to solve the above problems, an image composition device of the present invention includes an image acquisition unit that acquires an original image including a background region and a subject, and a background image to be combined with the original image, and a background that is the color of the background region. A background color acquisition unit that acquires a color, a background candidate coordinate specification unit that specifies, as background candidate coordinates, coordinates of pixels that are candidates for the background region from the original image based on the background color, and based on the background color A distance calculation unit that calculates a distance between the determined predetermined pixel value and each pixel value of the background candidate coordinates in the original image; and an image processing parameter storage unit that stores an image processing parameter corresponding to the distance For each pixel of the background candidate coordinates in the background image, the image processing parameter corresponding to the distance at the background candidate coordinates is obtained from the image processing parameter storage unit. A background image generation unit that performs image processing using the image processing parameters and generates a background image for synthesis, and a foreground image obtained by removing the background color from the pixel values of the background candidate coordinates in the original image And an image composition unit for compositing the background image.

  In this configuration, the coordinates of a pixel having a pixel value that approximates the background color are specified as the background candidate coordinates, and image processing is performed on the pixels at the background candidate coordinates in the background image. At this time, a distance between a predetermined pixel value determined based on the background color and each pixel value of the background candidate coordinates in the original image is calculated, and an image processing parameter determined according to the distance is used. That is, in this configuration, the degree of approximation (distance) with the background color is obtained for each pixel of the background image, and the image processing with the intensity corresponding to the distance is performed on the pixel of the background image. By synthesizing the background image that has been subjected to image processing in this way, it is possible to reduce a sense of incongruity in the transmissive region and the reflective region in the subject. In particular, it is preferable to use a process for reducing the sharpness of the image as the image processing, and to set the image processing parameter so that the sharpness decreases as the distance increases. As a result, a clear background image is synthesized in an area where the background is relatively clear, such as a transparent area, and an unclear background image is synthesized in an area where the background is unclearly reflected, such as a reflective area. As a result, a sense of incongruity can be reduced.

  In one preferred embodiment of the image composition device of the present invention, the predetermined pixel value is determined according to a distance between the background color and each pixel value of the background candidate coordinates in the original image, and The image composition unit uses an image obtained by subtracting the predetermined pixel value from the pixel value of the background candidate coordinates in the original image as the foreground image. In this configuration, the predetermined pixel value is determined according to the distance between the background color and each pixel value of the background candidate coordinates in the original image. Thereby, the process for generating the foreground image and the process in the distance calculation unit can be made common, and the amount of calculation can be reduced.

  In general, the reflection of the background area into the reflection area is greatly influenced by the pixel values of the background area near the boundary between them. Therefore, it is preferable to adjust image processing parameters based on the magnitude of influence when performing image processing on a background image. Therefore, in one preferred embodiment of the image composition device of the present invention, the background image generation unit adjusts an image processing parameter based on a boundary position of a background candidate region composed of the background candidate pixels. Thereby, the uncomfortable feeling of reflection in the reflection region can be reduced.

  The technical features of the image composition apparatus of the present invention can also be applied to similar image composition methods. For example, an original image including a background region and a subject, a background image to be combined with the original image, a step of acquiring a background color that is a color of the background region, and the original image based on the background color Specifying the coordinates of pixels that are candidates for the background region as background candidate coordinates, a predetermined pixel value determined based on the background color, and each pixel value of the background candidate coordinates in the original image And calculating an image processing parameter corresponding to the distance in the background candidate coordinate for each pixel of the background candidate coordinate in the background image, and using the image processing parameter to determine an image Processing to generate a composition background image; and a foreground image obtained by removing the background color from the pixel values of the background candidate coordinates in the original image; A step for combining the serial composite background image, can be applied to an image synthesis method comprising a. Of course, the above-described additional features of the image composition apparatus can also be applied to such an image composition method.

It is a perspective view of a reception device using the image composition device of the present invention. It is a functional block diagram of the reception unit of the reception device. It is an example of an original image. It is a flowchart showing the flow of a specific process of background candidate coordinates. It is a flowchart showing the flow of a production | generation process of a background candidate pixel conversion image. It is the figure which represented typically the process which converts the pixel value of a background candidate pixel into a predetermined pixel value. It is a flowchart showing the flow of a calculation process of distance. It is an example of the spatial filter for a blurring process. It is a graph showing the relationship between distance and an image processing parameter. It is a flowchart showing the flow of a process of the production | generation of a synthetic | combination background image. It is a figure showing the relationship between distance and a synthetic | combination ratio. It is a flowchart showing the flow of a production | generation process of a synthesized image. It is a flowchart showing the flow of the whole process. It is the figure which represented typically the process which changes the filter coefficient of a spatial filter.

  The image composition apparatus of the present invention will be described below with reference to the drawings. In this embodiment, the image composition apparatus is configured as a part of the image print receiving apparatus A.

  FIG. 1 is a perspective view of the accepting device A. FIG. As shown in the figure, the accepting apparatus A of the present invention acquires image data for image printing from a customer and image print order contents (number of prints, print size, print paper type, etc.) as order information. It has a function and a function of collecting a fee for image printing. Therefore, the reception device A is composed of an upper reception unit AU (an example of the image composition device of the present invention) and a lower fee reception unit AD.

  The fee receiving unit AD is provided at an upper position of the vertically long case 10, and includes a coin receiver 11 having a coin insertion hole 11A for receiving coins, a banknote receiver 12 having a banknote insertion hole 12A for receiving banknotes, And a change dispenser 13 provided at a lower position of the case 10 and provided with a change discharge port 13A for discharging change.

  On the other hand, the receiving unit AU includes a semiconductor drive 2 for acquiring image data from various semiconductor storage media including a semiconductor memory such as a flash memory, a disk drive 3 for acquiring image data from a disk-type storage medium such as a CD-R, A receipt printing apparatus 4 that prints and discharges a receipt, a connector 5 that connects to a device such as a mobile phone with a camera and acquires image data, and a touch panel 6 that performs various displays and enables input from a customer. A display 7 formed on the surface is provided.

  FIG. 2 is a functional block diagram of functional units configured in the main body 1 of the reception unit AU. As is apparent from the figure, the receiving unit AU includes a receiving unit 15 that performs various input / output operations during image print order receiving processing and image composition from the customer, and various information necessary for delivering the image print to the customer. A receipt creation unit 16 that creates a printed receipt R and an image composition module 20 that causes the reception unit AU to function substantially as an image composition device are provided. The reception unit AU includes a memory 17 composed of a semiconductor memory for storing image data and the like. In the reception process, the reception unit 15 acquires an image obtained by photographing the background region and the subject (hereinafter referred to as an original image) and a background image to be combined with the original image, and develops them on the memory 17. Therefore, the reception unit 15 in the form of the present embodiment functions as an image acquisition unit in the present invention.

  The image composition module 20 specifies a background color acquisition unit 21 that acquires a background color that is the color of the background area, and coordinates of pixels having pixel values close to the background color from the original image as background candidate coordinates based on the background color. A background candidate coordinate specifying unit 22, a background candidate pixel conversion unit 23 that converts a pixel value of a background candidate coordinate in the original image into a predetermined pixel value and generates a background candidate pixel converted image, and a pixel value of a background candidate coordinate in the original image And a distance calculation unit 24 that calculates the distance between the pixel value of the background candidate coordinates in the background candidate pixel converted image, an image processing parameter storage unit 25 that stores various parameters, and the background candidate coordinate pixels in the background image, Image processing parameters corresponding to the distance at the background candidate coordinates calculated by the distance calculation unit 24 are acquired from the parameter storage unit 25, and image processing is performed using the acquired image processing parameters. A background image generation unit 26 that generates a background image for synthesis, and an image synthesis unit 27 that synthesizes the foreground image obtained by removing the background color from the pixel values of the background candidate coordinates in the original image and the background image for synthesis. Yes.

  As described above, the reception unit 15 acquires at least an original image and a background image to be combined with the original image in the reception process. FIG. 3 is an example of the original image 30. As is clear from the figure, the original image 30 includes at least a background region 31 and a subject 32. The original image 30 is taken by placing a subject in front of a single-color background curtain, similar to an image used for normal chroma key processing. For this reason, the pixel values of the pixels constituting the background region 31 are ideally a single color and practically an approximate color. It is assumed that the original image in the embodiment is taken with a green backdrop as a background.

  The background color acquisition unit 21 acquires a background color. In general, the background color is known because the original image is taken in a studio or the like. In this case, the background color can be stored in a non-volatile memory (not shown), and the background color acquisition unit 21 can acquire the background color therefrom. On the other hand, when the background color is unknown, the customer is requested to specify one point or position area in the background area, and the background color acquisition unit 21 determines the pixel value of the point or the average value of the pixel values of the area. Can be obtained as a background color. Of course, the background color may be acquired using other methods.

  The background candidate coordinate specifying unit 22 specifies the coordinates of a pixel having a pixel value close to the background color (hereinafter referred to as a background candidate pixel) among the pixels in the original image as the background candidate coordinates. That is, the background candidate coordinate specifying unit 22 has a function of specifying the coordinates of a pixel affected by the background color among the pixels of the original image.

  Since the ideal background area has a single pixel value, it is sufficient to use only the pixel having the pixel value acquired by the background color acquisition unit 21 as the background candidate pixel. The background area is not a single color but is composed of approximate colors. Therefore, it is necessary to specify a pixel having a color close to the background color as the background candidate pixel. Some subjects are affected by the background color. For example, a transparent (or translucent) plastic product such as a plastic bottle or a lace fabric has a transparent background, and thus is affected by the background color while being a subject. In addition, since the background is reflected in an object such as metal having a strong specular reflection or a whitish material, such a subject also has a pixel value affected by the background color. In the example of FIG. 3, the portion of the plastic bottle is a transmission region 33 that transmits the background color. In addition, the lower part of the trousers is a reflection area 34 in which the background color is reflected. In order to realize synthesis without a sense of incongruity, it is necessary to specify such a transmissive region 33 and a reflective region 34 as background candidate pixels.

Therefore, the background candidate coordinate specifying unit 22 in the form of the present embodiment specifies background candidate pixels by the process shown in FIG. First, the background color acquired by the background color acquisition unit 21 is converted into a uniform color space (# 01). In the embodiment, the original image, the background image, and the like are expressed in the RGB space, and the HSV space is used as the uniform color space. A known method can be used for conversion from the RGB space to the HSV space. In addition, the HSV coordinate that is the cylindrical coordinate system is converted to the xyz coordinate that is the orthogonal coordinate system so that the actual calculation can be easily handled. Therefore, unless otherwise specified, the coordinate in the HSV space is the coordinate (x, y, z) = (S × in the rectangular coordinate system when the coordinate in the cylindrical coordinate system is (H, S, V). cos (H), S × sin (H), V). Therefore, the background color obtained in the HSV space is expressed as μ = (x _b , y _b , z _b ).

Next, one pixel value is acquired from the original image developed on the memory 17 (# 02). In the background candidate coordinate specifying process, it is necessary to check all the pixels of the original image. Therefore, the acquisition of pixel values from the original image is preferably in raster scan order (memory 17 address order). Also, the pixel value selected here is converted into the HSV space in the same manner as the background color (# 03). The pixel value in the HSV space obtained here is expressed as fg = (x _f , y _f , z _f ). An HSV original image representing the original image in the HSV space is generated using the pixel value fg in the HSV space obtained here, and stored in the memory 17.

ここで、Σは分散ベクトル[σ_x，σ_y，σ_z]^Tの分散共分散行列である。 In this embodiment, it is assumed that the pixel value of the background region and the pixel value of the pixel affected by the background color follow a multidimensional (three-dimensional) normal distribution centered on the background color. That is, the pixel values of the background candidate pixels are normally distributed N (x _b , σ _x ), N (y _b , σ _y ), N (z _b ) with respect to each orthogonal axis (x, y, z) in the HSV space. , Σ _z ) and assuming that the pixel value of the background candidate pixel in the HSV space is represented by α, the probability density function f (α) of the background candidate pixel in the HSV space is represented by the following equation.

Here, Σ is a variance covariance matrix of variance vectors [σ _x , σ _y , σ _z ] ^T.

  Based on the value obtained by substituting fg obtained in step # 03 for α in the equation (1), the probability p that fg is a background candidate pixel is obtained (# 04). Next, the probability p is compared with a predetermined threshold TH (# 05). If the probability p is equal to or greater than the threshold TH (Yes branch of # 05), the pixel selected in step # 02 is a background candidate pixel. The coordinates of this pixel are stored in the memory 17 in a list format as background candidate coordinates (# 06).

  Thereafter, it is determined whether or not there is an unprocessed pixel (# 07). If there is an unprocessed pixel (Yes branch of # 07), the process proceeds to # 02 and the above process is applied to all pixels of the original image. Against.

Note that when the background candidate pixels are specified, the background candidate pixels and the other pixels may be presented to the customer in an identifiable manner so that the variance values σ _x , σ _y , and σ _z can be adjusted. Thereby, it is possible to deal with the distribution of pixel values in different transmissive areas and reflective areas for each image.

In addition, when the background area is green as in the present embodiment, the specification of the background candidate pixels may not be successful at the boundary between the background area and the skin color of the subject. In this case, the normal distribution N _y representing the yellow distribution is defined, and the problem at the boundary between the background and the skin color region can be solved by performing the same processing as described above using the normal distribution N _y. .

  In the present embodiment, the HSV space is used as the uniform color space, but other uniform color spaces such as HVC and Lab space may be used. Further, the background candidate pixels are specified in the uniform color space, but may be performed in the RGB space or other color spaces.

  The background candidate pixel conversion unit 23 generates a background candidate pixel converted image by converting the pixel value of the background candidate pixel in the original image into a predetermined pixel value. In the present embodiment, a background candidate pixel converted image is generated by the processing of the flowchart of FIG. Note that the background candidate pixel converted image is newly generated while retaining the original image (HSV original image), instead of replacing the pixel value of the original image.

First, one background candidate coordinate is selected from the background candidate coordinates stored in the memory 17 (# 11), and the pixel value fg = (x _f , y _f , z _f ) of the background candidate coordinate in the HSV original image is set. Obtain (# 12).

Next, the background candidate pixel fg is converted to fg ′ (predetermined pixel value in the present invention) by the following calculation (# 13).

This operation is performed to convert to the background candidate pixel fg mapping on a plane z = z _b, the background color on the plane are moved to the origin. The fg ′ obtained by this conversion removes the influence of the background color and expresses the original color of the subject. FIG. 6 is a diagram schematically showing this conversion. In addition, the ellipse in a figure represents the contour line of the above-mentioned probability density function. As is clear from the figure, a background candidate pixel having the same pixel value as that of the background color μ (at least the same hue and saturation) is converted to μ ′ = O (origin) by the calculation of Expression (2). Further, the pixel value fg of the background candidate pixel having a pixel value different from the background color μ is converted to fg ′ by the calculation of Expression (2). As is apparent from the figure, fg ′ is a value obtained by shifting fg by (μ′−μ). At this time, the probability that this background candidate image is the background color, that is, the larger the value obtained by substituting the background candidate pixel value fg into the probability density function f is adjusted so that fg ′ approaches the origin. It doesn't matter.

When the above processing for one background candidate coordinate is completed, the presence / absence of an unprocessed background candidate pixel is determined (# 14). If there is an unprocessed background candidate pixel (Yes branch of # 14), the processing is performed. Is transferred to # 11 and the above-described processing is repeated. As a result, a background candidate pixel conversion image is generated on the memory 17. In the above description, the z component of the pixel value fg ′ of the converted background candidate pixel is set to z _b , but the difference value (z _f −z _b ) from the background color may be used similarly to the other components. Absent.

  The distance calculation unit 24 obtains the distance between the pixel value of the background candidate coordinates in the original image and the pixel value of the background candidate coordinates in the background candidate pixel replacement image. Specifically, the distance is calculated by the process of the flowchart of FIG.

  First, one background candidate coordinate is selected from the memory 17 (# 21). Next, the pixel value of the selected background candidate coordinates is acquired from the HSV original image and the background candidate pixel converted image (# 22, 23), and the distance between these pixel values is obtained (# 24). In this embodiment, since the pixel values are expressed in a uniform color space, the Euclidean distance (2-norm square root) in the HSV space is used here. Of course, 1-norm (sum of deviation absolute values of each component) or other distances may be used. The distance obtained here is stored in the memory 17 in association with the background candidate coordinates.

  When the distance calculation for one background candidate coordinate is completed, it is determined whether or not there is an unprocessed background candidate coordinate (# 25). If there is an unprocessed background candidate coordinate (Yes branch of # 25), the process is performed. The process proceeds to # 21 and the above process is repeated.

  The background image generation unit 26 acquires image processing parameters corresponding to the distance in the background candidate coordinates from the parameter storage unit 25 for the pixels of the background candidate coordinates in the background image, and performs image processing using the image processing parameters. A background image for application synthesis is generated.

  The image processing in the present invention is performed on background candidate pixels in the background image in order to reduce the uncomfortable feeling that occurs in the transmissive area and the reflective area when the original image and the background image are combined. Therefore, the image processing in the present invention is preferably processing for reducing the degree of image sharpness, such as reduction in contrast and blurring. In this embodiment, blurring processing using a spatial filter is performed, but other image processing may be applied.

  FIG. 8 is an example of a spatial filter that realizes the blurring process. In this example, the size is 3 × 3, and the filter coefficient other than the center is 1. On the other hand, the central filter coefficient gradually decreases from (a) to (c). These are filters having a large blurring effect in order from (a). Thus, in the spatial filter, the strength of the filter effect can be easily changed by changing the filter coefficient. In this embodiment, the strength of the filter effect is changed by changing the filter coefficient at the center of the filter. However, the filter effect may be changed by changing a filter coefficient other than the filter coefficient at the center or changing the size of the filter. Absent.

The median value of the spatial filter corresponding to the distance between pixel values (equivalent to the distance obtained by the distance calculation unit 24) is stored as an image processing parameter in the parameter storage unit 25 according to the present embodiment. FIG. 9 is a graph showing the relationship between the distance between pixel values and image processing parameters. The horizontal axis represents distance, and the vertical axis represents image processing parameters. In this example, the parameter storage unit 25 shows a case where the filter coefficient at the center of the filter of FIG. 8 is stored in association with the distance between the pixel values. As is apparent from the figure, the image processing parameter is 4 for the distance L ₁ to L ₂ , the image processing parameter is 2 for the distance L ₂ to L ₃ , and the distance is L ₃ or more. The image processing parameter is 1, and is 0 when the distance is less than L ₁ . As described above, the blurring effect increases as the filter coefficient at the center of the filter decreases. Therefore, this graph shows that the setting is such that the effect of image processing increases as the distance between pixel values increases. As a result, the effect of the image processing increases as the distance from the background color increases, that is, as the influence of the background color decreases. That is, since the distance is small in the background area and the transmission area in the subject with high transmittance, image processing is not performed on areas corresponding to those areas in the background image, or weak image processing is performed. On the other hand, since the distance is large in a transmissive region in a subject with low transmittance and a reflective region in a subject with low reflectance, strong image processing is performed on regions corresponding to those regions in the background image. . Therefore, even if the synthesis background image processed in this way is synthesized, the uncomfortable feeling in the transmissive region and the reflective region can be reduced. If the image processing parameter is 0, it means that image processing is not performed. Therefore, image processing is not performed on pixels having pixel values that are very close to the background color.

  Below, the flow of the process of the background image generation part 26 is demonstrated using the flowchart of FIG. First, one background candidate coordinate and a distance associated with the background candidate coordinate are acquired from the memory 17 (# 31, 32). Next, an image processing parameter corresponding to the acquired distance is acquired from the parameter storage unit 26 (# 33). Then, the acquired image processing parameter is set to the filter coefficient at the center of the spatial filter, the spatial filter is positioned so that the center of the spatial filter and the background candidate coordinates of the background image match, and the filter coefficient of the background image and the spatial filter And the pixel value of the background image subjected to image processing is obtained (# 34). Since the spatial filter in FIG. 8 is not normalized, it is necessary to divide the value obtained by the convolution operation by the sum of the filter coefficients in the actual operation.

  When the process for one background candidate coordinate is completed, the presence / absence of an unprocessed background candidate coordinate is determined (# 35). If there is an unprocessed background candidate coordinate (Yes branch of # 35), the process is # The process proceeds to 31 and the above-described processing is repeated. As a result, a composite background image subjected to image processing is generated.

  The image composition unit 27 synthesizes the foreground image obtained by removing the background color from the pixel value of the background candidate coordinates in the original image and the composition background image to generate a composite image. In this embodiment, the background candidate pixel converted image is in a state in which the background color is removed from the pixel value of the background candidate pixel converted image, and therefore the background candidate pixel converted image is used as the foreground image.

In the present embodiment, image synthesis uses a soft chroma key that synthesizes the pixels of the foreground image and the background image at a predetermined ratio instead of the hard chroma key that uses any of the pixels of the foreground image and the background image. Therefore, in the present embodiment, the parameter storage unit 25 stores the composition ratio corresponding to the distance between the pixel values (corresponding to the distance calculated by the distance calculation unit 24). FIG. 11 is an example of the composition ratio stored in the parameter storage unit 25. In the figure, the horizontal axis represents the distance between pixel values, and the vertical axis represents the composition ratio. Note that the composition ratio is a ratio of combining the pixel value of the background image with the pixel value of the foreground image, and assuming that the pixel value of the foreground image is fg, the pixel value of the background image is bg, and the composition ratio is α. The pixel value c of the image is
c = (1−α) × fg + α × bg (3)
Is required. As is apparent from the figure, when the distance is small, that is, when the pixel value of the background candidate pixel is close to the background color, the composition ratio is set to be close to 1. On the other hand, the composition ratio is set to be close to 0 as the distance increases, that is, as the pixel value of the background candidate pixel moves away from the background color. Thus, the pixel value of the composite image corresponding to the background region or the transparent region in the subject having a high transmittance greatly contributes to the pixel value of the composite background image. On the other hand, the pixel value of the composite image corresponding to the transmissive region in the subject with low transmittance and the reflective region in the subject with low reflectance is a value that is moderately contributed by the pixel value of the background image for synthesis, reducing the sense of incongruity. be able to.

  The flow of image composition processing will be described below using the flowchart of FIG. First, one pixel value fg is selected from the foreground image on the memory 17 (# 41), and it is determined whether the coordinates of this pixel value fg are background candidate coordinates (# 42). When the coordinates of the pixel value fg are not the background candidate image (No branch of # 42), the composition background image is not synthesized, and the pixel value of the foreground image is used as the pixel value c of the synthesized image. c is set to the pixel value fg (# 48). On the other hand, when the coordinates of the pixel value fg are background candidate coordinates (Yes branch of # 42), the following processing is performed to synthesize the pixel value of the foreground image and the pixel value of the background image for synthesis.

  First, the distance associated with the coordinates of the pixel value fg is acquired from the memory 17 (# 43), and the composition ratio α corresponding to the distance is acquired from the parameter storage unit 25 (# 44). Then, a pixel value bg having the same coordinates as the pixel value fg is obtained from the background image for synthesis (# 45). Based on these values, the pixel value c of the composite image is obtained by Equation (3) (# 46).

  When the process for one pixel value is completed, it is determined whether there is an unprocessed pixel (# 47). If there is an unprocessed pixel (Yes branch of # 47), the process proceeds to # 41, and the above-described process is performed. The process is repeated. By this process, a composite image is generated on the memory 17.

  The composite image generated as described above is sent to a printing apparatus (not shown) together with order information received by the receiving unit 15 to create an image print.

  Next, the overall flow of image composition processing according to the present invention will be described with reference to the flowchart of FIG. First, a customer who brings an image recording medium (CD-R, flash memory, etc.) on which an original image and a background image are recorded operates the touch panel 6 of the accepting device A so that the accepting device A can be accepted. Transition.

  When the accepting device A becomes ready to accept, when the customer sets an image recording medium in one of the semiconductor drive 2, the disk drive 3, and the connector 5, the accepting device A receives the image data recorded on the image recording medium or The thumbnail image data is read and displayed as a list on the display 7. On the other hand, when the customer designates the original image and the background image by operating the touch panel 6, the accepting unit 15 acquires the designated original image and background image from the image recording medium and develops them in the memory 17 (# 51). .

  When the original image and the background image are acquired by the receiving unit 15, the background color acquiring unit 21 acquires the pixel value of the pixel located at a predetermined coordinate of the original image as the background color (# 52). In the form of the present embodiment, the upper left coordinate of the original image is used as the predetermined coordinate, but other coordinates may be used as long as the coordinates are highly likely to belong to the background area. Further, a plurality of points may be acquired, and an average value of these pixel values may be used as the background color. The background color acquired in this way is stored in the memory 17.

  When the background color is acquired by the background color acquisition unit 21, the coordinates of the background candidate pixel are specified by the background candidate coordinate specification unit 22 (# 53). When the background candidate coordinate specification unit 22 completes the specification of the background candidate coordinates, the background candidate pixel conversion unit 23 is notified to that effect, the pixel values of the background candidate coordinates in the original image are converted, and a background candidate pixel conversion image is generated. (# 54). When the generation of the background candidate pixel conversion image is completed, this is notified to the distance calculation unit 24.

  Upon receiving the notification, the distance calculation unit 24 calculates the distance between the pixel value of the background candidate coordinate in the original image and the pixel value of the background candidate coordinate in the background candidate pixel converted image, and stores it in the memory 17 in association with each background candidate coordinate. (# 55). When the calculation of the distances for all the background candidate coordinates is completed, the background image generation unit 26 is notified to that effect.

  Receiving the notification, the background image generation unit 26 acquires the image processing parameter corresponding to the distance associated with each background candidate coordinate from the parameter storage unit 25, and performs image processing on the pixel of the background candidate coordinate of the background image. As a result, a background image for synthesis is generated (# 56). When the generation of the composition background image is completed, the image composition unit 27 is notified to that effect. When the generation of the composition background image is completed, the image composition unit 27 is notified to that effect.

  Upon receiving the notification, the image composition unit 27 determines whether or not the coordinates of the pixels of the foreground image are the background candidate coordinates, and if the coordinates are the background candidate coordinates, the composition ratio corresponding to the distance associated with the coordinates. And a synthesized image is generated by synthesizing the pixel value of the foreground image and the pixel value of the synthesis background image in accordance with the synthesis ratio (# 57).

  For example, in the example of FIG. 3, both the transmissive region 33 and the reflective region 34 are background candidate pixels, and the pixel value of the transmissive region 33 in the original image is close to the background color, and the reflective region 34 in the original image Assume that the pixel value is far from the background color. At this time, in the composition background image generation process, a large value is set for the image processing parameter for the pixel corresponding to the transmissive area 33, and thus the transmissive area 33 in the composition background image is subjected to weak image processing. Become. On the other hand, since a small value is set for the image processing parameter for the pixel corresponding to the reflection area 34, the reflection area 34 in the composition background image is subjected to strong image processing. Therefore, the transmissive region 33 in the composition background image retains the sharpness, but the reflective region 34 is in a state in which the sharpness is lowered. Further, in the generation of the composite image, the composite ratio of the pixels corresponding to the transmissive area 33 is large, and the composite ratio of the pixels corresponding to the reflective area 34 is small. Therefore, the transparent area 33 in the synthesized image is synthesized with a high percentage of clear background images, and the reflective area is synthesized with a low percentage of unclear background images. As a result, even if there is a transmitted or reflected portion in the subject, it is possible to perform image composition with little discomfort.

  When the above composition process and all the reception processes are completed, the receipt R is printed out by the receipt creation unit 16 and the receipt printing apparatus 4 (# 58). The receipt is recorded with various information when the customer receives the image print, and the customer can receive the image print with the receipt R.

  The composite image created by the above processing is transmitted from the receiving device A to a printing device (not shown) (# 59), and an image print is created by the printing device.

[Another embodiment]
(1) In the embodiment described above, the filter coefficients other than the center of the spatial filter are set to the same value (= 1). However, the filter coefficients may be different depending on the characteristics of the image. For example, as shown in FIG. 14A, if there is a reflection region 34 that is in contact with the background region 31 below, the pixel value below the reflection region 34 (near the boundary with the background region 31) is the background region. 31 is affected by the pixel value above 31 (near the boundary with the reflection region 34). At this time, it is desirable to set a large filter coefficient corresponding to a pixel position that greatly affects the target pixel cp. For example, when the filter coefficient as shown in FIG. 14B is set, the influence of the pixel value of the background area 31 in contact with the reflection area 34 can be increased. Thereby, a more natural reflection can be expressed.

(2) In the above-described embodiment, the predetermined pixel value is determined according to the distance between the background color and each pixel value of the background candidate coordinates in the original image. It does not matter as a value. For example, it can be a background color. Even in this case, since the same distance relationship as that in the above-described embodiment can be obtained, the generation of the background image for synthesis and the generation of the combined image can also obtain the same effects as those in the above-described embodiment. . However, in this case, the image composition unit 27 needs to generate the background candidate pixel converted image in the above-described embodiment as the foreground image.

  The present invention can be used for an image composition technique using a chroma key technique.

A: Reception device AU: Reception unit 1: Body 2: Semiconductor drive 3: Disk drive 4: Receipt certificate printing device 5: Connector 6: Touch panel 7: Display 15: Reception unit (image acquisition unit)
16: Receipt certificate creation unit 17: Memory 20: Image composition module 21: Background color acquisition unit 22: Background candidate coordinate specification unit 23: Background candidate pixel conversion unit 24: Distance calculation unit 25: Parameter storage unit 26: Background image generation unit 27: Image composition unit

Claims (5)

An image acquisition unit for acquiring an original image including a background region and a subject and a background image to be combined with the original image;
A background color acquisition unit that acquires a background color that is a color of the background region;
A background candidate coordinate specifying unit that specifies, as background candidate coordinates, coordinates of pixels that are candidates for the background region from the original image based on the background color;
A distance calculation unit that calculates a distance between a predetermined pixel value determined based on the background color and each pixel value of the background candidate coordinates in the original image;
An image processing parameter storage unit for storing an image processing parameter corresponding to the distance;
For each pixel of the background candidate coordinates in the background image, the image processing parameter corresponding to the distance at the background candidate coordinates is acquired from the image processing parameter storage unit, and image processing is performed using the image processing parameter. A background image generation unit that generates a background image for synthesis,
An image composition device comprising: an image composition unit that composes a foreground image obtained by removing the background color from a pixel value of the background candidate coordinates in the original image and the composition background image.   The image synthesizing apparatus according to claim 1, wherein the image processing is a process of reducing a sharpness of an image, and the image processing parameter is set so that the sharpness decreases as the distance increases. . The predetermined pixel value is determined according to a distance between the background color and each pixel value of the background candidate coordinates in the original image,
The image composition device according to claim 1 or 2, wherein the image composition unit uses an image obtained by subtracting the predetermined pixel value from a pixel value of the background candidate coordinates in the original image as the foreground image.   The image composition according to any one of claims 1 to 3, wherein the background image generation unit adjusts an image processing parameter based on a boundary position of a background candidate region configured from the background candidate pixels. apparatus. Obtaining an original image including a background region and a subject and a background image to be combined with the original image;
Obtaining a background color that is the color of the background region;
Identifying the coordinates of pixels that are candidates for the background region from the original image based on the background color as background candidate coordinates;
Calculating a distance between a predetermined pixel value determined based on the background color and each pixel value of the background candidate coordinates in the original image;
For each pixel of the background candidate coordinates in the background image, an image processing parameter corresponding to the distance at the background candidate coordinate is determined, and image processing is performed using the image processing parameter to obtain a composition background image. Generating step;
A step of combining the foreground image obtained by removing the background color from the pixel value of the background candidate coordinates in the original image and the combining background image;

Images