JP2009237927A - Image composition method and device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enable an image composition device, which obtains a composite image by compositing acomposition image having transmission information with a background image, to obtain a result having reality while reducing computational loads. <P>SOLUTION: This image composition device includes: a color average image generator 18 for generating a plurality of color average images B<SB>0</SB>to B<SB>3</SB>different in grainsize from a background image Go by performing blurring processing; a background blurring processor 15 for selecting a weighting coefficient corresponding to transmission information Oa of each pixel of the pixel in a composite image O and calculating a weighting sum of pixel values of the plurality of average color images B<SB>0</SB>to B<SB>3</SB>in pixels corresponding to the pixel to thereby generate a blurred background image Gb with the weighting sum defined as a pixel value; and an image composition processor 16 for compositing the composite image O with the blurred background image Gb in accordance with the transmission information Oa by α blending to obtain a composition result image M. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a technology for realizing augmented reality (AR) and mixed reality (MR), and in particular, for a real-world captured image captured by an imaging device such as a camera, CG ( The present invention relates to an image composition method and apparatus for superimposing and displaying a virtual object generated using computer graphics technology or a real object photographed in advance.

  There is a technique for synthesizing an object such as a virtual object that does not originally exist in a captured image as if the object exists in the captured image in the real world. In this case, a captured image of the real world is used as a background image, and an image of an object such as a virtual object (referred to as a composite image) is superimposed on the background image. Whether the object to be synthesized with the background image is an object generated by CG or a real-world object obtained by pre-photographing, an object like glass or plastic There is a case where an object through which is transmitted is combined with a background image as a combined image. In such a case, an α (alpha) value, an α mask, and the like are known as a method for expressing the light transmittance of an object to be synthesized. In commercially available image editing software and the like, an α mask, an α channel, and the like can generally be used.

  Patent Document 1 (Patent No. 4007160 “Image processing method and apparatus, and image processing program and recording medium on which the image processing program is recorded”) by the applicant of the present application has an image (composite image) in which an α mask is set. ) Is synthesized with a background image, a method for obtaining a better synthesis result is shown. The outline is that a portion with high transparency in the composite image is combined without blurring the background image, and a portion with low transparency in the composite image is combined with the background image blurred. FIG. 9 shows an example of image composition shown in Patent Document 1, and shows a case where an image of a drinking water plastic bottle is used as a composite image O having transmission information and is synthesized with a background image Go. Yes. The composite image O includes color information Oc indicating the color of each pixel (pixel) and opacity information Oa indicating the opacity of each pixel as an α mask. In the opacity information (α mask) Oa, the white portion is a portion with high opacity. As a result of combining such a combined image O with the background image Go, a combined result image M is generated. The transparent portion in the plastic bottle is expressed as a region where the background image can be seen in the synthesis result image M.

  10 and 11 show an example in which such a synthesized image is synthesized on a background image, which is an image of a desk having a striped pattern, using a teapot image assumed to be made of glass or the like as a synthesized image having transmission information. Show. In the case shown in FIG. 10, since the composite image is simply combined with the background image based on the α mask, the striped pattern of the background image is clearly seen at the lower part of the teapot, and lacks reality. Therefore, Patent Document 1 shows that the result shown in FIG. 11 can be realized by blurring and synthesizing the background image for the portion with low transparency by using the expression (1).

Here, x and y are coordinate values in each image, PS (x, y) is a pixel value at pixel (x, y) in the combined result image, and PC (x, y) is a color image (color information). Oc), and PT (x, y) is a pixel value in a transparency image (an image obtained by subtracting opacity information Oa from 1 for each pixel), and PZ is a pixel in the background image Go. Value. K is a constant indicating the maximum range of adjacent pixels used for calculating blur, and k is given by k = K × PT (x, y).

In the synthesized result image shown in FIG. 11, the striped pattern of the background image is expressed as blurred in the region below the teapot, so that an illusion that the image is “reflected” is given to the human eye. It has succeeded in expressing a feeling.
Japanese Patent No. 4007160

  However, the method described in Patent Document 1 described above has a problem that it takes time to perform the calculation because the process of “blurring” is performed for each pixel.

  FIG. 12 conceptually illustrates the processing in equation (1). Here, the following matters can be pointed out in the calculation of the pixel unit in the composition.

  (1) The background image blurring process is performed by obtaining a color average of areas according to opacity. Therefore, the blur calculation cannot be started unless the mask of the opacity (or transparency) of the composite image is determined.

  (2) In the opacity mask, when there are many regions with high opacity, the pixel area for obtaining the average color becomes large. Since the calculation of the color average of the pixel area is repeated many times, the calculation amount increases. That is, regarding the opaque mask of the composite image, the calculation time increases when the ratio of the opaque area is large, and the calculation time decreases when the ratio of the opaque area is small, resulting in a large variation in the calculation processing time. In particular, when a synthesis process is performed in real time on a background image captured by a video camera, a situation in which the frame rate is not constant appears significantly according to the calculation time.

  (3) In the calculation process for obtaining the color average of the background image for adjacent pixels, the calculation for obtaining the color average is performed independently for each common region, and as a result, many unnecessary calculations are included.

  An object of the present invention is to provide an image synthesizing method and apparatus capable of simplifying calculation processing while taking advantage of the technique disclosed in Patent Document 1. Specifically, it is to achieve the following items.

(1) Even if the opaque mask is not fixed, the background image blurring calculation can be started.
(2) The time required for calculation is made uniform regardless of the size of the opaque region.
(3) When calculating the color average, the calculation which overlaps in the same part is not generated.

  The image composition method of the present invention is an image composition method for obtaining a composite result image by compositing a composite image having transmission information with a background image, and a plurality of color average images having different particle sizes from the background image by performing blurring processing For each pixel in the composite image, a weighting coefficient corresponding to the transmission information of the pixel is selected, and a weighted sum of pixel values of a plurality of color average images in the pixel corresponding to the pixel is calculated. As a result, a blurred background image having a weighted sum as a pixel value is generated, and the synthesized image is synthesized with the blurred background image according to the transmission information by α blending to obtain a synthesized result image.

  An image composition apparatus according to the present invention is an image composition method for obtaining a composite result image by compositing a composite image having transmission information with a background image, and a plurality of color average images having different particle sizes from the background image by performing blurring processing For each pixel in the composite image, a weighting coefficient corresponding to the transmission information of the pixel is selected, and a weighted sum of pixel values of a plurality of color average images in the pixel corresponding to the pixel is calculated. By obtaining a background image blur processing unit that generates a blurred background image having a weighted sum as a pixel value, and synthesizing the synthesized image with the blurred background image according to the transmission information by α blending, a synthesized result image is obtained. Image composition processing means.

  According to the present invention, by adopting the above-described configuration, (1) even if an opaque mask is not determined, background image blurring calculation can be started, and (2) calculation is required regardless of the size of the opaque region. The time can be equalized, and (3) when calculating the color average, it is possible to prevent the occurrence of calculations that overlap in the same part, thereby obtaining a realistic composite result image while reducing the calculation process. Has the effect of becoming

  Next, a preferred embodiment of the present invention will be described with reference to the drawings.

  In general, the “blurring” process for an image is performed by calculating the color average (pixel value average) of pixels within a predetermined range in the vicinity of the target pixel for each pixel constituting the image, using the pixel as the target pixel. This is a process for obtaining a pixel value. In the above-described patent document 1, when performing blurring processing on a background image, the range of pixels to be averaged in blurring processing is set as opacity information (or transparency information) of corresponding pixels in the composite image. It was decided according to. In this method, blurring processing must be performed for each pixel in the background image while changing the size of the range to be averaged, so that a large amount of calculation processing is required.

  Therefore, in this embodiment, the concept of “granularity” in the blurring process is introduced. The granularity indicates how much of the pixel value average (color average) of pixels in the blurring process is to be obtained. In the present embodiment, the range in the image subjected to the blurring process (this range). Are called the “granularity range”). In other words, if a certain granularity corresponds to the size of m × m pixels, the granularity range is m × m pixels, and each of the images subjected to the blurring process with the granularity has a size of m × m pixels. And the pixel values of the pixels in the area are the same for each area of the size of m × m pixels, and the pixels of the pixels in the area in the original image The average value. Further, n is an integer of 0 or more, and the level of granularity is expressed as granularity n. An image that has been subjected to the blurring process with the concept of grain size is referred to as a color average image having a grain size n. Of course, the image before the blurring process may be a grayscale image instead of a color image. In such a case, it is also called a color average image. However, in this case, the color average image is also a grayscale image.

  In this embodiment, the color average image calculated once for a certain granularity level is reused when obtaining a color average image with a coarser granularity level. FIG. 1 conceptually shows this process. n indicates a level of granularity, a small square indicates a pixel, and a square with a thick line indicates a particle size range at the level of the granularity.

First, in the case of n = 0, in the color average image, the color information of the pixel unit of the background image Go is held as it is. That is, the color average image B _{0 with} n = 0 is the same as the background image Go.

For the color average image B _{1 with} n = 1, the color average of 4 pixels (2 × 2 pixel area) in the color average image B ₀ is obtained and the value is the value of each pixel within the granularity range with n = 1. And

For n = 2 and later, four areas of the respective areas (granularity range at n−1) obtained in the color average image B _n−1 are grouped into one group, and the color average is obtained as the color average image B _n . . In this case, since the color average in each area has already been calculated in the process of obtaining the color average image _n −1, in practice, only the average of the values (RGB values or luminance values) for each of the four areas is obtained. It's okay. The above operation is expressed by a general expression as shown in Expression (2).

Here, Average {} is a function for calculating the average of elements in {}.

Hereinafter, for convenience of explanation, it is assumed that the maximum value of n is 3, and color average images B ₀ , B ₁ , B ₂ , B ₃ are used. FIG. 2 shows a process for obtaining B ₀ , B ₁ , B ₂ , and B ₃ . In FIG. 2, the smallest square area indicates the granularity area at each granularity level.

  By such a calculation, it is possible to avoid redundantly performing the addition of the same region in the process of obtaining the color average and to save the calculation. In addition, since the calculation result of the previous step can be reused as it is, the calculation process can be performed more efficiently than in the conventional method.

In the present embodiment, the blurred background image Gb is generated from the color average images B _{0 to} B ₃ as described above. The blurred background image Gb is an image obtained by performing, for each pixel in the composite image O, a blurring process corresponding to the transmission information Oa of the pixel on the corresponding pixel of the background image Go. As described, the blurred background where the blur in the background image is small for the part with high transparency in the composite image and the blur in the background image is large for the part with low transparency in the composite image The image is Gb. Specifically, for each pixel in the composite image, a weighting coefficient corresponding to the transmission information Oa of the pixel is selected to obtain a weighted sum of the pixel values of the color average images B _{0 to} B _{3 in} the pixel corresponding to the pixel. Thus, the blurred background image Gb having the weighted sum as a pixel value is generated. That is, assuming that the coordinates of the pixel to be calculated are (x, y), the pixel value of the blurred background image Gb (x, y) at the coordinates (x, y) is obtained by the following equation.

Gb (x, y) = W ₀ (Oa (x, y)) · B ₀ (x, y)
+ W ₁ (Oa (x, y)) · B ₁ (x, y)
+ W ₂ (Oa (x, y)) · B ₂ (x, y)
+ W ₃ (Oa (x, y)) · B ₃ (x, y)
Sum {W ₀ (Oa (x, y)), W ₁ (Oa (x, y)), W ₂ (Oa (x, y)), W ₃ (Oa (x, y))} = 1
Here, Sum {} is a function for calculating the sum of the values in {}, Oa (x, y) is the value of the transmission information Oa at the coordinates (x, y) of the composite image O, and i = 0, 1 as 2,3, Bi (x, y) is the pixel value at the coordinates (x, y) of the color average image B _i. W _i is a weighting coefficient that gives an influence on the calculation result for each of the prepared color average images B _{0 to} B ₃ , and the total sum of each W _i is 1. Suppose that it is defined in advance.

FIG. 3 shows a conceptual diagram of Gb (x, y) calculation processing. In the above description, the color average images B _{0 to} B ₃ are all images having the same number of pixels as the background image Go. However, in a color average image having a large granularity (coarse granularity), the number of pixels included in one granularity range. Increases, and the burden of the process of copying the same pixel value to each pixel within the granularity range becomes heavy. Therefore, when storing in a memory or the like, the data is stored in units of the granularity range, that is, the size of the color average image is reduced as the granularity level increases, and the pixel value of the blurred background image Gb is set. At the time of calculation, it is conceivable to reduce the calculation load by performing conversion calculation of reference coordinates.

Next, a description will be given weighting factors W _i as described above. Figure 4 is an illustration how to have a weight of how much each W _i depending on the opacity at the coordinates (x, y) as an example, is represented as having been defined on the map. This is merely an example, and it is preferable that the mapping can be manually changed for each site according to the installation location of the camera that captures the background image and the captured image determined accordingly. In any case, when the opacity of the object included in the composite image is high (that is, when the transparency is high), the background image behind the object is easy to see, and thus an image close to the original background image Go. The weight of W ₀ is increased so that B ₀ affects more strongly. On the contrary, when the opacity of the object is low (that is, when the transparency is low), it is difficult to see the background image behind the object, so the image B ₀ close to the original background image Go has a weaker effect. Thus, the weight of W ₀ is lowered.

  With the above processing, the pixel value Gb (x, y) in the background image subjected to the blurring process, that is, the blurred background image Gb can be obtained. After that, the color information Oc (x, y) of the composite image and the blurred background image Gb (x, y) are superimposed by a normal α blend process using the value of the transmission information Oa (x, y). By combining, a combined result image M is obtained.

  By performing the processing on the coordinates (x, y) so far for all the pixels in the region covered by the composite image, the same effect as that shown in Patent Document 1 can be obtained. Although the technique of this embodiment is inferior to that described in Patent Document 1 in terms of the degree of strictness of blurring, it is possible to dramatically reduce the load of calculation processing compared to that described in Patent Document 1. .

FIG. 5 is a block diagram illustrating an example of a configuration of an image composition device that performs the above-described processing. This image synthesizing device synthesizes a synthesized image O having transparency information stored in the synthesized image storage unit 12 with an image (background image Go) photographed by the camera 11 and outputs the synthesized image O to the display device 17 as a synthesized result image M. To do. This image synthesizing device determines the arrangement location of the synthesized image O with respect to the background image Go, and outputs the color information Oc and the transmission information Oa of the synthesized image O, and the synthesized product arranging device 13 and the color average images B _{0 to} B ₃ . Temporary storage memory 14, color average image generator 18 that generates color average images B _{1 to} B ₃ from the background image Go according to the above-described procedure, and transmission information of each pixel in the composite image O select a weighting factor W _i corresponding to by obtaining a weighted sum of the pixel values of the color average image B ₀ .about.B ₃ in the pixel corresponding to the pixel, blurred already background image Gb of the weighted sum as the pixel value A background image blur processing unit 15 for generating the image, and an image composition for synthesizing the synthesized image O with the blurred background image Gb according to the transmission information Oa by α blending to obtain and output a synthesized result image M And a processor 16. Since the color average image B ₀ is the background image Go as described above, the background image Go from the camera 11 is parallel to the color average image generator 18 and the storage area of the color average image B ₀ in the memory 14. Sent.

In the embodiment described above, in order to obtain one synthesis result image M, four types of color average images B _{0 to} B ₃ having different particle sizes are required. However, when the input background image Go is a moving image composed of a plurality of continuous frames, it is not necessary to obtain all of the color average images B _{0 to} B ₃ for each frame of the moving image.

Examining the relationship between the color average image and the granularity, the color average image having a fine granularity, for example, the color average images B ₀ and B _1, are detailed image information and have a large influence on the image quality, and thus are calculated in units of one frame. You should do it. On the other hand, a color average image with a coarse particle size, for example, the color average images B ₃ and B ₄ , has only blurred image information with unclear outlines. Moreover, a large weight is given to the color average images B ₃ and B ₄ when the opacity of the composite image O is large. In such a case, the background image of the composite image O is obtained. The influence of the blurred background image Gb on the image quality is not so great. Therefore, even if there is a moving object in the moving image serving as the background image, it can be considered that the influence is small in the color average images B ₃ and B ₄ which are relatively blurred images. In other words, for the color average images B ₃ and B ₄ with large blurring, it is not necessary to calculate strictly for each frame, and the calculation load can be further reduced by performing decimation for every several frames. it can. As a specific example, “the color average images B ₀ and B ₁ are updated every frame, the color average image B ₃ is updated every two frames, and the color average image B ₄ is updated every four frames”. It can be considered.

  When all the processing is implemented by software for the image composition apparatus of the present embodiment, it is effective to reduce the load by thinning the processing in this way.

When the calculation result of the blurring process can be processed in a pipeline manner, the process of generating the color average image _Bn from the color average image _Bn-1 can be executed with a time difference as follows.

B ₀ is the latest background image Go given from the camera.
B ₁ is calculated with reference to B ₀ one frame before. (1 frame delay from B ₀ )
B ₂ is calculated with reference to B ₁ one frame before. (2 frames behind B ₀ )
B ₃ is calculated with reference to B ₂ one frame before. (3 frames behind B ₀ )
If such pipeline processing can be performed, the color average images B _{0 to} B ₄ can be calculated not in series but in parallel. In an environment where multi-process is possible, or in hardware implementation, calculation is performed. This can contribute to shortening the time.

FIG. 6 shows an example of the configuration of an image composition device that executes pipeline processing in this way. The illustrated apparatus is similar to the apparatus shown in FIG. 5 except that three converters 21 to 23 for executing pipeline processing are provided instead of the color average image generator. It is different from what is shown in. Converter 21 generates the color average image B ₁ performs color averaging process by reading the color average image B ₀ stored in the memory 14, and stores the generated color average image B ₁ to the memory 14. Similarly, the converter 22 reads the color average image B ₁ stored in the memory 14, generates the color average image B ₂ and stores it in the memory 14, and the converter 23 stores the color average image stored in the memory 14. The image B ₀ is read and a color average image B ₁ is generated and stored in the memory 14.

In the above description, four types of color average images B _{0 to} B ₃ are generated. However, a color average image B _n (n ≧ 4) with a coarser grain size may be generated.

With the methods described above, the following items listed as items to be achieved are realized:
(1) Even if the opaque mask is not fixed, the background image blur calculation can be started.
(2) Make the time required for calculation uniform regardless of the size of the opaque area.
(3) When calculating the color average, the calculation which overlaps in the same part is not generated.

  The image synthesizing apparatus described above can also be realized by reading a computer program for realizing it into a computer such as a personal computer (PC) or a server computer and executing the program. A program for performing image composition in the above-described procedure is read into a computer by a recording medium such as a CD-ROM or via a network.

  Next, application examples of the above-described image composition method and apparatus of the present invention will be described.

[Application Example 1]
FIG. 7 shows an example in which a PC (personal computer), a large screen monitor, and a moving image camera that acquires a background image are combined to perform image synthesis in real time. In this example, with respect to an object (object) or character (person image) to be a composite image, the image information and α mask information are stored in advance in the PC when the object or character is viewed from many viewpoints. It is assumed that

(Mounting by conventional technology)
For comparison, first, a case will be described in which synthesis processing is performed by software using the conventional technique.

  First stage: The user stands in front of a large-screen monitor so that the marker is photographed by the camera, such as wearing or holding the marker placed in front of the system. The video camera captures one frame of the user, the marker, and their background, and sends the image data to the PC.

  Second stage: The PC receives captured image data, and calculates the position, size, and orientation of the marker from the image data by automatic image recognition processing. In accordance with the position, size, and orientation of the marker determined as described above, the PC specifies the type of object or character to be combined, and selects the stored data that is closest to the viewpoint from among the stored data used for combining, and the captured image Determine the position of the composition within.

  Third stage: When the position and size of the object to be combined are determined, the area where the blurring process should be performed on the captured image is determined. The PC performs a blurring process on each pixel in this area using the α mask information of the composite object.

  Fourth stage: A composite (object or character) is superposed by α blending as a background image on which the photographed image for which the blurring process has been completed is placed behind.

  Fifth stage: Depending on the application settings, if mirror reversal processing is required, mirror reversal is performed, or if mirror reversal is not necessary, the PC can display the composite result image on a large monitor. To display. Thereafter, the process returns to the first stage and is repeated.

  The above is the processing flow when the conventional technology is implemented by software.

  In this case, if the marker is far from the camera or the composite is small, the pixel area that needs to be blurred is small, so the amount of calculation required for the blurring itself is small and the screen is updated. Frame rate will be faster. On the other hand, when the marker is close or the composite is large, it is necessary to perform blurring processing over almost the entire captured image. As described in the above section [Problems to be solved by the invention], in the blurring process between adjacent pixels, there is a possibility that overlapping calculations may be repeated in vain, which takes time and results in the process. Image update frame is slow. Moreover, it is necessary to process sequentially from the first stage, and the blurring process cannot be started unless the position of the composite is determined.

(Implementation by the method of the present invention)
Next, the case where it implements by the method based on this invention is described.

  First stage: the same as in the conventional method described above.

Second stage: The PC performs the following two processes in parallel:
[2a] Perform marker recognition processing as in the second stage of the prior art;
[2b] A plurality of images (color average images) obtained by sequentially blurring the captured images shown in the present proposal are generated.

  Third stage: The PC generates a blurred background image according to a weighting factor while referring to a plurality of generated color average images and an α mask where the composite is placed.

  Fourth stage: A composite (object or character) is superimposed by α blending as an image with the blurred background image placed behind.

  Fifth stage: Same as the above-mentioned conventional method.

  As described above, when the method of the present invention is applied, whether or not the blurring process covers the entire image while avoiding useless duplication calculation, The amount of calculation is constant. Therefore, it is possible to avoid the problem that the frame rate varies. In recent PCs equipped with a multi-core CPU and an expansion board that supports parallel arithmetic processing can be used, in the second stage, the blur processing (in parallel) without waiting for the position of the image object to be specified ( Color average image generation process) can be started, and the entire process can be efficiently performed.

[Application 2]
FIG. 7 shows a system in which a server for performing image composition processing is provided on a network to provide an image composition processing service. A processing server 31 that performs image composition by the method of the present invention is connected to a data network 30 such as the Internet, and a mobile phone 32 and a PC 33 are connected. The mobile phone 32 and the PC 33 can access the processing server 31 by e-mail or using HTTP (Hypertext Transfer Protocol) as a protocol.

  First, a paper booklet including a marker in a page is distributed to users by means such as store installation or mailing. The user separates the page on which the marker is printed from the booklet and uses it for photographing. The user can send an image captured by the camera-equipped mobile phone 32 to the processing server 31 by e-mail or HTTP. Alternatively, an image taken with a digital camera can be sent to the processing server by e-mail or HTTP using the PC 33. At this time, a product number or the like for designating an object to be combined can be sent together.

  The processing server 31 automatically recognizes a marker from the received image, performs an image composition process using an object or character, and returns it to the user. In the illustrated example, when an image in which a marker and a coffee cup are arranged on a desk is sent to the processing server 31 as a transmission image (captured image), the image is made of glass on the marker by image synthesis processing in the processing server 31. An image in which a vase is arranged is generated as a combined result image, and the terminal side receives the combined result image as a received image.

  As described above, the user can obtain the resultant composite result image without providing the terminal with software for image recognition or image composition.

  In the case of using the conventional technique, there is a wasteful calculation in the blurring process, and thus it is easy to put a load on the processing server. In particular, in the case of a server client type system configuration, a situation in which a large number of users are accessing at the same time is assumed. Therefore, as the processing on the processing server side is lighter, the performance of the entire system is improved. By applying the method according to the present invention, the processing load on the processing server side is reduced. Therefore, even when a heavy system is built on the processing server side, the performance is the same as in the case of the prior art with a smaller number of units. Obtainable.

It is a figure which shows notionally the process in which the color average image from which a particle size differs is produced | generated sequentially. The particle size is a diagram illustrating a process of generating a different color average image _{B 0, B 1, B 2} , B 3. It is a figure explaining the process which calculates | requires the blurred background image Gb in the image composition method of one Embodiment of this invention. It is a diagram showing a relationship between opacity and the weighting factor W _i. It is a block diagram which shows the structure of the image composition apparatus of one Embodiment of this invention. It is a block diagram which shows the structure of the image composition apparatus of another embodiment of this invention. It is a figure which shows the example applied to the image synthesis in real time with respect to a moving image. It is a block diagram showing a system in which an image composition processing service is provided by a server on a network. It is a figure which shows the example which synthesize | combined the synthesized image which has transparency information with the background image. It is a figure which shows an example of a synthetic result image. It is a figure which shows an example of a synthetic result image. It is a figure which shows notionally the process in the image composition method shown by the patent 4007160 specification.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Camera 12 Composite image memory | storage part 13 Composite arrangement | positioning device 14 Memory 15 Background image blurring processor 16 Image composition processor 17 Display apparatus 18 Color average image generator 21-23 Converter 30 Data network 31 Processing server 32 Cellular phone 33 PC (Personal computer)

Claims (9)

An image composition method for obtaining a composite result image by compositing a composite image having transmission information with a background image,
A color average image generation step of generating a plurality of color average images having different particle sizes from the background image by performing a blurring process;
For each pixel in the composite image, a weighting coefficient corresponding to the transmission information of the pixel is selected, and a weighted sum of pixel values of the plurality of color average images in the pixel corresponding to the pixel is obtained. Generate a blurred background image with pixel values,
The blended image is synthesized with the blurred background image according to the transmission information by α blending to obtain a synthesized result image.
Image composition method. The background image is a color average image with a particle size of 0, n is an integer of 1 or more,
In the color average image generation step, the color average of the particle size n is obtained by obtaining the color average of a predetermined number of particle size ranges at the particle size (n-1) for the color average image of the particle size (n-1). The image synthesizing method according to claim 1, wherein processing is generated to thereby recursively generate a color average image with a plurality of granularities.   The background image is a moving image composed of a plurality of continuous frames, and a color average image having a particle size coarser than the particle size i is generated by thinning out every predetermined frame with i as a predetermined threshold value. 3. The image composition method according to 2. The background image is a moving image composed of a plurality of continuous frames;
In the color average image generation step, by obtaining a color average for the predetermined number of particle size ranges at the particle size (n−1) with respect to the color average image of the particle size (n−1) in the previous frame, The image composition method according to claim 2, wherein a color average image having a particle size n in a current frame is generated. An image composition method for obtaining a composite result image by compositing a composite image having transmission information with a background image,
A color average image generating means for generating a plurality of color average images having different particle sizes from the background image by performing a blurring process;
For each pixel in the composite image, a weighting coefficient corresponding to the transmission information of the pixel is selected, and a weighted sum of pixel values of the plurality of color average images in the pixel corresponding to the pixel is obtained. Background image blur processing means for generating a blurred background image as a pixel value;
image synthesis processing means for synthesizing the synthesized image with the blurred background image according to the transmission information by α blending to obtain a synthesized result image;
An image synthesizing apparatus. The background image is a color average image with a particle size of 0, n is an integer of 1 or more,
The color average image generation means obtains a color average of a predetermined number of particle size ranges at the particle size (n-1) for the color average image of the particle size (n-1), thereby obtaining a color average of the particle size n. The image synthesizing apparatus according to claim 5, wherein the image synthesizing apparatus generates a process and thereby recursively generates a color average image with a plurality of granularities.   The background image is a moving image composed of a plurality of continuous frames, and a color average image having a particle size coarser than the particle size i is generated by thinning out every predetermined frame with i as a predetermined threshold value. 6. The image composition device according to 6. The background image is a moving image composed of a plurality of continuous frames;
A memory for holding the plurality of color average images;
The color average image generating means is configured to output the color for the predetermined number of granularity ranges at the granularity (n−1) with respect to the color average image of the granularity (n−1) in the previous frame stored in the memory. The image synthesizing apparatus according to claim 6, further comprising a conversion unit that generates a color average image having a granularity n in a current frame by storing an average and stores the average image in the memory.   The program which makes a computer perform the image composition method of any one of Claims 1 thru | or 4.

Images