JP2009033224A - Device and program for generating composited image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a composited image generating device capable of creating a composited image, where required computational complexity is small, resolution near a junction portion does not deteriorate, and a change in density (gradation, color tones, or the like) is not prominent in the junction portion. <P>SOLUTION: The composited image generating device has: a geometrical conversion section for determining arrangement in the composited image of a plurality of inputted original images for composition; a key generation section for generating composited key image information indicating the mixing ratio of the plurality of original images for composition at positions, based on key image information for indicating whether the respective original images for composition exist at respective positions on the composited image; and a composite processing section for generating a composited image by compositing the plurality of original images for composition using the composited key image information. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a composite image generation apparatus and a composite image generation program for generating a composite still image or a composite moving image by aligning and combining a still image group or a moving image group captured by overlapping a part of a subject. . In particular, an apparatus and a program for generating a panoramic still image or a panoramic moving image based on a still image group or a moving image group obtained by photographing the same subject using a plurality of cameras or a single camera while dividing time About.

Using a plurality of cameras, a panoramic image or the like can be created by joining still image groups or moving image groups captured with slightly different orientations in a state where partial areas of the subject are overlapped. Similar panoramic images can be obtained by using a single camera to release the shutter multiple times (when taking still images) or taking images at different times (when taking moving images). Can be obtained.
At this time, even if the same subject is shot, if a different camera is used, or if conditions such as light rays differ depending on the shooting location and time, the shot image and video will be different because the brightness and color of the image will be different. When an image as it is taken is used, it does not naturally join, so it is necessary to apply a process that makes it difficult to see the boundary of the combined boundary portion.

  In the conventional technique, the processes necessary for this are an alignment process, a shading correction process, a joint point search process, and a joint point smoothing process. For registration processing, to search for the corresponding points in the image in order to superimpose the images, adjust the position and size in the geometric conversion process, and correct geometric distortion that differs slightly from image to image. I can do it. In the shading correction process, a histogram of the entire image to be combined is calculated, and the color tone of the entire image is adjusted by adjusting the luminance average and the color tone. By this processing, it is possible to reduce a difference in color tone that differs for each original image. However, it is difficult to completely match the density and color tone. For this reason, a slight difference in luminance and color tone occurs in the joining line. However, since this is easily perceived by the human eye as an image boundary line, it is necessary to search for a junction point to be a junction line by searching for a place where the density level difference is small so that the junction line is not noticeable. is there. Nevertheless, since luminance and color tone differences remain in the joining line, it is necessary to perform a density value smoothing process.

Non-Patent Document 1 creates a composite image so as to make the joint line as inconspicuous as possible by searching for the joint line as described above, in particular, by searching for a joint line in which a difference in density level between a plurality of original images becomes small. The method is described.
Mikio Takagi and Yoshihisa Shimoda (supervised), “New Edition Image Analysis Handbook”, 2004, University of Tokyo Press, p. 1161-p. 1167

However, the method described in Non-Patent Document 1 has a problem in that the calculation load of the joint line search process is large. Further, it is necessary to perform a smoothing process after the joint line search process, and as a result, there is a drawback in that the resolution of the image is reduced in the vicinity of the joint line.
The present invention is based on the recognition of this problem, and does not require processing with a large calculation load such as search processing for joint lines, and obtains a composite image in which joint lines are not conspicuous without reducing resolution. It is an object of the present invention to provide a composite image generation apparatus and a composite image generation program that can be used.

[1] In order to solve the above-described problem, a composite image generation apparatus according to an aspect of the present invention includes a geometric conversion unit that determines an arrangement of a plurality of input composite original images in a composite image, and the composite image. Key generation for generating composite key image information representing a mixing ratio of the plurality of composition original images at the position based on key image information representing whether or not each composition original image exists at each of the above positions And a synthesis processing unit that generates a synthesized image by synthesizing the plurality of source images for synthesis using the synthesized key image information.
According to this, based on the key image information indicating whether or not each original image for synthesis exists at each position on the synthesized image, that is, without requiring a joint line search process depending on the pixel value. Key image information can be generated. Then, by using the composite key image information representing the mixing ratio generated in this way, a simple calculation process called a sum of products operation of the pixel value of the composite original image and the value of the composite key image is performed. A composite image can be generated, and the above-described problems can be solved.

[2] Further, according to one aspect of the present invention, in the composite image generation device, the key generation unit may be configured to store the composite original image in a region where the plurality of composite original images overlap in the composite image. The synthetic key image information representing the mixture ratio according to the distance from the boundary line of the overlap value to the position is generated.
The overlapping value of the composition original image is a value representing how many composition original images overlap at a position on the composition image. That is, the boundary line of the redundancy value is a boundary line that divides whether or not any of the plurality of composition original images is present. According to this configuration, the mixing ratio in the case of compositing the composition original image depends on the distance from the boundary line. In other words, a composite key image that suppresses discontinuous changes in color tone and gradation between the original images for compositing, and obtains a composite image that makes it difficult for human eyes to see the joint line according to the degree of overlap of the original images for compositing. Information can be generated. At the same time, the composite key image information need not depend on the pixel value of the composite original image.

[3] Further, according to one aspect of the present invention, in the composite image generation apparatus, the key generation unit represents the mixture ratio according to a redundancy value of the composition original image in the composite image. It is characterized in that synthetic key image information is generated.
According to this, it is possible to use a mixing ratio according to the redundancy value. In other words, when the overlap value is large, the mixing ratio of the original images for synthesis is lowered and the overlap value is small (for example, the overlap value is 1 at a position where only one current image exists and does not overlap). In this case, since the mixing ratio of the original images for synthesis can be increased, the color tone and gradation of the entire generated synthesized image can be balanced.

[4] In addition, according to an aspect of the present invention, in the composite image generation device, the key generation unit is configured to perform the mixing ratio according to an average value of the reciprocal of the overlap value in a predetermined neighborhood region of the position. The composite key image information representing the above is generated.
According to this, with a relatively small amount of calculation, it is possible to obtain synthetic key image information that changes continuously and smoothly while maintaining a mixing ratio according to the overlap value.

[5] Further, according to one aspect of the present invention, in the composite image generation apparatus, the key generation unit excludes an area where the original image for composition does not exist from an area near a predetermined position of the position. The average value of the reciprocal of the redundancy value is calculated.
According to this, the density of the image after synthesis is reduced by reducing the mixing ratio at a position including the area where the original image for synthesis does not exist in the vicinity of the area where the original image for synthesis exists. You can avoid being depressed.

[6] In the composite image generation device according to one aspect of the present invention, in the composite image generation device, the composite original image input is a moving image, and the composite key image generated by the key generation unit A composite key image information storage unit for storing information; and the composite processing unit uses the composite key image information read from the composite key image information storage unit to generate the plurality of video images for each frame of a moving image. The composition original image is synthesized.
According to this, if the composite key image information is calculated only once based on the layout of the composition original image, the composite key image information can be used in common for each frame of the moving image. Therefore, it is not necessary to calculate the composite key image information. This is a configuration that takes advantage of the advantage obtained by the present invention using a synthesis method that does not depend on the pixel value of the original image for synthesis.

  [7] A composite image generation program according to an aspect of the present invention includes a geometric transformation process for determining an arrangement of a plurality of composite original images to be input in a composite image, and each position on the composite image. A key generation process for generating composite key image information indicating a mixing ratio of the plurality of composite original images at the position based on key image information indicating whether or not the composite original image exists; and the composite key A computer is caused to execute processing in each of the combination processing steps of generating a composite image by combining the plurality of composite original images using image information.

  According to the composite image generation device and the composite image generation program of the present invention, when creating a composite image in which there is no unnatural change in color tone / gradation in the joint line portion and the joint portion is inconspicuous, the joint line search is performed. It does not require processing with a heavy calculation load. Further, the resolution is not lowered near the joint. Further, the density of the image does not drop even in the vicinity of a region where no image exists, and a good composite image can be obtained.

Hereinafter, the best embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram illustrating a functional configuration of the composite image generation apparatus according to the present embodiment. The composite image generation apparatus 10 creates a panoramic composite image using a plurality of input still images or moving images (hereinafter referred to as images together). As shown in the figure, the composite image generation apparatus 10 is configured to accept input of input images (still images or moving images) 1 to N. The composite image generation apparatus 10 performs a geometric conversion and density / color adjustment unit (geometric conversion unit) 11-1 on the input image 1 and a storage unit that stores the composition original image 13-1. The storage unit stores a key image (key image information) 15-1, a key generation unit 17-1, and a composite key multiplication unit 19-1. Further, the composite image generation apparatus 10 has the same configuration as the processing configuration for the input image 1 for each of the input images 2 to N. And the addition part 21 adds the image output from N synthetic key multiplication parts 19-1, 19-2, ..., 19-N, and becomes a structure which outputs a panoramic composite image ( The combination key multiplication units 19-1, 19-2,..., 19-N and the addition unit 21 are combined processing units).

Each image shown in FIG. 1 is an image on the xy plane, and pixel values at coordinates (x, y) are stored in the storage unit. The storage unit is realized by using, for example, a semiconductor memory or a magnetic hard disk device. For example, the pixel value at the coordinates (x, y) of the composition original image 13-1 is represented by d ₁ (x, y), and the pixel value of the key image 15-1 is e ₁ (x, y). in represented, pixel values of the outputted synthesized key image from the key generation unit 17-1 are represented by k _{1 (x,} y) _(synthetic key image information), panorama output from the addition section 21 The pixel value of the image is represented by P (x, y). The same applies to each series of input images 2 to N.

Next, a procedure of processing by the composite image generation apparatus 10 will be described.
First, the geometric conversion and density / color adjustment units 11-1, 11-2,..., 11-N receive input images 1, 2,. Adjust the position to be pasted by geometric transformation. This geometric transformation includes, for example, translation of an image in the x-axis direction or y-axis direction, enlargement or reduction in the x-axis direction or y-axis direction, rotation at an arbitrary angle, and the like. In addition, each of the geometric conversion and density / color adjusting units 11-1, 11-2,..., 11-N performs density / color adjustment in order to unify the color tone. In the density / color adjustment at this stage, the overall color tone of the image is unified, but the adjustment at the joint (boundary) portion where the images are bonded cannot be performed. In this way, the composition original images 13-1, 13-2, ..., 13-N are completed. These images are represented by d ₁ (x, y), d ₂ (x, y),..., D _N (x, y), respectively.

Geometric conversion and density / color adjustment units 11-1, 11-2,..., 11-N are key images 15-1, 15-2,. Also create each. These key images are represented by e ₁ (x, y), e ₂ (x, y),..., E _N (x, y), respectively. However, in d _t (x, y) (where 1 ≦ t ≦ N), in the region where the image exists (in FIG. 1, the region in which the pattern in the composition original image is painted), e _t (x , Y) has a value of 1, and in a region where no image exists (in FIG. 1, a region in the composition original image where no pattern is painted), the value of _et (x, y) is zero.
In the case of a monochrome image, the pixel value d _t (x, y) represents a gray scale, and takes the lowest value (zero) when it is black and takes the highest value when it is white. In the case of a color image, the pixel value d _t (x, y) is a value representing the lightness of each primary color of R (red), G (green), and B (blue), and is zero when black.
In both monochrome and color cases, the value of d _i (x, y) is zero (black) for areas where no image exists.

The adding unit 21 generates a panoramic composite image by superimposing the original images for synthesis 13-1, 13-2,..., 13-N, and at that time, the key generating units 17-1, 17-2. , ..., of each of the synthetic key image generated by the 17-N values _{k 1 (x, y),} k 2 (x, y), ···, k N (x, y) and, respectively Multiplying the pixel values d ₁ (x, y), d ₂ (x, y),..., D _N (x, y) of the original image for synthesis, and adding them together, the panorama synthesized image The pixel value P (x, y) is calculated. The method for obtaining the values k ₁ (x, y), k ₂ (x, y),..., K _N (x, y) of the composite key image will be described later. It acts to adjust the concentration of the.
The pixel value P (x, y) of the panorama composite image generated by the adding unit 21 is expressed by the following equation (1).

Note that the range of values k ₁ (x, y), k ₂ (x, y),..., K _N (x, y) of the composite key image is 0 or more and 1 or less.
That is, the value _k 1 of the synthetic key image _(x, y), k 2 (x, y), · · _·, k N (x, y) is the mixing of the synthesis original image at the coordinates (x, y) It represents the ratio. For example, the mixing ratio of the coordinate (x, y) composition original image 13-1 is k ₁ (x, y). The same applies to other composition original images. Note that the value of the composite key image is normalized so that k ₁ (x, y) + k ₂ (x, y) +... + K _N (x, y) = 1.

The example, the key generation unit 17-1, all the key images 15-1 and 15-2, ..., the value of 15-N _{(respectively, e 1 (x, y)} , e 2 (x, y) ,..., E _N (x, y)) as inputs, a composite key image for the composite original image 13-1 is generated. Similarly, each of the key generation units 17-2,..., 17-N receives the values of all the key images 15-1, 15-2,. A composite key image for an image is generated. The value of this composite key image is expressed by the following equation (2).

Here, 1 ≦ t ≦ N, and Ne (x, y) represents the vicinity of the coordinates (x, y). Further, p _t (i, j) and q _t (i, j) are expressed by the following expressions (3) and (4), respectively.

_Here, D t _(i, j) is the coordinates (i, j), the key image 15-t (its value is _e t (i, j)) in, or other key images are many overlapping This is the value of the degree of overlap. This D _t (i, j) is expressed by the following equation (5).

In other words, as expressed by the equation (5), at the coordinates (i, j), the composition original image 13-t (this code “13-t” is “13-1”, “13-2”,. .., And represents any one of “13-N” (the same applies to the following), D _t (i, j) = 1, and the composition original image 13 If there are other composition original images including -t, the number of composition original images including the composition original image 13-t is D _t (i, j). For example, if only the composition original image 13-t and another one composition original image exist, D _t (i, j) = 2. However, if the compositing original image 13-t does not exist at the coordinates (i, j) (that is, e _t (i, j) = 0), regardless of whether another compositing original image exists or not. D _t (i, j) = 0.

That is, as represented by the expression (2), the value k _t (x, y) of the composite key image for the composite original image 13-t has the composite original image 13-t at coordinates (x, y). In this case, the average value of the reciprocal of the degree of overlap of the image at each coordinate in the vicinity of the coordinates (x, y) (but only in the region where the composition original image 13-t exists) is obtained. Note that when the composition original image 13-t does not exist at the coordinates (x, y), the value k _t (x, y) of the composition key image is zero. Also, there only synthesized original image 13-t is in the vicinity (i.e., the value of the key image e _t corresponding to synthesized original image 13-t _(i, j) in the 1, other key images At coordinates (x, y) such that the value is 0), the value k _t (x, y) of the composite key image is 1.0. At the coordinates (x, y) where the composition original image 13-t and another composition original image exist in the vicinity, the value k _t (x, y) of the composition key image is in accordance with equation (2). The value is greater than 0 and less than 1.

What is necessary is just to determine suitably what range Neighboring Ne (x, y) in Formula (2) is made. A range within a certain distance from the coordinates (x, y) (in this case, the range is a circle) may be a neighborhood or a range other than a circle. Note that if the range in the vicinity is widened, changes in color tone and gradation in the vicinity of the junction point will be smooth, and if the range in the vicinity is narrowed, changes in color tone and gradation will be relatively steep. In the present embodiment, the size and shape of the range of the neighborhood Ne (x, y) are constant regardless of the coordinates (x, y). In general, it has been found by the inventors' experiments that good results can be obtained when about half of the overlapping region of the original images for synthesis is set as the neighborhood range. About half of the overlapping region is preferably within a range of 30% to 70%, for example. For example, it is still more preferable in it being in the range of 40% to 60%.
As expressed in Expressions (2) to (5), values k ₁ (x, y), k ₂ (x, y),..., K _N (x, y) of the combined key image are combined. This is due to the degree of overlap between the original images, in other words, depending on the distance from the boundary line of each composition original image in the overlapping region, and the pixel value (density (in the case of a color image, each It does not depend on the primary color density)) at all. That is, the key generation units 17-2,..., 17-N can generate a composite key image with a small amount of calculation without using any pixel value.

To summarize the synthesized key images represented by the equations (2) to (5), first, the value of the synthesized key image is a mixture corresponding to the distance from the boundary line of the overlap value of the original image for synthesis to the position. Represents the ratio. Further, the value of the composite key image represents a mixing ratio according to the overlap value of the composite original image. Then, the value of the composite key image represents a mixing ratio according to the average value of the reciprocal number of the overlap value of the original image for composition in a predetermined neighborhood area of the pixel position.
Further, as can be seen from the equation, when calculating the average value of the reciprocal of the above-described overlap value, the area where the original image for synthesis does not exist is excluded from the area in the vicinity of the pixel position. As a result, even in the case where the vicinity includes an area where no image exists, the area where no image exists does not affect the mixing ratio. In other words, there is an effect that a panorama composite image can be obtained while preventing the density of the image from being lowered near a region where no image exists.

Next, a specific operation example of the key generation unit 17-1 when N = 3 will be described.
FIG. 2 is a schematic diagram illustrating an operation example in which the key generation unit 17-1 generates a composite key image based on the key images 15-1, 15-2, and 15-3.

In this figure, only the self-compositing original image exists in the area indicated by the circled number “1” (for the key generation unit 17-1, the compositing original image 13-1 is the self-combining original image, and the key image 15-1 is a key image for a self-compositing original image), which is a region where no other compositing original images overlap. The area indicated by the circled number “2” is an area where the original image for self-compositing overlaps only the original image for synthesis 13-2. The region indicated by the circled number “3” is a region where the original image for self-compositing overlaps only with the original image for combining 13-3. The region indicated by the circled number “4” is a region where the original image for self-compositing overlaps with the original images for combining 13-2 and 13-3.
In the area indicated by circled "1" D _{1 (i,} j) = 1, in the area indicated by circled "2" area or circled "3" indicated by D _{1 (i,} j) = 2 and D ₁ (i, j) = 3 in the region indicated by the circled number “4”, and D ₁ (i, j) = 0 in the other regions.

The key generation unit 17-1 generates a composite key image using such D ₁ (i, j). What is important here is that the part having the original image for self-synthesis is separated from the part without the original image. It is a point that is processed. This is because the values of p _t (i, j) and p _t (i, j) are 0 when D _t (i, j) = 0 in the above equations (3) and (4). It corresponds to becoming. As a result, when the average value of the reciprocal of the degree of overlap of images is obtained using Equation (2), the region that does not have the original image for self-synthesis is excluded from the regions included in Ne (x, y). If the average value is calculated by adding the part without the original image for self-synthesis, the key level of the composite key image for the original image for self-synthesis is lowered in the vicinity of the joint line with the part without the image. However, the method of the present embodiment is characterized in that only the joint line between the images is smoothed, and the image density levels of the other portions are maintained at the original values.

  The processing procedure until the panorama composite image is generated based on the input image has been described above, but this method can be applied not only when the input image is a still image but also when it is a moving image. Even when the input image is a moving image, a key image (15-1, 15-2,..., 15-N) is generated based on each input image, and each key image is generated based on all the key images. The point that the key generation unit (17-1, 17-2,..., 17-N) generates a composite key image is the same as when the input image is a still image. The feature of the method of the present embodiment is that, as described above, the synthesized key image does not depend on the pixel value at all, and the synthesized key image is obtained only by the calculations represented by the equations (2) to (5). is there. Therefore, even when the input image is a moving image, a composite key image corresponding to each input image is created only once and stored in the composite key image storage unit, and is read out to all frames of the moving image. They may be used in common, and there is no need to create a composite key image for each frame of a moving image. Therefore, when a panoramic composite image is generated based on an input moving image, the amount of calculation required to generate the composite image is significantly smaller than a method of performing synthesis depending on the pixel value of each frame.

Next, another example of the composite key image will be described.
FIG. 3 is a schematic diagram illustrating an arrangement of each composition original image when the composite image generation apparatus 10 generates a panorama composite image based on a plurality of composition original images. In this figure, reference numerals 51, 52, and 53 are original images for synthesis, respectively, and these three original images for synthesis are arranged partially overlapping each other. Reference numeral 55 denotes a circle (indicated by a dotted line) representing the range of neighboring pixels, and Ne (x, y) appearing in the above equation (2) when the center of the circle 55 is (x, y) is It is a range inside the circle 55. A to G are sections provided for convenience of explanation, and the overlapping method of the original images for synthesis is different in each section on a line XX ′ (indicated by a one-dot chain line) crossing the panorama composite image. . In the section A, there is no composition original image, in the section B, only the composition original image 51 exists, in the section C, the composition original images 51 and 53 overlap, and in the section D, the composition original image 51 and 52 and 53 all overlap, and in the section E, the composition original images 52 and 53 overlap, in the section F, only the composition original image 52 exists, and in the section G, there is no composition original image.

FIG. 4 is a graph showing pixel values of the composite key image corresponding to the arrangement of the composite original images 51 to 53 shown in FIG. (A) in FIG. 4 is a graph showing the value of the composite key image k ₁ (x, y) corresponding to the composite original image 51. Similarly, (b) is a graph showing the value of the composite key image k ₂ (x, y) corresponding to the composite original image 52. Similarly, (c) is a graph showing the value of the composite key image k ₃ (x, y) corresponding to the composite original image 53. Each of the graphs (a) to (c) represents a change in the composite key image value on the alternate long and short dash line XX ′ shown in FIG. 3, the horizontal axis is the coordinate x, and the vertical axis is the composite key image. Value.

First, as shown in FIG. 4A, the value of the composite key image k ₁ (x, y) is 0 in the section A because the composite original image 51 does not exist. In section B, k ₁ (x, y) is 1 because only the composition original image 51 exists. Since the synthesis original image 53 in the synthesized original image 51 overlaps in a section C, as x increases, k 1 _(x, y) is gradually decreased from 1 while drawing a smooth curve 0.5 ( After reaching the reciprocal of “2”, 0.5 is maintained. In section D, since the composition original image 52 further overlaps, k ₁ (x, y) gradually decreases from 0.5 and reaches about 0.33 (the reciprocal of the degree of overlap “3”). It gradually decreases toward the right end of D and reaches zero. Since the composition original image 51 does not exist in the sections E to F, the value of the composition key image k ₁ (x, y) is 0.

Next, as shown in FIG. 4B, the value of the composite key image k ₂ (x, y) is 0 in the sections A to C because the composite original image 52 does not exist. In section D, since the composition original image 52 exists and overlaps with the composition original images 51 and 53, the value of k ₂ (x, y) gradually increases from 0 while drawing a smooth curve. Since it reaches about 0.33 (the reciprocal of the degree of overlap “3”) and the existence range of the composition original image 51 is to the right end of the section D, k ₂ (x, y) further gradually increases to 0.5. (The reciprocal of the degree of overlap “2”) is reached. In the left half region in the interval E, k ₂ (x, y) is maintained at 0.5. Since the composition original image 53 exists up to the right end of the section E, k ₂ (x, y) gradually increases from 0.5 and reaches 1 in the right half region in the section E. In the section F, since only the composition original image 52 exists, k ₂ (x, y) is 1. Since the composition original image 52 does not exist in the section G, the value of k ₂ (x, y) is 0.

Next, as shown in FIG. 4C, the value of the composite key image k ₃ (x, y) is 0 in the sections A to B because the composite original image 53 does not exist. In section C, since the composition original image 53 exists and the composition original image 51 overlaps, the value of k ₃ (x, y) gradually increases from 0 while drawing a smooth curve. After reaching 5 (the reciprocal of redundancy “2”), 0.5 is maintained. In the section D, the composition original image 53 exists, the composition original image 51 exists up to the right end of the section D, and the composition original image 52 exists up to the left end of the section D. Therefore, the value of k ₃ (x, y) is 0.5 at both ends of the interval D, and is approximately 0.33 near the center of the interval D (the reciprocal of the degree of overlap “3”). It has changed. In the section E, the composition original image 53 overlaps only the composition original image 52, and therefore k ₃ (x, y) is 0.5 (the reciprocal of the degree of overlap “2”) in the left half region of the section E. To maintain. Since the composition source image 53 exists up to the right end of the section E, k ₃ (x, y) gradually decreases in the right half of the section E and reaches 0 at the right end of the section E. In the sections F to G, k ₃ (x, y) is 0 because the composition original image 53 does not exist.

  Here, the change of the composite key image value in the x-axis direction has been described, and the description of the y-axis direction and the oblique direction has been omitted, but the same can be said for each direction.

  As described above, the first feature of the composite key image used in the present embodiment is that it depends only on the coordinates, and does not depend on the pixel value of the composite original image at all. Furthermore, the composite key image changes depending on the distance from the changing point where the degree of overlap of the composite original image changes (that is, the boundary line of the region of the composite original image). By doing so, it is possible to generate a natural-looking composite image with a small amount of calculation.

The second feature is that the reciprocal of the redundancy value is used as a reference. For example, in the section B in FIG. 4A, since the degree of overlap is 1, k ₁ (x, y) is 1. Further, for example, in the left half region of the section E in FIG. 4B, since the degree of overlap is 2, k ₂ (x, y) is 0.5. Further, for example, near the center of the section D in FIG. 4C, k ₃ (x, y) is about 0.33.
It should be noted that, although related to the second feature, is the total value of the values of the composite key image at a certain coordinate in the area where one or more original images for synthesis exist, fixed at 1.0? It is preferable that the value is close to 1.0.

The third feature is that the value of the composite key image changes smoothly while using the reciprocal of the redundancy value as a reference as described above. By doing in this way, it is possible to generate a synthetic image with a natural feeling even in a region where the joint of the original images for synthesis is difficult to see and the original images for synthesis are overlapping. However, this is not limited in that the degree of overlap changes from 0 to 1, and the composite key image may change discontinuously.
Note that the value of the composite key image smoothly changes means that, for example, the value k _t (x, y) of the composite key image is continuous in the xy plane in an area where one or more original images for synthesis exist. In addition, the value k _t (x, y) of the composite key image is a case where the differential value in an arbitrary direction on the xy plane is continuous. The composite key image represented by the expression (2) and the composite key images exemplified in FIGS. 4A to 4C satisfy these conditions. As a method of obtaining such a synthesized key image, in addition to using the formula (2), for example, an n-order function of x and y that passes through the reference points while using the reciprocal of the overlap value as a reference ( n may be, for example, 2 or 3) for each region, and a plurality of them may be connected.

In addition, you may make it implement | achieve the whole or one part function of the synthetic | combination image generation apparatus in embodiment mentioned above with a computer. In that case, a program for realizing this control function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be read into a computer system and executed. The “computer system” here includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, and a CD-ROM, and a storage device such as a hard disk built in the computer system. Further, the “computer-readable recording medium” dynamically holds a program for a short time, like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. It is also possible to include those that hold a program for a certain time, such as a volatile memory inside a computer system serving as a server or client in that case. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.
Further, the functions may be appropriately shared by a computer, hardware-implemented logic, a dedicated image processing processor, or the like.
In addition, a composition original image, a key image, a composition key image, a created panorama composition key image, and other intermediate calculation results required in the creation process are appropriately recorded in the storage unit. The storage unit is realized using, for example, a semiconductor memory, a magnetic hard disk device, a magneto-optical disk device, or the like. Geometric transformation and density / color adjustment units 11-1, 11-2,..., 11-N, key generation units 17-1, 17-2,. 19-1, 19-2,..., 19 -N, the addition unit 21, and the like appropriately read out information such as each image from the storage unit or write it into the storage unit.
When implemented using a computer, a storage unit in a memory space that can be addressed directly from the CPU may be used, or an external storage device via an input / output interface may be used as the storage unit.

The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.
For example, in the above embodiment, the composition original image data (d _t (x, y)) and the key image data (e _t (x, y)) are handled separately. By using only the data, information equivalent to the key image data may be represented depending on whether the data of the composition original image is zero or not. Further, by appropriately designing the expression of the values of the composition original image data and the key image data, the storage capacity required to store both may be reduced as a whole.

  The method used in the above embodiment (particularly, the synthesis key generation process) can be applied not only to panoramic image synthesis but also to synthesis of a plurality of images. Although the application range is wide, it can be applied to, for example, creating a wide-area photograph by joining a plurality of aerial photographs obtained by photographing the ground surface from an aircraft or satellite photographs photographed by artificial satellites. In addition, since it is possible to smoothly join objects having different quantitative levels, the present invention can also be applied to smooth joining of shaped objects.

It is a block diagram which shows the function structure of the synthesized image generation apparatus by this embodiment by embodiment of this invention. It is the schematic which shows the operation example in which the key production | generation part (17-1) produces | generates a synthetic | combination key image based on the key image (15-1, 15-2, 15-3) by the embodiment. It is the schematic which shows arrangement | positioning of each synthetic | combination original image at the time of producing | generating a panorama synthetic | combination image based on the some synthetic | combination original image by the embodiment. It is a graph showing the pixel value of the synthetic | combination key image corresponding to arrangement | positioning of several synthetic | combination original images shown in FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Composite image production | generation apparatus 11-1, 11-2, ..., 11-N Geometric conversion and density | concentration / color adjustment part (geometric conversion part)
13-1, 13-2, ..., 13-N Composition original images 15-1, 15-2, ..., 15-N Key images (key image information)
17-1, 17-2,..., 17-N Key generation units 19-1, 19-2,..., 19-N synthesis key multiplication unit (part of the synthesis processing unit)
21 Adder (part of synthesis processing unit)

Claims (7)

A geometric transformation unit that determines the arrangement of a plurality of composite original images to be input in the composite image;
Based on key image information indicating whether or not each of the composition original images exists at each position on the composite image, composite key image information indicating a mixing ratio of the plurality of composition original images at the position is generated. A key generator to
A synthesis processing unit that generates a synthesized image by synthesizing the plurality of synthesis original images using the synthesized key image information;
A composite image generation apparatus comprising: The key generation unit represents the mixing ratio in accordance with a distance from a boundary line of the overlap value of the composition original image to the position in a region where the plurality of composition original images overlap in the composite image. The composite image generation apparatus according to claim 1, wherein the composite key image information is generated. 3. The composite image according to claim 2, wherein the key generation unit generates the composite key image information representing the mixing ratio according to the overlap value of the composite original image in the composite image. Generator. The said key generation part produces | generates the said synthetic | combination key image information showing the said mixture ratio according to the average value of the reciprocal number of the said redundancy value in the area | region of the predetermined vicinity of the said position. Composite image generation apparatus. The key generation unit is configured to calculate an average value of the reciprocal of the redundancy value by excluding an area where the original image for synthesis does not exist from an area near a predetermined position of the position. The composite image generation apparatus described. The composite original image to be input is a moving image,
A composite key image information storage unit for storing the composite key image information generated by the key generation unit;
The composition processing unit synthesizes the plurality of composition original images for each frame of a moving image using the composition key image information read from the composition key image information storage unit. The composite image generation device according to any one of claims 1 to 5. A geometric transformation process for determining the arrangement of a plurality of composite original images to be input in the composite image;
Based on key image information indicating whether or not each of the composition original images exists at each position on the composite image, composite key image information indicating a mixing ratio of the plurality of composition original images at the position is generated. Key generation process,
A synthesis process for generating a synthesized image by synthesizing the plurality of source images for synthesis using the synthesized key image information;
A composite image generation program for causing a computer to execute the process of each process.

Images