JP4340836B2 - Image composition apparatus and image composition program - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image synthesizing apparatus and an image synthesizing program, and more particularly to a technique for creating a synthesized image by synthesizing a plurality of continuously shot images.
[0002]
[Prior art]
Conventionally, there has been proposed an electronic camera that synthesizes a photographed image and a recorded image recorded in a memory, or synthesizes two recorded images recorded in a memory to perform multiple exposure composition or chroma key composition (Japanese Patent Laid-Open No. Hei 6). -189255).
[0003]
Also, the main subject is extracted from each of the continuously shot images, and the changed portion (difference image) of the main subject image between the extracted frames is obtained and recorded in the memory. There has been proposed an electronic camera that obtains a multiple exposure image (composite image) with a small memory capacity by combining (Japanese Patent Laid-Open No. 2001-28726).
[0004]
[Problems to be solved by the invention]
However, the electronic camera described in Japanese Patent Laid-Open No. 6-189255 synthesizes two images (synthesis by addition or chroma key synthesis), and synthesizes a plurality of images taken continuously such as a moving image. It is not a thing. In addition, in the case of image composition by addition, when the number of additions increases, there is a problem that the image is blurred due to the movement of the main subject or the like and a high-quality image cannot be obtained.
[0005]
In addition, the electronic camera described in Japanese Patent Application Laid-Open No. 2001-28726 takes a difference image of the main subject, so that the moving part is thinned and added to synthesize the multiple exposure image electronically, resulting in a further blurred image. There is a problem that a high-quality image cannot be obtained.
[0006]
The present invention has been made in view of such circumstances, and an object of the present invention is to provide an image composition apparatus and an image composition program that can obtain a high-quality multiple exposure image from a plurality of continuously photographed images.
[0007]
[Means for Solving the Problems]
In order to achieve the above object, an image synthesizing apparatus according to claim 1 includes an extraction unit that extracts a main subject from a plurality of continuously shot images, and a plurality of pixel values that have only pixel values of the main subject extracted by the extraction unit. Image synthesizing means for creating an added image obtained by sequentially adding and synthesizing images in time series, wherein the pixels at the same coordinate position of a plurality of images having pixel values only for the main subject are relatively set for each of the plurality of images. And image compositing means for creating the added image by shifting and mapping on the memory.
[0008]
That is, instead of obtaining a normal multiple exposure image obtained by adding a plurality of images, a plurality of images each having a pixel value of only a main subject extracted from a plurality of images taken continuously are shifted in time series while shifting the images. An overlay image (composite image) is created by sequentially overlaying and compositing. This makes it possible to create a composite image that clearly shows the movement of the main subject.In particular, by overlaying and combining the main subject images as appropriate, the overlapping portion of the main subject images can be reduced. The image of each continuous main subject becomes clearer.
[0010]
According to a second aspect of the present invention, the image synthesizing unit creates a synthesized image formed by superimposing the superimposed image on a predetermined background image.
[0011]
According to a third aspect of the present invention, there is provided an image composition program for creating a composite image formed by combining a plurality of continuously shot images, wherein a main subject is obtained from the plurality of continuously shot images. And an image composition function for creating an added image obtained by sequentially adding and synthesizing a plurality of images having pixel values only in the extracted main subject in time series, and a plurality of images having pixel values only in the main subject. The image synthesizing function for creating the added image is realized by mapping the pixels at the same coordinate position of the image of the image on the memory while being shifted relative to each other for each of the plurality of images.
[0012]
The image composition program is used as application software in an electronic camera capable of continuous shooting, or used as application software such as a personal computer capable of storing and displaying continuously shot images.
[0013]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, preferred embodiments of an image composition device and an image composition program according to the present invention will be described in detail with reference to the accompanying drawings.
[0014]
FIG. 1 is a block diagram showing a hardware configuration example of a personal computer (hereinafter referred to as “personal computer”) having an image composition program according to the present invention.
[0015]
As shown in FIG. 1, a personal computer 1 includes a central processing unit (CPU) 10 that mainly controls the operation of each component, a main memory 12 that stores a control program for the device, and serves as a work area when the program is executed. An operating system (OS) of the personal computer 1, device drivers for peripheral devices connected to the personal computer 1, various application software including an image composition program according to the present invention, a hard disk device 14 storing user images, etc., and a CD- ROM drive 16, display memory 18 for temporarily storing display data, monitor device 20 such as a CRT monitor or a liquid crystal monitor for displaying images, characters, etc. from image data, character data, etc. from this display memory 18, keyboard 22, mouse 24 as a position input device, and the state of mouse 24 A mouse controller 26 that outputs signals such as the position of the mouse pointer on the data device 20 and the state of the mouse 24 to the CPU 10, and a USB (Universal Serial Bus) that can be connected to the digital camera 2 to input image data and the like. It comprises an interface 27 and a bus 28 for connecting the above components.
[0016]
The personal computer 1 having the above configuration is known except for the image composition program stored in the hard disk device 14 for performing the image composition processing according to the present invention. Is omitted. The image composition program is recorded on, for example, a CD-ROM included when the digital camera 2 is purchased. By setting this CD-ROM in the CD-ROM drive 16 of the personal computer 1, the personal computer 1 Can be installed on. It can also be downloaded from a server that provides an image composition program via communication means (not shown).
[0017]
Next, data transfer from the photographing by the digital camera 2 connected to the personal computer 1 to the personal computer 1 will be described.
(Step S1)
The digital camera 2 is fixed on a tripod or the like to shoot a still image that does not contain a subject. The captured still image is saved as a background image in a still image file format such as JPEG format in a memory card in the camera.
(Step S2)
Subsequent to the still image shooting in step S1, moving image shooting is performed by frame imaging of the subject with the digital camera 2 fixed. Save the shot video to a memory card in the camera in a video file format such as Motion JPEG or continuous still image file format.
(Step S3)
The digital camera 2 after photographing is connected to the personal computer 1 as shown in FIG. 1, and the still image file and the moving image file stored in the memory card in the digital camera 2 are transferred to the personal computer 1 and stored in the hard disk device 14.
[0018]
Next, image composition processing in the personal computer 1 will be described with reference to the flowchart shown in FIG.
[0019]
First, when the image composition processing program according to the present invention is started, a screen for selecting a still image file as a background image or a moving image file to be composed is displayed on the monitor device 20 of the personal computer 1 (not shown). ) A desired still image file and moving image file are selected on this screen (steps S10 and S11).
[0020]
Subsequently, n still images to be processed are selected from the moving image file selected in step S11, and the selected n still images are cut out from the moving image file and re-saved as continuous still images (step S12). , S13). For n still images, an arbitrary numerical value n may be input, and n still images may be automatically selected from the movie file at equal intervals, or manually while viewing each frame image of the movie. You may make it select one by one. Alternatively, a certain period in the moving image may be designated and all the frame images within the period may be selected.
[0021]
Next, the number i of the still image to be processed among the n still images is set to 1 (i = 1) (step S14). Note that n still images are numbered 1 to n in chronological order.
[0022]
A main subject is extracted from the i-th still image to be processed (step S15). The extraction of the main subject is performed based on the background image selected in step S10. That is, the color space distance between pixels at the same position of the i-th still image and the background image (the sum of the difference values of R, G, and B data of each pixel) is calculated, and pixels whose calculation results are equal to or greater than a specified value are calculated. Extracted as main subject candidate pixels.
[0023]
The main subject candidate pixel group extracted in this manner is subjected to image processing such as concatenation, and noise pixels are removed to obtain an image of only the main subject. The image of only the main subject is saved in the main memory 12 with the same pixel size as the original still image. That is, the area other than the main subject of the original still image is filled with pixels having no pixel value (for example, R, G, B = (-1, -1, -1)) and is the same as the original still image. Set to pixel size.
[0024]
Next, the image having the pixel value of only the main subject extracted in step S15 is added and mapped on the synthesized image memory area for one frame of the main memory 12, and the main subject is added and combined (step S16). That is, after mapping so that the pixels of the same coordinates (x, y) overlap each other in the consecutive i-th image i and i + 1-th image (i + 1), the pixel values of the pixels of both images overlap. The added pixel value is determined so that the pixel value of (i + 1) is adopted. In the case of an image of only the main subject extracted from the first still image, it is mapped on the memory as it is.
[0025]
The above overlay synthesis algorithm is shown below.
[0026]
Each pixel of the image i and the image (i + 1) is scanned, and each pixel value (Rcomp, Gcomp, Bcomp) of the composite image comp after addition is determined. For convenience, pixel values other than the main subject pixel are defined as (R, G, B) = (− 1, −1, −1).
[0027]
When both the pixel of the image i and the pixel at the same position of the image (i + 1) have the pixel value of the main subject, the pixel value of the image (i + 1) is adopted. That is,
(Ri, Gi, Bi) ≠ (-1, -1, -1) and (Ri + 1, Gi + 1, Bi + 1) ≠ (-1, -1, -1) → (Rcomp, Gcomp , Bcomp) = (Ri + 1, Gi + 1, Bi + 1)
When the pixel of the image i does not have the pixel value of the main subject and the pixel at the same position in the image (i + 1) has the pixel value of the main subject, the pixel value of the image (i + 1) is adopted. That is,
(Ri, Gi, Bi) = (− 1, −1, −1) and (Ri + 1, Gi + 1, Bi + 1) ≠ (−1, −1, −1) pixels → (Rcomp, Gcomp , Bcomp) = (Ri + 1, Gi + 1, Bi + 1)
When the pixel of the image i has the pixel value of the main subject and the pixel at the same position in the image (i + 1) does not have the pixel value of the main subject, the pixel value of the image i is adopted. That is,
(Ri, Gi, Bi) ≠ (-1, -1, -1) and (Ri + 1, Gi + 1, Bi + 1) = (-1, -1, -1) pixels → (Rcomp, Gcomp , Bcomp) = (Ri, Gi, Bi)
When both the pixel of the image i and the pixel at the same position of the image (i + 1) do not have the pixel value of the main subject, values (−1, −1, −1) having no pixel value after addition are set. That is,
(Ri, Gi, Bi) = (− 1, −1, −1) and (Ri + 1, Gi + 1, Bi + 1) = (− 1, −1, −1) pixels → (Rcomp, Gcomp , Bcomp) = (-1, -1, -1)
When the overlay synthesis of the main subject of the i-th image is completed as described above, it is determined whether or not the overlay synthesis for n images has been completed (step S17), and if not completed (i <n). Increments i by 1 (step S18) and returns to step S15. As a result, the main subject extraction process and the addition of the main subject are performed on the next still image in time series order.
[0028]
When the addition of n sheets is completed (i = n), the addition image is added to the background image and combined (step S19). In other words, all the portions of the superimposed image that have no pixel value are filled with the background image pixels.
[0029]
The composite image created in this way is stored in the hard disk device 14, displayed on the monitor device 20, or printed out by a printer (not shown).
[0030]
FIG. 3 is a creation image diagram of the composite image created as described above.
[0031]
Next, another embodiment of the overlay synthesis of the main subject image will be described.
[0032]
First, processing is performed in the same manner as described above until each main subject is extracted from n still images. A minimum rectangle including the image of the main subject extracted in this way is calculated, and an image (rectangular image) within the minimum rectangle is stored in the memory.
[0033]
The images shown in FIGS. 4A and 4B are an i-th rectangular image i and an i + 1-th rectangular image (i + 1), respectively.
[0034]
Next, the pixels are mapped so that the coordinates (x _i, y _i ) of the rectangular image i and the coordinates (x _{i + 1,} y _{i + 1} ) of the rectangular image (i + 1) overlap.
[0035]
For example, when aligning the rectangle image i so that the rectangle image (i + 1) overlaps the right half of the rectangle image i and overlaying the rectangle image i, the rectangle image i is set to Max-xi. ) By adding Max-xi / 2 to the x coordinate of the coordinates (x _{i + 1,} y _{i + 1} ) of each pixel, and shifting the coordinates to the right (x _{i + 1} + Max-xi / 2 _, y _{i +} Convert to ₁ ). Thereafter, after mapping so that pixels with the same coordinates overlap, the pixel value after addition is determined so that the pixel value of the image (i + 1) is adopted for the portion where the pixel values of the pixels of both images overlap.
[0036]
In this way, by combining the rectangular image i shown in FIG. 4A with the rectangular image (i + 1) shown in FIG. 4B shifted to the right and combining them, the combined image shown in FIG. can get.
[0037]
By sequentially performing such overlay composition on n rectangular images (1 to n), a composite image as shown in FIG. 5 can be created.
[0038]
In this embodiment, a still image (background image) that does not contain the main subject is shot separately from the movie. However, the present invention is not limited to this, and a movie that does not contain the main subject is shot. One frame of a moving image may be used as a background image. In addition, a plurality of still images are acquired from a moving image. However, the present invention is not limited to this, and a plurality of continuous images are captured in a continuous shooting mode in which images are continuously captured at a predetermined interval. It may be a picture.
[0039]
Further, as a method for extracting the main subject from the image, for example, a chroma key process is performed in which the main subject is photographed against a screen of a specific color such as a blue background and the main subject is extracted from the photographed image based on the specific color. May be adopted, or one of the pixels of the main subject in the image is pointed, the contour of the main subject is obtained from the continuity of the pointed pixel, and the main subject based on the contour You may employ | adopt the method of extracting. In the case of these main subject extraction methods, it is not necessary to separately shoot a background image that does not contain the main subject in order to extract the main subject.
[0040]
Furthermore, the image composition processing apparatus according to the present invention is not limited to a personal computer having the image composition processing program, and may be added as an image composition processing function in the digital camera.
[0041]
【The invention's effect】
As described above, according to the present invention, instead of obtaining a normal multiple-exposure image obtained by adding a plurality of images, a plurality of images of only the main subject extracted from a plurality of images taken continuously are Since an added image (synthesized image) is created by sequentially adding and synthesizing in order of series, a high-quality synthesized image in which the movement of the main subject can be clearly seen can be created.
[0042]
Further, in addition to the case where the main subject image is superimposed and combined without shifting, the main subject image may be superimposed and combined as appropriate. In this case, the overlapping portion of the main subject image can be reduced, and the individual images can be continuously displayed. The image of the main subject becomes clearer.
[Brief description of the drawings]
FIG. 1 is a block diagram showing an example of a hardware configuration of a personal computer having an image composition program according to the present invention. FIG. 2 is a flowchart showing an example of an image composition processing procedure according to the present invention. Fig. 4 is a diagram used to explain the method of adding and compositing images of the main subject by shifting the images of the main subject. Fig. 5 Shifting the images of a plurality of main subjects. Image of creating a composite image that is superimposed on top [Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 ... Personal computer, 2 ... Digital camera, 10 ... Central processing unit, 12 ... Main memory, 14 ... Hard disk drive, 20 ... Monitor device, 22 ... Keyboard, 24 ... Mouse, 27 ... Interface

Claims (3)

Extraction means for extracting a main subject from a plurality of images continuously shot;
Image synthesizing means for creating an added image obtained by sequentially superposing and synthesizing a plurality of images having pixel values only in the main subject extracted by the extracting means in a time-series order, and a plurality of images having pixel values only in the main subject. Image synthesizing means for creating the added image by shifting the pixels at the same coordinate position relative to each other for each of the plurality of images and adding and mapping on the memory;
An image composition device comprising:   2. The image synthesizing apparatus according to claim 1, wherein the image synthesizing unit creates a synthesized image formed by superimposing the superimposed image on a predetermined background image. An image composition program for creating a composite image formed by combining a plurality of images taken continuously,
A function of extracting a main subject from a plurality of images taken continuously;
An image composition function for creating an added image obtained by sequentially adding and synthesizing a plurality of images having pixel values only in the extracted main subject in chronological order, and the same coordinate positions of the plurality of images having pixel values only in the main subject An image composition function for creating the above-described added image by relatively shifting the pixels of each of the plurality of images and mapping on the memory,
An image composition program characterized by realizing the above.

Images