JP2012124917A - Imaging apparatus and method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent blur when a plurality of images are added.SOLUTION: A corresponding point detection portion 38 detects a corresponding point between one reference image and the other image except for the reference image among a plurality of images obtained by photographing by prescribed exposure time. A positioning portion 39 positions the reference image and the other image based on the corresponding point. An addition portion 40 generates an added image by adding the reference image and the other image after positioning.

Description

  The present invention relates to an imaging apparatus and method for performing processing for adding a plurality of images acquired by an imaging apparatus such as a digital camera, and a program for causing a computer to execute the imaging method.

  When shooting with a digital camera, if the shooting environment is dark, it is necessary to perform long exposure. However, if the exposure is performed for a long time, the S / N is lowered and the image quality is deteriorated due to the influence of the dark current of the CCD constituting the digital camera. For this reason, there has been proposed a digital camera capable of obtaining an image substantially equivalent to long-time exposure by sequentially accumulating image signals for each frame obtained from imaging means such as a CCD constituting the digital camera. (See Patent Document 1).

JP-A-5-236422

  However, when a plurality of images are added as described in Patent Document 1, there is a problem that the added images are blurred unless the plurality of images have the exact same angle of view.

  The present invention has been made in view of the above circumstances, and an object thereof is to prevent blurring when adding a plurality of images.

The first image adding device according to the present invention is a corresponding point detecting unit that detects corresponding points between one reference image and other images other than the reference image among a plurality of images acquired by photographing with a predetermined exposure time. When,
Alignment means for aligning the reference image and the other image based on the corresponding points;
Adding means for adding the reference image after alignment and one addition target image to be added among the other images to generate an addition image;
Brightness evaluation means for evaluating the brightness of the added image is provided.

  Here, the “corresponding point” means not only both points corresponding to the reference image and the other image, but also both a corresponding point of one image and a vector from the corresponding point to the corresponding point of the other image. Any information indicating the correspondence between the positions of the images may be used.

  In the first image addition device according to the present invention, the addition image is used as a new reference image until an addition image having a desired brightness is acquired or until all the plurality of images are added. Control means for controlling the addition means and the brightness evaluation means so as to sequentially change the addition target image and perform addition of the new reference image and the addition target image and evaluation of the brightness; You may do it.

The second image adding device according to the present invention acquires a plurality of images by photographing at a predetermined exposure time, and when adding the plurality of images to acquire the added image, the added image is set to a desired brightness. Corresponding point detection means for detecting corresponding points between one reference image and images other than the reference image among a plurality of images required for
Alignment means for aligning the reference image with all the other images based on the corresponding points;
An adding means for adding the reference image after alignment and all the other images to generate the added image is provided.

A first photographing apparatus according to the present invention includes photographing means for obtaining an image by photographing,
An image for obtaining a plurality of images by photographing with a predetermined exposure time, and calculating the number of images necessary to obtain the desired brightness for the added image when the plurality of images are added to obtain an added image. Sheet number calculating means;
Photographing control means for controlling the photographing means to obtain the calculated number of images;
Corresponding point detecting means for detecting corresponding points between one reference image and other images other than the reference image among the required number of images;
Alignment means for aligning the reference image with all the other images based on the corresponding points;
An adding means for adding the reference image after alignment and all the other images to generate the added image is provided.

A first photographing method according to the present invention includes a step of acquiring an image by photographing by photographing means;
A step of obtaining a plurality of images by photographing at a predetermined exposure time and adding the plurality of images to obtain an added image, and calculating a number of images necessary to obtain the desired brightness for the added image; When,
Controlling the imaging means to acquire the calculated number of images;
Detecting corresponding points between one reference image and other images other than the reference image among the required number of images;
Aligning the reference image with all the other images based on the corresponding points;
Adding the reference image after alignment and all the other images to generate the added image.

Corresponding point detection means for detecting a corresponding point between one reference image and other images other than the reference image among a plurality of images acquired by photographing with a predetermined exposure time. When,
Alignment means for aligning the reference image and the other image based on the corresponding points;
Adding means for adding the reference image after alignment and one addition target image to be added among the other images to generate an addition image;
Brightness adjustment means for adjusting the brightness of the added image to a desired brightness to generate a brightness-adjusted image;
Until all of the plurality of images are added, the addition image is used as a new reference image, and the addition target image is sequentially changed to add the new reference image and the addition target image and adjust the brightness. And a control means for controlling the addition means and the brightness adjustment means so as to generate a plurality of brightness adjusted images according to the number of additions.

In the third image adding device according to the present invention, display means for displaying the plurality of brightness adjusted images;
The image processing apparatus may further include selection means for receiving selection of one desired image among the plurality of brightness adjusted images.

  Further, in the third image adding device according to the present invention, the number of images required for setting the added image to a desired brightness, the number of the plurality of images being calculated based on ambient brightness. It is good.

A second photographing apparatus according to the present invention includes photographing means for obtaining an image by photographing,
An image for obtaining a plurality of images by photographing with a predetermined exposure time, and calculating the number of images necessary to obtain the desired brightness for the added image when the plurality of images are added to obtain an added image. Sheet number calculating means;
Photographing control means for controlling the photographing means to obtain the calculated number of images;
Corresponding point detecting means for detecting corresponding points between one reference image and other images other than the reference image among the required number of images;
Alignment means for aligning the reference image and the other image based on the corresponding points;
Adding means for adding the reference image after alignment and one addition target image to be added among the other images to generate an addition image;
Brightness adjustment means for adjusting the brightness of the added image to a desired brightness to obtain a brightness-adjusted image;
Until all the plurality of images are added, the addition image is used as a new reference image and the addition target image is sequentially changed to add the new reference image and the addition target image and to adjust the brightness. And a control means for controlling the brightness adjustment means so as to obtain a plurality of brightness adjusted images corresponding to the number of additions.

In the second photographing apparatus according to the present invention, display means for displaying the plurality of brightness adjusted images;
The image processing apparatus may further include selection means for receiving selection of one desired image among the plurality of brightness adjusted images.

According to a fourth image adding apparatus of the present invention, a corresponding point detecting unit that detects corresponding points between one reference image and other images other than the reference image among a plurality of images acquired by photographing with a predetermined exposure time. When,
Image determining means for determining an image necessary for addition among the other images based on the corresponding points;
Alignment means for aligning the reference image and the image necessary for the addition based on the corresponding points;
Adding means for adding the reference image after the alignment and the image necessary for the addition to generate an added image;
Brightness adjusting means for adjusting the brightness of the added image is provided.

A first image addition method according to the present invention detects a corresponding point between one reference image and other images other than the reference image among a plurality of images acquired by photographing with a predetermined exposure time,
Based on the corresponding point, the reference image and the other image are aligned,
Adding the reference image after alignment and one addition target image to be added among the other images to generate an addition image;
The brightness of the added image is evaluated.

In the second image addition method according to the present invention, when a plurality of images are acquired by photographing with a predetermined exposure time and the plurality of images are added to obtain an added image, the added image is set to a desired brightness. Detecting a corresponding point between one reference image and other images other than the reference image among a plurality of images necessary for
Based on the corresponding points, align the reference image with all the other images,
The reference image after alignment and all the other images are added to generate the added image.

A third image addition method according to the present invention detects corresponding points between one reference image and other images other than the reference image among a plurality of images acquired by photographing with a predetermined exposure time,
Based on the corresponding point, the reference image and the other image are aligned,
Adding the reference image after alignment and one addition target image to be added among the other images to generate an addition image;
Adjusting the brightness of the added image to a desired brightness to generate a brightness-adjusted image;
Until all of the plurality of images are added, the addition image is used as a new reference image, and the addition target image is sequentially changed to add the new reference image and the addition target image and adjust the brightness. And generating a plurality of brightness-adjusted images corresponding to the number of additions.

A fourth image addition method according to the present invention detects corresponding points between one reference image and other images other than the reference image among a plurality of images acquired by photographing with a predetermined exposure time,
Based on the corresponding points, determine an image necessary for addition among the other images,
Based on the corresponding points, align the reference image and the image necessary for the addition,
Adding the reference image after the alignment and the image necessary for the addition to generate an added image;
The brightness of the added image is adjusted.

  In addition, you may provide as a program for making a computer perform the 1st-4th image addition method by this invention.

  Moreover, you may provide as a program for making a computer perform the 1st imaging | photography method by this invention.

  According to the first image addition apparatus and method of the present invention, corresponding points between one reference image and other images other than the reference image are detected from among a plurality of images, and the reference image and the other are detected based on the corresponding points. Are aligned with each other, and the reference image after alignment and one addition target image to be added among other images are added to generate an added image. Then, the brightness of the added image is evaluated. Therefore, when evaluating the brightness of the added image, the positions of the reference image and the addition target image are aligned, so that the added image can be prevented from being blurred, and as a result, a high-quality added image can be obtained. Can do.

  Further, until the addition image having the desired brightness is acquired or until all of the plurality of images are added, the addition target image is sequentially changed while the addition image is used as a new reference image, and a new reference image and By performing addition with the addition target image and evaluating the brightness, it is possible to obtain a blur-free added image having a desired brightness.

  According to the second image addition apparatus and method of the present invention, when a plurality of images are acquired by photographing at a predetermined exposure time and the plurality of images are added to obtain an addition image, the desired brightness of the addition image is obtained. Corresponding points between one reference image and other images other than the reference image are detected from the plurality of images necessary for the determination, and based on the corresponding points, the reference image and all other images are detected. Alignment is performed, and the reference image after alignment and all other images are added to generate an added image. For this reason, at the time of shooting, the number of images necessary to acquire an added image having a desired brightness is calculated, and the number of images is acquired by shooting, thereby adding blur-free addition having a desired brightness. Images can be acquired.

  Here, if a plurality of images are added when a subject moves greatly or shakes during shooting of a plurality of images, the added image is blurred. In such a case, the degree of image blur differs depending on the number of images to be added.

  According to the third image adding apparatus and method of the present invention, corresponding points between one reference image and other images other than the reference image are detected from among the plurality of images, and the reference image and the other are detected based on the corresponding points. Are aligned with each other, and the reference image after alignment and one addition target image to be added among other images are added to generate an added image. Further, the brightness of the added image is adjusted to a desired brightness, and a brightness adjusted image is generated. Then, until all the plurality of images are added, the addition target image is sequentially changed while the addition image is used as a new reference image, and the addition of the new reference image and the addition target image and the adjustment of the brightness are performed. A plurality of brightness-adjusted images corresponding to the number of additions are generated.

  Here, the plurality of brightness-adjusted images have substantially the same brightness, and the degree of blur varies depending on the number of additions. For this reason, the user has a desired degree of blur from a plurality of brightness-adjusted images, that is, by selecting a brightness-adjusted image with less blur, an added image with less blur than desired brightness is selected. Can be acquired.

  In this case, displaying a plurality of brightness adjusted images makes it easy to select a desired image.

  According to the fourth image addition apparatus and method of the present invention, a corresponding point between one reference image and another image other than the reference image is detected from among a plurality of images, and another image is determined based on the corresponding point. The image necessary for the addition is determined from the above, the reference image and the image necessary for the addition are aligned, and the reference image after the alignment and the image necessary for the addition are added to generate an added image. Then, the brightness of the added image is adjusted. For this reason, it is possible to exclude an image having a small number of corresponding points or having a large difference in the corresponding points with the reference image from the addition target, thereby obtaining an added image with less blur.

1 is a schematic block diagram showing the configuration of a digital camera to which an image adding device according to a first embodiment of the present invention is applied. Flowchart of image addition processing in the first embodiment Flowchart of image addition processing in the second embodiment Schematic block diagram showing the configuration of a digital camera to which an image adding device according to a third embodiment of the present invention is applied. Flowchart of image addition processing in the third embodiment The figure which shows the state where the brightness adjusted picture is displayed on the monitor Diagram for explaining image addition The schematic block diagram which shows the structure of the digital camera to which the image addition apparatus by the 4th Embodiment of this invention is applied. Flowchart of image addition processing in the fourth embodiment Flowchart of processing for determining an image necessary for addition in the fourth embodiment

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a schematic block diagram showing a configuration of a digital camera to which an image adding device according to a first embodiment of the present invention is applied. As shown in FIG. 1, the digital camera 1 according to the first embodiment transmits an operation system 2 such as an operation mode switch, a zoom lever, up / down / left / right buttons, a release button, and a power switch, and the operation contents of the operation system 2 to the CPU 42. And an operation system control unit 3 that is an interface part.

  The imaging system 6 includes a focus lens 10a and a zoom lens 10b that constitute the photographing lens 10. Each lens can be moved in the optical axis direction by a focus lens driving unit 11 and a zoom lens driving unit 12 each including a motor and a motor driver. The focus lens drive unit 11 controls the movement of each lens based on the focus drive amount data output from the AF processing unit 30 and the zoom lens drive unit 12 based on the operation amount data of the zoom lever.

  The diaphragm 14 is driven by a diaphragm driving unit 15 including a motor and a motor driver. The aperture drive unit 15 adjusts the aperture diameter based on aperture value data output from the AE / AWB processing unit 31.

  The shutter 16 is a mechanical shutter and is driven by a shutter drive unit 17 including a motor and a motor driver. The shutter driving unit 17 controls the opening and closing of the shutter 16 according to a signal generated by pressing the release button and the shutter speed data output from the AE / AWB processing unit 31.

  A CCD 18 which is an image pickup device is provided behind the optical system. The CCD 18 has a photocathode in which a large number of light receiving elements are two-dimensionally arranged, and subject light that has passed through the optical system forms an image on the photocathode and is photoelectrically converted. In front of the photocathode, a microlens array for condensing light on each pixel and a color filter array in which filters of R, G, and B colors are regularly arranged are arranged. The CCD 18 outputs the charges accumulated for each pixel as a serial analog photographing signal line by line in synchronization with the vertical transfer clock and the horizontal transfer clock supplied from the CCD controller 19. The time for accumulating charges in each pixel, that is, the exposure time, is determined by an electronic shutter drive signal given from the CCD controller 19. The gain of the CCD 18 is adjusted by the CCD control unit 19 so that an analog imaging signal having a predetermined size can be obtained.

  The analog photographing signal captured from the CCD 18 is input to the analog signal processing unit 20. The analog signal processing unit 20 includes a correlated double sampling circuit (CDS) that removes noise from the analog signal, an auto gain controller (AGC) that adjusts the gain of the analog signal, and an A / D that converts the analog signal into a digital signal. It consists of a converter (ADC). Note that the processing performed by the analog signal processing unit 20 is referred to as analog signal processing. The image data converted into the digital signal is CCD-RAW data having R, G, and B density values for each pixel.

  The timing generator 21 generates a timing signal. By supplying this timing signal to the shutter drive unit 17, the CCD control unit 19, and the analog signal processing unit 20, the release button is operated, the shutter 16 is opened and closed, The capture of the charge of the CCD 18 and the processing of the analog signal processing unit 20 are synchronized.

  The flash control unit 23 causes the flash 24 to emit light during shooting.

  The image input controller 25 writes the CCD-RAW data input from the analog signal processing unit 20 in the frame memory 26.

  The frame memory 26 is a working memory used when various image processing (signal processing) described later is performed on the image data. For example, an SDRAM (Synchronous) that performs data transfer in synchronization with a bus clock signal having a fixed period. Dynamic Random Access Memory) is used.

  The display control unit 27 displays the image data stored in the frame memory 26 on the monitor 28 as a through image, or displays the image data stored in the recording medium 35 on the monitor 28 in the reproduction mode. It is. Note that through images are continuously photographed by the CCD 18 at predetermined time intervals while the photographing mode is selected.

  The AF processing unit 30 and the AE / AWB processing unit 31 determine shooting conditions based on the pre-image. This pre-image is an image represented by image data stored in the frame memory 26 as a result of the CPU 42 having detected a half-press signal generated by half-pressing the release button causing the CCD 18 to perform pre-photographing. is there.

  The AF processing unit 30 detects a focal position based on the pre-image and outputs focus drive amount data (AF processing). As a focus position detection method, for example, a passive method that detects a focus position using a feature that the contrast of an image is high when a desired subject is in focus can be considered.

  The AE / AWB processing unit 31 measures the subject brightness based on the pre-image, and based on the measured subject brightness, adjusts the ISO sensitivity, aperture value, shutter speed, etc. so that the captured image has a desired brightness. Then, ISO sensitivity data, aperture value data, and shutter speed data are determined as exposure setting values (AE processing), and white balance at the time of shooting is automatically adjusted (AWB processing).

  The image processing unit 32 performs image quality correction processing such as gradation correction, sharpness correction, and color correction on the image data of the main image, and CCD-RAW data as Y data that is a luminance signal and Cb that is a blue color difference signal. YC processing is performed for conversion into YC data comprising data and Cr data which is a red color difference signal. The main image is an image based on image data that is captured from the CCD 18 by main shooting performed when the release button is fully pressed and stored in the frame memory 26 via the analog signal processing unit 20 and the image input controller 25. is there.

  The compression / decompression processing unit 33 performs a compression process in a compression format such as JPEG on the image data of the main image that has been corrected and converted by the image processing unit 32 to generate an image file. A tag storing incidental information such as shooting date and time is added to the image file based on the Exif format or the like.

  The media control unit 34 controls the writing and reading of the image file by accessing the recording medium 35.

  The internal memory 36 stores various constants set in the digital camera 1, programs executed by the CPU 42, and the like.

  The digital camera 1 also includes an image number calculation unit 37, a corresponding point detection unit 38, a registration unit 39, an addition unit 40, and a brightness evaluation unit 41. Processing performed by the image number calculation unit 37, the corresponding point detection unit 38, the alignment unit 39, the addition unit 40, and the brightness evaluation unit 41 will be described later.

  The CPU 42 controls each part of the main body of the digital camera 1 in accordance with signals from the operation system 2 and various processing units such as the AF processing unit 30. Further, the CPU 42 controls the operations of the image number calculation unit 37, the corresponding point detection unit 38, the alignment unit 39, the addition unit 40, and the brightness evaluation unit 41 when performing addition processing described later. In addition, the CPU 42 divides the exposure time represented by the shutter speed determined by the AE / AWB processing unit 31 at the time of actual photographing, and controls the photographing system 6 to perform photographing several times at regular time intervals. . The predetermined time t0 is determined in advance. For example, if the frame rate of a moving image captured by the digital camera 1 is 30 frames / second, t0 = 30 ms. The CPU 42 constitutes a photographing control unit and a control unit.

  The data bus 43 is connected to various processing units, the frame memory 26, the CPU 42, and the like, and exchanges digital image data and various instructions.

  Next, the addition process in the first embodiment will be described. FIG. 2 is a flowchart of the addition process in the first embodiment. In a state where the digital camera 1 is in the shooting mode, the CPU 42 starts processing by performing pre-shooting, and the image number calculation unit 37 calculates the number of images (necessary number of images) N necessary for the addition processing ( Step ST1). The number-of-images calculation unit 37 calculates the required number of images N from the exposure time represented by the shutter speed determined by the AE / AWB processing unit 31 and the exposure time per shooting. For example, assuming that the exposure time represented by the shutter speed determined by the AE / AWB processing unit 31 is 1/10 seconds and the exposure time per shooting is 1/100 seconds, the required number of images N is 10 It becomes a sheet.

  Next, the photographing system 6 performs photographing according to an instruction from the CPU 42 (step ST2). Image data acquired by shooting is subjected to necessary processing in the image processing unit 32 and the like, and then stored in the frame memory (step ST3). Then, the CPU 42 determines whether or not the number of images stored in the frame memory has reached the required number N (step ST4), and if step ST4 is negative, the process returns to step ST2. As a result, N images are taken every 30 ms and N images are acquired.

  When step ST4 is affirmed, the corresponding point detection unit 38 sets one of the N images as the reference image B0, and N−1 images Si (i = i) other than the reference image B0 and the reference image B0. N-1) is detected (step ST5). Specifically, feature points such as edges on the reference image B0 and the image Si and corner portions of structures included in the image are extracted, and among the extracted feature points, correspondence is made between the reference image B0 and the image Si. A point is detected as a corresponding point. The corresponding points can be detected by various known methods such as the KLT method and template matching using a region around the feature point in the reference image B0 as a template. Further, as the reference image B0, for example, an image acquired by the first shooting may be used, but the reference image B0 is not limited to this.

Next, the alignment unit 39 performs alignment between the reference image B0 and the image Si (step ST6). That is, the alignment is performed by warping each image Si so that the corresponding point of each image Si matches the corresponding point of the reference image B0. Specifically, for each corresponding point, a shift amount necessary for matching the corresponding point between each image Si and the reference image B0 is calculated, and the following equation (1) is calculated based on the shift amount. From (4), the shift amount to the reference image B0 for each pixel of each image Si is calculated by the two-dimensional quintic polynomial of (4), and each image Si is warped to the reference image B0 for each pixel.

  Here, x and y are coordinates of corresponding points in each image Si, x ′ and y ′ are coordinates on the reference image B0, Δx and Δy are shift amounts to the reference image B0, n is a degree, aij, bij Is a coefficient. Note that the coefficient of polynomial approximation is obtained using the method of least squares. At this time, for a pixel whose coordinates after warping move to a position including a decimal point instead of an integer, a pixel value is obtained by first order approximation from four neighborhoods. That is, pixel values are distributed in proportion to the distance from the coordinates after warping to the coordinates of each pixel for the four pixels surrounding the coordinates after warping.

  Note that the alignment method is not limited to warping, and an arbitrary method such as affine transformation can be used.

  Next, the adding unit 40 sets the image Soi to be added as the first image among the images Si after alignment (n = 1: Step ST7), and sets the reference image B0 and the addition target image Soi after alignment. Pixels corresponding to each other are added to generate an added image Sai (step ST8). Further, the brightness evaluation unit 41 calculates an evaluation value of the brightness of the added image Sa1 (step ST9). As the brightness evaluation value, an average value of pixel values of all the pixels of the added image Sai, a maximum value of pixel values, or the like may be used.

  Next, the CPU 42 determines whether or not all the images Si have been added (step ST10). If step ST10 is affirmed, the added image Sai is recorded on the recording medium 35 (step ST11), and the process is terminated. To do. At this time, the compression / decompression processing unit 33 compresses the added image Sai and records it on the recording medium 35.

  If step ST10 is negative, it is determined whether or not the brightness evaluation value is less than a predetermined threshold value Th1 (step ST12). Note that the threshold Th1 may be set so that the added image Sai has the desired brightness based on the exposure setting value determined by the AE / AWB processing unit 31. If step ST12 is affirmed, the addition target image Soi is set to the next image (n = n + 1), the addition image Sai is set to a new reference image B0 (step ST13), and the process returns to step ST8 and returns to step ST8. The subsequent processing is repeated. If step ST12 is negative, the process proceeds to step ST11, the added image Sai is recorded on the recording medium 35, and the process ends.

  As described above, in the first embodiment, a corresponding point between one reference image B0 and another image Si other than the reference image is detected, and the alignment between the reference image B0 and the other image Si is detected according to the corresponding point. And the reference image B0 after alignment and one addition target image Soi to be added among the other images Si are added to evaluate the brightness of the addition image Sai. For this reason, when evaluating the brightness of the added image, the reference image B0 and the addition target image Soi are aligned, and as a result, it is possible to prevent the added image Sai from being blurred. Therefore, it is possible to acquire a high-quality added image Sai.

  Further, the addition image Sai is used as a new reference image B0 until the addition image Sai whose brightness evaluation value is equal to or greater than the threshold Th1 is acquired, or until all the plurality of images Si are added. By changing the image Soi, adding the new reference image B0 and the addition target image Soi, and evaluating the brightness, it is possible to obtain a blur-free added image having a desired brightness.

  In the first embodiment, the image number calculation unit 37 calculates the required number of images N. However, the number of images to be captured is set in advance, and the set number of images is acquired by shooting. It may be. In this case, the image number calculation unit 37 is unnecessary.

  Next, a second embodiment of the present invention will be described. The digital camera to which the image addition device according to the second embodiment is applied has the same configuration as the digital camera 1 to which the image addition device according to the first embodiment is applied, and only the processing to be performed is different. Then, the detailed description about a structure is abbreviate | omitted.

  FIG. 3 is a flowchart of the addition process in the second embodiment of the present invention. In the second embodiment, the processes from step ST21 to step ST28 are the same as the processes from step ST1 to step ST8 in the first embodiment, and thus detailed description thereof is omitted here.

  Subsequent to step ST28, the CPU 42 determines whether or not all the images Si have been added (step ST29). If step ST29 is negative, the addition target image Soi is set to the next image (n = N + 1), the added image Sai is set as a new reference image B0 (step ST30), and the process returns to step ST28 to repeat the processes after step ST28. When step ST29 is affirmed, the added image Sai is recorded on the recording medium 35 (step ST30), and the process is terminated.

  As described above, in the second embodiment, the number N of images necessary for setting the added image to a desired brightness, which is calculated based on the surrounding brightness, is calculated. Corresponding points between one reference image B0 and another image Si other than the reference image are detected, the reference image B0 and another image Si are aligned according to the corresponding points, and the reference image B0 after alignment And all other images Si are added. For this reason, it is possible to obtain a high-quality added image having a desired brightness and without blur.

  Next, a third embodiment of the present invention will be described. FIG. 4 is a schematic block diagram showing the configuration of a digital camera to which the image adding device according to the third embodiment of the present invention is applied. Note that in the third embodiment, the same reference numerals are assigned to the same configurations as those in the first embodiment, and detailed description thereof is omitted here. The digital camera 1A according to the third embodiment is different from the first embodiment in that a brightness adjusting unit 44 is provided instead of the brightness evaluating unit 41.

  Next, addition processing in the third embodiment will be described. FIG. 5 is a flowchart of the addition process in the first embodiment. In the third embodiment, the processes from step ST41 to step ST48 are the same as the processes from step ST1 to step ST8 in the first embodiment, and thus detailed description thereof is omitted here.

  Subsequent to step ST48, the added image Sai is stored in the frame memory 26 (step ST49). Then, the CPU 42 determines whether or not all the images Si have been added (step ST50). If step ST50 is negative, the addition target image Soi is set to the next image (n = n + 1), The added image Sai is set as a new reference image B0 (step ST51), the process returns to step ST48, and the processes after step ST48 are repeated.

  If step ST50 is affirmed, the brightness adjustment unit 44 adjusts the brightness of all the added images Sai stored in the frame memory 26 (step ST52). Here, in the present embodiment, since N images are acquired by photographing, N−1 additional images Sai are generated. Further, since the necessary number of images N is calculated so that an image having a desired brightness is acquired, the added image SaN-1 generated by adding all the images Si has a desired brightness. ing. On the other hand, the other added images Sai are darker as the number of added images is smaller. For this reason, the brightness adjusting unit 44 adjusts the brightness of each added image Sai so that the brightness of all the added images Sai becomes a desired brightness. Therefore, the brightness of the added image Sai with a smaller number of added images is adjusted more greatly.

  Note that, as a method of adjusting the brightness, a method of increasing the gain and a method of correcting the gradation can be considered, but either method may be used. Further, which method is used may be set by the user for the digital camera 1A.

  Then, all the added images Sai whose brightness has been adjusted are displayed on the monitor 28 (step ST53). FIG. 6 is a diagram illustrating the brightness-adjusted added image displayed on the monitor. Here, if a subject moves greatly or shakes during the shooting of the required number N of images, if a plurality of images are added, the added image is blurred. In such a case, the degree of image blur differs depending on the number of images to be added. For example, if the required number of images N is 5 as shown in FIG. 7, and the subject has moved between the third image S3 and the fourth image S4, all the images S1 When S5 is added, the image is blurred at the moving arm portion.

  In the third embodiment, the brightness of the added image Sai is adjusted to a desired brightness, and a plurality of brightness adjusted images corresponding to the added number are generated. For this reason, the added image generated by adding the images S1 to S3 has no blur and has a desired brightness. On the other hand, the added image generated by adding the images S1 to S4 or the images S1 to S5 has a desired brightness, but the arm portion is blurred.

  The CPU 42 determines whether or not the user has selected an image (step ST54), and if step ST54 is negative, the process returns to step ST53. When step ST54 is affirmed, the selected addition image Sai is recorded on the recording medium 35 (step ST55), and the process is terminated.

  Accordingly, the user can select a brightness adjusted image that has a desired degree of blur from a plurality of brightness adjusted images displayed on the monitor 28, i.e., has reduced blur. Therefore, it is possible to obtain an added image with less brightness and with less brightness.

  In the third embodiment, a plurality of brightness-adjusted added images are displayed on the monitor 28, and the added images selected by the user are recorded on the recording medium 35. However, all the added images are recorded. You may make it record on the medium 35 and make a user select the addition image desired later.

  In the third embodiment, the image number calculation unit 37 calculates the required number of images N. However, the number of images to be captured is set in advance, and the set number of images is acquired by shooting. It may be. In this case, the image number calculation unit 37 is unnecessary.

  Next, a fourth embodiment of the present invention will be described. FIG. 8 is a schematic block diagram showing the configuration of a digital camera to which the image adding apparatus according to the fourth embodiment of the present invention is applied. Note that the same reference numerals in the fourth embodiment denote the same components as those in the third embodiment, and a detailed description thereof will be omitted here. The digital camera 1B according to the fourth embodiment differs from the third embodiment in that an image determination unit 45 that determines an image necessary for addition is provided.

  Next, addition processing in the fourth embodiment will be described. FIG. 9 is a flowchart of the addition process in the fourth embodiment. Note that in the fourth embodiment, the processes from step ST61 to step ST65 are the same as the processes from step ST41 to step ST45 in the third embodiment, and thus detailed description thereof is omitted here.

  Subsequent to step ST65, the image determination unit 45 determines an image necessary for addition from the plurality of images Si (step ST66).

  FIG. 10 is a flowchart of an image determination process necessary for addition. First, the image determination unit 45 sets the image to be processed in the image Si as the first image (n = 1: step ST81), and determines whether or not the number of corresponding points is less than the threshold value Th2. (Step ST82). If step ST82 is affirmed, the image to be processed has a small number of corresponding points and is difficult to align and is not suitable for image addition. Therefore, the image is excluded from the addition target (step ST83). Proceed to step ST86.

  If step ST82 is negative, it is determined whether or not the distance between all corresponding points in the state where the reference image B0 and the image to be processed are superimposed is less than the threshold value Th3 (step ST84). If step ST84 is negative, since the corresponding point of the image to be processed must match the corresponding point of the reference image B0, the image needs to be largely deformed, so that it is not suitable for image addition. Proceeding to ST83, the image is excluded from the addition target.

  If step ST84 is affirmed, the image to be processed is determined as an image necessary for addition (step ST85), and it is further determined whether or not all images Si are necessary for addition (step ST86). . If step ST86 is negative, the process target is set to the next image (n = n + 1: step ST87), the process returns to step ST82, and the processes after step ST82 are repeated. When step ST86 is affirmed, the image determination process necessary for addition is terminated.

  Returning to FIG. 9, following step ST <b> 66, the alignment unit 39 performs alignment between the reference image B <b> 0 and the image Si (step ST <b> 67), and the addition unit 40 includes the addition target 40 in the image Si after alignment. The image Soi is set as the first image (n = 1: Step ST68), and the corresponding pixels of the reference image B0 after alignment and the addition target image Soi are added to generate an added image Sai (Step ST69). ).

  Then, the CPU 42 determines whether or not all the images Si determined to be necessary for addition have been added (step ST70). If step ST70 is negative, the addition target image Soi is changed to the next image. In addition to setting (n = n + 1), the added image Sai is set as a new reference image B0 (step ST71), and the process returns to step ST69 to repeat the processing after step ST69.

  When step ST70 is affirmed, the brightness adjustment unit adjusts the brightness of the added image Sai (step ST72), records the brightness-adjusted added image Sai on the recording medium 35 (step ST73), and ends the process. To do. Note that the brightness adjustment in the fourth embodiment is performed to set the brightness of the added image to a desired brightness in order to reduce the brightness of the added image by the amount of the image excluded from the addition target. Is.

  As described above, in the fourth embodiment, the image necessary for the addition is determined from the plurality of images Si based on the corresponding points, and the addition image is generated using only the image necessary for the addition. An image in which the corresponding point with the reference image is greatly deviated can be excluded from the addition target, whereby an added image with less blur can be acquired. In addition, since the brightness of the added image is adjusted, an added image having a desired brightness can be generated.

  In the fourth embodiment, as the process for adjusting the brightness of the added image, a method of increasing the gain and a method of correcting the gradation are conceivable, as in the third embodiment. It may be used. Further, which method is used may be set by the user for the digital camera 1B.

  In the fourth embodiment, the number-of-images calculation unit 37 calculates the required number of images N. However, the number of images to be photographed is set in advance, and the set number of images is acquired by photographing. It may be. In this case, the image number calculation unit 37 is unnecessary.

  In the first to fourth embodiments, the required number of images N is calculated based on the brightness of the pre-image. However, the required number of images N may be calculated based on the brightness of the through image. Good. In this case, in the shooting process in step ST2, etc., the main shooting is performed following the pre-shooting.

  As described above, the digital camera according to the embodiment of the present invention has been described. However, the computer includes the above-described image number calculation unit 37, corresponding point detection unit 38, alignment unit 39, addition unit 40, brightness evaluation unit 41, brightness. A program that functions as means corresponding to the adjustment unit 44 and the image determination unit 45 and performs processing as shown in FIGS. 2, 3, 5, 9, and 10 is also one embodiment of the present invention. A computer-readable recording medium in which such a program is recorded is also one embodiment of the present invention.

1, 1A, 1B Digital Camera 37 Image Number Calculation Unit 38 Corresponding Point Detection Unit 39 Positioning Unit 40 Addition Unit 41 Brightness Evaluation Unit 42 CPU
44 Brightness adjustment unit 45 Image determination unit

Claims (4)

Photographing means for acquiring an image by photographing;
An image for obtaining a plurality of images by photographing with a predetermined exposure time, and calculating the number of images necessary to obtain the desired brightness for the added image when the plurality of images are added to obtain an added image. Sheet number calculating means;
Photographing control means for controlling the photographing means to obtain the calculated number of images;
Corresponding point detecting means for detecting corresponding points between one reference image and other images other than the reference image among the required number of images;
Image determining means for determining an image necessary for addition among the other images based on the corresponding points;
Alignment means for aligning the reference image and the image necessary for the addition based on the corresponding points;
Adding means for adding the reference image after alignment and the image necessary for the addition to generate the added image;
An imaging apparatus comprising: brightness adjustment means for adjusting the brightness of the added image.   The image determining means has a second distance between the corresponding points when the number of the corresponding points is equal to or greater than a first threshold value and is superimposed on the reference image among the other images. 2. The photographing apparatus according to claim 1, wherein the photographing device is means for determining an image that is less than a threshold value as an image necessary for the addition. Acquiring an image by photographing with a photographing means;
A step of obtaining a plurality of images by photographing at a predetermined exposure time and adding the plurality of images to obtain an added image, and calculating a number of images necessary to obtain the desired brightness for the added image; When,
Controlling the imaging means to acquire the calculated number of images;
Detecting corresponding points between one reference image and other images other than the reference image among the required number of images;
Determining an image necessary for addition among the other images based on the corresponding points;
Aligning the reference image with the image required for the addition based on the corresponding points;
Adding the reference image after alignment and the image necessary for the addition to generate the added image;
Adjusting the brightness of the added image. A procedure for obtaining an image by photographing by a photographing means;
A procedure for obtaining a plurality of images by photographing at a predetermined exposure time, and calculating the number of images necessary to obtain the desired brightness for the added image when the plurality of images are added to obtain an added image. When,
A procedure for controlling the photographing means to acquire the calculated number of images;
Detecting a corresponding point between one reference image and other images other than the reference image among the required number of images;
A procedure for determining an image required for addition among the other images based on the corresponding points;
A procedure for aligning the reference image and the image necessary for the addition based on the corresponding points;
A procedure for generating the added image by adding the reference image after alignment and the image necessary for the addition;
A program causing a computer to execute a photographing method including a procedure for adjusting the brightness of the added image.

Images