JP4379334B2 - Image composition apparatus and electronic camera - Google Patents

Description

  The present invention relates to a technique for obtaining a composite image by superimposing a plurality of images.

  A technique for obtaining a multiple captured image by superimposing a plurality of captured images is known (see Patent Document 1).

JP 2003-298925 A

  Japanese Patent Application Laid-Open No. H10-260260 discloses that when a multiple captured image is obtained, the background portion of the captured image is replaced with a through image and displayed on a monitor. From the image displayed on the monitor by replacing the background of the image, it is not possible to confirm the overlay effect for the entire area of the image.

An image composition apparatus according to the present invention includes an image composition unit that obtains an image obtained by superimposing a first image to which a first gain is applied and a second image to which a second gain is applied, and changes the first gain. The first gain setting means for setting, the second gain setting means for changing and setting the second gain, and the first gain generated based on the data of the first image before the image composition means starts superimposing the images. Superimposing a first sub-image having a smaller number of data than one image and a second sub-image having a smaller number of data than a second image generated based on the data of the second image; A sub-image synthesizing unit that obtains a sub-image corresponding to the image after superimposition that will be obtained in step 1, a first sub-image to which the first gain is applied, a second sub-image to which the second gain is applied, Information indicating 1 gain, information indicating 2nd gain, and sub-image composition And display means for displaying the sub-image after superimposing the same screen by the image synthesizing means, starts the superposition of the images in accordance with the start instruction after the sub image synthesizing means to obtain a sub-image after superimposition characterized in that it.
An electronic camera according to the present invention includes the image composition device according to any one of claims 1 to 6 .

According to the present invention , the first gain to be applied to the first image, the second gain to be applied to the second image, the first sub-image to which the first gain is applied, and the second gain It can be confirmed from the applied second sub-image and the composite image thereof.

The best mode for carrying out the present invention will be described below with reference to the drawings.
FIG. 1 is a diagram showing a configuration of an electronic camera according to an embodiment of the present invention. In FIG. 1, a subject image is formed on the imaging surface of the image sensor 2 by the photographing lens 1. The image sensor 2 captures a subject image and outputs an image signal to the A / D conversion circuit 3. The A / D conversion circuit 3 converts the analog image signal input from the image sensor 2 into a digital image signal, and outputs the converted digital data to the image memory 5. The timing generator (TG) 4 supplies a timing signal to the image sensor 2.

  The image memory 5 is a memory for temporarily storing image data after digital conversion and image data after addition described later. The memory capacity of the image memory 5 is sufficiently secured so that a plurality of captured image data can be stored. The image processing circuit 6 performs various types of image processing such as interpolation processing, color temperature processing, gradation correction processing, and resolution conversion processing on the image data stored in the image memory 5. The gain circuit 7 applies gain (pixel multiplication) to the image data stored in the image memory 5. The gain value is set in the gain circuit 7 by the gain setting circuit 9.

  The adder circuit 8 adds a plurality of frames of image data multiplied by the gain circuit 7 (adds corresponding pixels), and outputs the added image data to the image memory 5. The compression circuit 11 performs compression processing on the image data that has been subjected to image processing by the image processing circuit 6 using a method such as JPEG. The display circuit 10 generates a display signal necessary for reproducing and displaying the captured image on the display device 13.

  The display device 13 is constituted by, for example, a TFT liquid crystal panel, and reproduces and displays an image based on a display signal generated by the display circuit 10. The display device 13 is configured to display an operation menu in addition to the image. The operation menu is used when performing various settings for the electronic camera. The display circuit 10 also generates a display signal necessary for displaying the operation menu.

  The memory 14 is a memory that temporarily stores the image data compressed by the compression circuit 11. The recording medium 15 is configured by a memory card that can be attached to and detached from the electronic camera. The electronic camera can store the image data stored in the memory 14 in the recording mode 15 in the shooting mode and can read out the image data stored in the recording medium 15 in the playback mode and the image composition mode to the memory 14. Composed. The electronic camera has a shooting mode for shooting an object, a playback mode for displaying a shot image on the display device 13, and an image composition mode for combining shot images.

  The CPU 12 controls each block constituting the electronic camera described above. The gain circuit 7, the gain setting circuit 9, the adding circuit 8, the image processing circuit 6, the compression circuit 11 and the display circuit 10 are particularly referred to as an image processing block. The operation member 16 includes a mode change switch, which will be described later, and a pointing device such as an enter key and a cross key. The operation member 16 generates operation signals corresponding to various operations and sends them to the CPU 12. The mode switch is a switch for switching between a shooting mode, a playback mode, an image composition mode, and the like.

(Shooting mode)
When the release button (not shown) is pressed down, the electronic camera that is switched to the shooting mode by the mode change switch drives the aperture (not shown) of the taking lens 1 to a predetermined aperture and opens the shutter (not shown). Drive open. As a result, a subject image is formed on the imaging surface of the imaging device 2. The electronic camera ends the exposure by driving the shutter (not shown) to be closed so that an exposure time corresponding to the exposure condition can be obtained.

  The charges accumulated in the image sensor 2 during exposure are sequentially read from the image sensor 2 and stored in the image memory 5 as image data after A / D conversion. The image data at this point is called RAW data and is data before image processing is performed.

  The RAW data is subjected to predetermined image processing and compression processing, and the captured image data after the compression processing is recorded on the recording medium 15 via the memory 14. Further, the captured image is reproduced and displayed on the display device 13 by a display signal generated using the image data after the image processing. It should be noted that the photographed image data can also be recorded as RAW data.

  The electronic camera records captured image data as an image data file. The image data file includes a captured image (referred to as a main image) and a reduced image (thumbnail image) of 160 × 120 pixels in which the data size (number of pixels) is suppressed from that of the main image. The thumbnail image is generated using the data of the main image. Among these, the recording format of the main image data is divided into, for example, the following four stages depending on the presence / absence of image processing and the difference in compression rate during compression processing. The electronic camera switches the recording format of the main image data according to the data recording format selected at the time of shooting. The data recording format is selected by, for example, the operation menu.

(1) “RAW” for recording image data without image processing
(2) “TIFF” for recording image data after predetermined image processing in uncompressed RGB-TIFF format
(3) "FINE" for recording image data after predetermined image processing at a compression ratio of about 1/5
(4) “NORMAL” for recording image data after predetermined image processing at a compression ratio of about 1/10
Generally, the (1) side has high image quality, and the (4) side has low image quality. The compression processing in the above (3) and (4) is performed in the JPEG format. The predetermined image processing is color temperature adjustment processing or the like. In the present embodiment, (1) is selected as the recording format of the main image data.

(Image composition mode)
The electronic camera that has been switched to the image composition mode by the mode changeover switch superimposes and composes two images (referred to as a first image and a second image) recorded on the recording medium 15. The image composition process is as follows. Image selection, 2. 2. Gain setting, Synthesis, 4. It is performed in the order of saving. FIG. 2 is a diagram illustrating an initial screen displayed on the display device 13 in the image composition mode.

1. The image selection CPU 12 receives an operation signal for selecting “Image 1” corresponding to the first image from the cross key constituting the operation member 16 in a state where the initial screen of FIG. 2 is displayed on the display device 13. Further, when an operation signal indicating selection is input from the enter key, the image selection screen shown in FIG. 3 is displayed on the display device 13 instead of the initial screen being displayed. In FIG. 3, six thumbnail images respectively read from the six image data files recorded on the recording medium 15 are displayed in the upper and lower two rows, three images. When an operation signal for selecting the upper center image (building night view) is input from the cross key and an operation signal indicating selection is input from the determination key, the CPU 12 sets the image as a first image, An image composition screen shown in FIG. 4 is displayed instead of the image selection screen being displayed on the display device 13. It should be noted that other thumbnail images other than the six thumbnail images displayed on the image selection screen can also be read out and displayed from the image data file newly recorded on the recording medium 15 by the cross key operation. Has been.

  In FIG. 4, the thumbnail image corresponding to the first image is displayed as “Image 1”, and the same thumbnail image as the first image is displayed as “Preview”. When the CPU 12 receives an operation signal for selecting “Image 2” corresponding to the second image from the cross key constituting the operation member 16 and further receives an operation signal indicating selection determination from the determination key, the display device 13 is displayed again in place of the display screen shown in FIG. When an operation signal for selecting the lower right image (month) is input from the cross key and an operation signal indicating selection is input from the enter key, the CPU 12 sets the image as a second image and displays it. An image composition screen shown in FIG. 5 is displayed instead of the image selection screen being displayed on the device 13.

  In FIG. 5, “Image 1” is a thumbnail image corresponding to the first image, “Image 2” is a thumbnail image corresponding to the second image, and “Preview” is the first image and the second image. A thumbnail image obtained by synthesizing two corresponding thumbnail images is displayed.

2. When the operation signal for selecting the gain “Gain 1” for the first image is input from the cross key constituting the operation member 16 and the operation signal for increasing or decreasing the gain is input, the gain setting CPU 12 receives the first image. Is changed from the initial value “1.0” to a value corresponding to the operation signal, and data indicating the changed gain value is sent to the gain setting circuit 9. When the gain setting circuit 9 sets the changed gain setting value in the gain circuit 7, the display device 13 reproduces and displays a thumbnail image corresponding to the first image to which the changed gain is applied.

  Similarly, when an operation signal for selecting a gain “Gain 2” for the second image is input from the cross key constituting the operation member 16 and an operation signal for increasing or decreasing the gain is input, the CPU 12 receives the second image. Is changed from the initial value “1.0” to a value corresponding to the operation signal, and data indicating the changed gain value is sent to the gain setting circuit 9. When the gain setting circuit 9 sets the changed gain setting value in the gain circuit 7, the display device 13 reproduces and displays a thumbnail image corresponding to the second image to which the changed gain is applied.

  FIG. 6 is a diagram illustrating an image composition screen when the gain setting value for the first image is changed to “0.7” and the gain setting value for the second image is changed to “1.3”. In FIG. 6, the exposure is corrected to the under side by changing the gain of the first image (building night view) to be smaller than 1, and the exposure is corrected by changing the gain of the second image (month) to be larger than 1. Is corrected to the over side.

  FIG. 7 is a diagram illustrating an image composition screen when the gain setting value for the first image remains “1.0” and the gain setting value for the second image is changed to “0.3”. In FIG. 7, when the gain of the second image (month) is changed to be smaller than 1, the exposure is corrected to the under side.

  The adder circuit 8 combines the two thumbnail images corresponding to the first image and the second image. The adder circuit 8 adds the corresponding pixels of the thumbnail image corresponding to the gained first image and the thumbnail image corresponding to the gained second image. The thumbnail image after addition (after synthesis) is displayed in real time as “Preview” in FIGS.

3. The composition CPU 12 receives an operation signal for selecting “composition” from the cross key constituting the operation member 16 in a state where the image composition screen as shown in FIGS. When an operation signal indicating selection decision is input from the key, synthesis of the main image corresponding to the first image and the main image corresponding to the second image is started.

  The CPU 12 reads two main image data from the recording medium 15 and stores the read main image data in the image memory 5. The gain circuit 7 displays a set gain (displayed in the image composition screen shown in FIGS. 5 to 7) for each of the main image corresponding to the first image and the main image corresponding to the second image. Value). The adding circuit 8 adds the corresponding pixels of the main image corresponding to the first image to which the gain has been applied and the main image corresponding to the second image to which the gain has been applied. The data of the main image after addition (after synthesis) is stored in the image memory 5 as synthesized image data.

4). The save CPU 12 is an operation signal for selecting “save” from the cross key constituting the operation member 16 after the above-described synthesis process 3 in a state where the image composition screen as shown in FIGS. When an operation signal indicating selection is input from the enter key, the composite image data according to 3 is recorded from the image memory 5 to the recording medium 15 via the memory 14. At this time, an image data file including a composite image corresponding to the main image and a composite image corresponding to the thumbnail image (generated by performing resizing processing on 160 × 120 pixels from the composite image corresponding to the main image) is recorded. . The data recording format corresponding to the main image follows the data recording format selected by the electronic camera in the shooting mode.

The embodiment described above will be summarized.
(1) The electronic camera superimposes and combines the first image and the second image in the image combining mode. The image composition process is as follows. Image selection, 2. 2. Gain setting, Synthesis, 4. Perform in order of saving. After the gain setting of 2 above (after the selection of the 1 image when the gain setting value may be the initial value “1.0”), a thumbnail image corresponding to the first image and a thumbnail image corresponding to the second image are obtained. The synthesized thumbnail image is displayed as “Preview” on the image composition screen (FIGS. 5 to 7). In general, the thumbnail image recorded in the image data file is YCbCr data represented by 8 bits, and the number of pixels is 160 × 120 (= 19200), which is very small compared to the main image (for example, several million pixels). . Furthermore, since the thumbnail image is subjected to interpolation processing and color temperature processing, it is not necessary to perform interpolation processing or color temperature processing for displaying “Preview”. Therefore, thumbnail images can be instantly added (synthesized) to display a thumbnail composite image as “Preview”, so the user can immediately check the composite image before performing image composition on the main image. it can.

(2) On the image composition screen (FIGS. 5 to 7), the selected two thumbnail images (thumbnail image before composition) and the thumbnail image after composition are displayed on the same screen. It becomes easy.

(3) Since the gains for two thumbnail images (thumbnail images before composition) can be changed and pixel multiplication is performed for all the pixels constituting the thumbnail, the adjustment state of exposure adjustment at the time of composition in the entire thumbnail image And the synthesis result can be confirmed.

(4) Since the recording format of the main image data is (1) the “RAW” format, the combined main image data can also be recorded in the “RAW” format. By synthesizing (adding) data in the “RAW” format, it is possible to suppress deterioration such as gradation characteristics to be multiplied at the time of synthesis as compared with the case of synthesizing (adding) data in “JPEG”. Furthermore, by recording the combined main image data in the “RAW” format, the combined image can be further combined with another image.

(5) In order to add (synthesize) thumbnail images and display “Preview”, no processing is required on the image data file recorded on the recording medium 15, so that the image of the composition source There is no risk of destroying the data file.

  In the above description, an example in which the recording format of the main image data is distinguished based on the presence / absence of image processing or the difference in compression rate at the time of compression processing is shown. You may make it distinguish by the difference.

  The thumbnail image data included in the image data file is used for “Preview” display, but if the image size is smaller than the main image data, the main image data is thinned out. The composition for displaying the “Preview” may be performed using image data other than the thumbnails generated in this way.

  In the above-described embodiment, an example in which two images of the first image and the second image are superimposed and combined has been described. However, the number of images to be combined is not limited to two images, and may be three images or six images.

  In the image selection screen (FIG. 3), six thumbnail images are displayed in two upper and lower stages, but the number of thumbnail images displayed on one screen is not limited to six images, and may be four images or nine images.

  In the above description, the example in which the image composition process is performed by the electronic camera has been described. In addition to the electronic camera, the image composition apparatus may be configured by causing an apparatus that handles electronic images such as a photo storage device, a photo stand, and a PDA to perform image composition processing.

  Correspondence between each component in the claims and each component in the best mode for carrying out the invention will be described. The image synthesizing means and the sub image synthesizing means are constituted by, for example, an adding circuit 8. For example, a thumbnail image corresponds to the first sub-image and the second sub-image. The sub-image after superimposition corresponds to, for example, a thumbnail image in which corresponding pixels are added. A display means is comprised by the display circuit 10 and the display apparatus 13, for example. The image synthesizing apparatus is constituted by an electronic camera, for example. For example, gain corresponds to the superposition ratio. The superimposition ratio setting means is configured by a gain setting circuit 9, for example. In addition, the above description is an example to the last, and when interpreting invention, it is not limited to the correspondence of the component of said embodiment and the component of this invention at all.

It is a figure which shows the structure of the electronic camera by one embodiment of this invention. It is a figure which illustrates the initial screen displayed in image composition mode. It is a figure which illustrates an image selection screen. It is a figure which illustrates an image composition screen. It is a figure which illustrates an image composition screen. It is a figure which illustrates the image composition screen at the time of changing a gain. It is a figure which illustrates the image composition screen at the time of changing a gain.

Explanation of symbols

DESCRIPTION OF SYMBOLS 2 ... Imaging device 5 ... Image memory 6 ... Image processing circuit 7 ... Gain circuit 8 ... Adder circuit 9 ... Gain setting circuit 10 ... Display circuit 11 ... Compression circuit 12 ... CPU
13 ... Display device 14 ... Memory 15 ... Recording medium 16 ... Operating member

Claims (7)

Image combining means for obtaining an image obtained by superimposing the first image to which the first gain is applied and the second image to which the second gain is applied;
First gain setting means for changing and setting the first gain;
Second gain setting means for changing and setting the second gain;
A first sub-image having a smaller number of data than the first image generated based on the data of the first image and the second image before the image synthesizing unit starts superimposing the image; A sub-image obtained by superimposing a second sub-image having a smaller number of data than the second image generated based on the data of the second image and obtaining a sub-image corresponding to the image after superimposition that would be obtained by the image synthesis means Image synthesis means;
The first sub-image to which the first gain is applied, the second sub-image to which the second gain is applied, information indicating the first gain, information indicating the second gain, and the sub-image Display means for displaying the sub-image superimposed by the combining means on the same screen ,
The image synthesizing device, wherein the image synthesizing unit starts superimposing the images in response to a start instruction after the sub-image synthesizing unit obtains the superimposed sub-image . The image composition apparatus according to claim 1 ,
The first sub-image is a thumbnail image included in a data file in which the first image is recorded,
The image synthesizing apparatus, wherein the second sub-image is a thumbnail image included in a data file in which the second image is recorded. The image composition device according to claim 1 or 2 ,
A superposition ratio setting means for setting a superposition ratio when the image composition means superimposes the image;
The image synthesizing apparatus, wherein the sub-image synthesizing unit superimposes the first sub-image and the second sub-image using the superimposition ratio set by the superimposition ratio setting unit. The image composition device according to claim 3 .
The image synthesizing apparatus is characterized in that recording of the superimposed image is started in response to a start instruction after the image synthesizing unit obtains the superimposed image. The image composition device according to claim 4 , wherein
The image composition apparatus according to claim 1, wherein the superimposed image recording format is an image recording format set in an electronic camera. In the image composition device according to any one of claims 1 to 5 ,
An image composition apparatus in which the image superimposed by the image composition means is recorded as RAW data. An electronic camera comprising the image composition device according to any one of claims 1 to 6 .

Images