JP6003081B2 - Imaging apparatus and imaging method - Google Patents

Description

  The present invention relates to an improvement of an imaging apparatus and an imaging method having a so-called comparatively bright combination processing function.

  In recent years, with the widespread use of digital cameras as imaging devices, the digital cameras have gradually advanced technically. For example, a plurality of images obtained by repeatedly capturing a subject image at a predetermined cycle during bulb photography. There has been proposed a digital camera in which an addition operation is performed on images to generate an addition image, and a past addition image is sequentially displayed so that a change in exposure amount of the addition image can be easily grasped (for example, Patent Document 1). reference).

In addition, digital cameras are used for various shooting applications, and many new image expression techniques that were not technically possible in the age of film cameras have been born.
One of them is a video expression technique that creates a new video using a large amount of still images obtained by fixing a digital camera to a tripod and performing interval shooting. For example, to create a photo with increased light trails of fireflies or a photo that leaves the light trails of stars moving in the night sky due to the diurnal motion of the earth, a video expression technique that performs a composition process called comparative bright synthesis There is.

  FIG. 20 is a conceptual diagram for explaining the outline of the comparatively bright combination process. In FIG. 20, (a) shows an image group as subject images acquired by interval shooting, (b) shows a photograph after comparatively bright combining processing, and (c) and (d) are comparatively bright combining processing. It is a schematic diagram of the buffer memory area for demonstrating these procedures.

  Here, FIG. 20A shows an image group G1 made up of landscape photographs including a night starry sky as subject images taken by interval shooting, and the number of the landscape photographs is 300, for example. Degree.

  In the comparatively bright combining process, as shown in FIG. 20C, N corresponding to the first subject image is stored in the combined image memory area α corresponding to one image as a buffer memory used for the comparatively bright combining process. = 1 is set as the initial comparison target image data.

  Next, as shown in FIG. 20D, N = i (corresponding to the second and subsequent subject images in the updated image memory area β corresponding to one image as a buffer memory used for the comparatively bright combination processing. Each image data of i = 2,..., N) is set as update image data.

  Next, in the comparative bright combination processing, the luminance value Pαi of the predetermined pixel area i of the composite image memory area α and the pixel area i of the updated image memory area β corresponding to the predetermined pixel area i of the composite image memory area α. Luminance value Pβi is read out, the luminance value Pαi is compared with the luminance value Pβi, and the data of the pixel having the larger luminance value is replaced with the predetermined pixel area i in the composite image memory area α.

  In the comparatively bright combination process, this replacement process is performed for all the pixel data (i = 1 to m) of each update image data. When this replacement processing is performed for N subject images, the maximum brightness is extracted for each pixel corresponding to each other of the image data constituting each subject image by this comparatively bright combining processing, and each pixel of the composite image memory area α is extracted. Data having the maximum luminance value is written in the area.

  When the synthesized image G2 is constructed using the luminance data written in each pixel area of the synthesized image memory area α after the comparatively bright synthesis process, as shown in FIG. 20B, a still image of the landscape and the starry sky Even so, the light trails of the stars caused by the diurnal motion of the Earth extend, but the night sky part without stars and the bright scenery coexisted without becoming pure white, as if they were exposed for a long time Images can be obtained.

  However, when shooting night scenes, the sky becomes brighter as it approaches the dawn, and the combined image G2 becomes brighter only by means of combining bright portions of luminance information. There is a problem that the star trajectory becomes blurred after the synthesis process.

  In addition, if a subject having extremely bright brightness is captured during the composition process, the entire composite image G2 is affected by the brightness of the subject, so that there is a problem that an intended photograph cannot be obtained.

  The present invention has been made in view of the above circumstances, and an imaging apparatus capable of constructing an intended composite image flexibly corresponding to a luminance change at the time of acquiring a certain time interval of a subject image, and An object is to provide an imaging method.

  The imaging apparatus according to the present invention includes an acquisition unit 104 that acquires a subject image as image data corresponding to one image at regular time intervals, and an update that stores the image data as update image data whose luminance is compared for each pixel. The comparison target image data corresponding to one image whose luminance is compared for each pixel corresponding to each pixel of the image memory area β and the pixel of the updated image memory area β is stored, and the luminance is compared after comparing the luminance for each pixel. Of the composite image memory area α for storing the comparison target image data in which the pixel data is replaced with the larger luminance as composite image data, and the update image data acquired by the acquisition unit 104 at regular time intervals The luminance data stored in the pixels of the composite image memory area α is increased when the comparison for each pixel of the update image data and the composite image data is performed during acquisition. In a square luminance data and an image processing unit 104 for replacement process.

When the update image data to be stored in the update image memory area β is the initial image data constituting the first subject image, the image processing means 104 obtains an average value of luminance as the initial image data and updates it. When the update image data to be stored in the image memory area β is update image data constituting the second and subsequent subject images, an average value of the brightness as the update image data is obtained, and the brightness of both image data The content of the replacement process is changed depending on whether the difference between the average values is greater or smaller than the threshold value.

  According to the present invention, it is possible to construct a composite image intended in a flexible manner in response to a change in luminance when a subject image is acquired at regular time intervals.

FIG. 1 is a front view of a digital camera as an imaging apparatus according to an embodiment of the present invention. FIG. 2 is a rear view of the digital camera shown in FIG. FIG. 3 is a top view of the digital camera shown in FIG. FIG. 4 is an internal system configuration diagram of the digital camera shown in FIG. FIG. 5 is an explanatory diagram of an initial setting screen for interval shooting shown in FIG. FIG. 6 is an explanatory diagram of items that can be selected on the setting screen shown in FIG. FIG. 7 is a main flowchart at the time of interval shooting according to the first embodiment of the digital camera shown in FIG. FIG. 8 is a flowchart of the shooting setting confirmation process shown in FIG. FIG. 9 is a flowchart of the comparatively bright combination process shown in FIG. FIG. 10 is a flowchart showing an example of the interval image file output process shown in FIG. FIG. 11 is a flowchart for explaining a first modification of the interval photographing process shown in FIG. FIG. 12 is a flowchart for explaining a second modification of the interval photographing process shown in FIG. FIG. 13 is a flowchart for explaining a third modification of the interval photographing process shown in FIG. FIG. 14 is an explanatory diagram of the second embodiment of the digital camera shown in FIG. 1, wherein (a) is a schematic diagram showing a state in which the composite image memory area is divided into a plurality of areas, and (b) is a schematic diagram showing the state. Schematic diagram showing a state where the update image memory area is divided into a plurality of areas, (c) is a schematic diagram showing pixel data written in each area of the composite image memory area, and (d) is an update. It is a schematic diagram which shows the pixel data written in each area of the memory area for images. FIG. 15 is an explanatory diagram illustrating an example of a MENU screen used for selecting a threshold value according to the second embodiment. FIG. 16 is an explanatory diagram schematically showing the concept of the auto setting shown in FIG. FIG. 17 is a main flowchart at the time of interval shooting according to the second embodiment of the digital camera shown in FIG. FIG. 18 is a flowchart of the comparatively bright combination process shown in FIG. FIG. 19 is an explanatory diagram illustrating an example of processing for each area. FIG. 20 is a conceptual diagram of comparatively bright combination processing.

Embodiments of an image recording apparatus according to the present invention will be described below with reference to the drawings.
(General configuration of digital camera)
1 is a front view showing an example of a digital camera (hereinafter referred to as a camera) as an image recording apparatus (imaging apparatus) according to the present invention, FIG. 2 is a rear view thereof, FIG. 3 is a top view thereof, and FIG. It is a block circuit diagram which shows the outline | summary of the system configuration inside a camera.

In FIG. 1, a release switch (release shutter) SW1, a mode dial SW2, and a sub liquid crystal display (also referred to as a sub LCD or display unit) 1 shown in FIG. The sub LCD 1 displays the number of shootable images.
When the release switch SW1 is pushed in the first stage, auto-focusing is executed, and in the second stage, shooting is executed.

  In front of the camera body (apparatus body), a lens barrel unit 7 including a photographing lens, an optical finder 4, a strobe light emitting unit 3, a distance measuring unit 5, and a remote control light receiving unit 6 are provided.

  On the back of the camera, as shown in FIG. 2, a power switch SW13, an LCD monitor (also referred to as a display unit) 10, an AFLED 8, a strobe LED 9, an optical viewfinder 4, a wide-angle zoom switch SW3, a telephoto zoom switch SW4, a self-timer Keys such as setting / deleting switch SW5, playback switch SW6, upward movement / strobe set switch SW7, rightward movement switch SW8, downward movement / macro switch SW9, leftward movement / image confirmation switch SW10, display switch SW11, menu / OK switch SW12, etc. A switch is provided. A memory card / battery loading chamber lid 2 is provided on the side of the camera body.

  In the camera of this embodiment, a screen for performing various settings can be displayed on the LCD monitor 10 by the menu / OK switch SW12. By operating the up / down / left / right buttons SW7 to SW10 on the screen for performing various settings, it is possible to shift to the continuous shooting mode (interval shooting mode).

  Further, by operating the playback switch SW6, the subject image can be displayed on the LCD monitor 10 using the captured image data, and the zoom button switch (the telephoto direction zoom switch TARE) can be displayed while the subject image is displayed. ) By operating SW4 and zoom button switch (wide-angle zoom switch WIDE) SW3, the subject image can be enlarged / reduced at an arbitrary magnification. Further, the enlarged / reduced portion of the subject image can be arbitrarily designated by operating the up / down / left / right buttons SW7 to SW10 in the enlarged / reduced state.

(Camera internal system configuration)
Next, the system configuration inside the camera will be described.
In FIG. 4, reference numeral 104 denotes a digital still camera processor (hereinafter also referred to as a processor).

  The processor 104 includes a CCD1 signal processing block 1041, a CCD2 signal processing block 1042, a CPU block 1043, a local SRAM 1044, a USB block 1045, a serial block 1046, a JPEG / CODEC block (block for JPEG compression / decompression) 1047, and a RESIZE block (image). 1048, a TV signal display block (block that converts image data to a video signal for display on a display device such as a liquid crystal monitor / TV) 1049, a memory card controller block ( Block 10410 for controlling a memory card for recording photographed image data. Each of these blocks is connected to each other via a bus line. The CCD1 signal processing block 1041 and the CCD2 signal processing block 1042 constitute part of an image processing means (DSP). The CPU block 1043 functions as control means described later.

  Outside the processor 104, RAW-RGB image data (image data in a state in which white balance setting and γ setting are performed), YUV image data (image data in a state in which luminance data and color difference data are converted), a JPEG image An SDRAM 103 for storing data (image data in a JPEG compressed state) is arranged, and the SDRAM 103 is connected to the processor 104 via a memory controller (not shown) and a bus line. The SDRAM 103 includes a composite image memory area α and an update image memory area β (with reference to FIG. 20) for performing comparatively bright combination processing described later.

  In addition to the processor 104, storage means (RAM 107), built-in memory (memory for storing captured image data even when no memory card is mounted on the memory card throttle) 120, control programs, parameters, and the like are stored. ROM 108 is provided, and these are also connected to the processor 104 by a bus line.

  When the camera power switch SW13 is turned on, the control program stored in the ROM 108 is loaded into the main memory (not shown) of the processor 104. The processor 104 controls the operation of each unit according to the control program, The parameter or the like is temporarily stored in the RAM 107 or the like.

  In addition, the ROM 108 stores a program for performing interval shooting and comparatively bright combination processing. In this embodiment, the comparatively bright combination processing is realized by software using a program, but it can also be configured by dedicated hardware.

  The lens barrel unit 7 includes a zoom optical system 71 having a zoom lens 71a as a lens system, a focus optical system 72 having a focus lens 72a as a lens system, a diaphragm unit 73 having a diaphragm 73a, and a mechanical shutter unit having a mechanical shutter 74a. A lens barrel 74 is provided.

  The zoom optical system 71, the focus optical system 72, the aperture unit 73, and the mechanical shutter unit 74 are driven by a zoom motor 71b, a focus motor 72b, an aperture motor 73b, and a mechanical shutter motor 74b, respectively.

Each of these motors is driven by a motor driver 75, and the motor driver 75 is controlled by a CPU block 1043 of the processor 104.
A subject image is formed on a CCD 101 as a solid-state image sensor by each lens system of the lens barrel unit 7, and the CCD 101 converts a subject image into an image signal and outputs a part of the image signal to the F / E-IC 102. Function as.

The F / E-IC 102 includes a CDS 1021 that performs correlated double sampling for image noise removal, an AGC 1022 for gain adjustment, and an A / D conversion unit 1023 that performs analog-digital conversion.
That is, the F / E-IC 102 performs predetermined processing on the image signal, converts the analog image signal into a digital signal, and outputs the digital signal to the CCD1 signal processing block 1041 of the processor 104. A CMOS can be used instead of the CCD 101.

These signal control processes are performed via the TG 1024 by the vertical synchronization signal VD and the horizontal synchronization signal HD output from the CCD1 signal processing block 1041 of the processor 104.
The TG 1024 generates a drive timing signal based on the vertical synchronization signal VD and the horizontal synchronization signal HD.

  The CPU block 1043 of the processor 104 controls the sound recording operation by the sound recording circuit 1151. The voice recording circuit 1151 records the amplified signal of the voice recording signal converted by the microphone 1153 by the microphone amplifier (AMP) 1152 according to the command. The CPU block 1043 also controls the operation of the audio reproduction circuit 1161. The audio reproduction circuit 1161 is configured to reproduce an audio signal stored in the memory as appropriate according to a command, output the audio signal to an audio amplifier (AMP) 1162, and output audio from the speaker 1163.

The CPU block 1043 further controls the strobe circuit 114 to emit illumination light from the strobe light emitting unit 3. In addition to this, the CPU block 1043 also controls the distance measuring unit 5.
The CPU block 1043 is connected to the sub CPU 109 of the processor 104, and the sub CPU 109 performs display control by the sub LCD 1 via the LCD driver 111. The sub CPU 109 is further connected to the buzzer 113, an operation key unit including the AF LED 8, the strobe LED 9, the remote control light receiving unit 6, and the operation switches SW1 to SW13.

  The USB block 1045 is connected to the USB connector 122, and the serial block 1046 is connected to the RS-232C connector 1232 via the serial driver circuit 1231. The TV signal display block 1049 is connected to the LCD monitor 10 via the LCD driver 117, and a video amplifier (amplifier for converting the video signal output from the TV signal display block 1049 into 75Ω impedance) 118 is provided. Via a video jack (jack for connecting the camera to an external display device such as a TV) 119. The memory card controller block 10410 is connected to a contact with the card contact of the memory card throttle 121.

  The LCD driver 117 serves to drive the LCD monitor 10 and convert the video signal output from the TV signal display block 1049 into a signal to be displayed on the LCD monitor 10. The LCD monitor 10 is used for monitoring the state of a subject before photographing, confirming a photographed image, and displaying image data recorded in the memory card or the built-in memory 120. The LCD monitor 10 can also display the contents of the composite image memory area α.

Before describing the operation of the comparatively bright combination process at the time of interval shooting, the initial setting on the screen at the time of the comparatively bright combination process at the time of interval shooting will be described.
(Initial setting for comparatively bright composite processing on the menu screen)
FIG. 5 shows the initial setting of the comparatively bright combination process at the time of interval shooting and the screen transition by the setting button operation. 6 shows types of items that can be selected on the setting screen shown in FIG. 5. FIG. 6A shows items that can be selected when the file output setting at the time of composite reset shown in FIG. 5 is selected. (B) shows items that can be selected when the interval image file output setting process shown in FIG. 5 is selected.

Hereinafter, with reference to FIG. 5 and FIG. 6, the initial setting of the comparatively bright combination process at the time of interval shooting will be described.
When the user turns on the power of the camera and sets various shooting settings as desired, the user performs predetermined operations with the MENU / OK SW (SW12) and the up / down / left / right buttons (SW7 to SW10). Is displayed on the LCD monitor 10.

The “setting screen 1-1” illustrated in FIG. 5 is a “time digit” of an interval shooting time interval (that is, a time from the start of exposure to the start of the next shooting exposure, hereinafter referred to as an interval shooting time interval t1). Is a screen that can be set.
The numerical value can be changed by operating the up / down buttons (SW7, SW9). When the left button (SW10) is operated, a transition is made to “setting screen 1-5”, and when the right button (SW8) is operated, a transition is made to “setting screen 1-2”.

  The “setting screen 1-2” illustrated in FIG. 5 is a screen on which “minute digits” of the interval time can be set. The numerical value can be changed by operating the up / down buttons (SW7, SW9). When the left button (SW10) is operated, the screen transitions to “setting screen 1-1”, and when the right button (SW8) is pressed, the screen transitions to “setting screen 1-3”.

  A “setting screen 1-3” illustrated in FIG. 5 is a screen on which “second digits” of the interval time can be set. The numerical value can be changed by operating the up / down buttons (SW7, SW9). When the left button (SW10) is operated, a transition is made to “setting screen 1-2”, and when the right button (SW8) is operated, a transition is made to “setting screen 1-4”.

  The “setting screen 1-4” illustrated in FIG. 5 displays the composite image data in the composite image memory area α before the initialization when the composite image memory area α is initialized during the composite image processing. This is a screen for setting whether or not to perform the operation of outputting as a file. By operating the up / down buttons (SW7, SW9), as shown in FIG. 6A, either “leave the composite image before reset” or “leave nothing” can be selected. When the left button (SW10) is operated, the screen transitions to “setting screen 1-3”, and when the right button (SW8) is pressed, the screen transitions to “setting screen 1-5”.

  A “setting screen 1-5” illustrated in FIG. 5 is a screen that allows the following five types of settings when performing composite image processing. By operating the up / down buttons (SW7, SW9), as shown in FIG. 6, “leave a normal still image”, “leave a composite still image (composite processed image)”, “leave a normal movie”, “composite” “Leave video (composite processed image)” or “Leave nothing” can be selected. When the left button (SW10) is operated, the screen transitions to “setting screen 1-4”, and when the right button (SW8) is pressed, the screen transitions to “setting screen 1-1”.

  The “setting screen 1-1” to “setting screen 1-5” shown in FIG. 5 can determine the initial settings for interval shooting and composite image processing by pressing MENU / OK SW (SW12) from any screen. After confirming the initial setting, interval shooting and composition processing are started by pushing the release shutter SW1 into the second stage (so-called full-press operation).

(Description of Example 1 of Main Flowchart at Interval Shooting)
FIG. 7 shows a main flowchart when the digital camera executes interval shooting.
Here, the processor 104 executes a comparison of luminance for each pixel of the current update image data and the composite image data among the update image data acquired by the acquisition unit at regular time intervals during the acquisition. It also functions as image processing means for replacing the luminance data stored in the pixel area of the composite image memory area α with the luminance data having the higher luminance (so-called comparatively bright combination processing).

The update image memory area β functions to store image data acquired from the imaging means at regular time intervals as update image data whose luminance is compared for each pixel.
The composite image memory area α stores comparison target image data corresponding to one image whose luminance is compared for each pixel corresponding to the pixel of the updated image memory area β, and the luminance after comparing the luminance for each pixel. The comparison target image data in which the pixel data is subjected to the replacement processing with the higher luminance is stored as a composite image.

  During interval shooting, as shown in FIG. 7, first, the compositing process flag is cleared (S.1), and the subject image compositing counter is initialized (S.2). Here, the in-combination flag is a flag for determining whether or not a comparatively bright combination process is being executed using image data corresponding to one subject image.

  Next, the processor 104 shifts to the photographing condition setting confirmation process shown in FIG. 8 (S.3). In this shooting condition setting confirmation process (shooting setting confirmation process), first, it is determined whether or not the auto ranging mode (AF mode) is on (S.31). Note that the ranging mode is determined to be on except in the manual mode.

  The processor 104 is an S.I. If yes in step 31, a known autofocus process (AF process) is executed (S.32). 33. The processor 104 is an S.I. If no in step 31, step S. 32 is skipped. 33.

The processor 104 is an S.I. In 33, it is determined whether or not the auto exposure mode (AE mode) is set. Note that, except for the manual mode, the auto exposure mode is determined to be on.
The processor 104 is an S.I. If YES in step 33, a well-known auto exposure process (AE process) is executed (S.34), and the shooting setting confirmation process is terminated. 4 The processor 104 is an S.I. If the answer is no in step 33, step S. 34 is skipped and the shooting setting confirmation process is terminated. 4

Next, the processor 104 measures the interval shooting time interval t1 set by the user and performs shooting at each time interval t1 (S.4).
That is, an exposure process is performed every time the interval shooting time interval t1 elapses (S.5), and the compositing process flag is checked (S.6). When the compositing process flag is cleared, the processor 104 acquires image data corresponding to one subject image from the CCD (imaging device) 101 (S.7). When the compositing process flag is set, the acquisition of image data is prohibited until the comparatively bright compositing process is completed as the comparatively bright compositing process is in progress.

Next, the processor 104 determines whether or not it is the “first” image (S.8). When the count value of the combination number counter is “0”, it is determined that the “1” shot is taken. S. 8, if “YES”, the average value of the brightness of the image data acquired by the first image is stored in, for example, a free area of a memory (appropriate memory may be used). (S.9). And S. 11
The average value of luminance is obtained by dividing the sum of the luminance of each pixel corresponding to one subject image by the total number of pixels (number of pixels corresponding to one subject image) used for the sum of the luminance. Desired.

  S. 8, in the case of “NO”, it is determined that the second and subsequent shots are taken, and the average value of the brightness of the image data acquired by the second and subsequent shots is set, for example, in another free area of the memory. (S.10).

  Next, the processor 104 compares the average luminance value of the first image data with the average luminance value of the second and subsequent image data from each empty area of the memory, and compares the luminance values of both image data. The average value data is read, and the difference between the average luminance value of the first image data and the average luminance value of the second and subsequent image data is calculated (S11).

Next, the processor 104 determines whether or not the absolute value of the difference between the average luminance values is equal to or greater than the threshold value M (S.12). In the case of the first image data, S.I. 12, the difference is determined to be equal to or less than the threshold value M.
Here, the threshold value M is used to determine the overall brightness of the subject image. For example, assuming interval shooting at night, the overall brightness of the subject image increases as the sky becomes brighter as it approaches the dawn. This is because the luminance data in the composite image memory area α is replaced with the image data composed of the subject image, and the meaning of performing the comparatively bright combination processing is lost.

  The processor 104 is an S.I. 12, if “YES” is determined, the number N of times the difference between the average value of the luminance of the subject image obtained by the first shooting and the average value of the luminance of the subject image obtained after the second shooting exceeds the threshold M 'Is counted (S.13). The initial value of the number N ′ is set to “0” when interval shooting is started.

  Next, the processor 104 performs S.I. The process shifts to 14, and it is determined whether or not the number N ′ is equal to or greater than the set number P. The set number of times P is set when the threshold value M is set to a large value and only the threshold value M is used for determination. The luminance change of the subject image due to natural conditions such as cloud movement, artificial illumination reflected on the cloud, This is because it is not preferable to forcibly end the interval shooting in such a case without considering the luminance change of the subject image due to the instantaneous reflection of light from the vehicle headlight, flashlight, or the like.

  Therefore, the threshold value M is set to a value that slightly exceeds the size that does not interfere with the replacement process in the comparatively bright combination process, and the set number P includes subject images that exceed the threshold value M. Even when the comparatively bright combination process is performed, the number of times that can maintain the image quality as the combined image is set.

The threshold value M and the set number P are of a property that can be adjusted by the ISO sensitivity and the shooting mode, and may be set by the camera manufacturer or by the user.
The processor 104 is an S.I. 14, if the number N ′ is equal to or greater than the set number P, interval shooting is forcibly terminated. If the number N ′ is less than the set number P, step S. 19
It should be noted that the forcible photographing end of the interval photographing may be configured to be performed only when set by the user.

If the determination is “NO” in S.12, the processor 104 determines whether or not the user has set to save the image data acquired every interval shooting time interval (fixed time interval) t1. (S.15), and if the image data storage setting is set by the user, a process (interval image file output process) of outputting the image data of the subject image acquired at every fixed time interval t1 is executed (S .16), a process of saving in the SD card as the storage medium or the built-in memory 120 is performed. If this setting is not made by the user, the process of S.16 is skipped and the process proceeds to S.17.
Details of the interval image file output will be described later.

In S.17, a compositing process flag is set. Subsequently, the processor 104 proceeds to the comparative bright combination process S.18. Details of the flowchart of the comparatively bright combination process will also be described later.
When the comparatively bright combination process ends, the processor 104 determines whether or not the user has performed an interval shooting end operation (S.19).

  When the user performs an interval shooting end operation, the processor 104 ends the interval shooting process. When the user does not perform the interval shooting end operation, the processor 104 determines that the interval shooting time interval t1 has elapsed. (S.20), and when the interval shooting time interval t1 has elapsed, the process proceeds to S.4, and the processes of S.4 to S.20 are executed.

Hereinafter, a flowchart of comparatively bright combination processing will be described for each embodiment with reference to FIG.
(Basic example of comparative bright combination processing)
FIG. 9 is a flowchart of a basic example of the comparatively bright combination process shown in FIG.
First, the processor 104 determines whether or not the count value of the synthesis number counter is “0” (S.18 1).

  When the count value of the synthesis number counter is “0”, the processor 104 sets the image data in the updated image memory area β in the synthesized image memory area α as comparison target image data, that is, initial synthesized image data. Copy for use (S.18 2).

If the count value of the combination number counter is not “0”, the processor 104 determines, for each pixel of the current update image data and the composite image data among the update image data acquired at regular time intervals by the acquisition unit. Brightness comparison is executed during the acquisition, and a comparatively bright combination process is executed in which the luminance data stored in the pixels of the composite image memory area α is replaced with the higher luminance data (S. 183).

  Subsequently, the processor 104 counts the count value of the synthesis number counter by “+1”, and then clears the synthesis process flag (S.184, S.185), ends the comparatively bright synthesis process, and FIG. The process which transfers to S.19 shown is performed.

  Through this series of processing, the light passing through the photographing lens is received by the image sensor (CCD), and the brightness of each pixel is controlled during execution of acquisition control for every predetermined period of image data corresponding to one image of the subject image. This makes it possible to construct a composite image by comparison, which is convenient because the user does not have to carry back the digital camera and perform a series of comparatively bright composite processes of the acquired subject images later.

  Furthermore, according to this example, since it is possible to store the image data of the subject image acquired at regular time intervals on the recording medium as will be described later, even after taking the digital camera home, etc., the personal computer ( It is also possible to correct these image data by using (not shown).

(An example of interval image file output processing)
FIG. 10 is a flowchart showing an example of the interval image file output process in step 16 shown in FIG. In the interval image file output process, the processor 104 executes the interval image file output process in accordance with the setting conditions set in FIG.

  That is, when the setting is made to leave the normal still image, the processor 104 outputs the image data in the updated image memory area β as a still image file every time the subject image is acquired by interval shooting (S.162). When the composite image is set to remain as a still image, the image data in the composite image memory area α is output as a still image file each time an object is acquired by interval shooting (S.163), and a normal moving image is output. If the setting is left, the image data in the updated image memory area β is output as a moving image file every time the subject is acquired by interval shooting (S.164), and the setting is made to leave the composite image as a moving image. Image data in the composite image memory area α is output as a moving image file each time a subject is acquired by interval shooting. To (S.165).

  According to the first embodiment, the conversion step (see S.7 in FIG. 7) for converting the subject image into image data corresponding to one image, and the image data converted in this conversion step (S.7) The acquisition step (see S.4 to S.20 in FIG. 7) for acquiring at regular time intervals, and the update image memory area β for storing the image data as update image data whose luminance is compared for each pixel. After the image data for update is read out, the comparison target image data corresponding to one image whose luminance is compared for each pixel corresponding to each pixel of the updated image memory area β is stored and the luminance for each pixel is compared An image processing step for reading out the composite image from the composite image memory area α for storing the comparison target image data in which the pixel data is replaced with the luminance having the higher luminance as the composite image data (FIG. 7). S.1 to S.20) and the read update image data and the composition image data read out are compared for each pixel corresponding to each other, and are combined with the higher luminance after the comparison for each pixel. A replacement processing step (refer to S.18 in FIG. 7 and S.181 to S.185 in FIG. 9) for replacing the data of the pixel in the image memory area α is executed.

  The image processing step (see S.1 to S.20 in FIG. 7) is an initial image determination step for determining whether the update image data is initial image data constituting the first subject image (FIG. 7). 7) and the initial image determination step (see S.8 in FIG. 7), when it is determined that the image data is the initial image data, the average value of the luminance of the initial image data is obtained, and the initial image determination step (FIG. 7) (refer to S.8 of Fig. 7), when it is determined that the image is not an initial image, the average value of the luminance of the image data for update is obtained assuming that the image data for update is update image data constituting the second and subsequent subject images. The difference between the average values of both luminances obtained in the average value calculating step (see S.9 to S.11 in FIG. 7) and the average value calculating step (see S.9 to S.11 in FIG. 7) is the threshold value M. Judge whether or not By the determination result, and a changing step of changing the processing content of the replacement process step (S18) (S.12~S.14).

In the image processing step (see S.1 to S.20 in FIG. 7), when the difference is smaller than the threshold value M, a step of executing replacement processing using the update image data acquired this time, and the difference is less than the threshold value. If it is larger, a step of shifting to a replacement process using an update image acquired next time may be executed.
The changing step (S.12 to S.14) is a counting step S.12 that counts the number of times that the difference is exceeded. 13 and a determination step (see S.14 in FIG. 7) for determining whether or not the number of times has exceeded the set number P. When the determination step determines that the set number P has been exceeded, a replacement processing step ( The process of S.18 in FIG.

(Modification 1)
The processor 104 may be configured to add a function of displaying a composite image being composited.
FIG. 11 is a flowchart illustrating an example of a process for displaying a composite image in the middle of synthesis. This composite image display processing function is the same as step S. 4 to step S.1. In the middle of the process 16, it is possible to add a process for determining whether or not the release switch SW <b> 1 is half-pressed and a composite image display process.

  If the release switch SW1 is pressed halfway during the interval shooting process shown in FIG. 7, the processor 104 determines whether or not the compositing process flag is set, as shown in FIG. The process waits until the compositing flag is cleared (S.31). Next, in the case of NO, the processor 104 determines whether or not the count value of the synthesis number counter is “0” (S.32), and when the count value of the synthesis number counter is “0”, “+1”. Wait until it is counted up.

  If the count value of the combination number counter is not “0” in S32, the processor 104 determines that the S.I. Then, the process proceeds to 33, where a composite image composed of image data in the composite image memory area α is output to the LCD monitor 10 and display data for the composite number is output to the sub LCD 1. Then, the processor 104 determines whether or not the display time of the composite image and the display time of the composite number have elapsed (S.34).

When the display time has elapsed, the processor 104 performs display erasure processing of the composite image and the composite number, and returns to the interval shooting processing shown in FIG.
According to the first modification, the user can confirm the composite image in the middle of the interval shooting process, which is convenient.

(Modification 2)
The processor 104 may be configured to add a function that allows the user to arbitrarily initialize a composite image being composited.
FIG. 12 is a flowchart illustrating an example of processing for initializing the composite image memory area α. The initialization process of the composite image memory area α is also performed in step S. of FIG. 4 to step S.1. In the middle of the process of 16, the process for determining whether or not the self-timer / delete SW5 has been operated and the process for initializing the composite image memory area α can be added.

When the setting / deleting SW5 of the self-timer is operated, the processor 104 determines whether or not the compositing process flag is set as shown in FIG. 12, and if yes, the compositing process flag is cleared. (S.41). Next, in the case of NO, the processor 104 initializes the count value of the synthesis number counter to “0” (S.42), and then determines whether or not to output the synthesized image as a file when the synthesized image is reset. (S.43).
When the composite image before the composite image initialization is stored, the composite image data in the composite image memory area α is output as a file (S.44). Returning to the flow shown in FIG.

(Modification 3)
FIG. 13 shows an alternative to forcibly terminating the interval shooting process when the difference between the average brightness value of the first image data and the average brightness value of the second and subsequent image data is equal to or greater than the threshold value M. Then, the image data of the update image used to obtain the difference between the second and subsequent images is corrected (S.12 ′), and the comparatively bright combination is performed using the corrected update image data. Since the other processing is the same as the processing shown in FIG. 7, the detailed description of the flowchart will be omitted, but the image data correction processing will be described briefly.

In order to correct the brightness of the image data for update, for each pixel, a correction formula of a linear function relationship or a quadratic function relationship may be prepared and corrected, or a two-dimensional lookup table corresponding to each pixel is prepared. Then, the correction amount for each pixel may be determined in advance by experiments.
According to these embodiments and modifications, it is possible to construct an intended composite image that flexibly responds to a change in luminance when a subject image is acquired at regular time intervals.

Also, if the composite image is output as a video file, for example, if you want to leave a star light trail, you can obtain original data for easily creating a moving image in which the star light trail gradually extends Can do.
Users with advanced techniques tend to fine-tune still images before creating moving image data, but they can withstand that demand.

  Even a user who is not good at post-processing by a personal computer can obtain a moving image (a so-called slow-speed moving image) obtained by thinning out a video over a long period of time by photographing only with a digital camera.

(Description of Example 2 of Interval Shooting)
The second embodiment is devised to avoid a problem when a subject having extremely bright luminance is captured instantaneously during the comparatively bright combination process.

In the second embodiment, the processor 104 serving as an image processing unit has a function as a region dividing unit that divides the composite image memory region α and the updated image memory region β into a plurality of areas.
As shown in FIGS. 14A to 14D, the composite image memory area α and the update image memory area β are divided into a plurality of areas αi (i = 1 to m), βi by the area dividing unit. (I = 1 to m). A known technique can be used for the area dividing means.

The processor 104 uses the luminance value of the pixel data in each area αi (i = 1 to m) of the composite image memory area α to use the average luminance value Pαi ′ for each area and each area βi of the updated image memory area β. An average luminance value Pβi ′ for each area is calculated using the luminance value of the pixel data (i = 1 to m).
The average luminance values Pαi ′ and Pβi ′ for each area are obtained by dividing the total luminance value of each pixel in the area by the total number of pixels in the area.

Next, the processor 104 calculates whether or not the absolute value of the difference between the average luminance values (Pαi′−Pβi ′) exceeds the threshold value PM.
As shown in FIG. 14D, for example, when a subject image Q having a high brightness exists in the area βi (i = 4), the average brightness value Pβ4 ′ (the sum of the brightness β4j of the area β4 is calculated as the area βi (i = 4). (the value divided by the total number of pixels in β4) is greater than the average luminance value Pα4 ′ of the area α4 corresponding to this area β4 (the value obtained by dividing the total sum of luminance α4j by the total number of pixels in the area α4). It becomes relatively large.

  In this case, if the comparatively bright combination processing is executed for all the pixels as they are, the subject image Q is instantaneously reflected and becomes a combined image G2 (see FIG. 20B), and the combined image intended by the user cannot be obtained. The inconvenience arises.

  Therefore, in the second embodiment, when update image data with high brightness is acquired instantaneously, the image data in the area where the subject image Q with high brightness exists is not used for the comparative bright combination processing. It was decided.

  According to the second embodiment, the image data of each pixel in an area other than the area where the high-bright subject image Q exists is adopted, so that the obtained image data of the subject image can be used as effectively as possible. is there.

  The threshold value PM can be set for each area, and is set to, for example, 15% of the average luminance value Pαi ′ of the composite image created at the present time. For example, when the average luminance value Pαi ′ = 100 of the composite image created at the present time, the threshold value PM = 15.

  This threshold value PM can be changed by operating the MENU / OK SW (SW12) to call up the threshold selection screen as shown in FIG. 15 and operating the up / down buttons (SW7, SW9).

  When the threshold value PM is increased and set to about 30% and 50%, for example, particularly large instantaneous luminance changes are excluded, and when the threshold value PM is set to low and set to about 5% and 10%, for example, relatively small instantaneous luminance is set. Changes are excluded.

  When the threshold value PM is set to auto, the processor 104 obtains an average luminance value Pαi ′ in each area αi of the composite image memory area α by calculation, and if the average luminance value Pαi ′ in each area αi is relatively low, When the threshold value PM is set to a large value and the average luminance value Pαi ′ of each area αi is relatively high, a process for setting the threshold value PM to a small value is performed.

  FIG. 16 is a schematic diagram conceptually showing the magnitude of the threshold value PM, where the symbol X1 is the median of the lower average luminance values of each area αi, and the symbol X2 is the higher average of each area αi. This is the median value of the luminance values, and the arrow X3 indicates a range where the threshold value PM is automatically set higher, and the arrow X4 indicates the range where the threshold value PM is automatically set lower.

  As a result, in each of the areas αi and βi, the lower average luminance corresponds to insensitivity to the luminance change, and the higher average luminance corresponds to the luminance change sensitively. A more flexible response to changes is possible.

Hereinafter, the interval shooting according to the second embodiment will be described with reference to the flowchart shown in FIG. 17 and the flowchart shown in FIG.
Note that the flowchart shown in FIG. 17 is a part of the flowchart shown in FIG. 7 and FIG. 13 that has been reprinted for convenience of explanation. Of the steps shown in FIG. 7 and FIG. Only the essential steps are shown in the description of Example 2, and of course, the steps shown in FIGS. 7 and 13 can be executed simultaneously.
First, as shown in FIG. 17, the processor 104 clears the compositing processing flag (S.1 ′), and initializes the subject image composition count counter (S.2 ′).

  Next, the interval shooting time interval t1 set by the user is measured, and an exposure process is performed at each interval shooting time interval t1 (S.4 '). Next, the processor 104 determines whether or not the compositing process flag is set (S.5 ′). If NO, the processor 104 acquires image data corresponding to one subject image from the CCD (imaging device) 101. After executing (S.6 ′), after setting the compositing process flag (S.7 ′), the comparatively bright compositing process is executed (S.8 ′).

  In the comparatively bright combination process, the processor 104 first determines whether or not the count value of the combination number counter is “0” (S. 81 ′). When the count value of the combination number counter is “0”, the image data in the updated image memory area β is copied to the combined image memory area α as in the conventional case (S.82 ′).

  Next, the count value of the synthesis number counter is incremented by “+1” (S.84 ′), the synthesis processing flag is cleared (S.85 ′), 9 ', S.M. 10 ′, it is determined whether or not the interval shooting time interval t1 has elapsed. Return to 3 '.

  The processor 104 is an S.I. If the count value of the combination number counter is not “0” in 81 ′, S.E. In 86 ', it is determined whether or not the threshold value PM is set.

  S. 86 ', if NO, In 83 ′, a comparison of the brightness of each pixel of the current update image data and the composite image data among the update image data acquired by the acquisition means at regular time intervals is performed during the acquisition, and the composite image The comparatively bright combining process of replacing the luminance data stored in the pixels in the memory area α with the luminance data having the higher luminance is performed by performing all the pixels in the synthesized image memory area α and all the pixels in the updated image memory area β. Run as before. Thereafter, the processor 104 performs S.I. The processing after 84 'is executed.

  S. In 86 ′, in the case of YES, the processor 104 calculates an average luminance value Pαi ′ for each area using image data (also referred to as pixel data) of pixels in each area α1 to αm of the composite image memory area α. At the same time, using the image data (also referred to as pixel data) in each of the areas β1 to βm of the updated image memory area β, the average luminance value Pβi ′ is calculated for each area (S.87 ′).

  Next, the processor 104 calculates whether or not the absolute value of the average luminance value difference Pαi′−Pβi ′ exceeds the threshold value PM, and the absolute value of the average luminance value difference Pαi′−Pβi ′ is greater than the threshold value PM. Is also larger (S.88 ′).

  The processor 104 is an S.I. 88 ', if NO, Then, the process proceeds to 83 ', and the process proceeds to the conventional comparative bright combination process. The processor 104 is an S.I. 88 ', if YES, S.P. Move to 89 '.

The processor 104 is an S.I. In 89 ′, the comparatively bright combination process is executed using the image data of each pixel in the updated image memory area β excluding the pixel data of the area βi exceeding the threshold value PM. The processing after 84 ′ is executed.
With this processing, it is possible to perform comparatively bright combination processing excluding subject images with extremely bright luminance.

(Modification 4)
When tag information is attached to the image group G1 shown in FIG. 20 and recorded on a recording medium as a recording unit, as shown schematically in FIG. Even when the subject image Q having a high luminance in G2 is reflected in the area α4, for example, the image group G1 is read from the recording means, and the image data in which the subject image Q having the high luminance exists is identified from the series of image groups G1. Then, image data obtained continuously in the vicinity of the image data with high brightness is appropriately selected by the user by the image selection means, and the image data of the area β4 in which the subject image Q with high brightness is reflected, Substitution processing is performed with the pixel data of the area β4 of another image data in which the high luminance subject image Q is not reflected, and as shown in FIG. 19B, the high luminance subject image Q is not reflected. Together It can be fabricated image G2.

According to the fourth modification, even if the synthesized image is not finally intended, the user can appropriately correct the luminance data later for each area, and therefore, it becomes even easier to use.
In addition, it is good also as a structure which a user can set a threshold value for every area arbitrarily.

104. Processor (acquisition means, image processing means)
α: Memory area for composite image β: Memory area for updated image M: Threshold value

JP 2009-239600 A

Claims (12)

Imaging means for converting a subject image into image data;
Obtaining means for obtaining the image data from the imaging means at regular time intervals;
The image data is stored as update image data whose luminance is compared for each pixel, and comparison target image data whose luminance is compared for each pixel corresponding to the pixel of the update image data is stored and for each pixel A memory for storing, as synthesized image data, comparison target image data in which the pixel data is replaced with the higher luminance after the luminance comparison of
Of the update image data acquired by the acquisition means at regular time intervals, this update image data and the composite image data are compared for each pixel and replaced with the higher brightness data Processing means,
The image processing means obtains an average luminance value as initial image data when the update image data is the first image data, and when the update image data is the second and subsequent image data, An imaging apparatus characterized in that an average value of luminance as image data is obtained, and the content of the replacement process is changed depending on whether the difference between the average values of luminance of both image data is larger or smaller than a threshold value.   The image processing means executes the replacement process using the update image data acquired this time when the difference is smaller than the threshold value, and is acquired next time when the difference is larger than the threshold value. The imaging apparatus according to claim 1, wherein a replacement process using an update image is executed.   The image processing means includes counting means for counting the number of times the difference exceeds the threshold value, and stops the replacement processing when a count value of the counting means exceeds a set number of times. Item 3. The imaging device according to Item 2.   When the difference is smaller than the threshold, the image processing means executes the replacement process using the update image data acquired this time, and when the difference is larger than the threshold, the update acquired this time The imaging apparatus according to claim 1, wherein after performing the replacement process by correcting the luminance value of the image data for replacement, the replacement process using the update image acquired next time is performed. The image processing means includes area dividing means for dividing a composite image memory area for storing the composite image data and an update image memory area for storing the update image data into a plurality of areas.
The image processing means obtains an average luminance value for each area in the composite image memory area and an average luminance value for each area in the updated image memory area, and also calculates a luminance value for each corresponding area. 5. The content of the replacement process is changed by obtaining a difference between the average values and determining whether or not the difference between the average values of the luminances exceeds a threshold value. 6. The imaging apparatus of Claim 1.   The imaging apparatus according to claim 5, wherein the threshold value for each area can be set for each area divided by the area dividing unit.   The threshold value for each area is obtained by calculating the average luminance value for each area using the luminance value in each area of the composite image memory area, and when the average luminance value is large, the threshold value is set small. The imaging apparatus according to claim 6, wherein when the average value is small, the threshold value is set large.   6. The image processing unit according to claim 5, wherein the captured image data is recorded in a recording unit, and replacement processing for each area can be executed using the image data read from the recording unit. Imaging device.   Image update means for selecting arbitrary update image data from the image data recorded in the recording means, and an area of the update image memory area is selected using the update image data selected by the image selection means. The image data in the area corresponding to the selected area of the composite image memory area is then replaced with the image data of the area of the updated image memory area. Imaging device. A conversion step of converting the subject image into image data;
An acquisition step of acquiring the image data converted by the conversion step at regular time intervals;
The image data is compared with the pixel of update image data whose luminance is compared for each pixel, and comparison target image data whose luminance is compared for each pixel corresponding to each other is stored, and the luminance is larger after the luminance comparison for each pixel An image processing step of reading out the composite image data from a memory that stores, as composite image data, comparison target image data in which the pixel data has been replaced with the brightness of
A replacement processing step of comparing the image data for update this time and the composite image data with each other corresponding to each other and replacing with the data having the larger luminance after the comparison for each pixel,
The replacement processing step includes an initial image determination step of determining whether or not the update image data is the first image data;
The average value of the luminance of the initial image data when the updated image data is determined to be image data of the first sheet by said initial image determination step, for the updating by the initial image determination step When it is determined that the image data is not the first image data, an average value calculating step for obtaining an average value of luminance as the update image data assuming that the update image data is image data after the second image When,
A change step of determining whether or not the difference between the average values of the two luminances obtained in the average value calculation step is larger than a threshold value, and changing the processing content of the replacement processing step according to the determination result;
An imaging method comprising:   The image processing step includes a step of executing a replacement process using the update image data acquired this time when the difference is smaller than the threshold value, and a next time when the difference is larger than the threshold value. The imaging method according to claim 10, further comprising: performing a replacement process using an update image.   The changing step includes a counting step that counts the number of times that the difference is exceeded, and a determining step that determines whether or not the number of times exceeds the set number of times, and when the determining step determines that the number of times has been exceeded, The imaging method according to claim 10, wherein the processing of the replacement processing step is stopped.