JP2006332934A - Electronic camera - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an electronic camera which can stop photographing in an appropriate timing when images formed by multiple photographing are synthesized and a synthesized image is formed, and can form the synthesized image even when multiple photographing are stopped on the way. <P>SOLUTION: The electronic camera comprises a determinator for determining image pick-up conditions, an image pick-up for photographing repeatedly an object image to form images of a plurality of frames according to the image pick-up conditions determined by the determinator, a judgment for judging stoppage of photographing by the image pick-up based on user's direction or an image information on the images of the plurality of frames formed by the image pick-up, a control unit for stopping photographing by the image pick-up based on either judgment by the judgment, and a synthesizer for synthesizing the images of the plurality of frames formed by the image pick-up to form a synthesized image. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an electronic camera that generates a composite image by combining images generated by a plurality of imaging operations.

Conventionally, when an image is captured by a camera, an image of the subject on the imaging surface may be blurred due to camera shake of the user or subject blur caused by movement of the subject. When the subject image is blurred, the blur of the subject image is recorded as a locus. Such a problem is more likely to occur as the exposure time becomes longer. Thus, a technique for correcting the above-described image blur by detecting a main subject in a plurality of images generated by a plurality of times of imaging, performing alignment, and generating a combined image (for example, is considered) , See Patent Document 1). Such a technique can be expected to reduce random noise.
Japanese Patent Laid-Open No. 2004-208024

However, in the above-described technique, since the image suitable for composition reaches the specified number of frames repeatedly, it is difficult for the user to understand the time until the end of the image capture, and the subject moves greatly on the way. Even if the user determines that it is impossible to combine frames, it cannot be handled.
When the electronic camera of the present invention generates a composite image by synthesizing images generated by multiple times of imaging, the imaging can be stopped at an appropriate timing, and when multiple times of imaging are stopped in the middle However, it is intended to make it possible to generate a composite image.

  An electronic camera according to the present invention is based on a determination unit that determines an imaging condition, an imaging unit that repeatedly captures a subject image and generates a plurality of frames according to the imaging condition determined by the determination unit, and a user instruction Determining whether to stop imaging by the imaging unit, and determining whether to stop imaging by the imaging unit based on image information of the images of the plurality of frames generated by the imaging unit, A control unit that stops the imaging by the imaging unit, and a synthesis unit that synthesizes the images of the plurality of frames generated by the imaging unit to generate a synthesized image.

Preferably, when the determination unit determines to stop imaging by the imaging unit, the control unit stops imaging by the imaging unit even during imaging of an arbitrary frame, or the current imaging You may make it stop imaging by the said imaging part, after the imaging of the inside flame | frame is complete | finished.
Preferably, the synthesizing unit synthesizes images of a plurality of frames excluding an image obtained when the imaging is stopped during the imaging of the frame among the images of the plurality of frames generated by the imaging unit, or You may make it synthesize | combine the image of all the frames produced | generated by the imaging part.

  Preferably, the determining unit independently determines an exposure time for each of a plurality of frames when the subject image is repeatedly captured by the imaging unit, and the combining unit is configured to combine the images of the plurality of frames. In addition, each of the images of the plurality of frames may be corrected based on a ratio between a target exposure amount based on the exposure time determined independently for each of the plurality of frames and an exposure amount of the image of each frame. .

Preferably, the determining unit determines an exposure time when the imaging unit repeatedly captures a subject image, and the combining unit combines the images of the plurality of frames and then performs target exposure based on the exposure time. The composite image may be corrected based on the ratio between the amount and the sum of the exposure amounts of the images in each frame.
Preferably, an operation unit that receives a user instruction is provided, and the determination unit may determine whether to stop imaging by the imaging unit based on the user instruction from the operation unit.

  Preferably, when the determination unit determines to stop imaging by the imaging unit, the control unit stops imaging by the imaging unit and discards an image of a frame generated until the imaging is stopped. May be.

  According to the electronic camera of the present invention, when a composite image is generated by synthesizing images generated by a plurality of times of imaging, the imaging can be stopped at an appropriate timing, and a plurality of times of imaging is stopped halfway. Even in this case, a composite image can be generated.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the present embodiment, a description will be given using a compact camera as an example of the electronic camera of the present invention. The technique of the present invention may be applied to a single-lens reflex digital camera, an electronic camera equipped with a moving image shooting function, and the like.
FIG. 1 is a functional block diagram of the electronic camera 100 of the present embodiment. As shown in FIG. 1, an electronic camera 100 includes an CCD 1 that is an image sensor that converts a subject image into an electrical signal, a photographing lens 2 that forms the subject image on the image sensor surface of the CCD 1, and an analog output from the CCD 1. In order to make a signal suitable for A / D conversion, a signal processing circuit A3 that adjusts the amplitude and offset of the signal, and an A / D conversion unit that converts an analog signal output from the signal processing circuit A3 into a digital signal 4. A signal processing circuit B5 that performs white balance adjustment, gradation characteristic processing, filter processing, color adjustment processing, resolution conversion processing, resizing, and the like on the image data digitized by the A / D conversion unit 4. TG6 for generating a signal for controlling the operation of the device and the timing of signal processing. The electronic camera 100 also displays a buffer / SDRAM 7 that is a memory for temporarily storing image data, an SDRAM control unit 8 that controls the buffer / SDRAM 7, shooting information, captured images, menus for various settings, and the like. The monitor 9, the display control circuit 10 for controlling the display information on the monitor 9, and the images of a plurality of frames held in the buffer / SDRAM 7 are compared to detect a shift, and the shift is combined to cancel the detected shift. A motion detection / motion compensation circuit 11 that performs processing and a bus switching circuit 12 that controls the bus are provided. The electronic camera 100 includes a gyro sensor 13 that detects the movement of the electronic camera 100, a CPU 14 that controls each unit and references data, a timer 15 that measures the time when each frame to be described later is actually captured, A shutter button 16 that is a switch for controlling a release that is a trigger for the camera, and an operation member for switching between normal shooting and shooting for shooting and combining a plurality of frames, and for shifting to a mode for performing various camera settings. A mode switching button 17 is provided. The electronic camera 100 also includes a program memory 18 that stores processing procedures of the CPU 14, an I / F unit 19 that mediates with a recording medium such as a memory card, a recording medium 20 that includes a micro drive, a memory card, and the like.

  The output of the CCD 1 is connected to the signal processing circuit A3, and the output of the signal processing circuit A3 is connected to the A / D conversion unit 4. The output of the A / D converter 4 is connected to the signal processing circuit B5, and the output of the signal processing circuit B5 is connected to the bus. The output of the TG 6 is connected to the CCD 1, the signal processing circuit A3, and the A / D conversion unit 4, and these units operate in synchronization with the signal from the TG 6. The buffer / SDRAM 7, SDRAM control unit 8, motion detection / motion compensation circuit 11, CPU 14, program memory 18, and I / F unit 19 are interconnected with the bus, and the bus is switched appropriately by the bus switching circuit 12. It is done. The buffer / SDRAM 7 and the SDRAM control unit 8 are connected to each other. Further, the output of the bus is connected to the display control circuit 10, and the output of the display control circuit 10 is connected to the monitor 9. The I / F unit 19 is connected to the recording medium 20. The CPU 14 detects the states of the gyro sensor 13, the shutter button 16, and the mode switching button 17 and is connected to the timer 15. The CPU 14 controls each unit according to the processing procedure stored in the program memory 16.

  When the shutter button 16 is pressed by the user, the CPU 14 detects this, determines the imaging conditions such as the number of imaging, the exposure time, the shutter speed, and starts the accumulation of electric charges by the CCD 1. The charge accumulated by the CCD 1 is read out in synchronization with the signal output by the TG 6 and is subjected to the above processing in the signal processing circuit A 3, A / D conversion unit 4, and signal processing circuit B 5, and then the buffer / It is held in the SDRAM 7.

  Hereinafter, the operation of the electronic camera 100 during execution of the synthesis mode, which is a feature of the present invention, will be described with reference to the flowchart of FIG. The synthesis mode is a shooting mode in which images generated by multiple times of imaging are combined to generate a combined image, and the selection of the combining mode is performed by a user operation via the mode switching button 17. The mode may be selected by another method (such as automatic selection of a shooting mode by a camera).

In step S <b> 1, the CPU 14 determines whether the start of imaging has been instructed by a user operation via the shutter button 16.
When the start of imaging is instructed, in step S2, the CPU 14 determines the number N of imaging times and imaging conditions. The CPU 14 determines imaging conditions such as a shutter speed based on photometric data, image information obtained by the CCD 1, and the like. Then, based on the shutter speed, the number N of times of photographing that hardly causes camera shake and that makes noise less noticeable is determined. Note that the determination of the imaging condition and the number N of imaging is performed in the same manner as in the known technique, and thus detailed description thereof is omitted. Furthermore, the CPU 14 determines the exposure time independently for each of N times of imaging. Here, as shown in FIG. 3, the following description will be given assuming that the exposure times of N = 6 and six imaging operations are determined as tB1, tB2,... TB6. As shown in FIG. 3, the exposure times tB1, tB2,... TB6 are not always uniform. This is because the respective exposure times are determined according to the camera shake characteristics determined empirically. Further, after the start of imaging, the respective exposure times may be finely adjusted according to the output of the gyro sensor 13.

In step S3, the CPU 14 initializes a counter (not shown) in the CPU 14, and sets the count value n to n = 1.
In step S4, the CPU 14 controls each unit according to the imaging conditions determined in step S2, and performs n-th imaging. It should be noted that adjustment values determined at the time of starting imaging, such as white balance, may be performed only during the first imaging.

In step S <b> 5, the CPU 14 holds the generated image in the buffer / SDRAM 7. At this time, the actually measured value (tAn) of the exposure time at the n-th imaging measured by the timer 15 is also recorded.
In step S <b> 6, the CPU 14 determines whether or not (current counter number n) = (shooting number N determined in step S <b> 2). If (current counter number n) = (number of photographing times N determined in step S2), the process proceeds to step S8, and (current counter number n) does not reach (number of photographing number N determined in step S2). If so, the process proceeds to step S7.

In step S7, the CPU 14 sets n = n + 1, returns to step S4, and repeats the processing from step S4 to step S6 until n = N.
When (current counter number n) = (the number of photographing times N determined in step S 2), in step S 8, the CPU 14 reads the N frame image held in the buffer / SDRAM 7, and the motion detection / motion compensation circuit 11. And performing motion detection, the N frame images are combined to generate a combined image.

  First, the CPU 14 reads an image of N frames held in the buffer / SDRAM 7, and based on the ratio between the exposure time determined independently for each frame in step S2 and the actual value of the exposure time at the time of image capturing of each frame. Thus, the image data of each frame image is corrected. That is, the image of frame n is corrected by multiplying by tBn / tAn. For example, the image of frame 2 is corrected by multiplying by tB2 / tA2. By such correction, even when an error occurs between the exposure time independently determined for each frame in step S2 and the exposure time at the time of image capturing, appropriate correction can be performed. Then, among the corrected N frame images, for example, alignment is performed using the frame image generated by the first imaging as a reference image. Since the alignment is performed in the same manner as a known technique such as a block matching method or a correlation gradient method, detailed description is omitted. Then, after alignment, N frames of images are added and combined to generate a combined image.

In step S9, the CPU 14 records the composite image generated in step S8 on the recording medium 20 via the I / F unit 19, and ends the series of processes.
Next, the operation of the electronic camera 100 when an instruction to stop imaging is given during execution of the synthesis mode, which is a feature of the present invention, will be described with reference to the flowchart of FIG. In the present embodiment, the user stops the imaging by pressing the shutter button 16 again after starting the imaging in the synthesis mode. The instruction to stop the imaging may be performed by releasing the shutter button 16 or may be performed via another operation member of the electronic camera 100. Further, it may be possible to automatically determine whether to stop imaging according to the output of the gyro sensor 13, or to perform subject recognition on the image, and automatically determine whether to stop imaging according to the result. You may make it do. For example, if it is configured to automatically determine whether to stop imaging according to the output of the gyro sensor 13, the user can instruct to stop imaging by intentionally shifting the framing. Further, even when the electronic camera 100 falls, imaging can be stopped. When an instruction to stop imaging is given, the CPU 14 performs the following process as an interrupt process. Hereinafter, a case where an instruction to stop imaging is performed in the middle of the frame 4 will be described as an example.

When the stop of imaging is instructed during the execution of the synthesis mode described in the flowchart of FIG. 2, the CPU 14 detects this and records the timer value and the frame number currently being captured in step S21. The frame number indicates which frame the image capturing is instructed to stop, and the timer value indicates at which position (timing) in the frame the cancellation is instructed.
In step S22, the CPU 14 confirms with the user whether or not to stop imaging. The CPU 14 displays a message on the monitor 9 via the display control circuit 10 and asks the user whether or not to stop imaging via an audio output unit (not shown). By making such an inquiry, it is possible to avoid inadvertently stopping imaging due to an erroneous operation by the user or an erroneous determination by the electronic camera 100. Then, when the user instructs not to stop imaging, the CPU 14 ends the interrupt process and continues the operation of the flowchart of FIG.

On the other hand, when the user instructs to stop the imaging, in step S23, the CPU 14 holds the image of the frame generated so far in the buffer / SDRAM 7 and stops the imaging.
Here, a first example and a second example of the stop timing will be described with reference to FIGS. 5 and 6. As shown in FIGS. 5 and 6, when an imaging stop instruction is issued in the middle of the frame 4 (during the fourth imaging), in the first example, as shown in FIG. The imaging is stopped after the imaging of the frame 4) is completed. As a result, images from frame 1 to frame 4 can be generated. On the other hand, in the second example, as shown in FIG. 6, the imaging is stopped immediately after the imaging stop instruction even during the imaging. As a result, the image of the frame 4 cannot be completely generated, but the imaging can be quickly stopped in accordance with the imaging stop instruction.

  In step S24, the CPU 14 confirms with the user whether or not to discard the image held in the buffer / SDRAM 7. The CPU 14 displays a message on the monitor 9 via the display control circuit 10 and asks the user whether or not to discard the image held in the buffer / SDRAM 7. By asking such a question, it is possible to prompt the user to quickly prepare for the next shooting or to generate a composite image using an image generated until the shooting is stopped.

If the user instructs to discard the image, in step S25, the CPU 14 discards the image held in the buffer / SDRAM 7 to end the interrupt process, and the operation of the flowchart of FIG. Also ends.
On the other hand, if the user instructs not to discard the image, in step S26, the CPU 14 synthesizes the image according to the timer value and the frame number.

  The image composition is performed in the same manner as step S8 in the flowchart of FIG. However, here, since the imaging is stopped in a state where the number N of times of imaging determined in step S2 in FIG. 2 has not been reached, it is necessary to compensate for the shortage. Therefore, the CPU 14 first reads out the four-frame images held in the buffer / SDRAM 7 in the same manner as step S8 in the flowchart of FIG. 2, and sets the exposure time independently determined for each frame in step S2 and the image of each frame. Each image data of the image of each frame is corrected based on the ratio of the exposure time at the time of imaging with the actual measurement value. By this processing, for example, as shown in the example of FIG. 6, even when imaging is in progress, it is possible to compensate for an image when imaging is stopped immediately after an imaging cancellation instruction. For example, in the example of FIG. 6, the image of frame 4 is corrected by multiplying by tB4 / tA4. The actual measurement value tA4 of the exposure time is shorter than the exposure time tB4 determined in step S2 of the flowchart of FIG. 2, but the image can be compensated by multiplying this ratio. Then, the corrected four-frame image is aligned as in step S8 in the flowchart of FIG. 2, and is synthesized by adding the four-frame images to generate a composite image.

  At this point, the composite image is a composite of four frame images. Therefore, as described with reference to the flowchart of FIG. 2 and FIG. 3, in order to obtain an image of the same level as that obtained by synthesizing images of 6 frames, it is necessary to compensate for the shortage. Therefore, the CPU 14 corrects the composite image based on the ratio between the exposure time determined in step S2 in the flowchart of FIG. 2 and the actual measurement value of the exposure time at the time of capturing an image of each frame. That is, correction is performed by multiplying by (tB1 + tB2 + tB3 + tB4 + tB5 + tB6) / (tA1 + tA2 + tA3 + tA4). The total of the actually measured exposure times (tA1 + tA2 + tA3 + tA4) is shorter than the total exposure time (tB1 + tB2 + tB3 + tB4 + tB5 + tB6) determined in step S2 in the flowchart of FIG. 2, but the image can be compensated by multiplying this ratio. Hereinafter, the compensated image is referred to as a final image.

Note that (tB1 + tB2 + tB3 + tB4 + tB5 + tB6) / (tB1 + tB2 + tB3 + tB4) may be used instead of (tB1 + tB2 + tB3 + tB4 + tB5 + tB6) / (tA1 + tA2 + tA3 + tA4).
Further, as shown in the example of FIG. 6, even in the middle of imaging, if there is a problem in terms of reliability or the like when the imaging is stopped immediately after the imaging cancellation instruction, the image In the example of FIG. 6, the images from frame 1 to frame 3 may be combined. Further, the user may select an image to be synthesized.

As described above, when composition is performed in accordance with the timing of the imaging stop instruction, in step S27, the CPU 14 records the final image generated in step S26 on the recording medium 20 via the I / F unit 19, and a series of processes. Exit.
As described above, according to the present embodiment, in accordance with the determined imaging conditions, the subject image is repeatedly captured to generate a plurality of frames, the stop of the imaging is determined based on the user instruction, and the imaging is performed. Based on the image information of the plurality of frames of images generated by the above, it is determined whether to stop imaging. Based on one of the determinations, the imaging is stopped and a composite image is generated by synthesizing a plurality of frames of images generated by the imaging. Therefore, when a composite image is generated by combining images generated by a plurality of imaging operations, it is possible to stop imaging at an appropriate timing.

Further, according to the present embodiment, when it is determined that the imaging is to be stopped, the imaging is stopped even during the imaging of an arbitrary frame, or the imaging is stopped after the imaging of the frame currently being captured is completed. . Therefore, it is possible to immediately stop imaging after determining to stop imaging. Therefore, it is possible to quickly prepare for the next shooting.
Further, according to the present embodiment, among the multiple frame images generated by the imaging, the multiple frame images except for the image when the imaging is stopped in the middle of the imaging of the frame are synthesized, or all the generated by the imaging are performed. Synthesize the frame image. Therefore, it is possible to generate a composite image even when a plurality of times of imaging are stopped in the middle.

  In addition, according to the present embodiment, the exposure time is determined independently for each of a plurality of frames when the subject image is repeatedly captured by imaging, and is determined independently for each of the plurality of frames before combining the images of the plurality of frames. Each of the images of the plurality of frames is corrected based on the ratio between the target exposure amount based on the exposure time and the exposure amount of the image of each frame. Therefore, since the composition can be performed after correcting the image to be synthesized to an appropriate exposure amount, the quality of the synthesized image can be improved.

In addition, according to the present embodiment, after determining the exposure time when the subject image is repeatedly captured by imaging and combining the images of a plurality of frames, the target exposure amount based on the exposure time and the exposure amount of the image of each frame The composite image is corrected based on the ratio to the sum of the two. Therefore, since the composite image can be corrected to an appropriate exposure amount, the quality of the composite image can be improved.
In addition, according to the present embodiment, an operation unit that receives a user instruction is provided, and it is determined to stop imaging based on the user instruction by the operation unit. Therefore, imaging can be stopped at a timing desired by the user.

Further, according to the present embodiment, it is determined to stop imaging, the imaging is stopped, and the images of the frames generated until the imaging is stopped are discarded. Therefore, it is possible to save the recording area and quickly prepare for the next shooting.
In the present embodiment, an example is shown in which correction is performed before and after image synthesis, but only one of them may be performed. Further, before image synthesis, correction may be performed in consideration of the total exposure time determined in step S2 in the flowchart of FIG. Moreover, it may replace with the correction | amendment which multiplies ratio, and you may make it correct | amend based on luminance information, and may consider luminance information when multiplying a ratio. In addition, correction may be performed by adding an error of shutter accuracy at the time of correction.

  Further, the present invention may be applied to an electronic camera that repeatedly captures images until an image suitable for synthesis reaches a specified number of frames. In the present invention, since it is possible to stop imaging at an appropriate timing, when the time until the end of imaging is difficult to understand or the subject moves greatly in the middle, it is impossible to combine in subsequent frames. Even if it is judged, it is possible to respond promptly.

It is a functional block diagram which shows the structure of the electronic camera 100 of this embodiment. It is a flowchart which shows operation | movement of the electronic camera 100 of this embodiment. It is a figure explaining composition of an image. It is a flowchart which shows operation | movement of the electronic camera 100 of this embodiment. It is a figure explaining composition of an image. It is a figure explaining composition of an image.

Explanation of symbols

1, CCD 11, motion detection / compensation circuit 14, CPU 15, timer 16, shutter button

Claims (7)

A determination unit for determining imaging conditions;
In accordance with the imaging conditions determined by the determination unit, an imaging unit that repeatedly captures a subject image and generates a plurality of frames of images,
A determination unit that determines whether to stop imaging by the imaging unit based on a user instruction, and determines whether to stop imaging by the imaging unit based on image information of the images of the plurality of frames generated by the imaging unit;
A control unit that stops imaging by the imaging unit based on any determination by the determination unit;
An electronic camera comprising: a combining unit that combines the plurality of frames of images generated by the imaging unit to generate a combined image. The electronic camera according to claim 1,
When the determination unit determines that the imaging unit stops the imaging, the control unit stops the imaging by the imaging unit even during the imaging of an arbitrary frame, or the imaging of the frame currently being captured An electronic camera characterized in that imaging by the imaging unit is stopped after the operation is completed. The electronic camera according to claim 1,
The synthesizing unit synthesizes a plurality of frame images excluding an image obtained when the imaging is stopped during the imaging of the frame among the plurality of frame images generated by the imaging unit, or is generated by the imaging unit. An electronic camera that combines images of all frames. The electronic camera according to claim 1,
The determining unit independently determines an exposure time for each of a plurality of frames when the subject image is repeatedly captured by the imaging unit,
The combining unit, based on the ratio of the target exposure amount based on the exposure time determined independently for each of the plurality of frames and the exposure amount of the image of each frame before combining the images of the plurality of frames, An electronic camera characterized by correcting each of multiple frame images. The electronic camera according to claim 1,
The determining unit determines an exposure time when the imaging unit repeatedly captures a subject image;
The synthesizing unit, after synthesizing the images of the plurality of frames, corrects the synthesized image based on a ratio between a target exposure amount based on the exposure time and a sum of exposure amounts of the images of the respective frames. And an electronic camera. The electronic camera according to claim 1,
It has an operation unit that accepts user instructions,
The electronic camera according to claim 1, wherein the determination unit determines whether to stop imaging by the imaging unit based on the user instruction from the operation unit. The electronic camera according to claim 1,
When the determination unit determines to stop imaging by the imaging unit, the control unit stops imaging by the imaging unit and discards an image of a frame generated until the imaging is stopped. .

Images