JP2015041879A - Imaging apparatus, imaging system, control method of imaging apparatus, program, and storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus capable of displaying a recording image and a middle image with high quality, by including image processing means for displaying and recording an image, obtained by interval shooting, while synthesizing.SOLUTION: An imaging apparatus has imaging elements for capturing an image, storage means for storing an image, shooting control means for shooting a plurality of images sequentially by using the imaging elements, synthesizing means for sequentially synthesizing the images captured sequentially by the shooting control means, image processing means for correcting the composite images generated sequentially by the synthesizing means based on the optical black value of the composite image, display control means for controlling display means to display the first composite images generated sequentially by the synthesizing means, and subjected to correction by the image processing means, while the shooting control means is capturing a plurality of images, and storage control means for storing the second composite images, produced by synthesizing the plurality of images, in the storage means.

Description

  The present invention relates to an imaging apparatus capable of interval shooting.

  2. Description of the Related Art In recent years, there has been known an imaging apparatus that generates a plurality of images by a series of imaging operations and performs comparatively bright combination processing on the generated images. The comparatively bright combining process is a process for generating a combined image using only bright pixels with respect to a reference image by comparing a plurality of images acquired by continuous shooting, for example.

  Patent Document 1 discloses an imaging apparatus that performs comparatively bright combining processing during interval shooting and displays an image being combined. Patent Document 2 discloses a configuration in which, when a plurality of composite images are generated, information related to the progress of generation is displayed as a progress bar on a display unit.

JP2013-62740A JP 2011-223292 A

  However, in the digital still camera of Patent Document 1, if interval shooting is continued for a long time, the temperature of the image sensor rises. As a result, there is a phenomenon in which a composite image displayed on the LCD monitor gradually becomes brighter (black floating), and it may not be possible to confirm minute lights such as fireflies and stars.

  Therefore, the present invention provides an imaging apparatus that includes an image processing apparatus that performs display processing and recording by performing synthesis processing on an image obtained by interval shooting, and can display a recorded image and an intermediate image with high quality. In addition, such an imaging system, an imaging device control method, a program, and a storage medium are provided.

  An imaging apparatus according to an aspect of the present invention includes an imaging device that captures an image, a storage unit that stores the image, a capturing control unit that sequentially captures a plurality of images using the imaging device, and the capturing control unit. Combining means for sequentially combining the images sequentially captured by the image processing means, and image processing means for performing correction processing based on the optical black value of the combined image for the combined images sequentially generated by the combining means; A display control means for displaying on the display means a first composite image after the correction processing by the image processing means, which is sequentially generated by the combining means during the photographing of the plurality of images by the photographing control means; Storage control means for storing in the storage means a second composite image obtained by combining the images.

  An imaging apparatus according to another aspect of the present invention includes an imaging element that captures an image, a storage unit that stores the image, an imaging control unit that sequentially captures a plurality of images using the imaging element, and the imaging control. An image processing unit that performs blacking processing for subtracting, for each image that is sequentially captured by the imaging control unit, a first image that is captured in a state where the image sensor is shielded before the plurality of images are captured by the unit. And a synthesis unit that sequentially performs synthesis processing on the images that are sequentially captured by the imaging control unit, and a first synthesis that is sequentially generated by the synthesis unit during imaging of the plurality of images by the imaging control unit. Display control means for displaying an image on the display means, and storage control means for storing in the storage means a second synthesized image obtained by synthesizing the plurality of images.

  An imaging system as another aspect of the present invention includes a lens device including an imaging optical system and the imaging device.

  According to another aspect of the present invention, there is provided a control method for an image pickup apparatus, a shooting step for sequentially shooting a plurality of images using an image sensor, and a composition for sequentially combining images captured in the shooting step. An image processing step for performing a correction process based on an optical black value of the composite image with respect to the composite image sequentially generated by the composite step; and during the photographing of the plurality of images by the photographing step A display control step for displaying on the display means the first composite image after the correction processing in the image processing step sequentially generated in the step, and a second composite image in which the plurality of images are combined is stored in the storage means. Storage step.

  According to another aspect of the present invention, there is provided a method for controlling an imaging apparatus, wherein a shooting step of sequentially shooting a plurality of images using an imaging device, and the imaging device is shielded from light before the shooting of the plurality of images in the shooting step. An image processing step for performing blacking processing for subtracting the first image captured in the captured state for each image sequentially captured in the capturing step, and sequentially for the images sequentially captured in the capturing step A composition step for performing composition processing; a display control step for displaying on the display means a first composite image that is sequentially generated in the composition step during photographing of the plurality of images in the photographing step; and the plurality of images Storing a second synthesized image synthesized.

  A program according to another aspect of the present invention is a program that can be executed by a computer in which a procedure of a control method of the imaging apparatus is described.

  A storage medium according to another aspect of the present invention is a computer-readable storage medium storing a program for causing a computer to execute each step of the method for controlling the imaging apparatus.

  Other objects and features of the present invention are illustrated in the following examples.

  According to the present invention, it is possible to provide an imaging apparatus that includes an image processing unit that performs display processing and recording by performing synthesis processing on an image obtained by interval shooting, and can display a recorded image and a halfway image with high quality. . In addition, such an imaging system, an imaging device control method, a program, and a storage medium can be provided.

It is a block diagram which shows the structure of the imaging device in a present Example. 6 is a flowchart illustrating a control method (image processing method) of the imaging apparatus according to the present exemplary embodiment. It is a flowchart which shows the interval imaging | photography by the imaging device in a present Example. It is a flowchart which shows the comparative bright combination process by the imaging device in a present Example.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  First, with reference to FIG. 1, the structure of the imaging device (digital camera) in a present Example is demonstrated. FIG. 1 is a block diagram illustrating the configuration of the imaging apparatus according to the present embodiment.

  In the imaging apparatus 100, 10 is a photographing lens (imaging optical system), 12 is a mechanical shutter having a diaphragm function, and 14 is an imaging element that photoelectrically converts a subject image (optical image) and outputs an electrical signal (analog signal). is there. The image sensor 14 captures an image. Reference numeral 16 denotes an A / D converter that converts an analog signal output from the image sensor 14 into a digital signal. Reference numeral 18 denotes a timing generation circuit that supplies a clock signal and a control signal to the image sensor 14, the A / D converter 16, and the D / A converter 26, and is controlled by the memory control circuit 22 and the system control circuit 50. . In this embodiment, the storage time can be controlled by using an electronic shutter that controls the reset timing of the image sensor 14 instead of the mechanical shutter 12 and can be used for moving image shooting and the like.

  Reference numeral 20 denotes an image processing circuit, which performs pixel interpolation processing, color conversion processing, noise removal processing, and edge enhancement processing for data enlargement / reduction with respect to data from the A / D converter 16 or data from the memory control circuit 22. And so on. The image processing circuit 20 performs wavelet transform processing for face detection on the image data from the memory control circuit 22. The image processing circuit 20 performs predetermined arithmetic processing using the captured image (image data) in order to perform AWB (auto white balance) processing. Then, the obtained calculation result is calculated as a WB (white balance) evaluation value, and color conversion of image data is performed based on the calculated WB evaluation value. Further, the image processing circuit 20 uses a captured image (image data) to calculate an AF evaluation value, an AE evaluation value, and an EF evaluation value for performing AF processing, AE processing, and EF processing, respectively. Predetermined arithmetic processing is performed. The system control circuit 50 controls the exposure control unit 40 and the distance measurement control unit 42 according to a predetermined algorithm based on the obtained AF evaluation value, AE evaluation value, and EF evaluation value.

  A memory control circuit 22 includes an A / D converter 16, a timing generation circuit 18, an image processing circuit 20, an image display memory 24, a D / A converter 26, a memory 30 (storage means), and a compression / decompression circuit. 32 is controlled. Data (image signal) output from the A / D converter 16 is stored in the image display memory 24 or the memory 30 via the image processing circuit 20 and the memory control circuit 22 or directly via the memory control circuit 22. Is done.

  Reference numeral 24 denotes an image display memory, 26 denotes a D / A converter, and 28 denotes an image display unit including a TFT-LCD. The display image data written in the image display memory 24 is displayed on the image display unit 28 (display means) via the D / A converter 26. An electronic viewfinder function can be realized by sequentially displaying captured images (image data) using the image display unit 28. The image display unit 28 can arbitrarily turn on / off the display based on an instruction from the system control circuit 50. When the display is turned off, the image display unit 28 can reduce the power consumption of the imaging device 100 (image processing device).

  Reference numeral 30 denotes a memory (storage means) for storing captured images such as still images and moving images, and has a sufficient storage capacity for storing a predetermined number of still images and a predetermined time of moving images. This makes it possible to write a large amount of images to the memory 30 at high speed even in continuous shooting or panoramic shooting in which a plurality of still images are continuously shot. The memory 30 can also be used as a work area for the system control circuit 50.

  A compression / decompression circuit 32 compresses and decompresses image data by adaptive discrete cosine transform (ADCT) or the like. The compression / decompression circuit 32 reads an image stored in the memory 30, performs compression processing or decompression processing, and writes the processed data in the memory 30. Reference numeral 40 denotes exposure control means for controlling the mechanical shutter 12 having an aperture function. The exposure control means 40 has a flash light control function in cooperation with the flash 48. Reference numeral 42 denotes distance measurement control means for controlling the focusing of the photographing lens 10. Reference numeral 44 denotes zoom control means for controlling zooming of the taking lens 10. 46 is a barrier control means for controlling the operation of the protection means 102 as a barrier. A flash 48 has an AF auxiliary light projecting function and a flash light control function. The exposure control means 40 and the distance measurement control means 42 are controlled using the TTL method. The system control circuit 50 controls the exposure control means 40 and the distance measurement control means 42 based on the calculation result of the captured image (image data) by the image processing circuit 20.

  Reference numeral 50 denotes a system control circuit (control means) that controls the entire imaging apparatus 100 (image processing apparatus). The system control circuit 50 includes an interval shooting unit 50a (shooting control unit), a comparatively bright combining unit 50b (synthesizing unit), an image processing unit 50c, and a display control unit 50d. The interval photographing unit 50 a sequentially photographs a plurality of images using the image sensor 14. The comparatively bright combining unit 50b sequentially performs comparatively bright combining processing (synthesizing processing) on the plurality of images sequentially captured by the interval photographing unit 50a. In the comparatively bright combination process, when performing the sequential combination process on the images sequentially captured by the interval photographing unit 50a, the images are compared and a composite image is generated by selectively using the one whose signal level is not low. It is processing.

  The image processing unit 50c performs predetermined image processing on the synthesized images (compositing images) that are sequentially generated by the comparatively bright synthesizing unit 50b in synchronization with the timing at which the interval shooting unit 50a sequentially captures images. The predetermined image processing is, for example, correction processing based on the optical black value of the composite image. The display control unit 50d is a composite image (composite image, first composite image) after correction processing by the image processing unit 50c, which is sequentially generated by the comparative bright combination unit 50b, during the shooting of a plurality of images by the interval shooting unit 50a. ) Is displayed on the image display unit 28 (display means).

  A memory 52 stores constants, variables, programs, and the like for operating the system control circuit 50. Reference numeral 54 denotes a display unit that includes a liquid crystal display device, a speaker, and the like that display an operation state, a message, and the like using characters, images, sounds, and the like in accordance with execution of a program by the system control circuit 50. The display unit 54 is installed at an easily visible position near the operation unit of the imaging apparatus 100 (image processing apparatus), and includes, for example, a combination of an LCD, an LED, a sounding element, and the like. The display unit 54 is partially installed in the optical viewfinder 104.

  Among the displayed contents of the display unit 54, the contents displayed on the LCD include single shot / continuous shooting display, self-timer display, compression rate display, recording pixel number display, recording number display, and remaining image number display. is there. In addition, shutter speed display, aperture value display, exposure correction display, flash display, red-eye reduction display, and macro shooting display are also possible. Further, a buzzer setting display, a battery remaining amount display, an error display, an information display with a multi-digit number, an attachment / detachment state display of the recording media 200 and 210, a communication I / F operation display, and the like are possible. Among the display contents of the display unit 54, what is displayed in the optical viewfinder 104 includes in-focus display, camera shake warning display, flash charge display, shutter speed display, aperture value display, exposure correction display, and the like. Reference numeral 56 denotes an electrically erasable / recordable nonvolatile memory, which is configured using, for example, an EEPROM.

  Reference numerals 60, 62, 64, 66, 70, 72, 74 are operation means for inputting various operation instructions of the system control circuit 50. These operation means are configured by a single or a combination of a switch, a dial, a touch panel, pointing by line-of-sight detection, a voice recognition device, and the like. Hereinafter, these operation means will be described in detail.

  Reference numeral 60 denotes a mode dial switch. The mode dial switch 60 switches and sets each function mode such as power-off, still image shooting mode, panoramic shooting mode, moving image shooting mode, snap moving image shooting mode, playback mode, multi-screen playback, PC connection mode, and TV reception mode. be able to. Reference numeral 62 denotes a first shutter switch (SW1). The first shutter switch 62 is turned ON during the operation of the shutter button, and instructs to start a shooting preparation operation such as AF (autofocus) processing, AE (automatic exposure) processing, AWB (auto white balance) processing, and the like.

  Reference numeral 64 denotes a second shutter switch (SW2). The second shutter switch 64 is turned on when the operation of the shutter button is completed, and instructs the start of a series of processing operations. In this series of processes, first, an image capturing process is performed in which a signal (image data) read from the image sensor 14 is written into the memory 30 via the A / D converter 16 and the memory control circuit 22. Further, development processing using calculation in the image processing circuit 20 and the memory control circuit 22 and processing for reading out image data from the memory 30 and performing compression processing in the compression / decompression circuit 32 are performed. Then, a recording process for writing image data to the recording medium 200 or the recording medium 210 is performed.

  Reference numeral 66 denotes a display changeover switch, which can perform display changeover of the image display unit 28. With this function, when photographing using the optical finder 104, it is possible to save power by cutting off the current supply to the image display unit 28 including a TFT-LCD or the like. An operation unit 70 includes various buttons and a touch panel. The operation unit 70 includes a menu button, a set button, a macro button, a multi-screen playback page break button, a flash setting button, a single shooting / continuous shooting / self-timer switching button, and the like. The operation unit 70 also includes a menu movement + (plus) button, menu movement-(minus) button, playback image movement + (plus) button, playback image-(minus) button, shooting image quality selection button, exposure correction button, date / time It also has time setting buttons.

  Reference numeral 72 denotes a zoom switch as zoom operation means for the user to give an instruction to change the magnification of the captured image. The zoom switch 72 includes a tele switch that changes the imaging angle of view to the telephoto side, and a wide switch that changes the imaging angle of view to the wide angle side. The operation of the zoom switch 72 serves as a trigger for performing an optical zoom operation by instructing the zoom control unit 44 to change the imaging field angle of the photographing lens 10. The operation of the zoom switch 72 also serves as a trigger for electronic change of the imaging angle of view by image cutting by the image processing circuit 20 or pixel interpolation processing. Reference numeral 74 denotes a moving image reproduction instruction unit. By operating the moving image playback instruction unit 74 within a predetermined period after moving image shooting, it is possible to start playback of the captured moving image.

  Reference numeral 80 denotes power supply control means. The power control unit 80 includes a battery detection circuit, a DC-DC converter, a switch circuit that switches a block to be energized, and the like, and detects whether or not a battery is installed, the type of battery, and the remaining battery level. Further, the power supply control means 80 controls the DC-DC converter based on the detection result and the instruction of the system control circuit 50, and supplies the necessary voltage to each part including the recording media 200, 201 for a necessary period. Reference numerals 82 and 84 denote connectors. Reference numeral 86 denotes power supply means configured to include a primary battery such as an alkaline battery or a lithium battery, or a secondary battery such as a NiCd battery, a NiMH battery, or a Li battery, and an AC adapter.

  Reference numerals 90 and 94 denote interfaces (I / F) with recording media such as memory cards and hard disks. Reference numerals 92 and 96 denote connectors for connecting to a recording medium such as a memory card or a hard disk. In this embodiment, the case where two interfaces and connectors for attaching the recording medium are described, but the present invention is not limited to this. The interface and connector for attaching the recording medium may include a single system or a plurality of systems other than two. Also, a configuration in which interfaces and connectors of different standards are combined may be used. The interface and the connector may be configured using an SD card, a PCMCIA card, a CF (Compact Flash (registered trademark)) card, or the like. In such a configuration, various communication cards such as a communication card such as a LAN card, a modem card, a USB card, an IEEE 1394 card, a SCSI card, and a PHS are connected. Thereby, image data and management information attached to the image data can be transferred bi-directionally between peripheral devices such as other computers and printers.

  Reference numeral 102 denotes a protection unit (barrier) that covers the imaging unit including the photographing lens 10 of the imaging apparatus 100 to prevent the imaging unit from being dirty or damaged. Reference numeral 104 denotes an optical viewfinder. The imaging apparatus 100 can perform imaging using only the optical viewfinder 104 without using the electronic viewfinder function of the image display unit 28. Further, the optical viewfinder 104 is provided with some functions of the display unit 54 such as an in-focus display, a camera shake warning display, a flash charge display, a shutter speed display, an aperture value display, and an exposure correction display.

  110 is a communication means. The communication unit 110 has various communication functions such as USB, IEEE 1394, LAN, and wireless communication. Reference numeral 112 denotes a connector for connecting the imaging apparatus 100 (image processing apparatus) to another device by the communication unit 110 or an antenna when using wireless communication. Reference numeral 200 denotes a recording medium such as a memory card or a hard disk. The recording medium 200 includes a recording unit 202 that includes a semiconductor memory, a magnetic disk, and the like, an interface 204 (I / F) with the imaging device 100, and a connector 206 that connects to the imaging device 100. Reference numeral 210 denotes a recording medium such as a memory card or a hard disk. The recording medium 210 includes a recording unit 212 that includes a semiconductor memory, a magnetic disk, and the like, an interface 214 with the imaging device 100, and a connector 216 that connects to the imaging device 100.

  In addition, although the imaging device 100 of a present Example is comprised by integrating the imaging lens 10, it is not limited to this. This embodiment can also be applied to an imaging system configured by combining an imaging device main body and a photographing lens (imaging optical system) that can be attached to and detached from the imaging device main body.

  Next, the comparatively bright combination process in the present embodiment will be described. As one of the shooting functions, there is a method of shooting a plurality of still images obtained by performing interval shooting while a digital camera is fixed to a tripod to create a composite image. Specifically, comparisons were made to create a photo that reflected the light trails of fireflies moving within the shooting screen and a photo that represented the light trails of stars moving in the night sky caused by the diurnal motion of the Earth. A bright synthesis process may be performed.

  The comparatively bright combination process in the present embodiment is performed according to the following procedure. First, interval shooting is started in accordance with a user instruction. When shooting is started, the first still image (first image) of a plurality of shots is shot and stored as a reference image in a memory area (memory 30) prepared for composition. Next, the second still image (second image) of interval shooting is taken, and the second still image is compared with the first still image stored in the compositing memory area. Specifically, the brightness of the corresponding pixels is compared, and the value of the pixel with the lower brightness is written back to the synthesis memory area. This process is performed for all the pixels, and the composite image of the first still image and the second still image is held in the compositing memory area. After that, for the third and subsequent still images (images), the luminance of the corresponding pixel of the still image to be processed is compared with the synthesized image in the synthesis memory region, and the pixel whose luminance is not low is compared with the synthesis memory region. The process of writing back to all pixels is performed. By repeating the above steps for all still images to be processed, image composition processing is sequentially performed.

  By performing the above-described comparatively bright combination processing using a large number of still images obtained by interval shooting of the digital camera, a combined image as if it has been exposed for a long time is obtained.

  However, as the shooting time of a large number of still images becomes longer, the dark current of the image sensor 14 increases, and a pixel signal is also output from a pixel that is not illuminated by the image sensor. As a result, when the comparatively bright combination process described above is performed, a problem arises that a dark portion of the combined image gradually floats with a signal value (black floating phenomenon).

  Therefore, blacking processing is performed as a countermeasure against dark current of the image sensor 14. Blacking processing is processing that subtracts a signal generated by dark current from an image captured by actual photographing by performing actual photographing and dark photographing using the image sensor 14 before and after photographing and performing subtraction processing. It is. Specifically, first, main shooting (shooting a main image) is performed. Subsequently, dark imaging is performed with the same exposure time, exposure sensitivity, and the like as in the actual imaging, with the same conditions, and the image sensor being shielded. As a result, the dark current component of the signal from the image sensor 14 is derived, and black subtraction processing is performed by performing subtraction processing from the main imaging performed previously. However, as mentioned above, when shooting dark light in the shaded state for this blacking process when shooting light trails of fireflies or light spots of stars, digital images of light trails during the dark shooting time period are digitally recorded. Cannot import to the camera. For this reason, the light trails of the fireflies and the stars are taken off. As a result, the light traces of the fireflies and the light traces of the stars become broken lines. Therefore, it is not preferable to perform the blacking process at every photographing in the interval photographing in which the comparative bright combination processing is performed.

  Therefore, in the digital camera (imaging device 100) in the present embodiment, in order to reduce the above phenomenon, dark photography is performed when interval photographing for performing comparatively bright composition on the main image (captured record image) is completed. To obtain a black image (second image). Then, blacking processing is performed with the comparatively bright composite image. That is, the interval photographing unit 50a photographs a second image in a state where the image sensor 14 is shielded after photographing a plurality of images. Then, the image processing unit 50c performs blacking processing for subtracting the second image from the second composite image. By performing this process, the black floating phenomenon is reduced in the comparatively bright composite image that is finally left.

  Further, in this embodiment, during the interval shooting, the progress of shooting and image processing can be confirmed by displaying an intermediate image (composite image) of comparatively bright combination on the display unit. However, dark photography cannot be performed on an image being combined. Therefore, in this embodiment, a black image (first image) obtained by performing dark shooting before interval shooting, which is shooting of the main image (shooting and recording image), is used for an intermediate image of comparative bright combination. To reduce the black floating phenomenon.

  However, during the comparative bright combination, the dark current gradually increases and the image becomes black as the number of interval shots in which comparative bright combination is performed increases. Therefore, in this embodiment, an intermediate image of comparatively bright combination is obtained from an OB (optical black) area that is arranged outside the effective pixel area on the image sensor 14 acquired in each shooting and is always in a light-shielded state at the time of shooting. The dark current value (OB value) is subtracted (OB subtraction). This subtraction process is performed before processing the comparatively bright composite intermediate image into a display image. This reduces the black floating of the display image. For this reason, it is possible to display a more preferable midway image (composite image) of comparative bright combination.

  In this case, not only the value of the dark current obtained from the OB area is subtracted, but also, for example, a subtraction process may be performed from an image captured in dark before the start of imaging. Alternatively, the same effect can be obtained even if image processing is arbitrarily performed using a color conversion processing circuit.

  With reference to FIG. 2, a control method (image processing method) of the imaging apparatus 100 in the present embodiment will be described. FIG. 2 is a flowchart illustrating a control method (image processing method) of the imaging apparatus 100. The flow of steps S105 to S108 in FIG. 2 is a sequence in which the OB subtraction is displayed on the follow-up image of the midway image in the present embodiment. Each step in FIG. 2 is executed by each unit in accordance with a command from the system control circuit 50 or an instruction from the system control circuit 50.

  First, in step S101, the system control circuit 50 performs dark shooting with the image sensor 14 shielded before performing interval shooting, and stores a black image (black image, first image) in the memory area. . This black image is subtracted from each captured image in a later step in order to make it difficult to see pixel scratches and the like that appear in the intermediate image for confirming the progress of interval shooting. Subsequently, in step S102, the system control circuit 50 starts interval shooting according to a user instruction. Thereby, the interval photographing unit 50a sequentially captures a plurality of images. Details of interval shooting will be described later with reference to FIG. At the same time, in step S103, the system control circuit 50 (comparative light composition means 50b) advances comparative light composition processing for a plurality of images obtained by interval shooting. The details of the comparatively bright combination process in this embodiment will be described later with reference to FIG.

  Next, in step S104, the system control circuit 50 determines whether the interval shooting has been completed by performing the interval shooting end operation, or whether the interval shooting has ended after the set time has elapsed. When the interval shooting is finished, the process proceeds to step S109. On the other hand, if the interval shooting has not ended in step S104, the system control circuit 50 continues the interval shooting. In step S105, the system control circuit 50 determines whether or not the halfway image display function is set to ON, that is, whether or not halfway image display is set. If it is not set to display an intermediate image, the process returns to step S102, and the system control circuit 50 continues the interval shooting. On the other hand, if it is set to display the REC view image, the process proceeds to step S106. Here, since the interval shooting itself must be performed at a constant interval regardless of whether or not the intermediate image is displayed, the processes of S105 to S108 and S102 may be performed in parallel.

  In step S106, the system control circuit 50 creates an intermediate image. At this time, the system control circuit 50 (image processing unit 50c) performs correction processing (predetermined image processing) based on the optical black value (OB value) on the combined image generated by the comparatively bright combining unit 50b. This correction processing is performed in synchronism with the timing of sequential shooting by the interval shooting means 50a. More specifically, the system control circuit 50 reads the OB value from the OB area of the image obtained by the interval shooting, and subtracts the OB value from the shot image (image signal), whereby the synthesized image is subjected to OB. Correction processing based on the value is performed (OB subtraction is performed). Subsequently, in step S107, the system control circuit 50 performs development processing on the image subjected to the subtraction process in step S106. Subsequently, in step S108, the system control circuit 50 reduces (resizes) the image developed in step S107 to a size for display on an LCD (Liquid Crystal Display) (image display unit 28). Then, a reduced image (resized image) is displayed on the image display unit 28. That is, the system control circuit 50 (display control means 50d) causes the image display unit 28 to display a halfway composite image after image processing by the image processing means 50c. After displaying the reduced image, the process returns to step S102.

  When interval shooting is completed in step S104, the system control circuit 50 performs dark shooting in step S109, and a black image (black image) for blacking processing of the main image (recorded image) is stored in the memory area. Store. This is because there is time for processing at the end of shooting. Subsequently, in step S110, the system control circuit 50 performs blacking processing from the comparatively bright composite image to reduce the black float of the image. In step S111, the system control circuit 50 performs development processing on the comparatively bright composite image, and sets the image as the final comparatively bright composite image. This is the end of this flow. The sequence shown in FIG. 2 allows the user to confirm the intermediate image of the comparatively bright combination, which improves usability.

  Next, with reference to FIG. 3, the interval shooting (step S102 in FIG. 2) in the present embodiment will be described in detail. FIG. 3 is a flowchart showing interval shooting. The flowchart of FIG. 3 is a main routine of interval shooting, and shows the timing for starting the comparatively bright combination process while performing interval shooting. Each step of FIG. 3 is mainly executed by the system control circuit 50 (interval photographing means 50a).

  First, in step S201, the system control circuit 50 initializes a synthesis processing flag. Subsequently, in step S202, the system control unit 50 determines and sets focus information, exposure information (exposure conditions), and the like before performing a series of interval shootings. Step S202 is passed through when the second and subsequent images are taken in one interval shooting. The in-combination processing flag is a flag indicating whether or not “comparative bright combination processing” is being executed.

  Next, in step S203, the system control circuit 50 starts measuring the interval shooting interval. This process is a process of operating an internal timer in order to measure the interval shooting interval. Subsequently, in step S <b> 204, the system control circuit 50 performs an exposure process using the exposure control means 40. The exposure process is a process for exposing the image sensor 14. At this time, there is a possibility that the comparatively bright combination processing is performed simultaneously. Therefore, the system control circuit 50 does not immediately read out data from the image sensor 14 even when the exposure is completed.

  Subsequently, in step S205, the system control circuit 50 determines whether or not the compositing process flag is set. That is, the system control circuit 50 determines whether or not the composition processing of the image captured last time in a series of interval shooting processing has been completed with reference to a variable in the program. If the composition processing of the previously captured image has not been completed and the composition processing flag is set (in the case of “Yes” in step S205), the processing waits until the composition processing is completed. On the other hand, when the composition processing of the previously photographed image is completed and the composition processing flag is cleared (in the case of “No” in step S205), the processing proceeds to step S206.

  In step S206, the system control circuit 50 controls the memory control circuit 22 (timing generation circuit 18) to read data from the image sensor (image sensor 14). This process is a process of reading an image obtained by exposure of the image sensor from the image sensor and storing it in the memory area (memory 30). At this time, a black drawing process for subtracting the black image (first image) acquired in step S101 and stored in the memory from the read image is performed. In other words, the image processing unit 50c subtracts the first image captured by the interval imaging unit 50a in a state where the image sensor 14 is shielded before imaging a plurality of images for each image sequentially captured by the interval imaging unit 50a. Blacking process is performed. As a result, pixel flaws and noise components existing in each image can be reduced from appearing in the combined image.

  Subsequently, in step S207, the system control circuit 50 sets a synthesis processing flag. This process is a process of setting a synthesis process in progress flag for determining whether or not the synthesis process is being executed in another program as a variable in the program.

  Subsequently, in step S208, the system control circuit 50 starts a comparatively bright combination process. The comparatively bright combination process is executed in parallel with interval shooting by multitasking. Details of the comparatively bright combination process will be described later.

  Subsequently, in step S209, the system control circuit 50 determines whether or not an operation for ending the interval shooting has been performed. When an operation to end the interval shooting is instructed by the user, when the interval shooting time elapses, or when the battery is depleted to a predetermined value or more (when “Yes” in step S209), the system control circuit 50 ends interval shooting. On the other hand, when the operation for ending the interval shooting is not performed, the total interval shooting time has not elapsed, and the next shooting timing has not come (in the case of “No” in step S209) ) Wait until interval shooting time elapses. If the next shooting timing comes after the interval shooting time has elapsed, the process returns to step S203 to start measurement of the interval shooting time, and the next image is shot.

  Next, with reference to FIG. 4, the comparatively bright combination process (step S103 in FIG. 2) in the present embodiment will be described in detail. FIG. 4 is a flowchart showing the comparatively bright combination process. Each step in FIG. 4 is executed based on a command from the system control circuit 50.

  First, in step S301, the system control circuit 50 determines whether or not shooting is the first shooting. This process is a process executed to determine a memory area used as a work memory for synthesis. This determination is generally made by providing a photographing number counter. The number of shots counter is initialized at the time of the first shooting after the interval shooting is started. When it is determined in step S301 that the shooting is the first shooting, the system control unit 50 stores the image in the work memory area (memory 30) for the first shooting image. On the other hand, if the shooting is for the second and subsequent shots, the process proceeds to step S303.

  In step S303, the system control circuit 50 stores the captured image in a work memory area (memory 30) different from the first work memory area. Subsequently, in step S304, the system control circuit 50 performs comparatively bright combination processing using the first captured image and the second captured image. In this embodiment, the comparatively brightly combined image is overwritten on the first work memory area. Subsequently, in step S305, the system control circuit 50 increments (+1) the photographic number counter. In step S306, the system control circuit 50 clears the synthesis processing flag and ends this flow.

  As described above, in this embodiment, the image processing apparatus combines and displays and records an image obtained by interval shooting, and the recorded image is blackened by a black image obtained after interval shooting. Process. As a result, an image in which the influence of pixel scratches and dark current is reduced is generated. In addition, when displaying an intermediate image during interval shooting, the black image processing is performed on the captured image using a black image acquired before interval shooting, thereby reducing the influence of pixel flaws on the intermediate image. In addition, by performing correction processing (predetermined image processing) based on the optical black value for the intermediate image (composite image obtained by combining up to the image that has been shot), the influence of black float due to dark current in the intermediate image Can be reduced.

  In the present embodiment, the predetermined image processing by the image processing unit 50c is correction processing based on the optical black value, but is not limited thereto. For example, color conversion processing for the composite image may be performed as the predetermined image processing.

  More preferably, the comparatively bright combining unit 50b performs comparatively bright combining processing using the first image and the second image among the plurality of images sequentially captured by the interval shooting unit 50a, and performs comparatively bright combining. The processing result is stored in the memory 30 as a first composite image. For the third image, a comparatively bright composite process is performed using this image and the composite image (first composite image) stored in the memory 30, and the result of the comparatively bright composite process is used as the second composite image. It is stored in the memory 30. As described above, the system control circuit 50 (storage control unit) stores the second combined image obtained by combining a plurality of images in the memory 30 (storage unit). Similarly for the fourth and subsequent images, this image and the synthesized image stored in the memory 30 (an image obtained by synthesizing the first to third images) are newly updated as a synthesized image.

  Preferably, the display control unit 50d causes the image display unit 28 to display the composite image after the image processing without stopping the processing of the interval photographing unit 50a in synchronization with the timing of sequential photographing by the interval photographing unit 50a. . Preferably, the image processing unit 50 c performs predetermined image processing on the composite image stored in the memory 30.

[Other Embodiments]
The present invention is also realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and the computer (or CPU, MPU, etc.) of the system or apparatus reads the program. To be executed. In this case, a computer-executable program describing the procedure of the imaging apparatus control method and a storage medium storing the program constitute the present invention.

  According to the present embodiment, it is possible to provide an imaging device, an imaging system, an imaging device control method, a program, and a storage medium that can perform comparatively bright combination processing during interval shooting and display a high-quality composite image. it can.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.

14 Image pickup device 30 Memory 50 System control circuit 50a Interval shooting means 50b Comparative bright combination means 50c Image processing means 50d Display control means 100 Imaging device

Claims (10)

An image sensor for taking an image;
Storage means for storing the image;
Shooting control means for sequentially shooting a plurality of images using the imaging device;
Synthesizing means for sequentially performing synthesizing processing on images sequentially photographed by the photographing control means;
Image processing means for performing correction processing based on an optical black value of the composite image for the composite images sequentially generated by the composite means;
Display control means for causing the display means to display the first composite image after the correction processing by the image processing means, which is sequentially generated by the combining means, during photographing of the plurality of images by the photographing control means;
An image pickup apparatus comprising: a storage control unit configured to store a second combined image obtained by combining the plurality of images in the storage unit.   The image processing means subtracts, for each image that is sequentially photographed by the photographing control means, a first image that is photographed in a state where the image sensor is shielded from light before photographing the plurality of images by the photographing control means. The imaging apparatus according to claim 1, wherein blacking processing is performed. An image sensor for taking an image;
Storage means for storing the image;
Shooting control means for sequentially shooting a plurality of images using the imaging device;
A blacking process is performed to subtract the first image captured with the image sensor shielded before the plurality of images are captured by the imaging control unit for each image sequentially captured by the imaging control unit. Image processing means;
Synthesizing means for sequentially performing synthesizing processing on images sequentially photographed by the photographing control means;
Display control means for causing the display means to display a first composite image sequentially generated by the combining means during shooting of the plurality of images by the shooting control means;
An image pickup apparatus comprising: a storage control unit configured to store a second combined image obtained by combining the plurality of images in the storage unit. The imaging control unit captures the second image in a state where the image sensor is shielded after capturing the plurality of images.
4. The imaging apparatus according to claim 1, wherein the image processing unit performs blacking processing for subtracting the second image from the second composite image. 5.   The synthesizing unit compares the images and selectively uses the one whose signal level is not low when performing sequential synthesis processing on the images sequentially photographed by the photographing control unit. The imaging apparatus according to claim 1, wherein synthesis processing is performed. A lens apparatus equipped with an imaging optical system;
An imaging system comprising: the imaging device according to claim 1. A shooting step of sequentially shooting a plurality of images using an image sensor;
A synthesis step of sequentially performing a synthesis process on the images sequentially captured in the imaging step;
An image processing step for performing a correction process based on an optical black value of the composite image with respect to the composite image sequentially generated by the composite step;
A display control step for displaying on the display means the first combined image after the correction processing by the image processing step sequentially generated in the combining step during the shooting of the plurality of images by the shooting step;
And a storage step of storing in the storage means a second combined image obtained by combining the plurality of images. A shooting step of sequentially shooting a plurality of images using an image sensor;
A blacking process is performed in which the first image captured in a state where the image sensor is shielded from light before capturing the plurality of images in the capturing step is subtracted for each image sequentially captured in the capturing step. An image processing step;
A synthesis step of sequentially performing a synthesis process on the images sequentially captured in the imaging step;
A display control step for displaying on the display means a first composite image that is sequentially generated in the combining step during shooting of the plurality of images in the shooting step;
And a storage step of storing a second combined image obtained by combining the plurality of images.   A computer-executable program in which a procedure of a control method for an imaging apparatus according to claim 7 or 8 is described.   A computer-readable storage medium storing a program for causing a computer to execute each step of the control method of the imaging apparatus according to claim 7 or 8.