JP2017022623A - Imaging apparatus and control method therefor, program, and storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus capable of uninterrupted shooting overtime with less noises when carrying out a bulb shooting while displaying a live view.SOLUTION: The imaging apparatus includes: an image pick-up device with plural unit pixels arranged each including at least two light receiving elements of a first light receiving element and a second light receiving element; and a reading part that reads out signals plural times for displaying an image from the second light receiving element before the image pick-up device completes imaging of a piece of image, and after the image pick-up device has completed the imaging of the piece of image, reads out signals for recording the image from the first light receiving element.SELECTED DRAWING: Figure 4

Description

  The present invention relates to an imaging apparatus and a control method thereof.

  The operation of a typical image sensor used in digital cameras, etc. includes an exposure operation in which received light is photoelectrically converted by a photodiode to accumulate charges, and then a voltage signal corresponding to the amount of accumulated charges is sent to the outside. There is a read operation to output. It is known that when the reading operation is performed, the temperature of the image sensor increases, and as a result, noise due to dark current increases. On the other hand, bulb shooting is performed when shooting a firework or a starry sky. At this time, there is a need for the user to perform a shooting end operation at a timing when appropriate exposure is performed while checking the shooting state in real time by a live view.

  In order to meet such needs and reduce noise due to dark current, Patent Document 1 reads out signals of pixels at different positions for each field, generates a composite image with information accumulated in units of pixels, A method of creating a display image with the latest accumulated information is proposed.

JP 2012-80457 A

  However, in the method disclosed in Patent Document 1, since the readout timing differs depending on the pixel position, the noise location due to the dark current is dispersed, but the influence of the dark current differs depending on the pixel position, and the noise feeling is uniform. There is a problem of not becoming. Further, as the number of times of reading increases, noise also increases.

  Further, when the image pickup device is of a destructive readout method in which the accumulated charge is cleared every reading, the charge cannot be accumulated in the pixel during the reading. Therefore, when shooting a moving subject such as fireworks or shooting stars, if a reading operation occurs midway, the trajectory may be interrupted even if the accumulation is divided into multiple times and combined later. It was.

  The present invention has been made in view of the above-described problems. The purpose of the present invention is to enable continuous shooting without bulb interruption and suppress increase in noise when bulb shooting is performed while displaying a live view. An imaging device that can be used is provided.

  An image pickup apparatus according to the present invention includes an image pickup element in which a plurality of unit pixels each having at least two light receiving elements of a first light receiving element and a second light receiving element are arranged, and one image by the image pickup element. Until the photographing is finished, the display image signal is read out from the second light receiving element a plurality of times, and after the photographing of the one image by the imaging element is finished, the first light receiving element Reading means for reading out an image signal for recording.

  ADVANTAGE OF THE INVENTION According to this invention, when performing bulb | ball imaging, performing live view display, it becomes possible to provide the imaging device which enables the imaging | photography which is not interrupted temporally and can also suppress the increase in noise.

1 is a block diagram illustrating a configuration of a digital camera according to a first embodiment. FIG. 3 is a diagram illustrating a configuration of a pixel of an image sensor. 6 is a flowchart showing an operation at the time of bulb photographing in the first embodiment. The figure which shows the change of the accumulation | storage state of the image pick-up element at the time of valve | bulb imaging | photography in 1st Embodiment. 6 is a flowchart showing an operation at the time of bulb photographing in the first embodiment.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

(First embodiment)
FIG. 1 is a block diagram illustrating a configuration of a digital camera 100 that is a first embodiment of an imaging apparatus according to the present invention. In FIG. 1, a photographing lens 103 is a lens group including a zoom lens and a focus lens, and forms a subject image. The diaphragm 101 adjusts the amount of light incident on the photographing lens 103. The image sensor 22 is composed of a CCD, a CMOS sensor, or the like, and converts an optical image into an electrical signal. The A / D converter 23 converts the analog signal output from the image sensor 22 into a digital signal. The barrier 102 covers the imaging system of the digital camera 100 including the photographing lens 103, the diaphragm 101, and the imaging element 22, thereby preventing the imaging system from becoming dirty or damaged.

  The image processing unit 24 resizes the data from the A / D converter 23 or the data from the memory control unit 15 such as predetermined pixel interpolation processing and reduction processing, color conversion processing, gamma correction processing, digital gain. Addition processing is performed. Further, a predetermined calculation process is performed using the captured image data, and the calculation result is transmitted to the system control unit 50. Based on the calculation result transmitted, the system control unit 50 performs exposure control, distance measurement control, white balance control, and the like. As a result, TTL (through the lens) AF (autofocus) processing, AE (automatic exposure) processing, EF (flash pre-emission) processing, and the like are performed. The image processing unit 24 performs predetermined calculation processing using the captured image data, and also performs TTL AWB (auto white balance) processing based on the obtained calculation result.

  Output data from the A / D converter 23 is directly written into the memory 32 that temporarily holds data via the image processing unit 24 and the memory control unit 15 or via the memory control unit 15. The memory 32 stores image data captured by the imaging unit 22 and converted into digital data by the A / D converter 23 and image data to be displayed on the display unit 28. The memory 32 has a storage capacity sufficient to store a predetermined number of still images, a moving image and audio for a predetermined time.

  The memory 32 also serves as an image display memory (video memory). The D / A converter 13 converts the image display data stored in the memory 32 into an analog signal and supplies the analog signal to the display unit 28. Thus, the display image data written in the memory 32 is displayed on the display unit 28 such as an LCD via the D / A converter 13. The digital signal once A / D converted by the A / D converter 23 and stored in the memory 32 is converted into an analog signal by the D / A converter 13 and sequentially transferred to the display unit 28 for display. Realized and can perform live view display with a through image.

  The nonvolatile memory 56 is an electrically erasable / recordable memory, and for example, an EEPROM is used. The nonvolatile memory 56 stores constants, programs, and the like for operating the system control unit 50. Here, the program is a program for executing various flowcharts to be described later.

  The system control unit 50 controls the entire digital camera 100. By executing the program recorded in the non-volatile memory 56 described above, each process of this embodiment to be described later is executed. A RAM is used as the system memory 52, and constants and variables for operation of the system control unit 50, a program read from the nonvolatile memory 56, and the like are expanded. The system control unit 50 also performs display control by controlling the memory 32, the D / A converter 13, the display unit 28, and the like.

  The system timer 53 measures the time used for various controls and the time of a built-in clock. The mode switch 60 connected to the system controller 50, the first shutter switch 62, the shutter button 61 having the second shutter switch 64, and the operation unit 70 input various operation instructions to the system controller 50.

  The mode selector switch 60 switches the operation mode of the system control unit 50 to any one of a still image recording auto mode, a still image recording manual mode, a valve mode, a moving image recording mode, a reproduction mode, and the like. The first shutter switch 62 is turned on when the shutter button 61 provided in the digital camera 100 is being operated, so-called half-pressing to generate a first shutter switch signal SW1 (shooting preparation instruction). In response to the first shutter switch signal SW1, shooting preparation operations such as AF (autofocus) processing, AE (automatic exposure) processing, AWB (auto white balance) processing, and EF (flash pre-flash) processing are started.

  The second shutter switch 64 is turned on when the operation of the shutter button 61 is completed, that is, when fully pressed, and generates a second shutter switch signal SW2 (shooting start instruction). Then, the second shutter switch signal SW2 is stopped by releasing the full press. In the still image recording auto mode and the still image recording manual mode, the system control unit 50 performs a series of shooting processes from reading a signal from the image sensor 22 to writing image data on the recording medium 200 by the second shutter switch signal SW2. Start the operation. In the bulb mode and the moving image recording mode, the system control unit 50 starts the photographing process operation by the second shutter switch signal SW2. Then, when the second shutter switch signal SW2 is stopped, the operation of the photographing end process until the image data is written to the recording medium 200 is started.

  Each operation member of the operation unit 70 is appropriately assigned a function for each scene by selecting and operating various function icons displayed on the display unit 28, and operates as various function buttons. Examples of the function buttons include an end button, a return button, an image advance button, a jump button, a narrowing button, and an attribute change button. For example, when a menu button is pressed, various setting menu screens are displayed on the display unit 28. The user can make various settings intuitively by using the menu screen displayed on the display unit 28, and the four-way key and the SET button in four directions.

  The controller wheel 73 is a rotatable operation member included in the operation unit 70, and is used when a selection item is designated together with the cross key. When the controller wheel 73 is rotated, an electrical pulse signal is generated according to the operation amount, and the system control unit 50 controls each unit of the digital camera 100 based on the pulse signal. In addition, the pulse signal can be used to determine the angle at which the controller wheel 73 has been rotated, the number of rotations, and the like. The controller wheel 73 may be any operation member that can detect a rotation operation. For example, it may be a dial operation member that generates a pulse signal by rotating the controller wheel 73 itself according to the rotation operation of the user. Further, it may be a touch sensor that detects the rotation operation of the user's finger on the controller wheel 73 without rotating the controller wheel 73 itself.

  The power control unit 80 includes a battery detection circuit, a DC-DC converter, a switch circuit that switches a block to be energized, and the like, and detects whether or not a battery is installed, the type of battery, and the remaining battery level. Further, the power control unit 80 controls the DC-DC converter based on the detection result and an instruction from the system control unit 50, and supplies a necessary voltage to each unit including the recording medium 200 for a necessary period.

  The power supply unit 30 includes a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li ion battery, an AC adapter, or the like. The recording medium I / F 18 is an interface with the recording medium 200 such as a memory card or a hard disk. The recording medium 200 is a recording medium such as a memory card for recording a captured image, and includes a semiconductor memory, a magnetic disk, or the like.

  FIG. 2 is a diagram schematically illustrating the configuration of the pixels of the image sensor 22. The image sensor 22 of the present embodiment is configured by arranging a plurality of unit pixels having one microlens. Two photodiodes are arranged in one pixel (unit pixel), and an image signal used for automatic focus adjustment (hereinafter referred to as imaging plane phase difference AF) by the imaging plane phase difference detection method can be generated. 2A schematically illustrates an example of a pixel configuration that does not correspond to the imaging surface phase difference AF, and FIG. 2B schematically illustrates an example of a pixel configuration that corresponds to the imaging surface phase difference AF. Here, in any case, it is assumed that a Bayer array primary color filter is provided. In the pixel configuration of FIG. 2B corresponding to the imaging surface phase difference AF, one pixel in FIG. 2A is divided into two in the horizontal direction on the paper surface, and two photodiodes A and B (one pixel ( A light receiving element). Note that the method of dividing one pixel shown in FIG. 2B into a plurality of light receiving elements is an example, and other division methods may be used, or different division methods may be used for each pixel.

  The light beam incident on each pixel is separated by a micro lens, and is received by two photodiodes (light receiving elements) provided in each pixel, so that two signals for imaging and AF (autofocus) are obtained by one pixel. Can be obtained. That is, signals (A signal and B signal) obtained by each of the two light receiving elements A and B in one pixel are two image signals for AF, and an addition signal (A + B signal) is an imaging signal. Further, in the imaging signal at the in-focus position, the sensitivity of the imaging signal of only one of the light receiving element A or the light receiving element B is halved with respect to the imaging signal that is the addition signal (A + B signal). It is a signal equivalent to only the imaging signal. It should be noted that a pair of image signals used in the imaging plane phase difference AF is also a plurality of pairs so that a pair of image signals used in the normal phase difference detection AF is generated by a pair of line sensors having a plurality of pixels. It is generated from the outputs of the light receiving element A and the plurality of light receiving elements B. Based on the AF signal, the image processing unit 24 performs correlation calculation on the two image signals, thereby calculating the image shift amount and various types of reliability information.

  FIG. 3 is a flowchart illustrating the operation of the system control unit 50 when the digital camera 100 according to the present embodiment captures one still image by performing bulb photographing.

  In FIG. 3, first, in step S300, it is determined whether or not to start shooting preparation. If the first shutter switch 62 is turned on, the process proceeds to step S301. If the first shutter switch 62 is not turned on, the process waits until the first shutter switch 62 is turned on. In step S301, preparation for shooting is performed. In this step, photometry is performed to determine the exposure amount, focus adjustment operation according to camera settings, white balance setting, and the like.

  In step S302, it is determined whether to start actual shooting. When the second shutter switch 64 is turned on, it is determined that shooting is started, and the process proceeds to step S303. If not, the process returns to step S300. In step S303, the image pickup device 22 starts an accumulation operation. The shutter 101 is opened and accumulation is started in each of the light receiving elements A and B. In step S304, it is determined whether or not to end shooting. The end of shooting is determined by whether or not the second shutter switch 64 is turned off. When the second shutter switch 64 is turned off, the process proceeds to step S305 to start an operation for ending the shooting. If the second shutter switch 64 is not turned off, the process proceeds to step S310 as a process for continuing shooting.

  In step S310, it is determined whether it is the update timing of the live view image that sequentially reads out the image signal (reads out a plurality of times) and displays it sequentially. The update of the live view image may be performed at a timing (fixed interval) for a certain period of time set in advance by the user, or the photometric result in step S301 or an image obtained at the previous update timing of the live view image. It may be determined whether or not to carry out according to the above. If it is time to update the live view image, the process advances to step S311. In step S311, the imaging signal of the light receiving element B is read. The imaging signal of the light receiving element B is read from the imaging element 22 and sent to the image processing unit 24 in a mode corresponding to the display device. At this time, the signal of the light receiving element B may be read according to the specification of the display unit 28 that displays the live view image, and it is not necessary to read out all the pixels by adding and thinning out the signal of the light receiving element B.

  In step S312, the live view image is synthesized. The currently displayed live view image is stored in the memory 32. The image data obtained by converting the imaging signal of the light receiving element B read in step S311 into a display image format and the image data read from the memory 32 are displayed. And add. In step S313, the composite result of the live view image generated in step S312 is written back to the memory 32, and the display on the display unit 28 is updated. When the display is updated, the process returns to step S304.

  On the other hand, in step S305, an image signal for recording is read from the light receiving element A. The signal accumulated in the light receiving element A within the imaging time is read out, the imaging signal is sent to the image processing unit 24, and the process proceeds to step S306. In step S306, a black image used for noise reduction processing is captured. When long-time shooting such as bulb shooting is performed, the influence of fixed noise of the image sensor on the image is increased. For this reason, the shutter 101 is closed and the black image is taken with the same shooting parameters as when the image for storage was taken. In step S308, the image processing unit 24 performs blacking processing for subtracting the black image pickup signal from the image pickup signal of the light receiving element A for each corresponding pixel to develop the image. Next, in step S308, an image file for recording is created according to the file format set by the user, and in step S309, the image file is stored in the recording medium 200.

  FIG. 4 is a diagram illustrating the relationship between the accumulation state of the image sensor 22 and time when performing bulb imaging in the imaging apparatus of the present embodiment. When performing bulb photography, the light receiving element A performs exposure in order to photograph a recording image. During the exposure period, since reading is not performed, the images 401 to 404 that are actually accumulated cannot be viewed. When the exposure is completed, an imaging signal is read from the light receiving element A, and a captured image 405 can be obtained. During the period when exposure is performed by the light receiving element A, the images 411 to 415 of the light receiving element B are periodically read out. The read image of the light receiving element B is added to the currently displayed live view images 421 to 425 to create a new display image, and the created display image is displayed.

  For example, a new display image 422 can be obtained by adding the image 412 obtained by the exposure of the first light receiving element B and the image 421 being displayed. As a result, the image obtained by each exposure of the light receiving element B is integrated as time passes, and the light receiving element A corresponding to each of the light receiving elements B is accumulated by being continuously exposed by bulb photography. It is close to the state of the signal. When the bulb photographing is finished by the user's operation, the exposure is finished, and a recording image pickup signal 405 is obtained. Thereafter, a black image is taken with the image sensor 22 shielded from light, and noise reduction processing is performed using the obtained black image 406 to obtain a recording image 426. A recording file is generated from the recording image 426 according to a predetermined file format and stored in the recording medium 200.

  As described above, in the digital camera according to the first embodiment, the light receiving element A is used for taking a recording image, and reading is not performed in the middle. The phenomenon of interruption can be avoided. In addition, by using the light receiving element B for capturing an image for live view display, the user can know the state of the captured image in real time.

(Second Embodiment)
There is a high need for shooting in scenes with a deep depth of field, such as landscape photography in the dark, such as starry sky photography and fireworks photography, as typical photography applications for bulb photography. On the other hand, in the imaging device 22 mentioned in the first embodiment, two photodiodes (light receiving devices) are arranged in one pixel, and the light receiving device A and the light receiving device B are used for the imaging surface phase difference detection method. Output of the phase difference information. Therefore, when shooting with a shallow depth of field, a difference occurs in the image pickup signals of the light receiving element A and the light receiving element B, and the image output for recording and the live view image may be greatly different. Also, in landscape photography in a dark place, the night sky occupies a large area of the work, so fixed noise in the image sensor greatly reduces the quality of the work. Therefore, in the second embodiment, an example will be described in which the difference between the image output for recording and the live view image is reduced as much as possible, and the influence of fixed noise on the recording image is reduced as much as possible. Hereinafter, a different part from 1st Embodiment of the digital camera 100 of 2nd Embodiment is demonstrated, and description is abbreviate | omitted about a common part.

  In the second embodiment, in the block diagram showing the configuration of the digital camera 100 of the first embodiment in FIG. 1, defective pixel information of the image sensor 22 is recorded in the nonvolatile memory 56. The defective pixel information is information indicating whether or not the pixel does not have the same performance as other pixels. The image sensor 22 has millions of pixels or more, but it is difficult in terms of manufacturing cost to make all the pixels have the same performance. And in order to improve a yield, the image pick-up element in which the number of defective pixels is below a reference value is handled as a non-defective product, and is dealt with by correcting by an image processing technique. This defective pixel appears as fixed noise in long-time shooting, and a pixel without light input becomes a bright spot, or always has a black spot with light input. Noise reduction processing is performed on such fixed noise. The defective pixel information may be recorded in the nonvolatile memory 56 at the time of manufacturing test. In addition, since defective pixels tend to increase with the passage of time, defective pixels may be detected automatically or by user operation at the stage of use, and information in the nonvolatile memory 56 may be updated. The defective pixel information is stored by providing an area for storing the total number of defective light receiving elements A and the total number of defective light receiving elements B.

  The mode changeover switch 60 is provided with a dark scene photographing mode separately from the bulb mode. Even in the dark landscape shooting mode, the shooting processing operation is started by the second shutter switch signal SW2 as in the bulb mode. Then, when the second shutter switch signal SW2 is stopped, the operation of the photographing end process until the image data is written to the recording medium 200 is started.

  FIG. 5 is a flowchart illustrating the operation of the system control unit 50 when shooting a dark scene in the digital camera of the second embodiment. 3 differs from the flowchart at the time of bulb photographing in FIG. 3 in three steps: photographing preparation in step S501, reading of a light receiving element with few defects in step S505, and reading of a light receiving element with many defects in step S511. In the shooting preparation step in step S501, photometry is performed to determine the exposure amount, white balance setting, and the like, and the focus position of the shooting lens 103 is fixed at infinity. Further, defective pixel information stored in the nonvolatile memory 56 is read out, and the light receiving element A and the light receiving element B are compared, and the light receiving element having a relatively large number of defects is received for live view display image capturing. Select as an element. The other is set as a light receiving element for image capturing for storage. In step S510, if it is the update timing of the live view image, the process proceeds to step S511, and the imaging signal is read from the light receiving element for capturing the live view display image with the larger number of defects set in step S501. In step S505, the storage image capturing light receiving element having the smaller number of defects set in step S501 is read.

  As described above, in the digital camera of the second embodiment, in the dark place shooting mode, the focal position is fixed at infinity, and output for storage due to the shallow depth of field. It can be avoided that the image and the live view image are greatly different. Further, by using the light receiving element having the smaller number of defects for taking a stored image, it is possible to avoid the fixed noise in the image pickup element from greatly degrading the quality of the work.

  Although the present invention has been described in detail based on preferred embodiments thereof, the present invention is not limited to these specific embodiments, and various forms within the scope of the present invention are also included in the present invention. included. A part of the above-described embodiments may be appropriately combined.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

100: Digital camera, 101: Shutter, 22: Image sensor, 24: Image processing unit, 28: Display unit, 50: System control unit, 56: Non-volatile memory, 60: Mode switch, 70: Operation unit, 200: Storage medium

Claims (12)

An imaging device in which a plurality of unit pixels each including at least two light receiving elements of a first light receiving element and a second light receiving element are arranged;
The image signal for display is read a plurality of times from the second light receiving element until the photographing of one image by the imaging element is completed, and the photographing of the one image by the imaging element is completed. A reading means for reading a recording image signal from the first light receiving element;
An imaging apparatus comprising:   The image pickup apparatus according to claim 1, further comprising storage means for storing information relating to defects of the first and second light receiving elements of the image pickup element.   Of the at least two light receiving elements, the light receiving element having a relatively small number of light receiving elements having defects in the plurality of unit pixels is used as the first light receiving element, and the number of light receiving elements having defects is relatively large. The imaging device according to claim 2, wherein the light receiving element is the second light receiving element.   The imaging apparatus according to claim 2, further comprising an update unit that detects a defect in the at least two light receiving elements and updates information stored in the storage unit.   The image signal for display is generated by adding the image signal to an image signal that has been used for display each time the image signal is read from the second light receiving element. Item 5. The imaging device according to any one of Items 1 to 4.   6. The imaging apparatus according to claim 1, wherein the recording image signal is recorded on a recording medium mounted on the imaging apparatus.   The imaging device according to claim 1, further comprising a control unit that adjusts a focal position of the photographing lens, wherein the control unit adjusts the focal position of the photographing lens to infinity. apparatus.   8. The imaging according to claim 1, further comprising a subtracting unit that captures a black image in a state where the image sensor is shielded from light and subtracts the black image from the recording image. apparatus.   The imaging apparatus according to claim 1, wherein the display image signal is sequentially read out at an interval shorter than a photographing time of the one image. A method for controlling an imaging apparatus including an imaging element in which a plurality of unit pixels each including at least two light receiving elements of a first light receiving element and a second light receiving element are arranged,
The image signal for display is read out and displayed from the second light receiving element a plurality of times until the photographing of one image by the image sensor is completed, and the one image is photographed by the image sensor. And a reading step of reading out an image signal for recording from the first light receiving element after the operation is completed.   A program for causing a computer to execute the control method according to claim 10.   A computer-readable storage medium storing a program for causing a computer to execute the control method according to claim 10.