JP2020031259A - Imaging apparatus and control method thereof - Google Patents

Abstract

To provide an imaging apparatus capable of achieving further improvements.SOLUTION: An imaging apparatus 100 includes: an imaging device 101 capable of nondestructive reading; and a control unit 102 configured to control the shooting of a target frame using the auxiliary image obtained by nondestructive reading in the target frame. For example, the control unit 102 may be configured to control the exposure time of the target frame using the auxiliary image. For example, the control unit 102 may be configured to sequentially perform multiple times of nondestructive reading in the target frame, and when the brightness of the auxiliary images exceeds a predetermined threshold, stop the exposure of the target frame.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an imaging device and a control method thereof.

  As an imaging device using an organic photoelectric conversion device, a technology described in Patent Document 1 is known.

JP 2008-042180 A

  In such an imaging device, further improvement is desired.

  Therefore, an object of the present disclosure is to provide an imaging device or a control method thereof that can achieve further improvement.

  An imaging device according to an aspect of the present disclosure includes an imaging element capable of nondestructive readout, and a control unit that controls shooting of the target frame using an auxiliary image obtained by nondestructive readout in the target frame. .

  The present disclosure can provide an imaging device or a control method thereof that can achieve further improvement.

FIG. 1 is a block diagram of the imaging device according to the first embodiment. FIG. 2A is a diagram illustrating an example of an external appearance of the imaging device according to Embodiment 1. FIG. 2B is a diagram illustrating an example of the external appearance of the imaging device according to Embodiment 1. FIG. 3 is a diagram illustrating a configuration of the imaging device according to the first embodiment. FIG. 4 is a circuit diagram showing a configuration of the pixel according to the first embodiment. FIG. 5 is a flowchart illustrating an operation of the imaging device according to the first embodiment. FIG. 6 is a diagram illustrating an operation of the imaging device according to the first embodiment. FIG. 7 is a flowchart illustrating an operation of the imaging device according to the second embodiment. FIG. 8 is a diagram illustrating an operation of the imaging device according to the second embodiment. FIG. 9 is a diagram illustrating an example of an auxiliary image according to the second embodiment. FIG. 10 is a diagram showing an example of a threshold setting screen according to the second embodiment. FIG. 11 is a flowchart illustrating an operation of the imaging device according to the third embodiment. FIG. 12 is a diagram for explaining a subject change determination process according to the third embodiment. FIG. 13 is a flowchart illustrating a modified example of the operation of the imaging device according to Embodiment 3. FIG. 14 is a flowchart illustrating a modified example of the operation of the imaging device according to Embodiment 3. FIG. 15 is a flowchart illustrating a modified example of the operation of the imaging device according to Embodiment 3. FIG. 16 is a flowchart illustrating the operation of the imaging device according to Embodiment 4. FIG. 17 is a diagram illustrating an operation of the imaging device according to the fourth embodiment. FIG. 18 is a diagram illustrating an example of an image selection screen according to the fourth embodiment. FIG. 19 is a diagram illustrating an operation of an imaging device according to a modification of the fourth embodiment.

  An imaging device according to an aspect of the present disclosure includes an imaging element capable of nondestructive readout, and a control unit that controls shooting of the target frame using an auxiliary image obtained by nondestructive readout in the target frame. .

  According to this, the imaging device can control the shooting of the frame using the auxiliary image obtained by the non-destructive readout, so that appropriate control can be performed according to the situation during the exposure period. Thus, the imaging device can achieve further improvement.

  For example, the control unit may control the exposure time of the target frame using the auxiliary image.

  According to this, the imaging apparatus can control the exposure time based on the information of the image obtained during the exposure period, so that the imaging apparatus is more appropriate than controlling the exposure time based on the information obtained before the start of the exposure. Exposure time can be controlled.

  For example, the control unit may sequentially perform a plurality of non-destructive readings on the target frame, and stop the exposure of the target frame when the luminance of the auxiliary image exceeds a predetermined threshold.

  According to this, the imaging device can appropriately control the exposure time based on the information of the image obtained during the exposure period.

  For example, the imaging device may further include a display unit, and the control unit may further display a user interface for setting the threshold on the display unit.

  According to this, the exposure setting according to the user's preference can be easily performed.

  For example, the control unit may perform automatic exposure of the target frame using the auxiliary image.

  According to this, the image pickup apparatus can perform automatic exposure based on information of an image obtained during the exposure period, so that the automatic exposure can be performed more appropriately than when performing automatic exposure based on information obtained before the start of exposure. Exposure can be performed.

  For example, the control unit sequentially performs a plurality of non-destructive readouts on the target frame, and detects a change in a subject in the target frame using a plurality of auxiliary images obtained by the plurality of non-destructive readouts. May be.

  According to this, the imaging device can detect, for example, a change in the subject during long-time exposure using the auxiliary image obtained by the nondestructive readout.

  For example, the control unit may issue a warning when a change in the subject is detected.

  According to this, the imaging device can notify the user of the occurrence of an abnormality during long-time exposure, for example.

  For example, the control unit may stop shooting when a change in the subject is detected.

  According to this, the imaging device can automatically stop shooting, for example, when an abnormality occurs during long-time exposure.

  For example, the control unit may perform re-photographing when a change in the subject is detected.

  According to this, the imaging apparatus can automatically perform re-imaging, for example, when an abnormality occurs during long-time exposure.

  For example, when a change in the subject is detected, the control unit may perform correction for reducing an influence of the change in the subject on an image of the target frame obtained by shooting.

  According to this, even when a change occurs in the subject, the imaging apparatus can generate an image in which the influence is reduced.

  For example, the imaging device further includes a display unit that displays the auxiliary image during the exposure period of the target frame, and the control unit further ends the exposure of the target frame during the exposure period of the target frame. May be accepted.

  According to this, the user can shoot an image of a desired luminance level with reference to the auxiliary image displayed during the exposure period.

  For example, the control unit may sequentially perform a plurality of non-destructive read operations on the target frame, and store a plurality of images obtained by the plurality of non-destructive read operations as a moving image.

  For example, the image sensor may be an organic sensor.

  A control method according to an aspect of the present disclosure is a control method of an imaging apparatus including an imaging element capable of nondestructive readout, and uses a supplementary image obtained by nondestructive readout in a target frame to generate a target frame. The method includes a control step of controlling photographing.

  According to this, the control method can control photographing of the frame using the auxiliary image obtained by the nondestructive readout, so that appropriate control can be performed according to the situation during the exposure period. Thus, the control method can achieve further improvement.

  Note that these comprehensive or specific aspects may be realized by a recording medium such as a system, a method, an integrated circuit, a computer program or a computer-readable CD-ROM, and the system, the method, the integrated circuit, and the computer program. And any combination of recording media.

  Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. Each of the embodiments described below shows a preferred specific example of the present disclosure. Therefore, numerical values, shapes, materials, constituent elements, arrangement positions of constituent elements, connection forms, and the like shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Therefore, among the components in the following embodiments, components that are not described in the independent claims that indicate the highest concept of the present disclosure are described as arbitrary components.

  Each drawing is a schematic diagram, and is not necessarily strictly illustrated. In addition, in each of the drawings, substantially the same configuration is denoted by the same reference numeral, and redundant description will be omitted or simplified.

(Embodiment 1)
[Configuration of Imaging Device]
First, a configuration of an imaging device according to an embodiment of the present disclosure will be described. FIG. 1 is a block diagram illustrating a configuration of an imaging device 100 according to the present embodiment. 2A and 2B are diagrams illustrating an example of the appearance of the imaging device 100. FIG. For example, as shown in FIGS. 2A and 2B, the imaging device 100 is a camera such as a digital still camera or a digital video camera.

  The imaging device 100 illustrated in FIG. 1 includes an imaging element 101, a control unit 102, a display unit 103, and a storage unit 104. The imaging device 101 is a solid-state imaging device (solid-state imaging device) that converts incident light into an electric signal (image) and outputs the obtained electric signal, and is, for example, an organic sensor using an organic photoelectric conversion device.

  The control unit 102 controls the image sensor 101. Further, the control unit 102 performs various kinds of signal processing on the image obtained by the imaging element 101, and displays the obtained image on the display unit 103 or stores the obtained image in the storage unit 104. Note that the image output from the control unit 102 may be output to the outside of the imaging device 100 via an input / output interface (not shown).

[Configuration of imaging device]
Next, the configuration of the image sensor 101 will be described. FIG. 3 is a block diagram illustrating a configuration of the imaging element 101.

  3 includes a plurality of pixels (unit pixel cells) 201 arranged in a matrix, a vertical scanning unit 202, a column signal processing unit 203, a horizontal reading unit 204, and a row. A plurality of reset control lines 205, a plurality of address control lines 206 provided for each row, a plurality of vertical signal lines 207 provided for each column, and a horizontal output terminal 208.

  Each of the plurality of pixels 201 outputs a signal corresponding to incident light to a vertical signal line 207 provided in a corresponding column.

  The vertical scanning unit 202 resets the plurality of pixels 201 via the plurality of reset control lines 205. Further, the vertical scanning unit 202 sequentially selects a plurality of pixels 201 on a row-by-row basis via a plurality of address control lines 206.

  The column signal processing unit 203 performs signal processing on the signals output to the plurality of vertical signal lines 207, and outputs the plurality of signals obtained by the signal processing to the horizontal reading unit 204. For example, the column signal processing unit 203 performs noise suppression signal processing represented by correlated double sampling, analog / digital conversion processing, and the like.

  The horizontal reading unit 204 sequentially outputs a plurality of signals that have been subjected to the signal processing by the plurality of column signal processing units 203 to the horizontal output terminal 208.

  Hereinafter, the configuration of the pixel 201 will be described. FIG. 4 is a circuit diagram illustrating a configuration of the pixel 201.

  As illustrated in FIG. 4, the pixel 201 includes a photoelectric conversion unit 211, a charge storage unit 212, a reset transistor 213, an amplification transistor 214 (source follower transistor), and a selection transistor 215.

  The photoelectric conversion unit 211 generates signal charges by photoelectrically converting incident light. A voltage Voe is applied to one end of the photoelectric conversion unit 211. Specifically, the photoelectric conversion unit 211 includes a photoelectric conversion layer made of an organic material. Note that this photoelectric conversion layer may include a layer made of an organic material and a layer made of an inorganic material.

  The charge storage unit 212 is connected to the photoelectric conversion unit 211 and stores the signal charge generated by the photoelectric conversion unit 211. Note that the charge storage unit 212 may be configured with a parasitic capacitance such as a wiring capacitance instead of a dedicated capacitance element.

  The reset transistor 213 is used to reset the potential of the signal charge. The gate of the reset transistor 213 is connected to the reset control line 205, the source is connected to the charge storage unit 212, and the reset voltage Vreset is applied to the drain.

  The definitions of the drain and the source generally depend on the circuit operation, and in many cases, cannot be specified from the element structure. In this embodiment, one of the source and the drain is referred to as a source for convenience, and the other of the source and the drain is referred to as a drain. However, the drain in this embodiment may be replaced with a source, and the source may be replaced with a drain.

  The amplification transistor 214 amplifies the voltage of the charge storage unit 212 and outputs a signal corresponding to the voltage to the vertical signal line 207. The gate of the amplification transistor 214 is connected to the charge storage unit 212, and the power supply voltage Vdd or the ground voltage Vss is applied to the drain.

  The selection transistor 215 is connected in series with the amplification transistor 214, and switches whether to output the signal amplified by the amplification transistor 214 to the vertical signal line 207. The gate of the selection transistor 215 is connected to the address control line 206, the drain is connected to the source of the amplification transistor 214, and the source is connected to the vertical signal line 207.

  In addition, for example, the voltage Voe, the reset voltage Vreset, and the power supply voltage Vdd are voltages commonly used in all the pixels 201.

  Further, the characteristics of the organic sensor include non-destructive readout and electronic ND (Neutral Density) control. Non-destructive readout is a process of reading out image data during an exposure period and continuing exposure. In conventional reading (hereinafter, referred to as destructive reading), it is necessary to terminate exposure when performing reading. That is, only one image could be obtained by one exposure. On the other hand, by using the nondestructive readout, it is possible to read out the image data exposed up to that time during the exposure period and continue the exposure. Thus, a plurality of images having different exposure times can be obtained by one exposure.

  The electronic ND control is a process for electrically controlling the transmittance of the image sensor. Here, the transmittance means a ratio of light converted into an electric signal in incident light. That is, by setting the transmittance to 0%, it is possible to electrically shield light. Specifically, the transmittance is controlled by controlling the voltage Voe shown in FIG. Thus, the exposure can be electrically terminated without using the mechanical shutter.

  The image sensor 101 may include a mesher, and may use the electronic ND control and the light blocking by the mechanical shutter in combination, or may use only the light blocking by the mechanical shutter.

  Here, an example in which the image sensor 101 is an organic sensor will be described. However, the image sensor 101 may be any device other than the organic sensor as long as it can perform non-destructive readout or electronic ND control. That is, the photoelectric conversion layer included in the photoelectric conversion unit 211 may be made of an inorganic material. For example, the photoelectric conversion layer may be made of amorphous silicon or chalcopyrite semiconductor.

[Operation of imaging device]
Next, the operation of the imaging device 100 according to the present embodiment will be described. The imaging apparatus 100 according to the present embodiment controls shooting of the target frame using the auxiliary image obtained by non-destructive reading in the target frame. Specifically, the imaging device 100 performs automatic exposure (AE: Auto Exposure) of the target frame using the auxiliary image.

  FIG. 5 is a flowchart illustrating a flow of the operation of the imaging device 100. FIG. 6 is a diagram for explaining the operation of the imaging device 100.

  As illustrated in FIG. 6, during a period before time t1 at which shooting of a still image is started, the imaging apparatus 100 shoots a live image that is a real-time moving image of a subject, and displays the live image on the display unit 103. are doing.

  At time t1, for example, based on a user operation, the imaging device 100 starts exposure for capturing a still image (S101). Next, the imaging device 100 acquires one or more auxiliary images by performing a predetermined number of non-destructive readings during the exposure period (S102). Although two non-destructive read operations are performed in FIG. 6, the number of non-destructive read operations may be arbitrary. Next, the imaging device 100 performs AE control using the obtained auxiliary image (S103).

  Here, conventionally, AE control was performed using a live image. Therefore, the imaging device 100 can perform the AE control using the auxiliary image by the same method as the AE control using the live image. Specifically, the imaging device 100 controls the exposure time T1 of the target frame using the auxiliary image. For example, the imaging device 100 detects the movement of the subject based on a difference between the plurality of auxiliary images, and controls the exposure time T1 based on the obtained movement. For example, the imaging device 100 sets the exposure time T1 to be shorter as the movement is larger. Alternatively, the imaging device 100 controls the exposure time T1 based on the luminance level of one or more auxiliary images. For example, the imaging apparatus 100 sets the exposure time T1 to be longer as the luminance level is lower.

  Next, at time t4 when the exposure time T1 set in step S103 has elapsed, the imaging device 100 ends the exposure (S104). For example, the image capturing apparatus 100 ends the exposure by performing the above-described electronic ND control or performing the light shielding by the mechanical shutter.

  Next, the imaging device 100 acquires an image by performing destructive reading (S105). In FIG. 6, the nondestructive readout is performed immediately after the end of the exposure.

  As described above, the imaging device 100 according to the present embodiment controls shooting of the target frame using the auxiliary image obtained by non-destructive reading in the target frame. Specifically, the imaging device 100 performs automatic exposure of the target frame using the auxiliary image. Here, when performing automatic exposure using a live image, there is a time difference between performing automatic exposure and capturing a still image. As a result, the accuracy of automatic exposure may decrease. On the other hand, in the method according to the present embodiment, since the time difference can be reduced, the accuracy of the automatic exposure can be improved.

  Further, the method of the present embodiment is different from the conventional method only in that an image used for AE control is changed from a live image to an auxiliary image obtained by non-destructive reading. Therefore, more accurate automatic exposure can be realized while diverting a part of the conventional automatic exposure algorithm.

  Further, here, the case where a still image is captured has been described as an example, but a similar method may be used when capturing a moving image.

(Embodiment 2)
In the present embodiment, another method for controlling the exposure time of a target frame using an auxiliary image will be described.

  FIG. 7 is a flowchart illustrating a flow of the operation of the imaging device 100 according to the present embodiment. FIG. 8 is a diagram for explaining the operation of the imaging device 100.

  First, the imaging device 100 determines a threshold value TH used for a determination process described later (S111). The details of this process will be described later.

  Next, the imaging device 100 starts exposure at time t1 (S112). At time t2 after a predetermined time has elapsed, the imaging device 100 acquires an auxiliary image by non-destructive reading (S113). Then, the imaging device 100 determines whether or not the luminance of the obtained auxiliary image has exceeded the threshold value TH set in step S111 (S114).

  If the luminance of the auxiliary image does not exceed the threshold value TH (No in S114), the imaging device 100 performs non-destructive readout again at time t3 after elapse of the time T2 (S113), It is determined whether the luminance has exceeded the threshold value TH (S114). Then, the imaging device 100 repeats these processes until the luminance of the auxiliary image exceeds the threshold value TH.

  When the luminance of the auxiliary image exceeds the threshold value TH (Yes in S114: time t5), the imaging device 100 ends the exposure (S115) and acquires an image by destructive reading (S116: time t6).

  FIG. 9 is a diagram illustrating an example of the auxiliary image and the luminance histogram obtained at each time. As shown in FIG. 9, the auxiliary images obtained at each time have different exposure times, and thus have different luminance levels. That is, the later the obtained time, the higher the luminance level.

  In the determination of step S114, the imaging device 100 determines, for example, whether the maximum value of the luminance of the auxiliary image has exceeded the threshold value TH. Note that the imaging device 100 may compare the threshold value TH with not only the maximum value of the luminance but also the average value or the median value of the luminance.

  Further, in FIG. 8, the imaging apparatus 100 ends the exposure immediately after determining that the luminance of the auxiliary image has exceeded the threshold value TH, but may end the exposure after a predetermined time. In this case, the threshold value TH is set lower by a difference corresponding to the time.

  Further, in FIG. 8, the interval T2 at which the nondestructive read is performed is constant, but may not be constant. For example, the imaging apparatus 100 may set an exposure time in advance by AE control or the like, and set a plurality of non-destructive read timings so as to shorten a non-destructive read interval at a time approaching the exposure time. Good. Alternatively, the imaging device 100 may shorten the non-destructive read interval as the luminance of the obtained auxiliary image approaches the threshold value TH. The time from the end of exposure to the time when destructive readout is performed may be arbitrary.

  Further, here, the case where a still image is captured has been described as an example, but a similar method may be used when capturing a moving image.

  As described above, the imaging device 100 sequentially performs the non-destructive reading a plurality of times in the target frame, and stops the exposure of the target frame when the luminance of the auxiliary image exceeds the predetermined threshold TH. Here, in a conventional general method, an exposure period is determined before photographing an image, and an image having a desired luminance level cannot always be obtained. On the other hand, by using the method of the present embodiment, it is possible to dynamically determine whether or not the luminance level of the image has reached a desired value during the exposure period, so that an image with the desired luminance level can be obtained with high accuracy. Can be.

  As a method of determining the threshold value TH in step S111, for example, the imaging device 100 determines the threshold value TH according to the currently set imaging mode. Alternatively, the imaging apparatus 100 may display a user interface for setting the threshold value TH on the display unit 103, and determine the threshold value TH based on a user instruction via the user interface user. FIG. 10 is a diagram illustrating an example of the user interface.

  For example, as shown in FIG. 10, a plurality of sample images having different luminance levels are displayed, and when the user selects any one of the sample images, the threshold value TH corresponding to the sample image is set. Note that the sample image may be an image stored in advance, or may be an auxiliary image obtained by non-destructive reading at the time of past photographing.

  FIG. 10 shows an example in which the user selects a sample image, but the luminance level may be selected using an operation bar or the like for changing the luminance level. Alternatively, an operation of increasing or decreasing the luminance level with respect to the current luminance level may be used.

  By providing such a user interface, the user can intuitively and easily perform an operation of selecting a luminance level.

  Note that the threshold TH need not be variable, and a fixed value may be used as the threshold TH.

(Embodiment 3)
In the present embodiment, another example of a process of controlling shooting of the target frame using an auxiliary image obtained by non-destructive reading in the target frame will be described. Specifically, the imaging device 100 according to the present embodiment sequentially performs a plurality of non-destructive readings on the target frame, and uses the plurality of auxiliary images obtained by the non-destructive reading on the target frame to perform The change of the subject in is detected. For example, the imaging device 100 detects an undesired change in the subject caused by a disturbance to the camera or a tripod falling when exposing a star night or the like for a long time.

  FIG. 11 is a flowchart illustrating a flow of the operation of the imaging device 100 according to the present embodiment. FIG. 12 is a diagram illustrating an example of a change in the luminance of the auxiliary image when the subject changes.

  First, the imaging device 100 starts exposure (S121). At time t1 after a predetermined time T3 has elapsed, the imaging device 100 acquires an auxiliary image by non-destructive reading (S122). Then, the imaging apparatus 100 determines whether or not the subject has changed using the plurality of auxiliary images obtained after the start of the exposure (S123). If it is determined that there is no change in the subject (No in S123) and the exposure period has not ended (No in S125), the imaging apparatus 100 performs non-destructive readout again at time t2 after the lapse of time T3. (S122), and it is determined again whether the subject has changed using the plurality of auxiliary images obtained up to that time (S123).

  On the other hand, when it is determined that the subject has changed (Yes in S123), the imaging device 100 issues a warning (S124). For example, the imaging device 100 displays on the display unit 103 a message indicating that the subject has changed. Note that the imaging device 100 may notify the user that the subject has changed by a warning sound or a voice message.

  Then, the imaging apparatus 100 repeats a series of these processes until the exposure period ends (S125).

  When the exposure period has ended (Yes in S125), the imaging device 100 ends the exposure and acquires an image by non-destructive reading (S126).

  Here, as shown in FIG. 12, as the exposure time increases, the luminance of the auxiliary image linearly increases. That is, the luminance increases at a constant slope. On the other hand, as shown at time t0, when the subject changes, the inclination of the luminance changes. FIG. 12 shows a case where an instantaneous disturbance occurs. Note that the inclination of the luminance also changes (increases or decreases) when the shooting scene changes due to the tripod falling or when an undesired subject enters the screen.

  Specifically, for example, the imaging device 100 detects the inclination of the luminance using a plurality of auxiliary images obtained at a plurality of times. Then, the imaging apparatus 100 calculates the inclination based on the luminance of the newly obtained auxiliary image and the luminance of the auxiliary image obtained immediately before, and determines the difference between the obtained inclination and the past inclination in advance. It is determined whether the threshold value is exceeded. If the difference exceeds the threshold, the imaging device 100 determines that the subject has changed. If the difference does not exceed the threshold, the imaging device 100 determines that the subject has not changed.

  For example, in the example shown in FIG. 12, at a time point before time t5, for example, a past inclination is calculated based on the luminance at time t3 and time t4. Then, the difference between the slope calculated based on the luminance at the time t4 and the time t5 and the past slope exceeds the threshold. Thereby, a warning is issued.

  The imaging device 100 performs the above determination using, for example, the average luminance of the auxiliary image. Note that the imaging device 100 may use the maximum luminance, or may use the average luminance or the maximum luminance of a specific region of the auxiliary image. In addition, the imaging device 100 may perform the above-described determination for each region including one or more pixels, and may determine that there is a change in the subject when it determines that there is a change in any of the regions.

  As described above, the imaging apparatus 100 according to the present embodiment sequentially performs a plurality of non-destructive readings on the target frame, and uses the plurality of auxiliary images obtained by the non-destructive reading on the target frame to perform the Detect changes in the subject. Thus, for example, a change in the subject during the long-time exposure can be detected, so that the user can be notified early of the failure of the imaging at the long-time exposure. Therefore, it is possible to prevent the exposure from being performed to the end even though the imaging has failed.

  As illustrated in FIG. 13, when it is determined that the subject has changed (Yes in S123), the imaging apparatus 100 may stop shooting (S124A). Thereby, for example, when an abnormality occurs during the long-time exposure, the imaging apparatus 100 can automatically stop the imaging, so that the exposure is performed to the end even though the imaging has failed. Can be prevented.

  Alternatively, as illustrated in FIG. 14, when it is determined that the subject has changed (Yes in S123), the imaging apparatus 100 may perform re-imaging (S124B). Accordingly, the imaging apparatus 100 can automatically perform re-imaging when an abnormality occurs during long-time exposure, for example.

  Alternatively, as illustrated in FIG. 15, when it is determined that there is a change in the subject, the imaging apparatus 100 performs correction for reducing the effect of the change in the subject on the image of the target frame obtained by shooting. You may. Specifically, when it is determined that there is a change in the subject (Yes in S123), the imaging device 100 stores change information regarding the change in the subject (S124C). Then, after acquiring the image of the target frame by non-destructive reading (S126), the imaging device 100 corrects the image using the change information (S127).

  For example, the imaging apparatus 100 determines a change for each area including one or more pixels, and stores information indicating the changed area and the amount of change as change information. Then, the imaging apparatus 100 performs correction on the changed area by subtracting or adding a luminance value corresponding to the amount of change.

  In the process illustrated in FIG. 15, the imaging apparatus 100 determines the change and saves the change information for each nondestructive read. However, after all the nondestructive read is performed, or the destructive read is performed. After that, the determination and the storage of the change information may be performed collectively.

  As described above, even when a change occurs in the subject, the imaging apparatus 100 can generate an image in which the influence is reduced.

  Although a plurality of operation examples when a change in the subject is detected have been described with reference to FIGS. 11 to 15, a plurality of these examples may be combined.

(Embodiment 4)
In the present embodiment, another method for controlling the exposure time of a target frame using an auxiliary image will be described. In the first and second embodiments, the method of automatically controlling the exposure time has been described. However, in the present embodiment, information for allowing a user to determine the exposure time is provided, and the exposure time is controlled based on a user operation. An example of control will be described. For example, the method of the present embodiment can be applied to long-time exposure of a still image.

  FIG. 16 is a flowchart illustrating a flow of the operation of the imaging device 100 according to the present embodiment. FIG. 17 is a diagram for explaining the operation of the imaging device 100.

  First, the imaging apparatus 100 starts exposure (S131). Next, the imaging device 100 acquires an auxiliary image by non-destructive reading (S132). Next, the imaging device 100 displays the obtained auxiliary image on the display unit 103 (S133).

  If there is no instruction to end the exposure from the user (No in S134) and the predetermined exposure period has not ended (No in S136), the imaging apparatus 100 performs the nondestructive readout again after a predetermined time has elapsed. Then, a newly obtained auxiliary image is displayed (S133).

  For example, as shown in FIG. 17, non-destructive reading is performed at a predetermined cycle T4, and the read auxiliary images are sequentially displayed. Note that the interval at which nondestructive reading is performed may not be constant.

  On the other hand, when there is an instruction to end the exposure from the user (Yes in S134), the imaging apparatus 100 stops the exposure (S135) and acquires an image by non-destructive reading (S137). When the predetermined exposure period has ended (Yes in S136), the imaging device 100 acquires an image by non-destructive reading (S137).

  As described above, the display unit 103 displays the auxiliary image during the exposure period of the target frame. In addition, the control unit 102 receives an instruction to end the exposure of the target frame during the exposure period of the target frame. Thereby, the user can stop the exposure at an arbitrary timing with reference to the auxiliary image displayed during the exposure period. Therefore, the user can easily take an image of a desired luminance level.

  Note that the method of instructing the end of the exposure by the user is not particularly limited. For example, the user can press the shutter button or operate the touch panel during the exposure period to receive the instruction of the end of the exposure.

  Also, here, an example in which the user controls the exposure time T1 during the exposure period has been described. However, during or after the exposure period, the user selects an arbitrary image from a plurality of auxiliary images already obtained. Is also good. In this case, for example, in step S133, the imaging device 100 displays the auxiliary image and saves the auxiliary image.

  In addition, for example, in accordance with a user operation, the imaging apparatus 100 displays a user interface for selecting one of a plurality of auxiliary images on the display unit 103 as illustrated in FIG. Specifically, if the exposure is being performed, a list of a plurality of auxiliary images obtained up to that timing is displayed. If the exposure has been completed, all the auxiliary images obtained by the exposure are displayed in a list. The user can select an image at an arbitrary luminance level by selecting one auxiliary image via the interface. When selection is performed after exposure, the displayed image may include an image obtained by destructive readout.

  The number of images to be selected is not limited to one, but may be plural. After the above selection, the imaging device 100 may delete the data of the auxiliary image that is not selected. If the selection is made during the exposure, the imaging apparatus 100 may end the exposure.

  Further, as shown in FIG. 19, the imaging apparatus 100 sequentially performs a plurality of non-destructive readings on a target frame, and stores a plurality of images obtained by the plurality of non-destructive readings as a moving image in the storage unit 104. Is also good. Further, the imaging apparatus 100 may reproduce the moving image and display the reproduced moving image on the display unit 103. That is, the image capturing apparatus 100 saves a plurality of images obtained by a plurality of nondestructive readouts in a moving image file format. For example, a known format including a moving image coding method using inter-screen prediction or the like is used.

  Further, the stored moving image is reproduced automatically after exposure is completed or based on a user operation.

  By storing a plurality of images obtained by non-destructive readout in a moving image format in this way, the shooting process itself during the exposure period can be stored as a moving image work. In addition, when using only a plurality of images obtained by destructive reading, it is necessary to generate a moving image by synthesizing a still image over time, but by using the above method, a still image synthesizing process is performed. Operations can be generated without performing Also, by storing a plurality of images in a moving image format, the data amount can be reduced as compared with the case of storing a plurality of still images.

  As described above, the imaging device according to the embodiment of the present disclosure has been described, but the present disclosure is not limited to this embodiment.

  For example, each processing unit included in the imaging device according to the above embodiment is typically realized as an LSI that is an integrated circuit. These may be individually integrated into one chip, or may be integrated into one chip so as to include some or all of them.

  Further, the integrated circuit is not limited to the LSI, and may be realized by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) that can be programmed after manufacturing the LSI, or a reconfigurable processor that can reconfigure the connection and setting of circuit cells inside the LSI may be used.

  Further, in each of the above embodiments, each component may be configured by dedicated hardware or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory.

  Further, the present disclosure may be realized as a control method executed by the imaging device.

  Further, the circuit configuration illustrated in the above circuit diagram is an example, and the present disclosure is not limited to the above circuit configuration. That is, a circuit capable of realizing the characteristic function of the present disclosure is also included in the present disclosure, similarly to the above circuit configuration. In addition, the numbers used above are merely examples for specifically describing the present disclosure, and the present disclosure is not limited to the illustrated numbers.

  The division of functional blocks in the block diagram is merely an example, and a plurality of functional blocks can be implemented as one functional block, one functional block can be divided into a plurality of functional blocks, and some functions can be transferred to other functional blocks. You may. Also, the functions of a plurality of functional blocks having similar functions may be processed by a single piece of hardware or software in parallel or time division.

  Further, the order in which the steps in the flowchart are executed is merely an example for specifically describing the present disclosure, and may be an order other than the above. Also, some of the above steps may be performed simultaneously (in parallel) with other steps.

  As described above, the imaging device according to one or more aspects has been described based on the embodiment, but the present disclosure is not limited to this embodiment. Unless departing from the spirit of the present disclosure, various modifications conceivable to those skilled in the art may be applied to the present embodiment, and a configuration constructed by combining components in different embodiments may be in the range of one or more aspects. May be included within.

  The present disclosure is applicable to an imaging device such as a digital still camera or a digital video camera.

REFERENCE SIGNS LIST 100 imaging device 101 imaging device 102 control unit 103 display unit 104 storage unit 201 pixel 202 vertical scanning unit 203 column signal processing unit 204 horizontal reading unit 205 reset control line 206 address control line 207 vertical signal line 208 horizontal output terminal 211 photoelectric conversion unit 212 Charge storage unit 213 Reset transistor 214 Amplification transistor 215 Selection transistor

Claims (14)

An image sensor capable of non-destructive readout,
A control unit that controls photographing of the target frame using an auxiliary image obtained by non-destructive reading in the target frame. The imaging device according to claim 1, wherein the control unit controls an exposure time of the target frame using the auxiliary image. The control unit includes:
Performing a plurality of non-destructive readouts sequentially in the target frame;
The imaging device according to claim 2, wherein when the luminance of the auxiliary image exceeds a predetermined threshold, exposure of the target frame is stopped. The imaging device further includes a display unit,
The imaging device according to claim 3, wherein the control unit further displays a user interface for setting the threshold on the display unit. The imaging device according to claim 1, wherein the control unit performs automatic exposure of the target frame using the auxiliary image. The control unit includes:
Performing a plurality of non-destructive readouts sequentially in the target frame;
The imaging device according to claim 1, wherein a change in a subject in the target frame is detected using a plurality of auxiliary images obtained by the plurality of nondestructive readouts. The imaging device according to claim 6, wherein the control unit issues a warning when a change in the subject is detected. The imaging device according to claim 6, wherein the control unit stops photographing when a change in the subject is detected. The imaging device according to claim 6, wherein the control unit performs re-imaging when a change in the subject is detected. The imaging apparatus according to claim 6, wherein, when a change in the subject is detected, the control unit performs correction for reducing an influence of the change in the subject on an image of the target frame obtained by shooting. The imaging device further includes:
A display unit that displays the auxiliary image during the exposure period of the target frame,
The imaging device according to claim 1, wherein the control unit further receives an instruction to end exposure of the target frame during an exposure period of the target frame. The control unit includes:
Performing a plurality of non-destructive readouts sequentially in the target frame;
The imaging device according to claim 1, wherein a plurality of images obtained by the plurality of nondestructive readouts are stored as moving images. The imaging device according to claim 1, wherein the imaging device is an organic sensor. A method for controlling an imaging device including an imaging element capable of nondestructive reading,
A control method including a control step of controlling photographing of a target frame by using an auxiliary image obtained by non-destructive reading in the target frame.

Images