JP7332376B2 - IMAGING DEVICE, CONTROL METHOD THEREOF, AND PROGRAM - Google Patents

Description

The present invention relates to an imaging apparatus, its control method, and program.

2. Description of the Related Art Conventionally, an imaging apparatus is known that has a live view (hereinafter also referred to as LV) function for displaying an output on a display while shooting. In the LV function, images for the LV (simply referred to as LV images) are generally shot and displayed at regular intervals, so the user can continuously shoot still images while checking the moving subject in the displayed LV images. It can be carried out.

When continuous shooting of still images is performed while using the LV function with an imaging device having one imaging sensor (hereinafter also referred to as continuous shooting), the imaging sensor repeatedly outputs image data for recording still images while outputting image data for display. I need to output image data. However, when a single image sensor is used, while still image shooting is performed and image data is output (also referred to as still image driving), LV display image shooting is performed and image data is output ( (also referred to as LV drive) cannot be performed. Therefore, when continuous shooting is performed during LV display, the LV image may not be obtained because the display driving time overlaps with the still image driving time. A blackout (hidden display) may occur and the display update frequency may decrease.

To solve this problem, when a continuous shooting operation is performed, image data for a reduced still image is displayed, and image data for the LV is displayed for a certain period of time according to the load state in the device. A technique for increasing the update frequency of the LV display during continuous shooting has been proposed (Patent Document 1).

JP 2012-44566 A

However, in the technique described in Patent Document 1, it is assumed that image data for still images is used when continuous shooting is performed. are displayed together. For this reason, if the still image and the LV image have different exposure times and subject motion blurring or a difference in brightness occurs, switching between the still image and the LV image to alternately display them may cause a sense of discomfort. On the other hand, if it is possible to make the most of the LV images that can be acquired while continuously shooting, it is possible to reduce the discomfort of the display in the LV function even if there is a difference in the exposure time between the still image and the LV image. There is

The present invention has been made in view of the above problems, and an object of the present invention is to provide a method for displaying images in the LV function even when the exposure time of the still image and the LV image are different when the still images are continuously shot while using the LV function. It is to realize a technology that can reduce the sense of discomfort.

In order to solve this problem, for example, the imaging device of the present invention has the following configuration. That is, in the case of repeatedly capturing images, an image capturing means for switching between a first image capturing for still image capturing and a second image capturing for live view display, each of which has an exposure time set for each, and performing image capturing; display means for sequentially displaying images based on imaging by imaging means on a display device, wherein the displayed image includes at least an image based on the second imaging; the first imaging and the second imaging; when the exposure time for the first imaging is shorter than a predetermined threshold in the case where the imaging is repeatedly performed by switching between the imaging of the first imaging, the image displayed by the display means includes the image based on the first imaging. a control means for controlling to include an image based on the first imaging in an image displayed by the display means when an exposure time for the first imaging is equal to or greater than the predetermined threshold; It is characterized by

According to the present invention, when still images are continuously shot while using the LV function, even if the exposure time of the still image and the LV image are different, it is possible to reduce the discomfort of the display in the LV function. Become.

1 is a block diagram showing a functional configuration example of a digital camera as an example of an imaging device according to Embodiment 1; FIG. 4 is a timing chart regarding LV display when continuous shooting of still images is performed in Embodiment 1. 4 is a flowchart showing a series of operations related to LV display control in Embodiment 1; Timing chart regarding LV display in the first continuous shot of still images in Embodiment 1 Timing chart regarding LV display after the second continuous shooting of still images in Embodiment 1 Timing chart regarding LV display at the end of continuous shooting of still images in Embodiment 1 Timing chart regarding LV display when performing continuous shooting of still images in Embodiment 2 Timing chart for LV display in the first continuous shot of still images in the second embodiment 10 is a flowchart showing a series of operations related to LV display control in Embodiment 2; Timing chart regarding LV display when performing continuous shooting of still images in Embodiment 3 Timing chart showing an example when the LV display interval is not changed during still image continuous shooting 14 is a flowchart showing a series of operations related to LV display control in Embodiment 3; Timing chart regarding LV display in the first continuous shot of still images in Embodiment 3 Timing chart for LV display after the second continuous shooting of still images in the third embodiment Timing chart regarding LV display at the end of continuous shooting of still images in Embodiment 3 Timing chart regarding LV display when performing continuous shooting of still images in Embodiment 4 Timing chart regarding LV display when performing continuous shooting of still images in Embodiment 5

(Embodiment 1)
Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. In addition, the following embodiments do not limit the invention according to the scope of claims. Although multiple features are described in the embodiments, not all of these multiple features are essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same or similar configurations are denoted by the same reference numerals, and redundant description is omitted.

In the following, an example using a digital camera capable of continuous shooting (continuous shooting) of still images while outputting LV images will be described as an example of an imaging device. However, the present embodiment is applicable not only to digital cameras but also to other devices capable of continuous shooting while outputting LV images. These devices may include, for example, digital video cameras, mobile phones including smartphones, game consoles, tablet terminals, medical devices, devices for in-vehicle systems, and the like.

(Configuration of digital camera)
FIG. 1 is a block diagram showing a functional configuration example of a digital camera 120 as an example of an imaging device according to this embodiment. One or more of the functional blocks shown in FIG. 1 may be implemented by hardware such as an ASIC or programmable logic array (PLA), or implemented by a programmable processor such as a CPU or MPU executing software. may It may also be implemented by a combination of software and hardware. Therefore, in the following description, even when different functional blocks are described as main entities, the same hardware can be implemented as the main entities.

The digital camera 120 includes, in addition to each unit to be described later, an AD converter that converts analog signals from the image sensor into digital signals, and a compression/decompression unit that compresses image data in JPEG or MPEG format or decompresses compressed data. Other configurations not shown, such as, may be included. It may also include an external recording unit for recording JPEG or MPEG on an external medium.

The imaging unit 100 includes an imaging element such as a CCD or CMOS sensor that converts an object image formed on an imaging surface by an imaging optical system into electrical signals and outputs image data. Image data from the imaging unit 100 is written to the temporary recording unit 107 via the data transfer unit 105 .

The image reduction unit 110 has one or more processing circuits that reduce image data for a still image to the size of an LV image. Also, the image reduction unit 110 is connected to the data transfer unit 105 , receives recorded image data captured by the imaging unit 100 , performs image processing, and transfers the processing result to the temporary storage unit 107 via the data transfer unit 105 . write out.

The recorded image processing unit 101 has one or more processing circuits for generating image data for still images. The recorded image processing unit 101 performs, for example, pixel correction, black level correction, shading correction, blemish correction, white balance adjustment, magnification chromatic aberration correction, gamma correction, luminance/color generation processing, geometric deformation, noise reduction, scaling, etc. process. Accordingly, appropriate image processing is performed on the image data. The recorded image processing unit 101 is also connected to the data transfer unit 105 , receives image data from the data transfer unit 105 , and writes the result of image processing to the temporary storage unit 107 via the data transfer unit 105 . .

The display image processing unit 109 has one or more processing circuits for generating LV images. As with the recorded image processing unit 101, it has a plurality of image processing functions. However, since it is necessary to complete the generation of the LV image during the display cycle, it is configured to perform simpler and faster image processing than the recorded image processing unit 101 . Further, the display image processing unit 109 is connected to the data transfer unit 105, receives image data from the data transfer unit 105, and outputs the result of image processing to the display unit 102 via the data transfer unit 105. . In the present embodiment, a case in which image processing units for still images and LV images are configured separately will be described as an example, but a common image processing unit may be used. In this case, for example, a configuration may be adopted in which time-division processing is performed such that still images are processed during a period in which LV images are not processed.

The display unit 102 has one or more circuits for display control. The display unit 102 is connected to the data transfer unit 105 , receives the image data processed by the display image processing unit 109 from the data transfer unit 105 , and applies a fixed color around the display image to match the size of the display device 103 . (For example, black-painted) pixels are added. It also executes a process of embedding the photographing time in the display data, a process of superimposing a so-called OSD (On-Screen Display) image, and the like. The display unit 102 executes a plurality of processes such as format conversion suitable for the display device 103 and transfers the display image to the display device 103 .

The display device 103 is, for example, an LCD (Liquid Crystal Display) or an EVF, and displays an image from the display unit 102 .

The control unit 104 has one or more processors that control the operation of the digital camera 120 . The control unit 104 expands the program recorded in the recording unit 108 into the memory of the temporary recording unit 107 and executes it, thereby issuing various instructions to each functional block constituting the digital camera 120 and performing various control processes. to run. Also, the control unit 104 controls the recorded image processing unit 101 , the display unit 102 , the data transfer unit 105 , the temporary recording unit 107 and the recording unit 108 which are connected via the bus 106 .

The data transfer unit 105 includes WRDMAC and RDDMAC (not shown) of a plurality of Direct Memory Access controllers that transfer data. The image data is output to the bus 106 by the WRDMAC and temporarily stored in the temporary recording unit 107 . The image data stored in the temporary recording unit 107 is output to the bus 106 by the RDDMAC, and the data is output to the recorded image processing unit 101 and the display unit 102 connected to the data transfer unit 105 .

The bus 106 has a system bus and a data bus (not shown), each of which has an independent bus configuration. The temporary recording unit 107 has a memory control unit and a memory (not shown), and writes data to the memory or reads data from the memory according to instructions from the control unit 104 or the data transfer unit 105 . The memory has a storage capacity sufficient to store a predetermined number of still images, moving images for a predetermined time, data such as audio, constants for the operation of the control unit 104, programs, and the like. , and is composed of, for example, a DRAM.

The recording unit 108 includes a nonvolatile memory control unit (not shown) and one or more nonvolatile memories. The nonvolatile memory control unit writes data to the nonvolatile memory and reads data from the nonvolatile memory in accordance with instructions from the control unit 104 . A nonvolatile memory is an electrically erasable/recordable memory, and for example, an EEPROM or the like is used. Constants for the operation of the control unit 104, programs, and the like are stored in the nonvolatile memory.

(Example of operation related to LV display during continuous shooting)
Next, with reference to FIG. 2, an operation example regarding LV display when performing continuous shooting of still images will be described. FIG. 2 is a timing chart showing LV display during continuous shooting of still images in which the release time is fixed and the continuous shooting interval is adjusted. The vertical direction in FIG. 2 indicates timing signals and images to be processed or output, and the horizontal direction indicates time.

First, the items arranged in the vertical direction in FIG. 2 will be described. “Imaging display V synchronization” indicates the timing of the imaging vertical synchronization signal representing the timing of reading the LV image from the imaging unit 100 . With this timing signal, for example, 60 frames of images are output from the imaging unit 100 per second.

“Still image synchronization” indicates the readout timing of still images from the imaging unit 100, and is a synchronization signal generated according to the still image shooting interval T4 or the continuous shooting interval T5. The still image shooting interval T4 is the interval from the start of shooting the first still image (still image 1) after the release to the start of shooting the second still image (still image 2). It represents the interval from the start of shooting the second and subsequent still images to the start of shooting the next still image. Timing 200 indicates the timing of photographing instruction by pressing the shutter button. In the example shown in FIG. 2, the still image synchronization signal is generated after the release time T0 from timing 200. In the example shown in FIG. The continuous shooting interval T5 will be described in detail in the description of the still image continuous shooting operation shown in FIG. Adjustments are made according to the output period (mainly exposure time).

“Sensor output image” indicates the start and end timings of reading an image from the imaging unit 100 and the sensor switching time. LV image readouts are indicated as "display 1" to "display 9", and still image readouts are indicated as "still image 1" to "still image 3". A lattice frame 201 is the switching time of the imaging unit 100 from the LV image to the still image, and a lattice frame 202 is the switching time of the imaging unit 100 from the still image to the LV image. The time difference between the LV images (displays 1 to 9) and the still images (still images 1 to 3) is due to the different number of pixels and the different exposure times. Note that hatched displays 2, 3, 6, and 9 are images in which the imaging unit 100 does not capture LV images, and indicate that they cannot be read out because they overlap with still image readout. .

“Display image processing” indicates execution timing of image processing of the display image processing unit 109 . At the timings shown in Display 1, 4, 5, 7, 8, which are images for LV images, and still images 1, 2, which are images for still images, the following processing is performed. That is, each image is read out from the imaging unit 100, recorded in the temporary recording unit 107, read out to the display image processing unit 109 via the data transfer unit 105, processed, and transferred again via the data transfer unit 105. It is recorded in the temporary recording unit 107 . Also, at the timings indicated by displays 1, 2 (still image 1), 4, 5, 6 (still image 2), 7, and 8, all of the image data for one screen from the imaging unit 100 is stored in the temporary recording unit 107. This indicates that the display image processing unit 109 has started processing before recording. Further, the data transfer unit 105 controls such that only the image data written by the imaging unit 100 is read by the recorded image processing unit 101 . That is, the display image processing unit 109 can be controlled so that the image capturing unit 100 does not read out before writing to the temporary recording unit 107 . Further, in the still images 1 and 2, the image data from the imaging unit 100 is input to the image reduction unit 110, reduced to the LV image size, and then recorded in the temporary recording unit 107. FIG. In the example shown in this timing chart, the display image processing for still images 1 and 2 is rate-determined by the sensor output image period, and the processing time is extended. If the bandwidth is not a problem for the system, the image data from the imaging unit 100 may be written to the temporary storage unit 107 and reduced to the LV image size by the display image processing unit 109 in the same way as the LV image. do not have.

“Recorded image processing” indicates execution timing of image processing of the recorded image processing unit 101 . At this timing, the still images 1 and 2 are read out from the imaging unit 100, recorded in the temporary recording unit 107, read out to the recorded image processing unit 101 via the data transfer unit 105, and processed. 105 and is recorded again in the temporary recording unit 107 . In still images 1 and 2, unlike the display image processing, most of the image data for one screen from the imaging unit 100 is recorded in the temporary recording unit 107 before being processed by the recorded image processing unit 101. is shown. For this reason, the start of the execution timing is delayed compared to the display image processing. It should be noted that if the bandwidth does not pose a problem for the system, reading of still images to the recorded image processing unit 101 may be started immediately after reading of still images from the imaging unit 100 is started in the same manner as for LV images.

"Display V synchronization" indicates the timing of the display vertical synchronization signal that starts display on the display device 103, and in this embodiment, it is a timing signal for displaying 60 frames of images per second, for example. Note that the display V synchronization is set to the shortest timing for displaying the sensor output image on the display device 103 as a system. In other words, the display V-synchronization occurs with a certain period of delay from the imaging-display V-synchronization.

“Display” indicates the timing for displaying the image processed by the display unit 102 on the display device 103 . In this example, since there are no frames corresponding to display 2 and display 3 (that is, the images of display 2 and display 3 in the "sensor output image" do not exist due to the output of still image 1), display 1 is displayed during period 206. displayed three times. Display 2 (still image 1) and display 6 (still image 2) are frames using still images as frames to be displayed. When a still image is used as a display frame during continuous shooting, displays 4, 5, 7, and 8 will be displayed one frame later than the normal LV display, but the LV display during continuous shooting will be smoothed. can do

As described above, according to the operation shown in FIG. 2, appropriate timings (display 2 (still image 1) and display 6 (display 2 (still image 1) and display 6 ( Still image 2)) allows a still image to be used for display. Then, it is possible to smoothen the display during continuous shooting of still images.

(Series of operations related to LV display control when performing continuous shooting of still images)
Next, a series of operations related to LV display control when performing continuous shooting of still images will be described with reference to FIG. This LV display control includes (i) processing for adjusting the imaging timing in still image shooting, and (ii) processing for adjusting the timing of LV display of the still image. Unless otherwise specified, the control unit 104 expands the program recorded in the recording unit 108 into the memory of the temporary recording unit 107 and executes it, or realizes it by controlling each unit of the digital camera 120. be done. Also, this processing is started when a still image capturing instruction (for example, pressing of the shutter) is received via an operation member such as a shutter button (not shown), and the first still image synchronization signal is detected. . A series of operations shown below correspond to operations for controlling displays 3 and 6 of the timing chart shown in FIG. 2, the still image shooting interval T4, and the continuous shooting interval T5.

In S<b>301 , the control unit 104 acquires subject brightness for capturing the first image. The control unit 104 can obtain subject brightness for the LV image using a known photometric technique such as using a photometric sensor (not shown). In S302, the control unit 104 adjusts the exposure time based on the obtained photometric value so that the exposure of the still image becomes the desired exposure (for example, according to preset user settings). Note that in S301 and S302, the control unit 104 calculates the subject brightness and adjusts the exposure time based on the LV image obtained before the instruction to capture a still image (for example, when the shutter is half-pressed). can be

In S303, the control unit 104 switches the driving of the imaging unit 100 from LV driving to still image driving. In the example shown in FIG. 2, the operation of this step corresponds to the operation during the period of the grid frame 201 .

In S304, the control unit 104 captures the first still image based on the exposure time obtained in S302. In S305, the control unit 104 determines whether or not the still image shooting is continuous shooting. For example, if the still image imaging instruction is a shutter button (not shown) being pressed, the control unit 104 determines that the imaging instruction is continuous shooting, and advances the process to step S306. Otherwise, it is determined that single-shot shooting is performed, and the process proceeds to S319.

In S306, the control unit 104 calculates a threshold according to the following calculation method, and determines whether or not the exposure time for still image shooting is equal to or greater than the threshold. Here, specific determination processing in this step will be described with reference to FIG. FIG. 4 shows in more detail the operation of photographing the first still image in the timing chart shown in FIG. Identical contents are provided with the same reference numerals.

In FIG. 4, the LV image output period (LV image shooting exposure time) is T2, and the still image output period (still image shooting exposure time) is T7. Furthermore, when the period for switching the driving of the imaging unit 100 from the LV image driving to the still image driving is T8, and the period for switching the driving of the imaging unit 100 from the still image driving to the LV image driving is T9, the threshold is expressed by (Equation 1).

Threshold = T1-T2-T8-T9 (Formula 1)

The control unit 104 calculates a threshold according to (Formula 1), and compares the still image output period T7 (exposure time for still image shooting) with the threshold. This comparison determines whether or not the sum of the exposure time (T7) for still image shooting, the exposure time (T2) for LV image shooting, and the drive switching time (T8 and T9) falls within the LV cycle (T1). means to judge. In other words, if the still image shooting with the exposure time (T7) is performed within the period of the LV cycle (T1), is it possible to perform the LV image shooting (in other words, is the LV image replaced with a still image? or) are considered. If it is determined that the still image output period T7 is equal to or greater than the threshold, the process proceeds to S307; otherwise, the process proceeds to S309. Note that if the still image output period T7 is less than the threshold in S306, the LV image can be obtained even if the still image is captured during T1. Therefore, the still image is not used as the LV image until a series of operations related to this processing is completed (continuous shooting is completed).

In S<b>307 , the control unit 104 determines the timing for displaying the still image on the display device 103 . In this embodiment, in order to reduce the delay time from capturing a still image to displaying an LV image using the still image, the control unit 104 displays the still image at the shortest timing of the display V synchronization signal after capturing the still image. Control to display on the display device. In the example shown in FIG. 4, the imaging of the LV image can be resumed from display 4, but the control unit 104 displays a still image (still image 1) at timing 401 of the display V synchronization signal synchronized with the imaging of display 4. It controls to display on the device 103 . The LV image generated by the imaging of display 4 is displayed on the display device 103 at the timing of the next display V synchronization signal. Therefore, the period from the start of capturing the LV image to the display of the LV image is longer than that before continuous shooting of still images. However, it is possible to reduce the difference between the period from the start of capturing a still image until the LV image is displayed and the period from the start of capturing the LV image to the display as the LV image. Movement will appear smoother. The display image processing unit 109 and the display unit 102 complete their processing by timing 401 of the display V synchronization signal. In S308, the display device 103 displays a still image (still image 1) based on the timing specified in S307 according to the instruction from the control unit 104. FIG.

In S309, the control unit 104 determines the timing of the subsequent still image capturing (that is, T4 and T5 of the subsequent still images). Here, the timing of capturing the second and subsequent still images will be described with reference to FIG. FIG. 5 shows in more detail the portion of the timing chart shown in FIG. 2 relating to the second still image. First, the time S1 of the timing signal 501 for the next imaging display V synchronization is calculated for the latest still image continuous shooting interval T4'. A known method can be used to calculate this time S1. Next, the control unit 104 calculates the center (accumulated center of gravity) T11 of the imaging period using the still image output period (exposure time) T7 according to (Equation 2), and further calculates the accumulated center of gravity according to (Equation 3). Time S2 of the still image synchronization timing signal 502 is calculated based on T11. That is, the time S2, which is the timing for starting the imaging of the still image 2, is adjusted so that the center of the still image output period is arranged based on the imaging timing of the LV image that cannot be imaged.

T11=T7/2 (Formula 2)

S2 = S1-T11 (Formula 3)

By capturing the second and subsequent still images at the timing of time S2 determined by (Equation 3), the LV image and the still image are accumulated at equal intervals. Also, the number of frames in which a still image is displayed as an LV image (that is, the number of times an LV image cannot be captured by a still image) is the minimum (one frame). Therefore, even if a still image is used as the LV image in the second and subsequent continuous shots, it is possible to realize smooth LV display with reduced display discomfort in the LV function.

In S310, the control unit 104 switches the driving of the imaging unit 100 from still image driving to LV image driving. In S311, the control unit 104 performs photometry on the LV image by a known method (similar to S301). Adjust exposure time.

In S<b>313 , the control unit 104 determines whether the LV image driving is completed. If it is determined that the LV image driving is completed, the process proceeds to S<b>314 . Otherwise, the process of S313 is repeated (waiting until the LV image drive is completed). That is, when switching the driving of the imaging unit 100 from the LV image driving to the still image driving, the control unit 104 waits until the LV image driving is completed. In this way, when the control unit 104 captures images at the still image capturing timing obtained in S309, the second and subsequent still images are captured while avoiding the capturing timing of the LV image.

In S314, the control unit 104 switches the driving of the imaging unit 100 from the LV image driving to the still image shooting driving. In S315, the control unit 104 captures a still image based on the exposure control obtained in S311. In S316, if the control unit 104 determines in S306 that the still image is to be used for LV display (the exposure time for still image shooting is equal to or greater than the threshold value), the process proceeds to S317 for LV display of the still image, Otherwise, the process proceeds to S318. In the present embodiment, the exposure time of the first still image is used to determine whether or not to use the second and subsequent still images for LV display. However, the threshold determination corresponding to S306 may be performed each time for the second and subsequent images to determine whether or not the still image is to be used as the LV image.

In S<b>317 , the display device 103 displays the second and subsequent still images according to the instruction from the control unit 104 based on the timing specified in S<b>307 . In S318, the control unit 104 determines whether continuous shooting of still images has ended. For example, if the shutter button continues to be pressed, the control unit 104 determines that continuous shooting of still images has not ended, and advances the process to S309, otherwise advances the process to S319. .

In step S<b>319 , the control unit 104 switches the driving of the imaging unit 100 from the still image driving to the LV image driving in response to completion of the continuous shooting of the still images. This switching period corresponds to the lattice frame 202 in the timing chart shown in FIG.

In S320, the control unit 104 restores the display timing of the LV image to the original timing because the continuous shooting of the still images is completed. In the example shown in FIG. 6, the control unit 104 discards the captured image of the display 7 captured at the timing of the captured display V synchronization signal 601 immediately after capturing the still image (still image 3). Then, the control is switched so that the image (display 8) captured at the timing of the next imaging display V synchronization signal 602 is displayed at the timing of display 8. FIG. Note that in the present embodiment, the display timing is switched immediately after the continuous shooting of still images is completed, but the switching may be performed after several frames have passed.

As described above, in the above-described embodiment, when the still image exposure time for continuous shooting of still images is short enough to capture the LV image within the same LV cycle, only the LV image is displayed. I made it By doing so, when still images are continuously shot while using the LV function, even if the exposure time of the still image and the LV image are different, it is possible to reduce the discomfort of the display in the LV function. .

Further, in the present embodiment, when the still image exposure time for continuous shooting of still images is so long that LV images cannot be captured within the same LV cycle, the number of LV images that cannot be captured is minimized. I adjusted the shooting timing of the still image. By doing so, the release time and the continuous shooting interval can be made constant, and the number of still images displayed in the LV can be reduced, so the LV display on the display device 103 can be smoothed. In other words, when still images are continuously shot while using the LV function, even if there is a large difference in the exposure time for shooting the still image and the LV image, it is possible to reduce the discomfort of the display in the LV function. .

(Embodiment 2)
Next, Embodiment 2 will be described. The second embodiment differs from the first embodiment in the display V synchronization of the LV display during continuous shooting of still images and the control of the display of the LV image. Therefore, the configuration of the digital camera 120 shown in FIG. 1 is the same as that of the first embodiment. Therefore, descriptions of the configurations and processes that are the same or substantially the same as those of the first embodiment will be omitted, and the points of difference will be mainly described.

(Example of operation related to LV display during continuous shooting)
FIG. 7 is a timing chart of LV display during still image continuous shooting in which the release time is fixed and the continuous shooting interval is adjusted. In FIG. 7, the same contents are denoted by the same reference numerals, and the same contents are assigned to "imaging display V synchronization", "still image synchronization", "sensor output image", "display image processing", and "recording image processing". Operation timing is the same as in FIG.

"Display V synchronization" indicates the timing of the vertical synchronization signal for starting display on the display device 103, and in this embodiment, for example, it is a timing signal for displaying 60 frames of images per second. However, in this embodiment, the “display V synchronization” is controlled to be the shortest timing for displaying the sensor output image on the display device 103 during live view shooting. In other words, control is performed so that display V synchronization occurs with a certain period of delay from imaging display V synchronization. However, in this embodiment, when timing 200 (that is, the timing at which the user presses the shutter button) occurs, the timing of the display V synchronization signal 703 that is display 2 (still image 1) is shifted to timing 702 . In subsequent display V synchronization, the V synchronization signal is output at a constant timing. A method of calculating the timing 702 and the timing 704 (the timing at which the display image processing for the still image 2 ends) will be described later.

In the "display" shown in FIG. 7, there are no frames corresponding to the display 2 and the display 3 (that is, the images of the display 2 and the display 3 in the "sensor output image" do not exist due to the output of the still image 1). Display 1 is displayed three times in the period. However, in the third frame (LV display 701), the display period is less than one frame (the display is interrupted in the middle) because the display V synchronization signal is switched. Note that the second frame and the third frame using the display 1 are non-updating periods of the display image, so even if the third frame is discontinued, the display will not be disturbed. Display 2 (still image 1) and display 6 (still image 2) are frames using still images as frames to be displayed. Even if a still image is used as a display frame during still image continuous shooting, displays 4, 5, 7, and 8 are displayed with the shortest display delay compared to the normal LV display, and the LV during still image continuous shooting is displayed. It is possible to smooth the display. That is, while reducing the number of frames of still images displayed instead of the LV image as in Embodiment 1, the display delay of the still image (still image 1) is further reduced (or the display time of display 1 and each LV display display delay) can be reduced.

(Series of operations related to LV display control when performing continuous shooting of still images)
Next, a series of operations related to LV display control when performing continuous shooting of still images in this embodiment will be described. As in the above-described embodiments, processing for adjusting imaging timing in still image shooting and processing for adjusting LV display timing of the still image are included. Unless otherwise specified, the control unit 104 expands the program recorded in the recording unit 108 into the memory of the temporary recording unit 107 and executes it, or is realized by controlling each unit of the digital camera 120. be. Also, this processing is started when a still image capturing instruction (for example, pressing of the shutter) is received via an operation member such as a shutter button (not shown), and the first still image synchronization signal is detected. . A series of operations shown below correspond to operations for controlling displays 2 and 6 of the timing chart shown in FIG. 2, the still image shooting interval T4, and the continuous shooting interval T5. In the following description, of the timing chart shown in FIG. 7, display 3, display 6, still image shooting interval T4, and continuous shooting interval T5 are controlled. Only differences from the first embodiment will be described below with reference to the flowchart shown in FIG.

First, the control unit 104 executes the processes of S301 to S306 in the same manner as in the first embodiment, and then, in S901, the control unit 104 determines the timing for displaying the still image on the display device 103. FIG. Here, the display timing determined by the control unit 104 in S901 will be described with reference to FIG. First, the control unit 104 preliminarily calculates the imaging timing (S2) of the still image 2, which is determined in S309 in the first embodiment, according to (Equation 2) and (Equation 3) in this step. Then, according to (Equation 4), the shortest display timing S3 of the display 6 is calculated from the imaging timing (S2) of the still image 2 and T12 (the period from the start of outputting the sensor output image to the completion of the display image processing). . That is, for the second and subsequent still images (still image 2), this means first calculating the timing from the start of still image capturing until display becomes possible. Note that the value of T12 is, for example, recorded in a table or the like in association with the exposure time that can be set for still image shooting and the time of T12 for each exposure time, and may be obtained by referring to

S3 = S2 + T12 (Formula 4)

Next, assuming that the output completion time of the sensor output image of the still image 1 is S4, the time S5 of the display synchronization V signal 702 may be the time that satisfies (Equation 5) and (Equation 6).

S5 = S3 - N x T1 (Formula 5)

S4≦S5 (Formula 6)

Here, the control unit 104 calculates the coefficient N that makes S4 to S5 the shortest. In the example shown in FIG. 8, the display delay is the shortest at N=3, and the time is S4.

The control unit 104 further executes S308 to S320 including S902, and then ends this series of operations. Note that in S902, the control unit 104 may determine the still image shooting timing using S2 already calculated in S901.

As described above, in the present embodiment, when the still image exposure time for continuous shooting of still images is long enough that LV images cannot be captured within the same LV cycle, the number of LV images that cannot be captured is I adjusted the shooting timing of the still image so as to minimize it. In addition, while considering the timing of displaying the second and subsequent still images, the timing is changed so as to reduce the delay in displaying the still images. By doing so, the release time and the continuous shooting interval are made constant, the number of still images displayed on the LV is reduced, and the display delay of the images is reduced, so that the LV display on the display device 103 can be smoothed. can be done.

(Embodiment 3)
Next, Embodiment 3 will be described. The third embodiment differs from the above-described embodiments in that the display cycle of the LV images in the LV display during continuous shooting of still images is changed. Therefore, the configuration of the digital camera 120 shown in FIG. 1 is the same or substantially the same as the embodiment described above. Descriptions of configurations and processes that are the same or substantially the same as those of the above-described embodiment will be omitted, and differences will be mainly described.

The display unit 102 in this embodiment has a configuration for changing the display cycle of the LV images. During the continuous shooting period of still images, the cycle (also referred to as LV display cycle) at which the display unit 102 transfers display images to the display device 103 is changed according to the exposure time of the still images. Control of the LV display cycle will be described later.

(Example of operation related to LV display during continuous shooting)
FIG. 10 is a timing chart of LV display during still image continuous shooting in which the release time is fixed and the continuous shooting interval and LV display cycle are adjusted. In FIG. 10, the same reference numerals are given to the same contents as in FIG.

In the "imaging display V synchronization" in FIG. 10, the LV display cycle T1 of the imaging display V synchronization before the start of still image continuous shooting is, for example, about 16 ms at 60 fps (60 frames of images are output per second). Become. On the other hand, in order to improve the smoothness of the live view display, the period T3 of the imaging display V synchronization during continuous shooting of still images is changed to a value different from T1 according to the exposure time of the still images. A method for calculating T3 will be described in a series of operations related to LV display control, which will be described later.

In the example of this embodiment, the "sensor output image" indicates the output timing of the LV image as display 1 to display 10, and the output timing of the still image as still image 1 to still image 3. FIG. Note that hatched displays 2, 3, 4, 6, 8, and 10 are images that are not captured for LV images by the imaging unit 100, and cannot be read out because they overlap with still image readout. It is shown that.

In the “display image processing”, the above-described display image processing is performed on Display 1, 5, 7, and 11, which are images for LV images, and Still Images 1, 2, 3, and 4, which are images for still images. It indicates that In this timing chart, the display image processing for still images 1, 2, 3, and 4 is rate-determined by the sensor output image period, and the processing time is extended. "Recorded image processing" indicates that still images 1, 2, 3, and 4 are subjected to the above-described recorded image processing.

In the "display V synchronization", the cycle T3 of the display V synchronization during still image continuous shooting is changed in accordance with the exposure time of the still image (at the issuance interval of the display V synchronization before Yes) indicates that the interval is changed to be different from T1.

By changing the display V synchronization interval (LV display cycle) as in the present embodiment, variations in the frame interval of LV image display can be reduced and LV image display can be smoothed. On the contrary, FIG. 11 shows a timing chart when the LV display interval is not changed during continuous shooting of still images, as an example of variation in the frame interval of LV image display. As in FIG. 10, whether or not still images 1 to 4 are used as LV images is determined according to the exposure time of the still images, but the LV display cycle is T1 even during continuous shooting. Focusing on "display", when displaying the LV image (for example, display 1 or display 5), it switches in one frame, whereas when displaying a still image (for example, still image 1), the same image is displayed in two frames. Since the frames are displayed in succession, the smoothness of the display is reduced.

In the "display" shown in FIG. 10, since the subsequent frames (that is, display 2 and display 3 in the sensor output image) do not exist due to the shooting of the still image, display 1 is displayed four times before the still image 1 is displayed. ing. Display 2 (still image 1), display 6 (still image 2), and display 8 (still image 3) are frames for displaying still images. When a still image is used as a display frame during still image continuous shooting, displays 5 and 7 will be displayed one frame later than the normal LV display, but the display time of the still image as a display frame is shortened, smooth LV display can be achieved during still image continuous shooting. As described above, in this embodiment, the display timings of still images 1 and 2, the still image shooting interval T4, and the continuous shooting interval T5 are adjusted by a series of operations described later. As a result, still images are used for display at appropriate timings (display 3, display 6, and display 8 in the sensor output image), and the LV display cycle (T3) during continuous shooting is adjusted.

(Series of operations related to LV display control when performing continuous shooting of still images)
A series of operations related to LV display control when continuous shooting of still images is performed according to the present embodiment will be described with reference to FIG. 12 . As in the first embodiment, unless otherwise specified, the control unit 104 expands the program recorded in the recording unit 108 into the memory of the temporary recording unit 107 and executes it, or the digital camera 120 It is realized by controlling each part of Also, this processing is started when a still image capturing instruction (for example, pressing of the shutter) is received via an operation member such as a shutter button (not shown), and the first still image synchronization signal is detected. . In this process, the displays 3 and 6 shown in FIG. 10, the still image shooting interval T4, the continuous shooting interval T5, and the LV display cycle T3 during continuous shooting are determined.

First, the control unit 104 executes the processes of S301 to S305 in the same manner as in the first embodiment. FIG. 13 shows in more detail the operation related to the first still image shooting in the timing chart shown in FIG. 10, and the same contents are given the same reference numerals.

In S<b>306 , the control unit 104 calculates the threshold according to the above-described (Equation 1), and determines whether or not the exposure time for still image shooting is equal to or greater than the calculated threshold. The control unit 104 advances the process to S1202 when determining that the threshold value is exceeded, and otherwise advances the process to S1202.

In step S<b>1201 , the control unit 104 determines the timing for displaying still images on the display device 103 and the LV display cycle during still image continuous shooting. Here, a still image (still image 1) is displayed at the timing of the shortest display V synchronization signal after still image shooting. In the example shown in FIG. 13, the display of the LV image can be restarted from the shooting timing of display 5 in the "sensor output image". However, the control unit 104 causes the display device 103 to display a still image (still image 1) at the timing 401 of the display V synchronization signal synchronized with the imaging of the display 5. FIG. The display image processing unit 109 and the display unit 102 complete each processing by the timing of this display V synchronization signal. Furthermore, the control unit 104 determines the LV display period T3 during continuous shooting of still images according to (Equation 7).

T3 = (T2+T7+T8+T9)/2 (Formula 7)

For example, the exposure time (T2) for LV image shooting is 2 ms, the exposure time (T7) for still image shooting is 32 ms, and the switching time (T8) from LV imaging to still image shooting and the time from still image shooting to LV imaging are The switching time (T9) is 1 ms, and the number of continuous shooting frames per second is 15. In this case, T3 is 18 ms. Further, the control unit 104 executes S308 in the same manner as in the above embodiment, and displays the first still image (still image 1) on the display device 103. FIG.

In S<b>1202 , the control unit 104 determines the timing of capturing the second and subsequent still images (for example, still image 2 and still image 3 ). FIG. 14 shows in more detail the operation of the second still image (still image 2) in the timing chart shown in FIG. ing.

The control unit 104 first determines the still image shooting interval T4. Specifically, the control unit 104 first calculates the time S1 of the timing signal 1401 of the imaging display V synchronization one before the shortest still image shooting interval indicated by T4' in FIG. A known method can be used to calculate this time S1. The control unit 104 calculates the accumulation center of gravity T11 using the still image exposure time T7 according to the above-described (Equation 2), and further calculates the time S2 of the still image synchronization timing signal 1402 according to (Equation 8). . That is, the time S2, which is the timing to start capturing the still image 2, is adjusted so that the still image output period is arranged based on the timing (time of S1+T3) after the LV display cycle T3 after adjustment has elapsed. .

T11=T7/2 (Formula 2)
S2=S1+T3-T11 (Formula 8)

The control unit 104 further executes S310 to S320, and then terminates this series of operations. Note that in S320, the control unit 104 returns the display cycle of the LV images from T3 to T1 after the end of continuous shooting. In the example shown in FIG. 15, the control unit 104 discards the captured image of display 9 captured at the timing of the captured display V synchronization signal 1501 immediately after capturing the still image (display 8 (still image 3)). Then, the control is switched so that the image of the display 10 imaged at the timing of the next image pickup display V synchronization signal 1502 is displayed during the period of the display 10 in the "display". The control unit 104 causes the LV display cycle to return to T1 after the imaged display V synchronization signal 1502, and displays the imaged display 11 during the display 11 period of "display" without delay. In this embodiment, the display timing is switched at 602 immediately after the continuous shooting of still images is completed, but the switching may be performed after several frames.

As described above, in this embodiment, during still image continuous shooting, the display V synchronization interval (LV display cycle) is changed to the LV display cycle T3 (for example, T2+T7+T8+T9) according to the still image exposure time T7. made it By doing so, the frame interval of the LV image display becomes the LV display period T3, and the variation in the frame interval is reduced, so that the LV image can be displayed smoothly.

(Embodiment 4)
Furthermore, Embodiment 4 will be described. In the fourth embodiment, not only the LV display cycle but also the LV image exposure time is adjusted according to the still image exposure time during still image continuous shooting, thereby obtaining a smoother LV display. It differs from the third embodiment. Therefore, the configuration of the digital camera 120 shown in FIG. 1 is the same or substantially the same as the embodiment described above. Descriptions of configurations and processes that are the same or substantially the same as those of the above-described embodiment will be omitted, and differences will be mainly described.

In the present embodiment, as shown in FIG. 16, the control unit 104 adjusts the exposure time (T2′) of the LV image during continuous shooting of still images to be equal to the exposure time (T7) of the still image. Further, along with the change in the exposure time of the LV image, the LV display cycle during continuous shooting of still images is changed to the display cycle (T3') calculated according to (Equation 9).

T3' = (T2' + T7 + T8 + T9)/2 (Formula 9)

In the example shown in FIG. 16, the exposure time of the LV image in the "sensor output image" is changed from T2 before continuous still image shooting to exposure time T2' equal to still image exposure time T7 during continuous still image shooting. . Further, the LV display period T3 is also changed to T3' calculated from the above equation (9) in accordance with the change in the exposure time of the LV image.

As described above, in the present embodiment, during continuous still image shooting, the exposure time of the LV image is adjusted according to the still image exposure time (for example, so that the exposure time of the LV image approaches or matches the still image exposure time T7). was adjusted. As a result, although the LV display cycle is longer than in the case of the third embodiment (eg, FIG. 10), unevenness in exposure and noise between frames due to the difference in exposure time between the LV image and the still image is reduced, resulting in a smoother image. LV display becomes possible.

(Embodiment 5)
Furthermore, Embodiment 5 will be described. In the fifth embodiment, LV image display when the still image exposure time (T7) is longer than the imaging display V synchronization (T1) in the operation of the fourth embodiment will be described. Therefore, the configuration of the digital camera 120 shown in FIG. 1 is the same or substantially the same as the embodiment described above. Descriptions of configurations and processes that are the same or substantially the same as those of the above-described embodiment will be omitted, and differences will be mainly described.

In this embodiment, as shown in FIG. 17, for example, the imaging display V synchronization (T1) before continuous shooting of still images is set to 16 ms, and the still image exposure time (T7) is set to 80 ms. In this way, when the still image exposure time is longer than the imaging display V synchronization (T1) before continuous still image shooting, the LV image capturing is interposed between the exposure times of the two still images. The number of shots per second may decrease.

Therefore, in this embodiment, in such a case, that is, when the control unit 104 determines that the conditional expression according to (Equation 10) is satisfied, the LV image is not captured, and only the still image is used as the LV display image. use.

T7>Tc (equation 10)

Here, Tc is a predetermined fixed value. For example, an integral multiple value such as twice or three times the LV image V synchronization (T1) before still image continuous shooting may be set. At this time, as in the third and fourth embodiments, the LV display cycle T3 is equal to the still image exposure time (T7).

As described above, in this embodiment, in addition to the fourth embodiment, when the exposure time (T7) of the still image is longer than the period indicated by the predetermined threshold value, the LV image is not captured. Only still images are used as LV display images. By doing so, it is possible to prevent LV display blackout (due to displaying an LV image that has not been shot) and to continue LV display during still image continuous shooting.

(Other embodiments)
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in the computer of the system or apparatus reads and executes the program. It can also be realized by processing to It can also be implemented by a circuit (for example, ASIC) that implements one or more functions.

The invention is not limited to the embodiments described above, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the claims are appended to make public the scope of the invention.

DESCRIPTION OF SYMBOLS 100... Imaging part 102... Display part 103... Display device 104... Control part

Claims (13)

imaging means for performing imaging by switching between first imaging for still image imaging and second imaging for live view display, each of which has an exposure time set for each exposure time, when imaging is repeatedly performed;
display means for sequentially displaying images based on imaging by said imaging means on a display device, wherein said images to be displayed include at least images based on said second imaging;
When the repeated imaging is performed by switching between the first imaging and the second imaging, if the exposure time for the first imaging is shorter than a predetermined threshold value, the When the image based on the first imaging is not included and the exposure time for the first imaging is equal to or greater than the predetermined threshold value, the image based on the first imaging is included in the image displayed by the display means. and a control means for controlling the image pickup apparatus. 2. The imaging apparatus according to claim 1, wherein said predetermined threshold is a threshold for determining whether said first imaging and said second imaging are possible within a predetermined period. . The predetermined threshold value includes a cycle of displaying an image based on an image captured by the image capturing means on the display device, a time for switching between the first image capturing and the second image capturing, and an exposure time for the second image capturing. 3. The image pickup apparatus according to claim 2, wherein the image pickup apparatus is obtained based on: When an image based on the first imaging is included in an image to be displayed by the display means, the control means controls the imaging means to reduce the number of times the second imaging cannot be performed due to the first imaging. 4. The imaging apparatus according to any one of claims 1 to 3, wherein timing of said first imaging is controlled. The control means controls the timing of the first imaging such that the center of the imaging period of the first imaging is positioned at the timing of the second imaging in which imaging cannot be performed. 5. The imaging device according to claim 4. The control means performs the first imaging when the first imaging immediately after the imaging instruction by the user overlaps with the second imaging, and controls the first imaging and the 6. The image pickup apparatus according to claim 4, wherein when the second image pickup overlaps with the second image pickup, control is performed so as to perform the second image pickup. When the image displayed by the display means includes the image based on the first imaging, the control means can display the image based on the first imaging on the display means after the first imaging is started. 7. The imaging apparatus according to any one of claims 4 to 6, wherein control is performed so as to change the timing of displaying the image based on the first imaging based on the period until . When an image based on the first imaging is included in the image displayed by the display means, the control means controls that after performing the first imaging, the second imaging is compared with before performing the first imaging. 8. Control is performed so as to lengthen a period from the start of imaging until the image based on the second imaging can be displayed on the display means. The imaging device according to . When the image to be displayed by the display means includes the image based on the first imaging, the control means displays the image based on the imaging by the imaging means on the display device according to the exposure time for the first imaging. 8. The imaging apparatus according to any one of claims 4 to 7, wherein the display cycle is controlled. When an image based on the first imaging is included in an image displayed by the display means, the control means controls the exposure time of the second imaging so that the exposure time of the second imaging approaches the exposure time of the first imaging. 10. The imaging apparatus according to claim 9, wherein the exposure time for imaging of is controlled. When the image displayed by the display means includes an image based on the first imaging, and when the exposure time for the first imaging is longer than a second threshold larger than the predetermined threshold, the control means 11. The imaging apparatus according to claim 9, wherein the image displayed by the display means does not include the image based on the second imaging. imaging means for performing imaging by switching between first imaging for still image shooting and second imaging for live view display, each of which has an exposure time set for each, when imaging is repeatedly performed; display means for sequentially displaying images based on imaging by a display device , wherein the images to be displayed include at least the images based on the second imaging ,
When the control means performs the repeated imaging by switching between the first imaging and the second imaging, if the exposure time for the first imaging is shorter than a predetermined threshold value, the display means displays When the image to be displayed does not include the image based on the first imaging and the exposure time for the first imaging is equal to or greater than the predetermined threshold, the image to be displayed by the display means is based on the first imaging. A control method for an imaging device, comprising: a control step for controlling to include an image. A program for causing a computer to function as control means for the imaging apparatus according to any one of claims 1 to 11.

Images