JP7170452B2 - Imaging device and its control method - Google Patents

Description

The present invention relates to an imaging device and its control method.

Improvements in the performance of imaging devices have made it possible to shoot at a frame rate higher than 30 fps, which is the frame rate of general moving images. For example, in order to obtain contrast AF evaluation values at high speed, imaging apparatuses are known that shoot moving images at a high frame rate such as 60 fps or 120 fps.

On the other hand, the higher the frame rate, the smaller the difference between frames. Therefore, when obtaining an evaluation value based on a difference between frames such as a motion vector, there is a problem that the processing for obtaining the evaluation value increases and it becomes difficult to obtain a significant evaluation value.

JP-A-2005-311699

In Patent Document 1, movement between frames is detected based on the inter-frame difference of the through image, and when the movement is small, the through image shooting frame rate is reduced to less than 30 fps, thereby improving the visibility of the through image. A configuration for reducing power consumption while maintaining is disclosed. However, if the shooting frame rate is lowered, like the contrast AF evaluation value described above, the acquisition interval of evaluation values acquired from individual frames will be extended, and the advantage of high frame rate shooting will be lost.

SUMMARY OF THE INVENTION An object of the present invention is to provide an imaging apparatus capable of appropriately obtaining an evaluation value based on a difference between frames while maintaining a shooting frame rate, and a control method thereof.

The above objects are provided by acquisition means for acquiring a motion picture having a first frame rate, image processing means for applying image processing to the motion picture having the first frame rate, and motion picture having the first frame rate. thinning means for thinning and outputting constituent frames according to a thinning amount; and detection means for detecting a difference amount between a plurality of frames output by the thinning means, wherein the thinning means detects the difference amount when the difference amount is less than a threshold value. This is achieved by an imaging device characterized by increasing the thinning amount.

ADVANTAGE OF THE INVENTION According to this invention, the imaging device which can acquire the evaluation value based on the difference between frames appropriately, and its control method can be provided, maintaining a imaging|photography frame rate.

1 is a block diagram showing a functional configuration example of a digital camera according to a first embodiment; FIG. Flowchart relating to evaluation value generation operation of the digital camera according to the first embodiment Schematic diagram showing control of thinning amount in the first embodiment Block diagram showing an example of the functional configuration of a digital camera according to the second embodiment. Flowchart relating to evaluation value generation operation of the digital camera according to the second embodiment Schematic diagram showing control of thinning amount in the second embodiment Block diagram showing a functional configuration example of a digital camera according to a third embodiment Flowchart relating to evaluation value generation operation of the digital camera according to the third embodiment Schematic diagram showing control of thinning amount in the third embodiment

The invention will now be described in detail on the basis of its exemplary embodiments with reference to the accompanying drawings. It should be noted that the described embodiments are merely examples and do not limit the scope of the present invention. For example, an embodiment in which the present invention is applied to a digital camera will be described below. However, a digital camera is just one example of equipment to which the present invention can be applied. The present invention can be implemented in any electronic device capable of shooting moving images. Such electronic devices include, but are not limited to, imaging devices such as digital cameras and digital video cameras, as well as personal computers, tablet terminals, mobile phones, game machines, drive recorders, robots, and drones.

● (first embodiment)
FIG. 1 is a block diagram showing a functional configuration example of a digital camera 100 as an imaging device according to the first embodiment. A lens 101 is an imaging optical system and includes an aperture and a focus lens. Lens 101 may be replaceable. The image sensor 102 has a large number of pixels arranged two-dimensionally, and an optical image formed on the imaging surface by the lens 101 is converted into an analog image signal by converting it into an electric signal by a photoelectric conversion unit of each pixel. do.

An A/D converter 103 converts an analog image signal output from the image sensor 102 into a digital image signal (image data) 104 . The control unit 115 has one or more programmable processors, for example, reads a program stored in the ROM 118 into the RAM 117 and executes it, and realizes the functions of the digital camera 100 by controlling each functional block. Although omitted in FIG. 1 for simplification of the drawing, the control unit 115 is connected to all functional blocks.

The RAM 117 is used as a system memory, video memory, buffer memory, etc. for executing programs.
The ROM 118 stores programs executable by the programmable processor of the control unit 115, various setting values, unique information of the digital camera 100, GUI data, and the like. The ROM 118 may be electrically rewritable.

A thinning adjustment unit 105 controls the thinning amount of the image data 104 output from the A/D converter 103 . Specifically, when the image sensor 102 is shooting a moving image at a predetermined shooting frame rate, the thinning adjustment unit 105 thins (discards) some frames as necessary and outputs the remaining frames. . The thinning adjustment unit 105 can also be said to be an extraction unit that extracts and outputs some frames from the image data 104 output from the A/D converter 103 as necessary. The thinning adjustment unit 105 outputs frames to the memory 116 and the evaluation value generation unit 106 . The memory 116 holds one frame of image data most recently output by the thinning adjustment unit 105 .

Evaluation value generator 106 generates an evaluation value based on comparison between frame 109 received directly from thinning adjustment unit 105 and past frames obtained from memory 116 . Here, the latest frame output by the thinning adjustment unit 105 is assumed to be the current frame, and the frame output by the thinning adjustment unit 105 immediately before the current frame is assumed to be the previous frame. The evaluation value generation unit 106 acquires the current frame from the thinning adjustment unit 105 and the previous frame from the memory 116, and generates an evaluation value based on comparison between the current frame and the previous frame.

The evaluation value generated by the evaluation value generation unit 106 is not particularly limited as long as it is an evaluation value based on a difference between frames. Here, motion vectors between frames are detected and used as the evaluation value. Other evaluation values such as the amount of difference between frames may be used. A motion vector can be detected using any known technique, but in this embodiment, template matching is used as an example to detect a motion vector.

That is, the evaluation value generation unit 106 divides the image of the previous frame horizontally and vertically to generate a plurality of partial images. Then, the evaluation value generating unit 106 performs template matching using each partial image as a template and the current frame as a reference image, and searches for an area having the highest degree of similarity with the partial image within the reference image. The evaluation value generation unit 106 sets a vector whose starting point is the coordinates of the central point of the partial image and whose ending point is the coordinates of the central point of the area found by the search that has the highest degree of similarity to the partial image as a motion vector for the partial image. Detect as When there is no motion of the partial image between frames, the start point and end point of the detected motion vector are the same. In this manner, the evaluation value generator 106 detects motion vectors for each partial image. Note that the method of generating a partial image is just an example, and for example, a characteristic region such as a face region may be extracted and used as a partial image (template).

The evaluation value generation unit 106 generates information for each detected motion vector (for example, start point and end point coordinates and detection reliability) as an evaluation value, and outputs the information to the control unit 115, the thinning adjustment unit 105, and the tracking processing unit 114. do. Note that the detection reliability may be, for example, the degree of similarity in template matching, but may be another known reliability index.

The control unit 115 may perform some control according to the evaluation value generated by the evaluation value generation unit 106 (depending on the magnitude of the motion vector). For example, the control unit 115 calculates representative vectors from the detected motion vectors according to predetermined conditions. Then, for example, when the magnitude of the representative vector is less than the threshold, the control unit 115 reduces the frame rate of the through image displayed on the display unit 112 from the normal frame rate. /or the display 112 can be controlled.

The thinning adjustment unit 105 determines (updates) the thinning amount of the image data 104 based on the evaluation value generated by the evaluation value generation unit 106 . The thinning amount may be the number of consecutive frames that are not output (the number of frames existing between adjacent output frames). For example, a thinning amount of 3 means that three consecutive frames are not output (thinning) and the fourth frame is output. Details of the operation of the thinning adjustment unit 105 will be described later.

A recording image processing unit 107 applies image processing for recording to the image data 104 output from the A/D converter 103 . Specifically, the recording image processing unit 107 performs processing (encoding processing, data file generation processing, etc.) to generate an image data file. Note that the image processing unit for recording 107 may generate the image data for recording in consideration of the information on the subject area notified from the tracking processing unit 114 . The recording image processing unit 107 stores the generated image data file in the RAM 117 . The recording unit 111 reads the image data file from the RAM 117 and records it in a recording medium such as a memory card.

A display image processing unit 108 applies image processing for display to the image data 104 output from the A/D converter 103 to generate display image data. Specifically, the display image processing unit 108 performs processing (scaling processing, etc.) according to display on the display unit 112 in addition to so-called development processing such as white balance adjustment, color interpolation, and gradation conversion on the image data 104 . is applied to generate display image data. Note that the display image processing unit 108 may generate display image data in consideration of the subject region information notified from the tracking processing unit 114 . The display image processing unit 108 stores the display image data in the VRAM area of the RAM 117 . The display unit 112 D/A-converts display image data read from the RAM 117 and displays the data on a display device such as an LCD.

Although the recording image processing unit 107 and the display image processing unit 108 are shown as separate functional blocks in FIG. 1, one image processing unit is configured to generate recording image data and display image data. You may Also, image processing commonly applied by the recording image processing unit 107 and the display image processing unit 108 is executed by one image processing unit, and processing specific to generation of recording or display image data is executed by the other image processing unit. It may be configured so that the image processing section performs subsequent processing.

At least one of the recording image processing unit 107 and the display image processing unit 108 generates an AF evaluation value used for automatic focus detection (AF) and an AE evaluation value used for automatic exposure control (AE) from image data. It can be output to the control unit 115 . The AF evaluation value may be an evaluation value for contrast AF or an evaluation value for phase difference detection AF. The control unit 115 uses the AF evaluation value to drive the focus lens of the lens unit 101 to perform focus adjustment. Further, the control unit 115 determines exposure conditions using the AE evaluation value, and controls the drive timing of the image sensor 102, the opening amount of the aperture of the lens unit 101, and the like.

Further, by generating an AF evaluation value and an AE evaluation value for the subject area notified from the tracking processing unit 114, the control unit 115 can adjust the focus so that the subject area is in focus, and the subject area can be properly exposed. Exposure conditions can be determined so that

The subject detection unit 113 detects a predetermined subject area from the recording image data. The subject area may be, for example, a characteristic area such as a person's face, or an image area designated by the user. The subject detection unit 113 outputs information such as the position and size of the detected subject area as a detection result.

The tracking processing unit 114 estimates and records the position of the subject detected by the subject detection unit 113 in the image based on the detection result of the subject detection unit 113 and the evaluation value (motion vector) generated by the evaluation value generation unit 106 . The image processing unit 107 for display and the image processing unit 108 for display are notified. Note that the tracking processing unit 114 is an example of a configuration that uses the evaluation value generated by the evaluation value generation unit 106, and the evaluation value may be used for any other processing.

Note that the thinning adjustment unit 105, the evaluation value generation unit 106, the subject detection unit 113, and the tracking processing unit 114 can be realized by hardware circuits such as FPGA and ASIC. Alternatively, it may be implemented by a programmable processor reading a program stored in ROM into RAM and executing it. Substantially, the microprocessor that constitutes control unit 115 may also implement the functions of these blocks.

Next, the evaluation value generation operation of the digital camera 100 during moving image shooting will be described using the flowchart shown in FIG. Moving image shooting may be, for example, for generating a through image for causing the display unit 112 to function as an electronic viewfinder (EVF) during shooting standby, or may be for moving image recording. Here, it is assumed that the moving image shooting frame rate is higher than the general frame rate of 30 fps, such as 60 fps, 120 fps, or 240 fps. Also, the shooting frame rate of moving images does not depend on the thinning amount.

Note that other operations (driving control of the lens unit 101 and the image sensor 102, operation control of the recording image processing unit 107 and/or the display image processing unit 108, recording unit 111 and/or display unit 112) performed during moving image shooting operation control, etc.) will be omitted. These operations may be similar to those performed in common digital cameras. In addition, control of the display frame rate based on the motion vector in the control unit 115 and tracking processing of the subject by the subject detection unit 113 and the tracking processing unit 114 at the time of moving image shooting for recording are also executed.

In S<b>201 , the thinning adjustment unit 105 acquires one frame of moving image data (hereinafter simply referred to as a frame) from the A/D converter 103 .

In S202, the thinning adjustment unit 105 determines whether or not the frame acquired in S201 is a thinning target frame (that is, a frame to be discarded). Details of the determination process will be described later. If the frame acquired in S201 is determined to be a frame to be thinned out, the thinning adjustment unit 105 returns the process to S201 without outputting the frame, and acquires the next frame. On the other hand, if the frame acquired in S201 is not determined to be a frame to be thinned out, the thinning adjustment unit 105 outputs the frame acquired in S201 and advances the process to S203.

In S203, the evaluation value generation unit 106 generates and outputs evaluation values as described above.
In S<b>204 , the thinning adjustment unit 105 determines (updates) the thinning amount or the frame to be output next based on the evaluation value supplied from the evaluation value generation unit 106 . Details of the operation of the thinning adjustment unit 105 in S204 will be described later.

In S205, the thinning adjustment unit 105 determines whether or not the moving image shooting has been completed.

Next, operations of the thinning adjustment unit 105 in S202 and S204 will be described.
The thinning adjustment unit 105 determines the frame thinning amount according to the evaluation value (here, motion vector) generated by the evaluation value generation unit 106 and adjusts the frame rate supplied to the evaluation value generation unit 106 . As an example in this embodiment, the thinning adjustment unit 105 determines the thinning amount according to the number of motion vectors whose magnitude exceeds the threshold.

As a specific example, the thinning adjustment unit 105 determines the thinning amount as follows.
・If the number of motion vectors whose magnitude exceeds the threshold is 0, increase the thinning amount. ・If the number of motion vectors whose magnitude exceeds the threshold increases from the previous time, change the thinning amount in the same direction as last time. If the number of motion vectors whose magnitude exceeds the threshold decreases from the previous time, change the thinning amount in the opposite direction from the previous time.

Note that the amount of change in the amount of thinning per thinning is determined in advance. It is set to 1 here. An increase in the thinning amount corresponds to a decrease in the output frame rate to the evaluation value generation unit 106 . When the amount of difference between frames is used as the evaluation value, if the amount of difference is less than the threshold, the amount of thinning is increased. It can be carried out.

A specific description will be given with reference to the schematic diagram shown in FIG. The initial value of the thinning amount is 0, and thinning adjustment section 105 outputs all frames output from A/D converter 103 . Also, although the threshold is set to 0 here, the threshold may be greater than 0 as long as it can be detected that the magnitude of motion is insufficient.

The evaluation value generation unit 106 acquires the second frame 302 from the thinning adjustment unit 105 and the first frame 301 from the memory 116, and detects motion vectors. Then, the evaluation value generation unit 106 outputs information on the detected motion vector. Here, it is assumed that the magnitudes of the motion vectors detected from the first frame and the second frame are both zero.

The thinning adjustment unit 105 calculates the magnitude of each motion vector from the motion vector information supplied as the evaluation value. Then, since the number of motion vectors whose magnitude exceeds the threshold value (here, 0) is 0, the thinning adjustment unit 105 increases the thinning amount by 1 to 1. Therefore, the thinning adjustment unit 105 outputs the fourth frame 304 without outputting the third frame.

The evaluation value generation unit 106 acquires the fourth frame 304 from the thinning adjustment unit 105 and the second frame 302 from the memory 116, and detects motion vectors. Then, the evaluation value generation unit 106 outputs information on the detected motion vector.

The thinning adjustment unit 105 calculates the magnitude of each motion vector from the motion vector information supplied as the evaluation value. As a result, the number of motion vectors whose magnitude exceeds the threshold is 20, which is an increase from the previous time. do. Therefore, the thinning adjustment unit 105 outputs the seventh frame 307 without outputting the fifth and sixth frames.

The evaluation value generation unit 106 acquires the seventh frame 307 from the thinning adjustment unit 105 and the fourth frame 304 from the memory 116, and detects motion vectors. Then, the evaluation value generation unit 106 outputs information on the detected motion vector.

The thinning adjustment unit 105 calculates the magnitude of each motion vector from the motion vector information supplied as the evaluation value. As a result, the number of motion vectors whose magnitude exceeds the threshold is 40, which is an increase from the previous time. do. Therefore, the thinning adjustment unit 105 outputs the 11th frame 311 without outputting the 8th, 9th, and 10th frames.

The evaluation value generation unit 106 acquires the 11th frame 311 from the thinning adjustment unit 105 and the 7th frame 307 from the memory 116, and detects motion vectors. Then, the evaluation value generation unit 106 outputs information on the detected motion vector.

The thinning adjustment unit 105 calculates the magnitude of each motion vector from the motion vector information supplied as the evaluation value. As a result, the number of motion vectors whose magnitude exceeds the threshold is 30, which is less than the previous time. and Therefore, the thinning adjustment unit 105 outputs the 14th frame 314 without outputting the 12th and 13th frames.

The evaluation value generation unit 106 acquires the 14th frame 314 from the thinning adjustment unit 105 and the 11th frame 311 from the memory 116, and detects motion vectors. Then, the evaluation value generation unit 106 outputs information on the detected motion vector.

The thinning adjustment unit 105 calculates the magnitude of each motion vector from the motion vector information supplied as the evaluation value. As a result, the number of motion vectors whose magnitude exceeds the threshold is 50, which is an increase from the previous time. do. Therefore, the thinning adjustment unit 105 outputs the 16th frame 316 without outputting the 15th frame.

The thinning adjustment unit 105 sequentially determines (updates) the thinning amount in the same manner thereafter. If the number of motion vectors whose magnitude exceeds the threshold increases, it is considered that the motion is properly detected. On the other hand, if the number of motion vectors whose magnitude exceeds the threshold decreases by increasing the thinning amount, the motion of the subject is large compared to the time between two frames for motion detection, and the motion is appropriately detected. Decrease the amount of decimation because it is considered that it is no longer possible. Also, when the number of motion vectors whose magnitude exceeds the threshold is reduced by reducing the thinning amount, the motion of the subject is small relative to the time difference between the two frames for motion detection, and the motion is detected appropriately. Since it is considered that there is no more, the amount of thinning is increased.

Here, the thinning amount is changed each time an evaluation value is generated. For example, if the change in the number of motion vectors whose magnitude exceeds the threshold is less than the threshold, the thinning amount is maintained. good too. Alternatively, an upper limit may be set for the size of the motion vector, and when a motion vector having a size exceeding the upper limit is detected, control may be combined to reduce the thinning amount.

As described above, in this embodiment, the frame rate of the image used for generating the evaluation value is controlled independently of the frame rate of the image used for recording and display, and based on the evaluation value. Therefore, it is possible to appropriately acquire evaluation values based on differences between frames without losing the advantages of shooting at a high frame rate, such as improving AF and AE performance by shortening the acquisition interval of AF evaluation values and AE evaluation values. becomes possible.

● (Second embodiment)
Next, a second embodiment of the invention will be described. FIG. 4 is a block diagram showing a functional configuration example of a digital camera 100' as an imaging device according to the second embodiment, and the same reference numerals are given to the configurations common to the first embodiment. In this embodiment, the detection result of the subject detection unit 113 is also output to the thinning adjustment unit 105, and the thinning adjustment unit 105 determines (updates) the thinning amount in consideration of the detection result of the subject detection unit 113.

Differences from the first embodiment will be described below. FIG. 5 is a flowchart relating to the evaluation value generation operation during moving image shooting of the digital camera 100', and the same processing as in the first embodiment is given the same reference numerals as in FIG. Since S201 to S203 are the same as those in the first embodiment, description thereof is omitted.

In step S<b>501 , the thinning adjustment unit 105 determines whether a subject area has been detected from the detection result of the subject detection unit 113 . The thinning adjustment unit 105 advances the processing to S204 if it is determined that the subject area has been detected, and to S502 if it is not determined.

In S502, the thinning adjustment unit 105 sets the thinning amount to zero. If the thinning amount is set to 1 or more when the subject is not detected, when the subject is detected next in the frame to be thinned, a delay occurs before the evaluation value reflecting the subject is obtained. Therefore, the thinning amount is set to 0 when the subject is not detected. The thinning adjustment unit 105 advances the process to S205.
Since the processing after S204 is the same as in the first embodiment, the description is omitted.

The operation of the thinning adjustment unit 105 of this embodiment will be specifically described using FIG. 6, which is similar to FIG. The initial value of the thinning amount is 0, and thinning adjustment section 105 outputs all frames output from A/D converter 103 .

The object is not detected from the first frame 601 to the fifth frame 602, and as a result, the thinning amount is maintained at the initial value of zero. During this time, the evaluation value generated by the evaluation value generation unit 106 does not include a motion vector whose magnitude exceeds 0, but the thinning amount is set to 0 in S502 instead of S204.

A subject is detected in the sixth frame 603, and when it is recognized, the thinning adjustment unit 105 performs thinning based on the evaluation value generated by the evaluation value generation unit 106 using the fifth frame 602 and the sixth frame 603 in S204. Determine (update) the amount. In this case, the number of motion vectors whose magnitude exceeds the threshold value (here, 0) is 10, and is increasing. Therefore, the thinning adjustment unit 105 outputs the eighth frame 604 without outputting the seventh frame.

After that, the subject is continuously detected up to the 16th frame 605, and the same thinning amount control as in the first embodiment is executed. Then, when the subject is no longer detected in the twentieth frame 606, the thinning adjustment unit 105 sets the thinning amount to 0 again.

According to the present embodiment, thinning is not performed (the thinning amount is set to 0) when the subject is not detected. An evaluation value that reflects the subject can be immediately obtained. In this embodiment, when the subject is no longer detected, thinning is not performed. may be controlled.

● (Third Embodiment)
Next, a third embodiment of the invention will be described. FIG. 7 is a block diagram showing an example of the functional configuration of a digital camera 100'' as an imaging device according to the third embodiment. Components common to those in the first and second embodiments are denoted by the same reference numerals. A digital camera 100″ of this embodiment has a configuration in which a threshold value setting unit 701 is added to the configuration of the second embodiment.

A threshold setting unit 701 sets a threshold for evaluation values in the thinning adjustment unit 105 . The threshold setting unit 701 may be a memory area that the thinning adjustment unit 105 can refer to. Note that the threshold may be directly set in the thinning adjustment unit 105 by the control unit 115, in which case the threshold setting unit 701 is unnecessary. The thinning adjustment unit 105 determines (updates) the thinning amount in consideration of the evaluation value 110 generated by the evaluation value generation unit 106 and the set threshold. The object detection result in the object detection unit 113 may or may not be taken into consideration.

Differences from the first and second embodiments will be described below. FIG. 8 is a flowchart relating to the evaluation value generation operation during moving image shooting of the digital camera 100″, and the same processing as in the first and second embodiments is given the same reference numerals as in FIGS. S201 to S203 are the same as in the first embodiment, and S501 to S502 are the same as in the second embodiment, so description thereof will be omitted.

If it is determined in S501 that the subject has been detected, the thinning adjustment unit 105 advances the processing to S801. In S801, the thinning adjustment unit 105 determines whether or not the number of motion vectors whose magnitude exceeds 0 is less than a threshold.

The determination in S801 is a determination as to whether or not an appropriate quality evaluation value has been generated, and the threshold is also determined in advance according to the purpose. If the number of motion vectors whose magnitude exceeds 0 is not less than the threshold (is greater than or equal to the threshold), it is determined that an appropriate thinning amount (frame interval) is set, and in S802 the thinning adjustment unit 105 selects the current Decide to keep the thinning amount. This is a control that gives priority to avoiding the risk of lowering the quality of the evaluation values over the possibility of further improving the quality of the evaluation values by changing the thinning amount.

On the other hand, if the number of motion vectors whose magnitude exceeds 0 is less than the threshold, the thinning adjustment unit 105 determines in S801 that the number of motion vectors whose magnitude exceeds 0 is less than the threshold a predetermined number of times (permissible continuous number of times). The thinning adjustment unit 105 advances the processing to S804 if it is determined that the determination has been made continuously exceeding the predetermined number of times, and to S204 if it is not determined.

In S804, the thinning adjustment unit 105 determines that the current thinning amount is inappropriate, and initializes the thinning amount. Since the initial value is 0 here, it is equivalent to no thinning. Note that the predetermined number of times in S803 can be set to any number of times equal to or greater than two. As described above, if the state of the evaluation value does not improve even after the thinning amount adjustment based on the evaluation value described in the first embodiment is continued to some extent, the thinning amount is initialized and the thinning amount adjustment is restarted from the beginning. .
Since the processing after S204 is the same as in the first embodiment, the description is omitted.

The operation of the thinning adjustment unit 105 of this embodiment will be specifically described using FIG. 9, which is similar to FIG. The initial value of the thinning amount is 0, and thinning adjustment section 105 outputs all frames output from A/D converter 103 . Also, the threshold in S801 is 25, and the predetermined number of times (permissible consecutive number of times) in S803 is two. Also, for ease of explanation and understanding, it is assumed that the subject continues to be detected.

The thinning amount is controlled for the first frame 301 to the seventh frame 307 in the same manner as in the first embodiment. By generating the evaluation value using the seventh frame 307 and the fourth frame 304, the number of motion vectors exceeding the magnitude of 0 exceeds the threshold, and the thinning adjustment unit 105 maintains the thinning amount at 2. do.

Next, although the number of motion vectors whose magnitude exceeds 0 is reduced by generating evaluation values using the tenth frame 901 and the seventh frame 307, the number of motion vectors still exceeds the threshold. Keep the thinning amount at 2.

Generating evaluation values using the 13th frame 903 and the 10th frame 901 further reduces the number of motion vectors whose magnitude exceeds 0 and falls below the threshold. However, the number of motion vectors exceeding 0 is less than the threshold for the first time and does not exceed the allowable number of consecutive motions. In this embodiment, when the number of motion vectors exceeding 0 decreases while the thinning amount is maintained, the thinning amount is increased, and when the number increases, the thinning amount is decreased. Therefore, the thinning adjustment unit 105 increases the thinning amount from two to three.

Generating evaluation values using the seventeenth frame 905 and the thirteenth frame 903 further reduces the number of motion vectors whose magnitude exceeds zero. Below threshold. However, the number of motion vectors exceeding 0 has been below the threshold value two times in a row, and does not exceed the allowable number of consecutive motions. Here, since the number of motion vectors exceeding 0 decreased as a result of increasing the thinning amount last time, the thinning adjustment unit 105 decreases the thinning amount to 2.

Generating an estimate using the 20th frame 907 and the 17th frame 905 increases the number of motion vectors with magnitudes greater than 0, but still below the threshold. As a result, the number of motion vectors exceeding 0 has fallen below the threshold three times in a row, exceeding the allowable number of consecutive times. Therefore, the thinning adjustment unit 105 initializes the thinning amount to zero. The thinning adjustment unit 105 then outputs the 21st frame 909 .

In the present embodiment, when an appropriate quality evaluation value is obtained, the quality evaluation value above a certain level can be obtained more stably by maintaining the thinning amount. On the other hand, if an appropriate quality evaluation value is not obtained, the same effect as in the first embodiment can be obtained by changing the thinning amount based on the evaluation value. Furthermore, if the situation does not improve by controlling the thinning amount based on the evaluation value, the thinning amount is initialized, so the situation where an appropriate evaluation value cannot be obtained continuously for a long time is automatically avoided. can do.

It should be noted that instead of initializing the thinning amount, the amount of change in the thinning amount per time may be increased. Also, the amount of change in the thinning amount per time in other cases may also be changed according to the evaluation value. For example, if the number of motion vectors whose magnitude exceeds 0 increases when the thinning amount is increased by one, then the thinning amount may be increased by two. Similar control can be achieved when the number of motion vectors with magnitudes greater than 0 is reduced. These are examples, and the amount of change may be changed according to other criteria.

(Other embodiments)
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by processing to It can also be implemented by a circuit (for example, ASIC) that implements one or more functions.

DESCRIPTION OF SYMBOLS 100, 100', 100'' --- digital camera, 105 -- thinning adjustment unit, 106 -- evaluation value generation unit, 107 -- recording image processing unit, 108 -- display image processing unit, 113 -- subject detection unit, 115 -- control unit

Claims (5)

acquisition means for acquiring a video having a first frame rate ;
image processing means for applying image processing to a moving image having a first frame rate;
thinning means for thinning and outputting frames constituting a moving image having the first frame rate according to a thinning amount;
detection means for detecting a difference amount between the plurality of frames output by the thinning means;
The imaging apparatus, wherein the thinning means increases the thinning amount when the difference amount is less than a threshold value. further comprising subject detection means for detecting a predetermined subject from the moving image,
2. The imaging apparatus according to claim 1 , wherein said thinning means reduces said thinning amount when said subject is not detected by said subject detecting means. further comprising display means;
3. The image processing means according to claim 1, wherein the image processing means applies image processing to the moving image having the first frame rate to generate image data to be displayed on the display means. Imaging device. A control method for an imaging device executed by the imaging device, comprising:
a capturing step of capturing a video having a first frame rate ;
an image processing step of applying image processing to a moving image having a first frame rate;
a thinning step of thinning and outputting frames constituting a moving image having the first frame rate according to a thinning amount;
a detection step of detecting a difference amount of a plurality of frames output by the thinning step;
In the thinning step, the thinning amount is increased when the difference amount is less than a threshold value. A program for causing a computer of an imaging device to function as each means of the imaging device according to any one of claims 1 to 3 .

Images