JP6873810B2 - Imaging device, control method of imaging device, and program - Google Patents

Description

The present invention relates to an image pickup device for capturing an image, a control method for the image pickup device, and a program.

In recent years, image quality has been improved in image pickup devices such as video cameras, and there is an increasing demand for cameras having a wide dynamic range that can express gradations up to high-luminance areas while preventing black crushing in dark areas. In addition, video cameras are becoming more polarized between commercial and consumer use, and in the case of small and inexpensive video cameras for consumer use, image sensors that do not have sufficient dynamic range performance are often used. On the other hand, as a countermeasure function when an image sensor that does not have a sufficient dynamic range is used, some cameras are equipped with a function that shifts the dynamic range of the image sensor according to the subject. With this function, when a high-brightness subject exists in the screen and overexposure occurs, the proper exposure is shifted under to shift the dynamic range of the image sensor to the high-brightness side, but the brightness of the entire image is not changed. The gradation conversion characteristic is changed so that the amplification factor of the dark part is high. In addition, with this function, when there are not many high-brightness subjects on the screen, the proper exposure is controlled to be overexposed and the dynamic range of the image sensor is shifted to the low-brightness side, while the amplification factor of dark areas is small. To control. Unlike the exposure control, such a dynamic range control is used for controlling the gradation of the high-luminance portion and the low-luminance portion. Further, in this case, in order to maintain the quality as a moving image, control is performed so as to have a very slow responsiveness.

On the other hand, it is also desired to expand the subject range in which the exposure can be controlled, and along with this, the number of models that can use the ND (Neutral Density) filter in the consumer video camera is increasing. When the ND filter is used, an operation of switching to a state in which the ND filter is used mechanically or electrically is performed based on an operation instruction from the photographer. As a result, the amount of light incident on the image sensor is changed at an arbitrary timing desired by the photographer. At this time, on the video camera side, by performing other exposure control in the direction of canceling the exposure change due to the change in the amount of light at the time of switching the ND filter, stable exposure can be obtained even before and after the switching of the ND filter. In addition, Patent Document 1 discloses a technique relating to an exposure control method by switching an ND filter or the like.

Japanese Unexamined Patent Publication No. 2004-7580

Here, when another exposure control is performed in the direction of canceling the exposure change by the ND filter, the exposure change may occur without being completely canceled. For example, if the ND filter is used when the subject is too bright and is out of the exposure control range, the exposure will change from the state where it was originally out of the exposure control range to the proper exposure state within the exposure control range. become. In this case, the degree of exposure to the subject changes significantly, and when the above-mentioned dynamic range control is performed, it takes time to shift to an appropriate dynamic range state. For example, with the exposure control method described in Patent Document 1, it is possible to stabilize the exposure before and after switching the ND filter, but until the dynamic range is reached while maintaining the quality of the moving image after switching the ND filter. It is not possible to save time. In this way, with the current video cameras, it is not possible to shoot in the proper dynamic range immediately after switching the ND filter, and for example, you miss the opportunity to shoot with the optimum image quality at the decisive moment. There is a risk that things will happen.

Therefore, an object of the present invention is to enable shooting in an appropriate dynamic range state immediately after an exposure change occurs.

The present invention is an imaging device that images a subject, the brightness acquisition means for acquiring the brightness information of the subject, the exposure control means for controlling the exposure at the time of imaging, and the imaging based on the brightness information. It has a calculation means for calculating the dynamic range set in the device, and the calculation means calculates the first dynamic range set in the current imaging device and when the exposure is changed to switch the exposure. It is characterized in that a second dynamic range set in the image pickup apparatus is calculated in relation to the exposure control of the above.

According to the present invention, it is possible to take an image in an appropriate dynamic range immediately after an exposure change occurs.

It is an external view of the video camera which is an example of the image pickup apparatus of this embodiment. It is a figure which shows an example of the ND filter mechanism. It is a figure which shows the internal structure example of the video camera of this embodiment. It is a figure which shows the main internal processing of a system control part. It is a program diagram at the time of exposure determination. It is a flowchart of a basic dynamic range calculation. It is a figure which shows an example of a small area division. It is a flowchart of dynamic range calculation in this embodiment. It is a flowchart of another dynamic range calculation.

Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.
FIG. 1 is a schematic external view showing a digital video camera (hereinafter referred to as a video camera 100) which is an example of the image pickup apparatus of the present embodiment. In FIG. 1, the display unit 28 is a display device that displays images and various information, and displays captured images, recorded images, and the like. The recording switch 61 is an operation unit for the user to give a shooting instruction. The mode changeover switch 60 is an operation unit for the user to give an instruction to change various modes. The user can give an instruction such as switching between the camera mode and the playback mode by pressing the mode changeover switch 60. The connector 112 is a connection connector portion that connects between the connection cable and the video camera 100. The operation unit 70 is composed of operation members such as various buttons and cross keys that receive various operations from the user. The power switch 72 is an operation unit for the user to switch between power on and power off. The recording medium 200 is a removable recording medium such as a memory card or a hard disk, and captured image data or the like is recorded.

Further, the video camera 100 of the present embodiment has, for example, an ND filter mechanism capable of switching the ND filter by a turret type as shown in FIG. 2A, or an ND filter by an insertion / removal type as shown in FIG. 2B. It is equipped with an ND filter mechanism that can switch between.

In the ND filter mechanism of FIG. 2A, ND filters 282, 283, 284 and ND filters having different light transmittances (filter densities) are attached to the turret portion 280 which is rotatable in the direction indicated by the arrow 285. A through hole portion 281 that is not arranged is provided. The through hole portion 281 to which the ND filter is not arranged corresponds to a light transmittance of 100%. Then, in the ND filter mechanism of FIG. 2A, by rotating the turret portion 280, switching of the ND filter arranged on the optical path of the imaging optical system (switching including the through hole portion 281 having no ND fill) is performed. Will be done. By switching the ND filter in this way, the ND filter mechanism shown in FIG. 2A can adjust the amount of light passing through the optical path of the image pickup optical system, that is, the amount of light incident on the image pickup element of the image pickup sensor. The example of FIG. 2A shows a state in which the through hole portion 281 is arranged on the optical path of the imaging optical system of the video camera 100 by rotating the turret portion 280. In this case, the optical image by the imaging optical system is formed directly on the image sensor 220 (imaging sensor) through the through hole portion 281 without passing through the ND filter. Further, in the case of the present embodiment, in the ND filter mechanism shown in FIG. 2A, the turret unit 280 is rotated according to the mechanical or electrical operation via the operation unit 70 by the user, so that the ND filter is filtered. Is switched.

The ND filter mechanism of FIG. 2B is capable of inserting and removing the ND filter 291 in the direction indicated by the arrow 293 on the optical path 290 of the imaging optical system. The figure on the left side of FIG. 2B shows the state before the ND filter 291 is loaded in the optical path 290 of the imaging optical system, and the figure on the right side shows the ND on the optical path 290 due to the loading of the ND filter 291. The state in which the filter 291 is arranged is shown. The ND filter mechanism shown in FIG. 2B can be replaced with an ND filter having a different light transmittance (density). In the case of the ND filter mechanism of FIG. 2B, the amount of light passing through the optical path of the image pickup optical system can be adjusted, that is, the amount of light incident on the image sensor can be adjusted by inserting and removing or replacing the ND filter. Further, in the case of the present embodiment, in the ND filter mechanism shown in FIG. 2B, the ND filter is inserted and removed in response to a mechanical or electrical operation by the user via the operation unit 70. Suppose you are.

FIG. 3 is a block diagram showing an example of the internal configuration of the video camera 100 of the present embodiment.
In FIG. 3, the photographing lens 103 is a lens group including a zoom lens and a focus lens of an imaging optical system, and forms a subject image on an imaging element of an imaging sensor. The diaphragm 101 is a diaphragm used for adjusting the amount of light. The ND filter mechanism 104 has the configurations shown in FIGS. 2 (a) and 2 (b), and is used when the amount of incident light via the imaging optical system is dimmed by switching the ND filter. The barrier 102 covers the imaging system including the photographing lens 103 of the video camera 100 to prevent the imaging system including the photographing lens 103, the aperture 101, and the imaging unit 22 from being soiled or damaged.

The image pickup unit 22 is an image pickup element such as a CCD or a CMOS element that constitutes an image pickup sensor, and converts an optical image formed on the image pickup element via an image pickup optical system into an analog image signal. Further, the imaging unit 22 also has a function of controlling charge accumulation by an electronic shutter, changing the analog gain, changing the reading speed, and the like. The A / D converter 23 converts the analog image signal input from the imaging unit 22 into digital image data and outputs it.

The image processing unit 24 performs predetermined pixel interpolation processing, resizing processing such as reduction, and color conversion on the image data supplied from the A / D converter 23 or the image data supplied via the memory control unit 15. Performs various image processing such as processing, gamma correction processing, and addition of digital gain. Further, the image processing unit 24 performs a predetermined calculation process using the captured image data, and sends the calculation result to the system control unit 50. The system control unit 50 has a CPU and the like, and performs exposure control, distance measurement control, white balance control, and the like based on the calculation results supplied from the image processing unit 24. As a result, so-called TTL (through-the-lens) AF (autofocus) processing, AE (autoexposure) processing, AWB (auto white balance) processing, and the like are performed.

Further, the image data output from the A / D converter 23 is directly written to the memory 32 via the image processing unit 24 and the memory control unit 15 or via the memory control unit 15. The memory 32 temporarily stores image data captured by the imaging unit 22 and digitally converted by the A / D converter 23, image data to be displayed on the display unit 28, and the like. The memory 32 has a storage capacity sufficient to store moving images and sounds for a predetermined time.

Further, the memory 32 also serves as a memory (video memory) for displaying an image. The D / A converter 13 converts the image data for image display read from the memory 32 and supplied via the memory control unit 15 into an analog image signal and supplies the image data to the display unit 28. As a result, the display unit 28 displays the display image stored in the memory 32. The display unit 28 has a display such as an LCD (liquid crystal display), and displays an image corresponding to an analog image signal supplied from the D / A converter 13. The video camera 100 converts the captured image data temporarily stored in the memory 32 from the imaging unit 22 via the A / D converter 23 into an analog image signal by the D / A converter 13 and displays the display unit. It is made possible to sequentially transfer to 28 and display it. As a result, the display unit 28 of the video camera 100 functions as a so-called electronic viewfinder and can display a through image.

The non-volatile memory 56 is a memory that can be electrically erased and recorded, and for example, EEPROM is used. The non-volatile memory 56 stores constants, programs, and the like for the operation of the system control unit 50. The program referred to here includes a program for executing various flowcharts described later in the present embodiment. The system memory 52 includes, for example, a RAM. In the system memory 52, constants and variables for the operation of the system control unit 50, a program read from the non-volatile memory 56, and the like are expanded.

The system control unit 50 controls the entire video camera 100. By executing the program recorded in the non-volatile memory 56 described above, each process of the present embodiment described later is realized. That is, the system control unit 50 controls the accumulation time of the imaging unit 22 and the reading of the captured image signal to the A / D converter 23, and the reading of the captured image signal by the imaging unit 22 to the recording medium 200 is a moving image. Controls a series of operations up to writing data. Further, the system control unit 50 controls the display by controlling the memory 32, the D / A converter 13, the display unit 28, and the like.

Further, various operation instructions are input to the system control unit 50 from the mode changeover switch 60, the operation unit 70, and the like. For example, when the mode changeover switch 60 is operated by the user, the system control unit 50 controls to sequentially switch the types of images displayed on the display unit 28. Each operation member of the operation unit 70 is appropriately assigned a function for each scene when various function icons displayed on the display unit 28 are selected and operated, and is used as various function buttons. In the case of the present embodiment, the system control unit 50 switches from the normal shooting display mode to the characteristic comparison display mode capable of notifying the photographer of the excess or deficiency of gradation according to the operation of the operation unit 70 by the user. Etc. are controlled. In addition, the operation unit 70 is also assigned, for example, a menu button, a cross key, an end button, a back button, an image feed button, a jump button, a narrowing down button, an attribute change button, and the like. For example, when the menu button is pressed, the system control unit 50 causes the display unit 28 to display various configurable menu screens. In this case, the user can intuitively make various settings by using the menu screen displayed on the display unit 28, the up / down / left / right four-direction direction instruction keys and the SET buttons arranged on the cross keys.

Further, in the present embodiment, when the ND filter mechanism 104 has the configuration shown in FIG. 2A, the system control unit 50 rotates the turret unit 280 based on the signal from the operation unit 70, thereby rotating the turret unit 280. Controls the switching of the ND filter. Further, when the ND filter mechanism 104 has the configuration shown in FIG. 2B, the system control unit 50 switches the ND filter by inserting and removing the ND filter based on the signal from the operation unit 70. To control.

The system timer 53 is a time measuring unit that measures the time used for various controls and the time by the built-in clock.
The power supply control unit 80 is composed of a battery detection circuit, a DC-DC converter, a switch circuit for switching a block to be energized, and the like, and detects whether or not the power supply unit 30 has a battery installed, the type of battery, and the remaining battery level. Further, the power supply control unit 80 controls the DC-DC converter based on the detection result and the instruction of the system control unit 50, and supplies a necessary voltage to each unit including the recording medium 200 for a necessary period. The power supply unit 30 includes a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li ion battery, an AC adapter, or the like, and is connected to the power supply control unit 80.

The external I / F 18 is an interface for communicating with a recording medium 200 such as a memory card or a hard disk, and communicating with an external display device 202 via a connector 112. The recording medium 200 is composed of a semiconductor memory card, a magnetic disk, or the like for recording a captured image. An external display device 202 such as a display device is connected to the connector 112. The video camera 100 transmits image data to the external display device 202 via the external I / F18 and further via the connector 112. The external display device 202 connected via the connector 112 receives the image data transmitted from the video camera 100 and displays an image based on the image data.

Next, the operation of the video camera 100 of the present embodiment will be described.
FIG. 4 shows a video related to each processing block in the system control unit 50, which is the main processing in the video camera 100 of the present embodiment, each configuration in the image processing unit 24, and each processing block in the system control unit 50. It is a figure which showed each part in a camera 100.
As shown in FIG. 4, the system control unit 50 includes an ND control unit 300, an exposure control unit 301, a dynamic range control unit 303, a dynamic range calculation unit 302, and a brightness acquisition unit as main processing blocks according to the present embodiment. It has 304. Further, the image processing unit 24 has a gain control unit 241 and a gamma circuit 242 as main configurations related to the processing block of the system control unit 50.

When the ND filter mechanism 104 has the configuration shown in FIG. 2A, the ND control unit 300 rotates the turret unit 280 based on the signal from the operation unit 70, and causes the ND filter 284 to 284 on the optical path of the imaging optical system. Control is performed so that any of the above or the through hole portion 281 is arranged. Further, when the ND filter mechanism 104 has the configuration shown in FIG. 2B, the ND control unit 300 controls the insertion / removal and switching of the ND filter based on the signal from the operation unit 70. In the following description, the ND filter configurations of FIGS. 2 (a) and 2 (b) are not distinguished, and the presence / absence (including insertion / extraction) of the ND filter on the optical path of the imaging optical system and the switching of the ND filter are made. The arrangement is collectively referred to as ND filter switching. Then, when the ND control unit 300 performs the switching control of the ND filter, the ND control unit 300 also outputs the control information to the exposure control unit 301.

The exposure control unit 301 controls the exposure at the time of imaging. Further, the exposure control unit 301 receives control information from the ND control unit 300, and various exposure parameters (for example, aperture value, exposure time, shooting sensitivity) in a direction of canceling a change in the amount of incident light on the image sensor due to ND filter switching. Calculate the control amount to control. Then, the exposure control unit 301 sends the information of the exposure value determined based on the control amount to the dynamic range control unit 303. However, in the control of the exposure parameter, the change in the amount of light due to the switching of the ND filter cannot be completely canceled, and the change in the exposure may occur.

Here, FIG. 5A is a program diagram for determining the shutter speed, gain (ISO sensitivity), and aperture of the electronic shutter with respect to the subject luminance 14 [Bv] when the ND filter is not used (CLEAR). Is shown. Further, FIG. 5B shows a program for determining the shutter speed, gain, and aperture with respect to the subject luminance 14 [Bv] when an ND filter having a 1/4 density (light transmittance is 25%) is used. The diagram is shown. As shown in FIG. 5A, when the ND filter is not used (CLEAR), the subject brightness 14 [Bv] covers the exposure controllable range (3 [Bv] to 13 [Bv]) on the program diagram. Over. On the other hand, as shown in FIG. 5B, when a 1/4 density ND filter is used, the subject brightness 14 [Bv] is the exposure controllable range (5 [Bv] to 15 [Bv] on the program diagram. Bv]) can be seen. At this time, the amount of exposure exceeding the controllable range of exposure corresponds to the amount of change in brightness + 1 [ΔBv], and when the exposure control unit 301 performs exposure control in the direction of canceling the change in amount of light due to switching of the ND filter, the brightness An exposure change corresponding to the amount of change -1 [ΔBv] occurs. In addition, an exposure change may occur in addition to the exposure range on the program diagram described above. For example, it is conceivable to change the exposure target aiming at an image quality effect for preventing image quality deterioration due to noise generated at high sensitivity. Even in such a case, the change in the amount of light due to the switching of the ND filter cannot be completely canceled, and a sudden change in exposure, that is, a sudden change in exposure occurs.

Therefore, in the present embodiment, in the system control unit 50, the information regarding the exposure change amount is transmitted from the exposure control unit 301 to the dynamic range calculation unit 302. Further, the brightness acquisition unit 304 acquires the brightness of the subject of the captured image. As a method of acquiring the brightness of the subject, in the present embodiment, a method of acquiring the brightness value of each pixel of the captured image signal is used, but for example, the information of the brightness of the subject measured by an external exposure meter or the like is acquired. You may. The brightness information of the subject acquired by the brightness acquisition unit 304 is sent to the dynamic range calculation unit 302.

The dynamic range calculation unit 302 calculates the dynamic range set in the current video camera 100, and also calculates the dynamic range related to the exposure control when the exposure change that suddenly switches the exposure is performed in the exposure control unit 301. .. Specifically, the dynamic range calculation unit 302 sets the video camera 100 of the present embodiment based on the brightness information from the brightness acquisition unit 304 and the information on the amount of change in exposure from the exposure control unit 301. Calculate the dynamic range. In particular, in the case of the present embodiment, the dynamic range calculation unit 302 determines whether or not the brightness of the image captured before and after the exposure control changes based on the brightness information and the exposure change amount information. Judgment is made, and the dynamic range calculation method is changed according to the judgment result. The dynamic range calculation method will be described in detail later. The dynamic range information calculated by the dynamic range calculation unit 302 is sent to the dynamic range control unit 303.

The dynamic range control unit 303 controls the aperture 101, the image pickup unit 22, and the image processing unit 24 based on the dynamic range information calculated by the dynamic range calculation unit 302 and the exposure value received from the exposure control unit 301. Calculate the value. That is, the dynamic range control unit 303 controls the dynamic range by controlling at least one of the exposure and the gradation conversion characteristic. At this time, the dynamic range control unit 303 allocates the exposure value determined by the exposure control unit 301 to the exposure and gradation conversion characteristics based on the dynamic range calculated by the dynamic range calculation unit 302. .. That is, the dynamic range control unit 303 allocates the exposure value in order to control the exposure value received from the exposure control unit 301 to the dynamic range calculated by the dynamic range calculation unit 302. For example, when the dynamic range of the image pickup sensor is shifted to the high brightness side, the dynamic range control unit 303 causes the image pickup unit 22 to set an underexposure in the image pickup sensor based on the exposure value from the exposure control unit 301. Controls the exposure time or shooting sensitivity of. At this time, the dynamic range control unit 303 controls the gamma circuit 242 gain rate to be increased by the control value for controlling the gain control unit 241 and the gamma circuit 242 of the image processing unit 24. On the other hand, when the dynamic range of the image pickup sensor is shifted to the low brightness side, the dynamic range control unit 303 determines the exposure value of the image pickup sensor to be overexposed based on the exposure value from the exposure control unit 301. Controls the exposure time or shooting sensitivity in. At this time, the dynamic range control unit 303 controls the gamma circuit 242 to have a non-linear conversion characteristic with a low gain rate by the control values for controlling the gain control unit 241 and the gamma circuit 242. In this way, the system control unit 50 controls the gradation by performing exposure control and gain rate control while shifting the dynamic range of the image pickup sensor according to the brightness of the subject.

Subsequently, the dynamic range calculation method in the dynamic range calculation unit 302 will be described in detail with reference to the flowcharts of FIGS. 6 and 6 and thereafter.
First, in the flowchart of FIG. 6, a method of calculating the dynamic range from the luminance signal (brightness information) of the captured image will be described as an example of a basic dynamic range calculation method that does not consider the change in the amount of light due to the switching of the ND filter. .. The processing of the flowchart of FIG. 6 may be executed by the hardware configuration, or may be partially realized by the software configuration and the rest by the hardware configuration. When the processing is executed by the software configuration, the processing of the flowchart of FIG. 6 is realized by expanding the program stored in the non-volatile memory 56 of FIG. 3 into the system memory 52 and executing the processing by the system control unit 50. Will be done. The program according to the present embodiment is prepared in advance in the non-volatile memory 56, but may be read from a detachable semiconductor memory or downloaded from a network such as the Internet (not shown) in the system memory 52. May be loaded. Further, in the following description, steps S400 to S406 of each process in FIG. 6 are abbreviated as S400 to S406. These things are the same in other flowcharts described later.

Here, the brightness acquisition unit 304 divides the captured image into a plurality of small regions in order to acquire detailed luminance information (brightness information) of the captured image, and the peak luminance value (hereinafter referred to as the peak luminance value) for each of the small regions. , Peak brightness) and average brightness value (hereinafter referred to as average brightness) are acquired. FIG. 7 shows an example in which the captured image is divided into 16 small areas. In the following description, the area numbers i (i = 1, 2, ..., 16) are sequentially assigned to each small area, and the last area number (or the number of divisions) is expressed as win_num. The peak brightness may be acquired for each small region, and the average brightness may be acquired by further dividing each of these small regions into 16.

As the process of S400, the dynamic range calculation unit 302 determines whether or not the area number i of the small area currently being processed is smaller than the last area number win_num. When the dynamic range calculation unit 302 determines (Yes) that the area number i of the small area to be processed is smaller than the last area number win_num, the dynamic range calculation unit 302 proceeds to S401. On the other hand, when the dynamic range calculation unit 302 determines (No) that the area number i of the small area to be processed has become the last area number win_num, the dynamic range calculation unit 302 proceeds to S405.

When proceeding to S401, the dynamic range calculation unit 302 compares the peak brightness of the small area of the area number i to be processed with the predetermined threshold Th1 and determines whether or not the peak brightness exceeds the threshold Th1. To do. Then, when the dynamic range calculation unit 302 determines that the peak luminance exceeds the threshold Th1 (Yes), the process proceeds to S402, while the dynamic range calculation unit 302 determines that the peak luminance does not exceed the predetermined threshold Th1 (No). In that case, the process proceeds to S404.

When proceeding to S404, the dynamic range calculation unit 302 increments the area number i, so that the small area of the next area number is set as the processing target and the processing is returned to S400.

When it is determined in S401 that the peak brightness exceeds the predetermined threshold Th1 and the process proceeds to S402, the dynamic range calculation unit 302 compares the average brightness of the small area of the area number i with the predetermined threshold Th2, and the average brightness is increased. It is determined whether or not the threshold Th2 has been exceeded. That is, in S402, the average brightness and the threshold Th2 are compared with respect to the small region determined in S401 that the peak brightness exceeds the threshold Th1. Then, when the dynamic range calculation unit 302 determines (Yes) that the average brightness exceeds the threshold Th2, the process proceeds to S403, while when it determines (No) that the average brightness does not exceed the threshold Th2. Proceeds to S404.

Proceeding to S403, the dynamic range calculation unit 302 calculates the average luminance Ave by adding the average luminance of each small region after the processing up to this point. That is, in the dynamic range calculation unit 302, the average used for the dynamic range calculation is calculated by adding only to the small region in which the peak brightness is determined to exceed the threshold Th1 in S401 and the average brightness is further determined to exceed the threshold Th2 in S402. Calculate the brightness Ave. After S403, the dynamic range calculation unit 302 proceeds to S404 for processing.

When it is determined in S400 that the area number i has become the last area number win_num and the process proceeds to S405, the dynamic range calculation unit 302 adds the average luminance Ave added in S403 and the average luminance added in S403. Divide by the number of small areas made. That is, in the calculation of S405, the average brightness value AVE of all the small areas including the high brightness area in the captured image is calculated. After S405, the dynamic range calculation unit 302 proceeds to S406.

Proceeding to S406, the dynamic range calculation unit 302 obtains the difference between the average luminance AVE calculated in S405 and the predetermined threshold value Th3, and the control value of the dynamic range according to the magnitude of the difference (hereinafter, D range). Determine the level.)
The above is an example of a basic dynamic range calculation method when the change in the amount of light due to switching of the ND filter is not taken into consideration.

Next, an example of calculating the dynamic range in the present embodiment in consideration of the change in the amount of light due to the switching of the ND filter will be described below.
As described above, when an exposure change due to a change in the amount of light due to the switching of the ND filter, that is, a sudden change in exposure occurs, the degree of exposure jump in the subject changes significantly, so that it is necessary to change the dynamic range. On the other hand, if the dynamic range is changed suddenly, the exposure and gradation are suddenly switched, giving an unnatural impression in terms of image quality, and there is a risk that the quality as a moving image cannot be maintained.

By the way, when the ND filter is switched, the optical path of the imaging optical system is temporarily blocked by the above-mentioned turret portion 280 or the insertion / removal of the ND filter, so that the captured image is temporarily disturbed. However, even if the image quality is disturbed due to the optical path blocking at the time of switching the ND filter, the user recognizes that the disturbance of the image quality is caused by the switching of the ND filter, which is unnatural. It is unlikely that you will be impressed.

For this reason, when the image quality is temporarily disturbed due to the optical path blocking when switching the ND filter, for example, even if the image quality is deteriorated due to a change in the dynamic range, it may give an unnatural impression to the user. It is thought that there is almost no such thing.
Therefore, in the present embodiment, the dynamic range is changed instantaneously while the image quality is temporarily disturbed due to the switching of the ND filter, and immediately after the switching of the ND filter, an appropriate dynamic range is used. Make it possible to obtain images with high image quality.

In order to realize such dynamic range control, the dynamic range calculation unit 302 has both the current D range level and the D range level immediately after the light amount change of the ND filter from the stage before the light amount change by switching the ND filter. To be calculated.
Hereinafter, the D range level calculation process in the present embodiment will be described with reference to the flowchart of FIG.

In S500 of FIG. 8, the dynamic range calculation unit 302 suddenly changes the exposure due to the change in the amount of light (that is, the change in brightness) due to the switching of the ND filter from the information on the amount of change in exposure acquired from the exposure control unit 301. Determine whether or not to do so. When the dynamic range calculation unit 302 determines in S500 that a sudden exposure change occurs due to ND filter switching (Yes), the process shifts to S501, while the dynamic range calculation unit 302 determines that no exposure change occurs (No). Proceed to normal control. The description of normal control will be omitted.

When the process proceeds to S501, the dynamic range calculation unit 302 acquires the exposure change amount calculated by the exposure control unit 301 when the light amount changes due to the ND filter switching, and proceeds to S502.

Proceeding to S502, the dynamic range calculation unit 302 calculates the threshold values Th1', Th2', and Th3' based on the exposure change amount acquired from the exposure control unit 301. The threshold values Th1', Th2', and Th3'are different from the above-mentioned threshold values Th1, Th2, and Th3 in the current exposure in order to predict the dynamic range assuming the exposure state after the change in the amount of light due to the switching of the ND filter. It is a new threshold value calculated by adding the amount. Specifically, the threshold values Th1', Th2', and Th3'are the following equations (1), (2), and equations using the threshold values Th1, Th2, Th3 and the exposure change amount, that is, the brightness change amount ΔBv). Calculated according to (3). The threshold values Th1', Th2', and Th3'may be calculated by other empirical coefficients or mathematical formulas. After S502, the dynamic range calculation unit 302 proceeds to S503 for processing.

Th1'= Th1 × 2 ^ ΔBv equation (1)
Th2'= Th2 × 2 ^ ΔBv equation (2)
Th3'= Th3 × 2 ^ ΔBv equation (3)

Since the processes from S503 to S506 are the same as the processes from S400 to S403 in the flowchart of FIG. 6 described above, their description will be omitted. The brightness average Ave obtained by the processes from S503 to S506 is the average brightness used for calculating the dynamic range before the exposure change according to the switching of the ND filter. After S506, the dynamic range calculation unit 302 proceeds to S507 for processing. Further, when it is determined (No) in S503 that the area number i becomes the last area number win_num, the dynamic range calculation unit 302 proceeds to S511 for processing.

When proceeding to the processing of S507, the dynamic range calculation unit 302 compares the peak brightness of the small area of the area number i with the threshold value Th1'calculated in S502, and the peak brightness of the small area of the area number i is the threshold value Th1. Determine if it exceeds'. Then, the dynamic range calculation unit 302 proceeds to S508 when it is determined (Yes) that the peak luminance exceeds the threshold Th1', and on the other hand, when it is determined (No) that it does not exceed the threshold Th1', the dynamic range calculation unit 302 proceeds. The process proceeds to S510.
When the process proceeds to S510, the dynamic range calculation unit 302 increments the area number i to set the small area of the next area number as the processing target and returns the processing to S503.

When the process proceeds to S508, the dynamic range calculation unit 302 compares the average brightness of the small area of the area number i with the threshold Th2'calculated in S502, and the average brightness of the small area of the area number i sets the threshold Th2'. Determine if it has been exceeded. That is, in S508, the average brightness and the threshold Th2'are compared with respect to the small region where the peak brightness exceeds the threshold Th1'in S507. Then, when the dynamic range calculation unit 302 determines (Yes) that the average brightness exceeds the threshold Th2', the process proceeds to S509, while the dynamic range calculation unit 302 determines (No) that the average brightness does not exceed the threshold Th2'. In that case, the process proceeds to S510.

Proceeding to S509, the dynamic range calculation unit 302 calculates the average luminance Ave'by adding the average luminance values of the small regions that have been processed up to this point after the change in the amount of light due to the ND filter switching. That is, in the dynamic range calculation unit 302, the average brightness Ave used for the dynamic range calculation is added only to a small region where the peak brightness exceeds the threshold Th1'in S507 and the average brightness exceeds the threshold Th2'in S508. 'Calculate. The brightness average Ave'obtained by the above-mentioned processes S507 to S509 is the average brightness used for calculating the dynamic range assuming the exposure state after the change in the amount of light due to the switching of the ND filter. After S509, the dynamic range calculation unit 302 proceeds to S510 for processing.

Further, when it is determined in S503 that the area number i has become the last area number win_num and the process proceeds to S511, the dynamic range calculation unit 302 adds the average luminance Ave added in S506 to the average luminance in S506. Divide by the number of small areas where That is, in the calculation of S511, the average brightness value AVE of all the small areas including the high brightness area in the captured image is calculated. After S511, the dynamic range calculation unit 302 proceeds to S512.

Proceeding to S512, the dynamic range calculation unit 302 obtains the difference between the average luminance AVE calculated in S511 and the above-mentioned threshold value Th3, and determines the D range level according to the magnitude of the difference. The D range level obtained by these S511 and S512 is the current D range level before the exposure change according to the switching of the ND filter. After S512, the dynamic range calculation unit 302 proceeds to S513 for processing.

The processing of S513 and the next S514 is a processing of calculating the expected D range level with respect to the expected average luminance value after the change in the amount of light due to the switching of the ND filter, using the threshold value Th3'calculated in S502.
In S513, the dynamic range calculation unit 302 divides the average luminance Ave'added in S509 by the number of small regions to which the average luminance is added in S509. In this calculation of S513, the expected average brightness value average brightness AVE'of all the small areas including the high brightness area in the captured image is calculated.

In S514 following S513, the dynamic range calculation unit 302 obtains the difference between the average luminance AVE'calculated in S513 and the threshold Th3'calculated in S502, and the D range level according to the magnitude of the difference. To determine. The D range level obtained by these S513 and S514 is a D range level predicted assuming an exposure state after a change in the amount of light due to ND filter switching. After S514, the dynamic range calculation unit 302 ends the processing of the flowchart of FIG.

As described above, when the exposure change due to the change in the amount of light due to the switching of the ND filter occurs, the dynamic range calculation unit 302 of the present embodiment has the current D range level and the D range immediately after the change in the amount of light due to the switching of the ND filter. Calculate two levels at the same time. Then, the information of these two D range levels is sent to the dynamic range control unit 303.

The dynamic range control unit 303 momentarily switches the dynamic range based on these two D range levels within the ND filter change period from the start to the completion of the ND filter switching, so that the image pickup unit 22 and the image processing unit 24 is controlled. The control of the image pickup unit 22 and the image processing unit 24 (gain control unit 241 and gamma circuit 242) is the same as described above except that the dynamic range is instantaneously switched, and thus a specific description thereof will be omitted.

According to the present embodiment, by instantaneously switching the dynamic range within the ND filter change period when switching the ND filter, shooting can be resumed immediately with the optimum image quality even after the light intensity is changed due to the ND filter switching, which is a decisive moment. Can be prevented from being missed. That is, according to the present embodiment, the ND filter can be automatically set according to the brightness of the shooting environment, and the occurrence of frames that cannot be used in the moving image can be reduced as much as possible.

Further, in the present embodiment, it is also possible to calculate the D range level by processing the flowcharts shown in FIGS. 9 (a) and 9 (b). In FIG. 9A, the processes from S600 to S606 are the same as the processes from S500 to S506 in FIG. 8, and the processes from S608 to S613 are the same as the processes from S509 to S514 in FIG. Therefore, their description will be omitted. However, in the case of FIG. 9A, in S602, the threshold Th4 different from the above-mentioned threshold Th2'is also calculated. The details of the calculation process of the threshold value Th4 will be described later. Further, in FIG. 9A, when No is determined in S604 and S605 and after the processing of S606, the dynamic range calculation unit 302 proceeds to S607. FIG. 9B shows the detailed processing of S607 of FIG. 9A.

Here, when the switching of the ND filter determined in S600 is a switching from the ND filter used state to the ND filter non-used state, the above-mentioned peak brightness is between the ND filter used state and the non-used state. It increases by a rate corresponding to the difference in light transmittance of. In this case, depending on the brightness of the subject or the like, the peak brightness assumed after switching to not using the ND filter may exceed the maximum value of the code amount (for example, 255 for an 8-bit signal). However, in reality, a signal exceeding the maximum value of the code amount is not observed (output). Therefore, there is a discrepancy between the D range level calculated before the ND filter switching and the D range level calculated after the ND filter switching, and the optimum image quality cannot be realized immediately after the ND filter switching.

Therefore, when the dynamic range calculation unit 302 proceeds to the processing of S607, first, as the processing of S620, whether or not the peak luminance of the small region of the region number i is the maximum value (= saturation signal) of the code amount, that is, the peak. It is determined whether or not the brightness exceeds the maximum value of the code amount. Then, when the dynamic range calculation unit 302 determines that the peak luminance becomes the maximum value of the code amount (Yes), the process proceeds to S623, while the dynamic range calculation unit 302 determines that the peak luminance does not reach the maximum value of the code amount (Yes). If so, the process proceeds to S621.

The processing of S621 and S622 is the same processing as that of S507 and S508 of FIG. 8 described above, and the description thereof will be omitted. When the dynamic range calculation unit 302 determines No in S621 and S622, the process proceeds to S609 in FIG. 9A, and when it determines Yes in S622, the dynamic range calculation unit 302 proceeds to S608 in FIG. 9A. Proceed with processing.

On the other hand, when proceeding to the processing of S623, the dynamic range calculation unit 302 does not compare the peak luminance with the above-mentioned threshold th1', and evaluates the average luminance using a threshold Th4 different from the above-mentioned threshold Th2'. Do. The threshold Th4 is a value larger than the threshold Th2', and as an example, in the threshold calculation step of S602, based on the ideal peak brightness and the average brightness when an ideal normal distribution is assumed and a saturation signal does not exist. Can be calculated. Conversely, the threshold Th4 may be calculated based on the relationship between the peak brightness and the average brightness in the empirical distribution.

Then, in S623, the dynamic range calculation unit 302 compares the average brightness of the small area of the area number i with the threshold value Th4, and determines whether or not the average brightness exceeds the threshold value Th4. Here, in the case of a small region including pixels whose peak brightness exceeds the maximum value of the code amount, as described above, the signal whose peak brightness exceeds the maximum value of the code amount is not output. The exact average brightness of is not calculated. Therefore, in the processing of S623, by using the threshold Th4 having a value larger than the threshold Th2'for comparison, the evaluation for the pseudo average brightness corresponding to the average brightness of the small region where the peak brightness exceeds the maximum value of the code amount is evaluated. I try to do it. When the dynamic range calculation unit 302 determines in S623 that the average brightness exceeds the threshold value Th4 (Yes), the dynamic range calculation unit 302 proceeds to S608 in FIG. 9A. On the other hand, when the dynamic range calculation unit 302 determines (No) that the average brightness does not exceed the threshold value Th4, the dynamic range calculation unit 302 proceeds to S609 in FIG. 9A.

In this way, the dynamic range calculation unit 302 performs the processes of FIGS. 9 (a) and 9 (b) to set the D range level even when there is a small region where the peak brightness exceeds the maximum value of the code amount. It becomes possible to calculate.

In the above-described embodiment, an example in which exposure control is performed according to a change in the amount of light due to ND filter switching has been described, but the present invention is not limited to this example.
For example, if the aperture mechanism does not have sufficient resolution and the aperture value can only be switched step by step, the exposure control using the aperture mechanism also causes a momentary change in exposure as described above. To do. Therefore, by performing the same dynamic range control as described above when the exposure is changed momentarily by the aperture mechanism, shooting can be resumed immediately with the optimum image quality, and it is possible to prevent the decisive moment from being missed.
Further, for example, the present invention is similarly effective in exposure control using a device that changes the amplification factor of an image signal stepwise by switching an analog circuit on an image sensor. That is, when the amplification factor of the image signal is changed stepwise by switching the analog circuit on the image sensor, the dynamic range is controlled in the same manner as described above to obtain the optimum image quality. Shooting can be resumed immediately, and it is possible to prevent missing a decisive moment.
Further, for example, regarding the exposure time, for example, when increasing the number of frames in which the charge is continuously accumulated, the exposure time may not be gradually increased or decreased, and the exposure may change instantaneously. The present invention is also applicable in such cases. That is, when the exposure time is changed and the exposure is changed momentarily, by controlling the dynamic range in the same manner as described above, it is possible to immediately resume shooting with the optimum image quality and prevent the decisive moment from being missed. Can be done.
It should be noted that these are examples, and the present invention is not limited to these specific examples, and various forms (aperture value control, exposure time control, shooting sensitivity control) within a range that does not deviate from the gist of the present invention are also the present invention. include. In addition, the above-mentioned examples of exposure control are not limited to the cases where they are individually performed, but some of them may be used in combination as appropriate.

The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

The above-described embodiments are merely examples of embodiment in carrying out the present invention, and the technical scope of the present invention should not be construed in a limited manner by these. That is, the present invention can be implemented in various forms without departing from the technical idea or its main features.

22: Imaging unit, 24: Image processing unit, 50: System control unit, 100: Digital video camera, 101: Aperture, 104: ND filter mechanism, 241: Gain control unit, 242: Gamma circuit, 300: ND control unit, 301: Exposure control unit, 302: Dynamic range calculation unit, 303: Dynamic range control unit, 304: Brightness acquisition unit

Claims (18)

An imaging device that captures a subject
A brightness acquisition means for acquiring the brightness information of the subject, and
An exposure control means that controls the exposure during imaging,
It has a calculation means for calculating a dynamic range set in the image pickup apparatus based on the brightness information.
The calculation means calculates the first dynamic range set in the current image pickup apparatus, and sets the second dynamic range in the image pickup apparatus in relation to the exposure control when the exposure is changed when the exposure is switched. An imaging device characterized by calculating a range. The exposure control means includes exposure control according to the exposure change by switching the shooting sensitivity, exposure control according to the exposure change by switching the exposure time, exposure control according to the exposure change by switching the aperture value, and light transmission. The imaging device according to claim 1, wherein at least one exposure control is performed among the exposure controls corresponding to the exposure change due to the switching of the rate. The imaging device according to claim 2, wherein the switching of the light transmittance is performed by switching at least a case where the ND filter is arranged and a case where the ND filter is not arranged on the optical path of the imaging optical system. The imaging device according to claim 3, wherein the switching of the light transmittance by the ND filter includes switching and arranging a plurality of ND filters having different light transmittances on the optical path of the imaging optical system. The exposure control means changes the shooting sensitivity in the direction of canceling the change in exposure due to the change in the light transmission rate when the exposure is controlled according to the change in the exposure due to the change in the light transmission rate. The imaging apparatus according to any one of claims 2 to 4, wherein at least one of exposure control for changing the time and exposure control for changing the aperture value is performed. The first dynamic range is a dynamic range calculated according to the brightness information before the exposure change that switches the exposure occurs.
The imaging device according to any one of claims 1 to 5, wherein the second dynamic range is a dynamic range set in the imaging device immediately after an exposure change in which the exposure is switched occurs. The calculation means determines whether or not the brightness of the image captured before and after the exposure change changes, and according to the result of the determination, the first dynamic range and the second dynamic range are obtained. The imaging apparatus according to claim 6, wherein it is determined whether or not to shift to the process of calculating. When the calculation means shifts to the process of calculating the second dynamic range together with the first dynamic range, the first is based on the amount of change that occurs in the brightness of the image captured before and after the exposure change. The imaging apparatus according to claim 7, wherein the process of calculating the dynamic range of 2 is changed. 8. The calculation means is characterized in that it determines whether or not to calculate the second dynamic range by comparing the threshold value calculated based on the amount of change in brightness with the brightness information. The imaging apparatus according to. The calculation means is
The first dynamic range is calculated based on the information of the brightness exceeding a predetermined first threshold value.
As the threshold value used for calculating the second dynamic range, a second threshold value is calculated based on the first threshold value and the amount of change in the brightness, and the brightness information exceeding the second threshold value is obtained. The imaging device according to claim 9, wherein the second dynamic range is calculated based on the calculation. The calculation means determines whether or not the brightness of the subject is saturated based on the brightness information, and if it is determined that the subject is saturated, a third threshold value larger than the second threshold value. 10. The imaging device according to claim 10, wherein the image is used for comparison with the brightness information when calculating the second dynamic range. The imaging device according to any one of claims 1 to 11, further comprising a control means for controlling the dynamic range set in the imaging device based on the calculated dynamic range. The imaging device according to claim 12, wherein the control means controls the dynamic range by controlling at least one of exposure and gradation conversion characteristics. 13. The thirteenth aspect of the present invention, wherein the control means controls the dynamic range of the image pickup apparatus by allocating an exposure value determined by the exposure control means based on the calculated dynamic range. Imaging device. The control means
When shifting the dynamic range of the image sensor that captures an image to the high-brightness side, control of the shooting sensitivity or exposure time to underexpose and control to increase the gain rate for the captured image signal. By controlling the dynamic range of the image pickup device,
When the dynamic range of the imaging sensor that captures an image is shifted to the low brightness side, the imaging is performed by controlling the shooting sensitivity or the exposure time so as to overexpose and controlling the gain ratio. The imaging device according to claim 14, wherein the dynamic range of the device is controlled. The control means changes the dynamic range set in the image pickup apparatus from the first dynamic range to the second dynamic range within the period in which the exposure is changed due to the exposure change in which the exposure is switched. The imaging apparatus according to any one of claims 12 to 15. It is a control method of an imaging device that images a subject.
The brightness acquisition process for acquiring the brightness information of the subject, and
An exposure control process that controls the exposure during imaging,
It has a calculation step of calculating a dynamic range set in the image pickup apparatus based on the brightness information.
In the calculation step, the first dynamic range set in the current image pickup apparatus is calculated, and the second dynamic range set in the image pickup apparatus is set in relation to the exposure control when the exposure is changed when the exposure is switched. A control method for an image pickup device, which comprises calculating a range. A program for causing a computer included in an image pickup apparatus to function as each means of the image pickup apparatus according to any one of claims 1 to 16.

Images