JP2017034473A - Photographing device and control method thereof - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging device and its control method capable of inhibiting an unnatural change of noise feeling from occurring in a time-lapse image.SOLUTION: An imaging device includes imaging means for periodically imaging an optical image, image processing means for performing noise reduction processing of an image acquired by the imaging means imaging the optical image, and determination means for determining whether or not to perform noise reduction processing, and performs the noise reduction processing of an image of the current frame and images to be acquired after the image of the current frame while changing a degree of the noise reduction processing in a case that there is a switching of execution and non-execution of the noise reduction processing.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a photographing apparatus and a control method thereof.

  In recent years, photographing apparatuses such as digital cameras have been able to be used with a battery for a long time by reducing power consumption. Furthermore, the number of images that can be taken has increased due to the increase in the capacity of storage for recording images. With the improvement in performance of such a photographing apparatus, it has become possible to easily photograph a low-speed moving image regardless of whether it is indoors or outdoors.

  When shooting time-lapse movies, it is expected that images that gradually change will be obtained. However, when a person, animal, vehicle, or the like that has accidentally passed is reflected, the image becomes discontinuous and unsightly. In order to solve such a problem, the previously captured image is compared with the image captured this time, and if the difference between them is less than or equal to the threshold, the image captured this time is adopted, and if these differences are greater than or equal to the threshold, A technique for repeating photographing a predetermined number of times has been proposed (Patent Document 1). According to Patent Document 1, it is possible to prevent discontinuous images from being taken.

  On the other hand, in the slow-speed moving image, it is possible to ensure a sufficiently long shooting time per frame as compared with a moving image having a high frame rate. For this reason, when shooting a slow-speed moving image, there are few restrictions on the reading time when reading an image from the image sensor, and it is possible to perform shooting with a high resolution and a high dynamic range. In addition, in the case of shooting by long-second exposure, it is possible to perform noise reduction processing after shooting each frame.

JP 2011-244336 A

  However, in the slow-speed moving image acquired by the conventional imaging device, an unnatural change in noise may be observed.

  An object of the present invention is to provide an imaging apparatus capable of suppressing an unnatural change in noise sensation from occurring in a slow-speed moving image and a control method therefor.

  According to one aspect of the present invention, an imaging unit that periodically captures an optical image, an image processing unit that performs noise reduction processing on an image acquired by the imaging unit by capturing the optical image, The determination means for determining whether or not to perform the noise reduction processing, and the determination means that the noise reduction processing is not performed for the image of the previous frame, but for the image of the current frame When the determination unit determines that the noise reduction process is performed, or the determination unit determines that the noise reduction process is performed on the image of the previous frame, If the determination means determines that the noise reduction processing is not performed, the image of the current frame and the current frame are changed while changing the degree of the noise reduction processing. After As to the images acquired perform the noise reduction processing, the image processing means image pickup apparatus characterized by and a control means for controlling is provided.

  According to the present invention, when switching between implementation and non-implementation of noise reduction processing at the time of long-second exposure occurs during shooting of a slow-speed moving image, noise is applied to the image while changing the degree of noise reduction processing. Perform reduction processing. For this reason, it is possible to prevent the noise feeling from changing unnaturally before and after switching between implementation and non-implementation of noise reduction processing during long-second exposure.

1 is a block diagram illustrating a configuration of an imaging apparatus according to a first embodiment of the present invention. It is a figure which shows notionally the noise reduction process at the time of long-second exposure. 3 is a flowchart illustrating an operation of the imaging apparatus according to the first embodiment of the present invention. It is a figure which shows an example of the judgment criteria of whether the noise reduction process at the time of long-second exposure is required. It is a graph which shows the blacking gain based on Formula (1), and the blacking gain based on Formula (2). It is a flowchart which shows operation | movement of the imaging device by 2nd Embodiment of this invention. It is a figure which shows notionally the noise reduction process at the time of long-second exposure. It is a figure which shows notionally the noise reduction process at the time of long-second exposure.

  When a time-lapse movie is taken over a long period of time, it may be taken from daytime to nighttime, or from nighttime to daytime. In outdoor shooting at night, the exposure time becomes very long, so noise reduction processing during long-second exposure may be required. On the other hand, in daytime shooting, the exposure time is sufficiently short, so that noise reduction processing during long-second exposure is not necessary. Therefore, switching between the implementation and non-execution of noise reduction processing during long-second exposure occurs in shooting a low-speed moving image that extends from daytime to nighttime or shooting a low-speed moving image that extends from nighttime to daytime. When the images before and after switching between the implementation and non-implementation of the noise reduction process at the time of long-second exposure are compared, a remarkable difference in noise feeling is observed. For this reason, when switching between implementation and non-implementation of noise reduction processing during long-second exposure occurs during shooting of a slow-speed moving image, the noise sensation changes unnaturally. The inventor of the present application has made the following inventions as a result of intensive studies in view of the above problems.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[First Embodiment]
An imaging apparatus and a control method thereof according to the first embodiment will be described with reference to FIGS. FIG. 1 is a block diagram illustrating the configuration of the imaging apparatus according to the present embodiment.

  The taking lens 103 includes a focus lens. The shutter 101 has an aperture function. The imaging unit 22 includes an imaging element such as a CCD or a CMOS element that converts an optical image into an electrical signal. A plurality of pixels (not shown) are arranged in a matrix on the light receiving surface (not shown) of the imaging unit 22. The A / D converter 23 converts the analog signal output from the imaging unit 22 into a digital signal. The barrier 102 covers the photographing lens 103 and the like, thereby preventing the imaging system including the photographing lens 103, the shutter 101, and the imaging unit 22 from being soiled or damaged.

  The image processing unit 24 performs resize processing such as predetermined pixel interpolation and reduction, color conversion processing, and the like on the data from the A / D converter 23 or the data from the memory control unit 15. In addition, the image processing unit 24 performs predetermined calculation processing using the captured image data, and transmits a calculation result obtained by the calculation processing to the system control unit 50. The system control unit 50 performs exposure control, distance measurement control, and the like based on the calculation result obtained by the image processing unit 24. As a result, TTL (through the lens) AF (autofocus) processing, AE (automatic exposure) processing, EF (flash pre-emission) processing, and the like are performed. The image processing unit 24 performs predetermined calculation processing using the captured image data, and also performs TTL AWB (auto white balance) processing based on the calculation result obtained by the calculation processing. The image processing unit 24 also performs noise reduction processing (noise reduction processing, NR processing) during long second exposure when long second exposure (long exposure) is performed. The noise reduction process at the time of long-second exposure is a process for removing fixed noise by capturing a black image after capturing the main image and subtracting the black image from the main image. Since such a process is a process of subtracting a black image, it is also referred to as a black drawing process. The black image is acquired in a state where outside light is blocked using the shutter 101, for example. The exposure time of the black image is set to be the same as the exposure time of the main image, for example.

  FIG. 2 is a diagram conceptually showing noise reduction processing during long-second exposure. FIG. 2A shows noise reduction processing during long-second exposure in the imaging apparatus according to the reference example, and FIG. 2B shows noise reduction processing during long-second exposure in the imaging apparatus according to the present embodiment. ing. As shown in FIG. 2A, the image pickup apparatus according to the reference example can execute only the on or off of the noise reduction process at the time of long-second exposure. On the other hand, in the imaging apparatus according to the present embodiment, as shown in FIG. 2B, the gain of the black image can be adjusted, so that the ratio of the black image to the main image can be changed stepwise. .

  Digital data output from the A / D converter 23 is written to the memory 32 via the image processing unit 24 and the memory control unit 15 or via the memory control unit 15. The memory 32 stores image data obtained by the imaging unit 22 and converted into digital data by the A / D converter 23 and image data to be displayed on the display unit 28. The memory 32 has a storage capacity sufficient to store a predetermined number of still images, a moving image and sound for a predetermined time.

  The memory 32 also serves as an image display memory (video memory). The D / A converter 13 converts the image display data output from the memory 32 into an analog signal, and transmits the analog signal to the display unit 28. As a result, the display image data written in the memory 32 is displayed by the display unit 28. The display unit 28 includes a display device such as an LCD. The display unit 28 displays an image corresponding to the analog signal from the D / A converter 13 using a display. The analog signal output from the imaging unit 22 is converted into a digital signal by the A / D converter 23, and the digital signal is temporarily stored in the memory 32. The digital signal temporarily stored in the memory 32 is converted into an analog signal by the D / A converter 13, and the analog signal is sequentially transmitted to the display unit 28. The display unit 28 displays an image corresponding to the analog signal sequentially transmitted from the D / A converter 13. The display unit 28 functions as an electronic viewfinder (through image display).

  The nonvolatile memory 56 is a memory that can electrically erase and record information, and for example, an EEPROM or the like is used. The nonvolatile memory 56 stores constants and programs for operating the system control unit 50. Such a program is a program for controlling the imaging apparatus according to the present embodiment. For example, an operation shown in FIG. 3 is executed by the program.

The system control unit 50 controls the entire imaging apparatus (hereinafter also referred to as “digital camera”) 100 according to the present embodiment. By executing the program recorded in the nonvolatile memory 56, each process described later is executed. For example, a RAM is used as the system memory 52. In the system memory 52, constants and variables for operating the system control unit 50, programs read from the nonvolatile memory 56, and the like are expanded. The system control unit 50 also performs display control by controlling the memory 32, the D / A converter 13, the display unit 28, and the like.
The system timer 53 measures the time used for various controls and the time of a built-in clock.

  The mode switch 60, the first shutter switch (not shown), the second shutter switch (not shown), and the operation unit 70 are operations for instructing the system control unit 50 to perform various operations. Means.

  The mode switch 60 switches the operation mode of the system control unit 50 to any one of a still image recording mode, a moving image recording mode, a reproduction mode, and the like. The first shutter switch is turned on when the shutter button 61 provided in the digital camera 100 is being operated, so-called half-press (shooting preparation instruction), and generates the first shutter switch signal SW1. Operations such as AF (autofocus) processing, AE (automatic exposure) processing, AWB (auto white balance) processing, and EF (flash pre-emission) processing are started by the first shutter switch signal SW1.

  The second shutter switch is turned on when the operation of the shutter button 61 is completed, so-called full press (shooting instruction), and the second shutter switch signal SW2 is generated. In response to the second shutter switch signal SW2, the system control unit 50 starts a series of photographing processing operations from reading of an image signal from the imaging unit 22 to writing of image data to the recording medium 200.

  Each operation member of the operation unit 70 is appropriately assigned a function for each scene by selecting and operating various function icons displayed on the display unit 28, and functions as various function buttons. Examples of the function buttons include an end button, a return button, an image advance button, a jump button, a narrowing button, and an attribute change button. For example, when a menu button is pressed, a menu screen on which various settings can be made is displayed on the display unit 28. For example, when a menu button is pressed in the moving image recording mode, the frame rate and resolution can be selected. Time-lapse moving image (time-lapse moving image) can also be selected on the menu screen. The user can make various settings intuitively using the menu screen displayed on the display unit 28, the four-way button, and the SET button.

  A controller wheel (not shown) is a rotatable operation member included in the operation unit 70, and is used when a selection item is instructed in the same manner as the direction button. When the controller wheel is rotated, an electrical pulse signal corresponding to the operation amount is generated. The system control unit 50 controls each unit of the digital camera 100 based on the pulse signal output from the controller wheel. The system control unit 50 can determine the angle at which the controller wheel is rotated, the number of rotations of the controller wheel, and the like based on the pulse signal output from the controller wheel. The controller wheel may be any operation member that can detect a rotation operation. For example, the controller wheel may be a dial operation member that generates a pulse signal as the controller wheel itself rotates in response to a user's rotation operation. Further, the controller wheel may be an operation member composed of a touch sensor. For example, the rotation of the user's finger on the controller wheel may be detected without rotating the controller wheel itself. Such a controller wheel is referred to as a touch wheel.

  The power control unit 80 includes a battery detection circuit, a DC-DC converter, a switch circuit that switches a block to be energized, and the like, and detects whether or not a battery is installed, the type of battery, and the remaining battery level. Further, the power supply control unit 80 controls the DC-DC converter based on these detection results and instructions from the system control unit 50, and supplies a necessary voltage to each unit including the recording medium 200 for a necessary period.

  The power supply unit 30 includes a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li battery, an AC adapter, or the like. The power switch 72 is for switching on / off the power of the digital camera 100. The recording medium 200 is configured by a memory card, a semiconductor memory, a magnetic disk, or the like. Writing of image data to the recording medium 200, reading of image data from the recording medium 200, and the like are performed via the interface 18.

  FIG. 3 is a flowchart illustrating the operation of the imaging apparatus according to the present embodiment. FIG. 3 shows a noise reduction process for removing fixed pattern noise at the time of shooting a low-speed moving image. The operation shown in FIG. 3 is executed under the control of the system control unit 50 which is a control means. The setting as to whether or not to shoot a slow-speed moving image is made before the start of shooting by a menu operation by the user.

  In step S <b> 201, the system control unit 50 determines whether or not shooting is finished. Conditions for determining the end of shooting include a case where conditions such as a shooting time have expired, a case where a shooting end operation performed by a user is accepted, and a case where an abnormality such as a drop in power supply voltage is detected. If the shooting end condition is satisfied (YES in step S201), the shooting process ends. On the other hand, if the conditions for ending the shooting are not satisfied, the process proceeds to step S202.

  In step S202, the system control unit 50 determines whether the elapsed time from the previous shooting has reached a preset interval time. The interval time is appropriately set according to the entire shooting time and according to the length at the time of moving image reproduction. For example, in order to shoot a moving image with a frame rate of 30 fps and 10 × speed reproduction, it is sufficient to shoot at a frame rate of 30/10 = 3 fps. The maximum exposure time when shooting at 3 fps is 1/6 second because it is equivalent to the time of one frame at 6 fps in consideration of black image shooting for noise reduction processing. If the interval time has elapsed (YES in step S202), the process proceeds to step S203. On the other hand, if the interval time has not elapsed (NO in step S202), the process returns to step S201.

  In step S <b> 203, the optical image formed on the light receiving surface of the imaging unit 22 by the imaging lens 103 is captured by the imaging unit 22, thereby acquiring the main image. In order to distinguish from a black image acquired for performing noise reduction processing during long-second exposure, an image acquired by photographing performed at this stage is referred to as a “main image”. When the photographing of the main image is completed, the process proceeds to step S204.

  In step S204, the system control unit 50 collects shooting information when the main image is shot in step S203. Examples of such shooting information include setting of shooting conditions, shooting environment information, and the like. More specifically, for example, the shutter speed, aperture value, ISO sensitivity, white balance setting, focal length, lens information used, temperature of the image sensor, and the like. When the collection of the shooting information is completed, the process proceeds to step S205.

  In step S205, the system control unit 50 determines whether or not it is necessary to perform noise reduction processing during long-second exposure based on the imaging information collected in step S204. FIG. 4 is a diagram illustrating an example of a criterion for determining whether or not it is necessary to perform noise reduction processing during long-second exposure. As a general tendency, fixed pattern noise becomes conspicuous as the exposure time Tv increases and the sensor temperature T increases. Therefore, it is determined whether or not to perform noise reduction processing during long-second exposure using the exposure time Tv and the sensor temperature T as parameters. If it is determined that it is necessary to perform noise reduction processing during long-second exposure (YES in step S205), the process proceeds to step S206. On the other hand, when it is determined that it is not necessary to perform the noise reduction process during the long second exposure (NO in step S205), the process proceeds to step S207.

Step S206 is a step for increasing the blacking gain step by step. On the other hand, step S207 is a step for decreasing the blacking gain step by step. These steps are characteristic steps in the present invention. The blacking gain indicates the degree of blacking, the minimum value is 0, and the maximum value is 1. In other words, the blacking gain is the ratio of the black image to the main image. When the main image is 1, the black image used when noise reduction processing is performed on the main image is greater than 0 and less than or equal to 1. The blacking gain is increased or decreased step by step, and when the blacking gain reaches the maximum value or the minimum value, the operation of increasing or decreasing the blacking gain is terminated. In order to reduce the amount of change of the black gain at the start point and end point of increase / decrease of the black gain, the black gain may be set based on the following equation (1). However, Formula (1) is applied in the process (step S206) which increases the blacking gain G in steps. N indicates the number of operations divided until the blacking gain reaches the maximum value, and n indicates the number of operations.
G = (1/2) × [sin {((n / N) − (1/2)) × π} +1] (1)

Further, the blacking gain G may be set based on the following equation (2). When the blacking gain is set based on Expression (2), the change rate of the blacking gain is constant.
G = n / N (2)

  FIG. 5 is a graph showing the blacking gain based on Expression (1) and the blacking gain based on Expression (2). The horizontal axis shows how many times the operation for increasing / decreasing the blacking gain is, and the vertical axis shows the value of the blacking gain. The blacking gain based on Expression (1) is plotted using the ♦ marks, and the blacking gain based on Expression (2) is plotted using the ■ marks.

  The larger the division number N, the smaller the amount of change in blacking gain. For this reason, the larger the number of divisions N, the smaller the change in noise feeling before and after switching between noise reduction processing during long-second exposure and non-execution.

  However, when the number of divisions N is too large, the amount of change in blacking gain becomes too small, and noise reduction processing at the time of long second exposure is sufficient for an image that should be sufficiently subjected to noise reduction processing at the time of long seconds exposure. The noise is conspicuous.

  On the other hand, when the number of divisions N is too small, the amount of change in blacking gain becomes too large, and the change in noise feeling becomes too large. When shooting a scene where the shooting interval time is short and the image changes slowly, it is not preferable that the division number N is too small. Therefore, the selection of the division number N is preferably adjusted according to the interval time.

When the black gain operation is completed, the process proceeds to step S208.
In step S208, it is determined whether or not the blacking gain is greater than zero. If the blacking gain is greater than 0 (YES in step S208), the process proceeds to step S209. On the other hand, when the blacking gain is 0 (NO in step S208), the process proceeds to step S201.

  In step S209, a black image is photographed in order to perform noise reduction processing during long-second exposure. In shooting a black image, the shutter 101 is completely closed, and shooting is performed with the same settings as the main image without taking in external light. When the shooting of the black image is completed, the process proceeds to step S210.

In step S210, noise reduction processing (NR processing) is performed. In the noise reduction process, the system control unit 50 drives the image processing unit 24 to perform image calculation processing. The output value of each pixel in the main image is R (h, v), the output value (pixel value) of each pixel in the black image is D (h, v), the blacking gain is G, and the black level offset component is O. Then, the synthesized image I (h, v) after the noise reduction process is expressed by the following formula (3).
I (h, v) = R (h, v) −G × D (h, v) + O (3)
When the noise reduction process during the long second exposure is completed, the process returns to step S201.

  Since the value of the black pull gain G multiplied by the output value D (h, v) of each pixel of the black image changes stepwise, the intensity of the noise reduction processing during long second exposure also changes stepwise.

  As described above, in this embodiment, the system control unit 50 determines that the noise reduction process is not performed on the main image of the previous frame, and the system control is performed when the noise reduction process is performed on the main image of the current frame. When the unit 50 determines, the operation is as follows. That is, the system control unit 50 performs the noise reduction process on the main image of the current frame and the main image acquired after the current frame while gradually increasing the degree of the noise reduction process. In this embodiment, the system control unit 50 determines that the noise reduction process is performed on the main image of the previous frame, but the system control unit does not perform the noise reduction process on the main image of the current frame. When 50 is determined, the operation is as follows. That is, the system control unit 50 performs noise reduction processing on the main image of the current frame and the main image acquired after the current frame while gradually decreasing the degree of noise reduction processing. That is, in this embodiment, when switching between implementation and non-implementation of noise reduction processing during long-second exposure occurs during slow-speed movie shooting, the degree of noise reduction processing during long-second exposure gradually changes. Let More specifically, the blacking gain is gradually changed. For this reason, according to the present embodiment, the intensity of the noise reduction process at the time of long-second exposure can be gradually changed, so that it is possible to prevent the noise sensation from changing abruptly.

[Second Embodiment]
An imaging apparatus and a control method thereof according to the second embodiment will be described with reference to FIGS. The same components as those of the imaging apparatus and the control method thereof according to the first embodiment shown in FIGS. 1 to 5 are denoted by the same reference numerals, and description thereof will be omitted or simplified.

  In the first embodiment, when switching between implementation and non-execution of noise reduction processing during long-second exposure occurs during shooting of a slow-speed moving image, the noise sensitivity is changed by changing the blacking gain G stepwise. Was prevented from changing rapidly. In this embodiment, when switching between implementation and non-implementation of noise reduction processing during long-second exposure during slow-speed movie shooting, the exposure time when shooting a black image is changed stepwise. This prevents the noise sensation from changing suddenly.

  The configuration of the imaging apparatus according to the present embodiment is the same as the configuration of the imaging apparatus according to the first embodiment described above with reference to FIG.

FIG. 6 is a flowchart illustrating the operation of the imaging apparatus according to the present embodiment. Steps S601 to S608 are the same as steps S201 to S208 described above with reference to FIG. The difference between the imaging device control method according to the first embodiment and the imaging device control method according to the present embodiment is the processing performed in step S609 and the processing performed in step S610. In the first embodiment, the exposure time for photographing the main image and the exposure time for the black image used for performing noise reduction processing on the main image are set to be the same (step S209). On the other hand, in the present embodiment, the exposure time when shooting the main image and the exposure time when shooting a black image for performing noise reduction processing on the main image are not set to be the same. The exposure time for taking an image is changed stepwise (step S609). The exposure time Dtv when shooting a black image can be calculated using the following equation (4), where Rtv is the exposure time when shooting the main image and G is the blacking gain.
Dtv = Rtv × G (4)
In step S609, a black image is taken with the exposure time Dtv obtained using equation (4).

In step S610, noise reduction processing during long second exposure is performed. In the noise reduction processing during long-second exposure, the system control unit 50 drives the image processing unit 24 to perform image calculation processing. If the output value of each pixel in the main image is R (h, v), the output value of each pixel in the black image is D ′ (h, v), and the black level offset component is O, the composite image after the noise reduction processing I (h, v) is represented by the following formula (5).
I (h, v) = R (h, v) −D ′ (h, v) + O (5)
When the noise reduction process during the long second exposure is completed, the process returns to step S601.

  7a and 7b are diagrams conceptually showing noise reduction processing during long-second exposure. FIGS. 7a and 7b schematically show the amount of light in the surrounding environment, the necessity of performing noise reduction processing during long-second exposure, and the exposure time when shooting a black image.

  FIG. 7a shows noise reduction processing during long-second exposure in the imaging apparatus according to the reference example. As shown in FIG. 7a, the main images 400 to 409 are periodically acquired in shooting a slow-speed moving image. That is, the main images 400 to 409 are sequentially acquired at a predetermined interval time. When photographing the main image 400, since the surroundings are bright, that is, the amount of light in the surrounding environment is larger than a predetermined value, black reduction, that is, noise reduction processing during long second exposure is not performed. When the surroundings become dark, that is, when the amount of light in the surrounding environment becomes smaller than a predetermined value, blackening, that is, noise reduction processing during long second exposure is performed. For this reason, black images 411 to 416 are acquired for performing noise reduction processing during long-second exposure for each of the main images 401 to 406. The exposure time when photographing the main image 401 and the exposure time when photographing the black image 411 are set to be the same. Similarly, the exposure times when the main images 402 to 406 are photographed and the exposure times when the black images 412 to 416 are photographed are set to be the same. By subtracting the black image 411 from the main image 401, a composite image 431 that has been subjected to noise reduction processing during long-second exposure is obtained. Similarly, by subtracting the black images 412 to 416 from the main images 402 to 406, respectively, synthesized images 432 to 436 subjected to noise reduction processing during long-second exposure are obtained. When photographing the main images 407 to 409, since the surroundings are bright, that is, the amount of light in the surrounding environment is larger than a predetermined value, black reduction, that is, noise reduction processing during long second exposure is not performed.

  As described above, in the imaging apparatus according to the reference example, the main image 400 is not subjected to the noise reduction process during the long second exposure, while the main image 401 is subjected to the noise reduction process during the long second exposure. Is called. Further, while the main image 406 is subjected to noise reduction processing during long-second exposure, the main image 407 is not subjected to noise reduction processing during long-second exposure. For this reason, in the imaging apparatus according to the reference example, the noise reduction process at the time of long-second exposure is suddenly performed, or the noise reduction process at the time of long-second exposure is not suddenly performed. For this reason, in the imaging apparatus according to the reference example, a sudden change in the noise sensation occurs before and after switching between implementation and non-implementation of noise reduction processing during long-second exposure.

  FIG. 7B shows a noise reduction process during long second exposure in the imaging apparatus according to the present embodiment. As shown in FIG. 7b, the main images 450 to 459 are periodically acquired when shooting the slow-speed moving image. That is, the main images 450 to 459 are sequentially acquired at a predetermined interval time. When the main image 450 is photographed, since the surroundings are bright, that is, the amount of light in the surrounding environment is larger than a predetermined value, black reduction, that is, noise reduction processing during long second exposure is not performed. The blacking gain at this time is zero. When the surroundings become dark, that is, when the amount of light in the surrounding environment becomes smaller than a predetermined value, blackening, that is, noise reduction processing during long second exposure is performed. The main image 451 is at a stage where noise reduction at the time of long-second exposure has just been required. For this reason, the exposure time when shooting the black image 461 is set sufficiently shorter than the exposure time when shooting the main image 451. Then, a composite image 471 is generated by subtracting the black image 461 from the main image 451. In order to gradually change the noise feeling of the image, the blacking gain is gradually increased. The ratio of the exposure time when shooting the black image 462 to the exposure time when shooting the main image 452 is higher than the ratio of the exposure time when shooting the black image 461 with respect to the exposure time when shooting the main image 451. It is set large. Then, a composite image 472 is generated by subtracting the black image 462 from the main image 452. In this way, composite images 471 and 472 that have been subjected to noise reduction processing during long second exposure while gradually increasing the blacking gain are obtained.

  The main images 453 to 456 are taken after the blacking gain reaches 1. For this reason, the exposure time when shooting the black images 463 to 466 is set to be the same as the exposure time when shooting the main images 453 to 456. By subtracting the black images 463 to 466 from the main images 453 to 456, respectively, synthesized images 473 to 476 that have been subjected to noise reduction processing during long-second exposure are obtained.

  The main image 457 is at a stage where noise reduction processing during long-second exposure is no longer necessary. If the noise reduction process at the time of long-second exposure is immediately stopped at this stage, the feeling of noise changes abruptly. Therefore, in this embodiment, the noise reduction process at the time of long-second exposure is not stopped immediately, but the blacking gain is decreased stepwise. Specifically, the exposure time when shooting the black image 467 is set shorter than the exposure time when shooting the main image 457. Then, a composite image 477 is generated by subtracting the black image 467 from the main image 457. The ratio of the exposure time when shooting the black image 468 to the exposure time when shooting the main image 458 is higher than the ratio of the exposure time when shooting the black image 467 to the exposure time when shooting the main image 457. Set small. Then, a composite image 478 is generated by subtracting the black image 468 from the main image 458. In this way, composite images 477 and 478 that have been subjected to noise reduction processing during long second exposure while gradually reducing the black gain are obtained.

  The main image 459 is captured after a sufficient amount of time has elapsed since it has become unnecessary to perform noise reduction processing during long-second exposure. The blacking gain at this time is zero. For this reason, noise reduction processing during long-second exposure is not performed on the main image 459.

  As described above, in the present embodiment, when switching between implementation and non-implementation of noise reduction processing during long-second exposure occurs during shooting of a slow-speed moving image, the shooting time of the black image is changed stepwise. For this reason, it is possible to prevent the noise sensation from changing abruptly also according to this embodiment.

  Although the present invention has been described in detail based on the preferred embodiments, the present invention is not limited to these specific embodiments, and various modifications within the scope of the present invention are also included in the present invention. It is. Moreover, you may combine a part of said embodiment suitably.

  In the above embodiment, the case where the noise reduction process is performed by subtracting the black image from the main image has been described as an example. However, the method of the noise reduction process is not limited to this. For example, the noise reduction processing may be performed by two-dimensional processing using a filter or three-dimensional processing using a plurality of frames of images. Then, the degree of noise reduction processing may be changed by changing the filter coefficient and the number of frames.

  In the above embodiment, the case where the ratio of the black image to the main image is changed stepwise has been described as an example. However, the ratio of the black image to the main image may be changed gradually.

(Other examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

22 ... Imaging unit 24 ... Image processing unit 28 ... Display unit 50 ... System control unit 60 ... Mode switch 70 ... Operation unit 100 ... Digital camera 101 ... Shutter

Claims (13)

An imaging means for periodically taking an optical image;
Image processing means for performing noise reduction processing on an image acquired by the imaging means by capturing the optical image;
Determining means for determining whether to perform the noise reduction processing;
The determination unit determines that the noise reduction process is not performed on the image of the previous frame, but the determination unit determines that the noise reduction process is performed on the image of the current frame, or The determination unit determines that the noise reduction process is performed on the image of the previous frame, but the determination unit determines that the noise reduction process is not performed on the image of the current frame. In this case, the image processing means is controlled to perform the noise reduction process on the image of the current frame and the image acquired after the current frame while changing the degree of the noise reduction process. Control means for
An imaging device comprising: The image processing means captures the optical image and captures the main image, which is the image obtained by the imaging means, and a black image obtained by the imaging means by capturing an image while blocking external light. The image pickup apparatus according to claim 1, wherein noise reduction processing is performed on the main image using. The exposure time of the main image and the exposure time of the black image used when performing the noise reduction processing on the main image are set to be the same,
The image processing means changes a ratio of the black image to the main image by changing a value to be multiplied by a pixel value of each pixel of the black image, and changes the black image in which the ratio to the main image is changed. The imaging apparatus according to claim 2, wherein the imaging apparatus subtracts from the main image. The image processing means changes a ratio of the black image to the main image by changing a ratio of the exposure time when the black image is captured with respect to an exposure time when the main image is captured. The imaging apparatus according to claim 2, wherein the black image whose ratio to the image is changed is subtracted from the main image. The determination unit determines that the noise reduction process is not performed on the main image of the previous frame, but the determination unit determines that the noise reduction process is performed on the main image of the current frame. In such a case, the ratio of the black image to the main image is increased. The imaging apparatus according to claim 3 or 4, wherein: The determination unit determines that the noise reduction processing is performed on the main image of the previous frame, but the determination unit determines that the noise reduction processing is not performed on the main image of the current frame. In this case, the ratio of the black image to the main image is reduced. The imaging apparatus according to claim 3 or 4, wherein The amount of change in the ratio when the ratio of the black image with respect to the main image is changed in accordance with the setting of the interval time in the regular imaging. The imaging apparatus according to item 1. An image pickup apparatus control method for periodically taking an optical image,
Acquiring an image by capturing the optical image;
Performing noise reduction processing on the image,
The noise reduction processing is not performed on the image of the previous frame, but the noise reduction processing is performed on the image of the current frame or the noise of the image of the previous frame is performed. When the noise reduction processing is not performed on the image of the current frame after the reduction processing is performed, the image of the current frame and the current frame are changed while changing the degree of the noise reduction processing. The noise reduction processing is performed on the acquired image. A method for controlling an imaging apparatus. In the step of performing the noise reduction processing, using the main image that is the image acquired by capturing the optical image and the black image acquired by performing imaging in a state where external light is blocked, The method for controlling an imaging apparatus according to claim 8, wherein noise reduction processing is performed on the main image. In the step of acquiring the black image, the black image is acquired with the same exposure time as the exposure time of the main image on which the noise reduction processing using the black image is performed,
The control of the imaging apparatus according to claim 9, wherein in the step of performing the noise reduction processing, the noise reduction processing is performed on the main image using the black image multiplied by a changed value. Method. The method for controlling an imaging apparatus according to claim 9, wherein, in the step of acquiring the black image, the black image is captured with an exposure time in which a ratio to an exposure time when the main image is captured is changed. .   The program for making a computer perform the control method of the imaging device of any one of Claims 8-11.   The computer-readable recording medium which memorize | stored the program for making a computer perform the control method of the imaging device of any one of Claims 8-11.

Images