JP2024035736A - Image capturing device, method of controlling the same, program, and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image capturing device that enables accurate focusing.

SOLUTION: An image capturing device is provided, comprising an image capturing unit for capturing an image of an object formed by an image capturing optical system, a signal processing unit for executing noise reduction processing on the image according to the images of a plurality of frames captured by the image capturing unit, an acquisition unit for acquiring an evaluation value indicative of contrast of an image subjected to noise reduction processing by the signal processing unit, and a control unit configured to control the focus position of the image capturing optical system according to the evaluation value, where the control unit determines at least either of a driving speed and focus position of the image capturing optical system according to intensity of the noise reduction processing.

SELECTED DRAWING: Figure 1

COPYRIGHT: (C)2024,JPO&INPIT

Description

The present invention relates to an imaging device, a method of controlling the imaging device, a program, and a storage medium.

2. Description of the Related Art Conventionally, 3DNR (Digital Noise Reduction) processing, which detects and removes noise by comparing a plurality of consecutive frames, is known as digital signal processing for reducing image noise.

Patent Document 1 discloses a technique for displaying focus information using an evaluation value stabilized by temporal filter processing.

JP2015-34869

When autofocus (hereinafter referred to as AF) control for controlling the focus position of a lens is performed using evaluation values stabilized by time filter processing as in Patent Document 1, the evaluation values are smoothed over multiple frames. As a result, a focus position shifted from the original focus position becomes a peak position (a position where the evaluation value is maximum), and the focus position may not be accurately detected.

Therefore, the problem to be solved by the present invention is to provide an imaging device that can accurately focus when performing AF control based on an image that has been subjected to 3DNR processing.

In order to solve the above problems, an imaging device according to one aspect of the present invention includes an imaging unit that captures an image of a subject using an imaging optical system, and an image capturing unit that captures an image of an image based on a plurality of frames of images captured by the imaging unit. a signal processing section that performs noise reduction processing; an acquisition section that obtains an evaluation value indicating the contrast of the image on which the noise reduction processing has been performed by the signal processing section; and a focus control unit for the imaging optical system based on the evaluation value. a control unit that controls a position, the control unit determining at least one of a driving speed of the imaging optical system and a focus position of the imaging optical system based on the intensity of the noise reduction process. It is characterized by

According to the present invention, it is possible to provide an imaging device that can focus accurately.

A diagram showing the configuration of an imaging device according to the first embodiment Flowchart showing AF operation related to a conventional imaging device Diagram showing the relationship between AF evaluation value and 3DNR Diagram showing the relationship between AF evaluation value and 3DNR Flowchart showing AF operation of the imaging device according to the first embodiment A diagram showing the hardware configuration of an imaging device according to the first embodiment

DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the accompanying drawings. The embodiment described below is an example of means for realizing the present invention, and should be modified or changed as appropriate depending on the configuration of the device to which the present invention is applied and various conditions. It is not limited to.

<Embodiment 1>
(Device configuration)
FIG. 1 shows the configuration of an imaging device according to this embodiment. The imaging device 100 includes a zoom lens 101, a focus lens 102, an aperture unit 103, a bandpass filter 104, a color filter 105, an image sensor 106, an AGC (Auto Gain Control) 107, an AD conversion section 108, a signal processing section 109, and a communication section 110. , an acquisition section 112, a control section 113, and a drive section 114.

The zoom lens 101, the focus lens 102, and the aperture unit 103 constitute an imaging optical system of the imaging apparatus 100. The zoom lens 101 changes its focal length by moving in the optical axis direction. The focus lens 102 changes the focus position by moving in the optical axis direction. The aperture unit 103 adjusts the amount of light incident on the image sensor 106.

The light that has passed through the imaging optical system forms an image of the subject on the imaging surface of the imaging element 106 via the bandpass filter 104 and the color filter 105. The bandpass filter 104 may be inserted into and removed from the optical path of the imaging optical system.

The image sensor 106 is an image capture unit that captures an image of a subject using an image capture optical system. Specifically, the light incident on the image sensor 106 is photoelectrically converted and the image of the subject is output as an analog electrical signal (image signal).

The AGC 107 amplifies the electrical signal output from the image sensor 106 based on a set amplification factor and outputs the amplified signal.

The AD conversion unit 108 converts the analog image signal amplified by the AGC 107 into a digital image signal and outputs the digital image signal to the signal processing unit 109.

The signal processing unit 109 performs various image processing on the digital image signal to generate an image. For image processing, 3DNR processing can be performed to reduce noise included in the digital imaging signal. It is also possible to perform noise reduction processing and white balance adjustment using spatial filtering. Note that the 3DNR processing is a process of detecting noise included in a digital image signal based on a comparison of signals of a plurality of consecutive frames outputted by the AD conversion unit 108, and smoothing the noise between the plurality of frames. This makes it possible to reduce noise on a pixel-by-pixel basis. Whether or not to perform 3DNR processing and the intensity of 3DNR processing can be arbitrarily set by the user. The number of frames used for smoothing in 3DNR processing depends on the set intensity. The higher the intensity, the more frames are used and therefore the more powerful the noise reduction effect can be obtained. On the other hand, the higher the intensity (that is, the greater the number of frames used), the more blurred the image may be when capturing a moving object, and the more responsive the evaluation value indicating contrast may be. be.

The communication unit 110 is a network interface for outputting the image generated by the signal processing unit 109 to the monitor device 110 connected by wired or wireless communication. Note that the image does not necessarily need to be output to the monitor device 110, and may be stored in a main storage device (not shown) such as a memory or an auxiliary storage device (not shown) such as a hard disk.

The acquisition unit 112 acquires an evaluation value indicating the contrast of the image generated by the signal processing unit 109. Specifically, it can be obtained by generating a specific frequency component (for example, a high frequency component) among the spatial frequency components of the image as an evaluation value. Here, it is assumed that an evaluation value of an image on which 3DNR processing has been performed by the signal processing unit 109 is acquired.

The control unit 113 performs AF control based on the evaluation value acquired by the acquisition unit 112. Further, when performing manual focus control, focus control is performed based on user operation information input via the communication unit 110. More specifically, focus control is performed by driving the focus position of the focus lens 101 of the imaging optical system in the optical axis direction. Switching between AF control and manual is also performed via the communication unit 110 based on user operation information, but it is also possible to switch between AF and manual automatically when predetermined conditions are met (for example, temperature focus correction, etc.). ). Although more detailed focus control will be described later, at least one of the driving speed and focus position of the imaging optical system is determined by the control unit 113 based on the strength of the 3DNR processing.

The drive unit 114 controls the focus lens 102 to the focus position instructed by the control unit 113.

(Conventional AF control)
FIG. 2 is a flowchart showing an example of conventional AF control. AF control by the control unit 113 is executed by a CPU (Central Processing Unit) of the imaging device reading a program stored in a nonvolatile storage medium such as a ROM (Read Only Memory). The hardware configuration of the imaging device 100 will be described later.

AF control consists of a method in which the imaging optical system is driven widely at high speed in order to specify a rough focus position (hereinafter referred to as the "hill climbing method"), and a method in which the imaging optical system is driven minutely in order to specify a highly accurate focus position. This is performed by combining methods (hereinafter referred to as wobbling methods).

After starting the AF operation, a wobbling method is performed. In the wobbling method, the focus lens 102 is slightly driven from the current focus position, the evaluation values before and after driving are compared, and the direction in which the evaluation value increases is identified. The specified direction is set as the direction of the in-focus position (focusing direction) seen from the focus position, and the focus lens 102 is gradually driven in the in-focus direction.

In step S201, a wobbling amount is set according to the depth. Generally, the amount of wobbling is kept within one depth to make changes in the image due to focus control less noticeable, but this is not the case if the current focus position is not near the in-focus position.

In step S202, it is driven to the front focus position, and in step S203, the change in evaluation value (increase or decrease) before and after the front focus drive is acquired. The front focus position is a position obtained by subtracting the amount of wobbling from the wobbling center position (the current focus position at the first time).

Subsequently, in step S204, it is driven to the rear focus position, and in step S205, a change in the evaluation value (increased or decreased) due to the rear focus drive is acquired. The rear pin position is a position obtained by adding the wobbling amount to the wobbling center position.

In step S206, the focusing direction is specified based on the results of the change in evaluation value before and after driving the front focus and the change in evaluation value before and after driving the rear focus. When the evaluation value increases before and after driving the front focus and decreases before and after driving the rear focus, the direction of the front focus is specified as the focusing direction. If the change in evaluation value is opposite between the front focus and the rear focus, the direction of the rear focus is identified as the in-focus direction. At the same time, the wobbling center position is shifted in the focusing direction.

In step S207, it is determined whether the focusing direction is continuously the same direction. If the direction is continuously the same, the process moves to the hill climbing method (step S208), and if not, it is determined whether the number of times the focusing direction is reversed is equal to or greater than a predetermined threshold (step S209). If the number of reversals is less than a predetermined threshold, the process returns to step S201 to continue the wobbling method, and if it is greater than or equal to the predetermined threshold, it is considered that the in-focus position has been reached, and the AF process ends.

In the hill climbing method (step S208), the focus lens 102 is driven at high speed in the focusing direction, and the focus position (peak position) at which the evaluation value is maximum is detected. While the focus lens 102 is being driven, the evaluation value and focus position are stored, and it is determined whether the difference between the stored evaluation value and the current evaluation value is greater than or equal to a predetermined threshold (step S210). If it is greater than or equal to the predetermined threshold, the hill climbing method is ended and the stored focus position is detected as the peak position (step S211). When the peak position is detected, the focus lens 102 is driven to the peak position (S212), and the process returns to the wobbling method (Step S201).

If it is less than the predetermined threshold in step S210, it is determined in step S213 whether the evaluation value has continuously decreased a predetermined number of times. If it is continuously descending, it is considered that the driving is in the opposite direction to the focusing direction, so the direction of mountain climbing is reversed (step S214) and the process returns to step S209.

(Relationship between evaluation value and 3DNR processing)
Figure 3 shows the transition of the evaluation value when the focus position of the imaging optical system (focus lens 102) is driven from 0 to 280 at a driving speed of 10 per frame, and the intensity of 3DNR processing (off/weak/strong). It is shown every time. Note that the focus position is set to 140 here.

A graph 301 shows evaluation values when the intensity of 3DNR processing is off, that is, when 3DNR processing is not performed. When 3DNR processing is not performed, the responsiveness is high because there is no influence from the previous frame, and 140, which is the in-focus position, is detected as the peak position.

A graph 302 is an evaluation value when the intensity of 3DNR processing is weak, and a graph 303 is an evaluation value when the intensity of 3DNR processing is strong. 3DNR processing is image noise reduction processing based on images of multiple frames, and smoothes the image at the current focus position and at least one past focus position. Therefore, the evaluation value obtained from the image on which 3DNR processing has been performed is affected by the past frame (previous frame), so the responsiveness of the evaluation value deteriorates, and the focus position that deviates from the original focus position becomes the peak position. Detected as . The amount of deviation increases as the strength of the 3DNR processing increases (that is, as the number of frames used for the 3DNR processing increases).

Due to this deterioration in responsiveness, in the above-described hill climbing method, the timing at which the hill climbing method ends is delayed, and overshoot is likely to occur. Furthermore, even if the lens is driven to the peak position detected by the hill climbing method, blur remains because the focus position is shifted from the original focus position. Furthermore, since the focusing direction is continuously in the same direction in the subsequent wobbling method, a hunting phenomenon may occur in which the method shifts to the hill-climbing method again.

FIG. 4 is a diagram showing the transition of evaluation values when the driving speed of the imaging optical system (focus lens 102) is halved (speed of 5 drives per frame) in the example of FIG. 3. When 3DNR processing is being performed (graphs 402 and 403), the amount of deviation between the in-focus position and the peak position is smaller than in FIG. 3. That is, the driving speed of the imaging optical system is determined so that the stronger the 3DNR intensity (that is, the larger the number of frames for 3DNR processing), the slower the driving speed of the imaging optical system. Thereby, the amount of deviation from the peak position to the in-focus position can be reduced.

However, decelerating the focus drive speed has side effects such as prolonging the focusing time and making the system susceptible to disturbances and noise. Furthermore, as shown in FIG. 4, even if the driving speed of the imaging optical system is slowed down, the deviation of the peak position from the original in-focus position cannot be completely suppressed. Therefore, the detected peak position is corrected. In this embodiment, the stronger the intensity of 3DNR processing (that is, the larger the number of frames for 3DNR processing), the larger the correction amount of the peak position is. For example, when the intensity of 3DNR processing is weak (graph 302), only one frame is corrected, and when the intensity of 3DNR processing is strong (graph 303), only three frames are corrected. As a result, the original focus position can be detected as the peak position, and by driving the focus position of the focus lens 102 to the peak position obtained by correction, it becomes possible to achieve accurate focusing.

(Operation explanation)
FIG. 5 is a flowchart showing an example of the AF operation of the imaging device according to the present embodiment. The operations in this flowchart are executed by the CPU of the imaging device 100 loading a program from the ROM. Note that steps similar to those in the description of the conventional AF operation are given the same reference numerals and the description thereof will be omitted.

Before executing the hill-climbing method in step S209, the driving speed of the imaging optical system is set in accordance with the intensity of the set 3DNR processing (step S501). As described above, the strength of the 3DNR processing can be arbitrarily set by the user. The strength of 3DNR processing depends on the user's settings (for example, off/weak/strong), the internal settings set inside the imaging device, the amount of noise detected or estimated from the image, the number of frames used in 3DNR processing, etc. Obtained based on. These may be acquired by the acquisition unit 112 together with the evaluation value.

In step S501, as described above, the drive speed of the imaging optical system is set to be slower as the intensity of the 3DNR becomes stronger. For example, the deceleration coefficient may be set using table data corresponding to setting values set by the user, or may be calculated from the user's setting values, the number of frames used in 3DNR processing, etc.

When the peak position is detected in step S211, the peak position is corrected according to the intensity of 3DNR (S502). In other words, at least one of the driving speed and the peak position (focus position) of the imaging optical system is determined based on the intensity of the noise reduction process. The peak position is determined by correcting the current focus position (ie, the detected peak position). As described above, the correction is performed such that the stronger the 3DNR intensity, the larger the correction amount. Alternatively, among the plurality of focus positions controlled by the control unit 113, a focus position earlier (past) than the current focus position is corrected to become the peak position. For example, if the number of frames used for 3DNR processing is three, the focus position controlled three frames before the current frame is set as the peak position. In this way, the focus position of the imaging optical system is determined based on the strength of the 3DNR processing so that the focus position that was driven (controlled) in the past compared to the current focus position becomes the peak position. Note that a plurality of focus positions in a plurality of frames past the current frame are temporarily stored in a storage unit such as a RAM (Random Access Memory) within the imaging device. As described above, the imaging device according to this embodiment is capable of accurate focusing.

(Hardware configuration)
FIG. 6 is a diagram showing an example of the hardware configuration of the imaging device according to this embodiment. The imaging device 100 includes an imaging optical system 601, a drive unit 114, an imaging element 106, a CPU 602, a RAM 603, and a ROM 604. The drive unit 114 and the image sensor 106 are the same as in FIG. Further, the imaging optical system 601 includes at least the zoom lens 101, focus lens 102, and aperture unit 103 shown in FIG.

The CPU 602 is a central processing unit for executing AF control of the imaging apparatus according to this embodiment. Among the blocks shown in FIG. 1, the functions realized by software (for example, the acquisition unit 112 and the control unit 113) are realized by the CPU 602 loading a program stored in the ROM 604 and executing it using the RAM 603 as a work space. . The RAM 603 stores, for example, focus positions for each of a plurality of frames.

<Other embodiments>
The present invention can be realized by reading and executing a program that implements one or more functions of the first embodiment described above. This program is supplied to a system or device via a network or storage medium, and is read and executed by one or more processors in the computer of that system or device. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

100 Imaging device 101 Zoom lens 102 Focus lens 103 Aperture unit 104 Bandpass filter 105 Color filter 106 Image sensor 107 AGC
108 AD conversion unit 109 signal processing unit 110 communication unit 111 monitor device 112 acquisition unit 113 control unit 114 drive unit

Claims (16)

an imaging unit that captures an image of a subject using an imaging optical system;
a signal processing unit that performs image noise reduction processing based on multiple frames of images captured by the imaging unit;
an acquisition unit that acquires an evaluation value indicating the contrast of an image on which the noise reduction process has been performed by the signal processing unit;
a control unit that controls a focus position of the imaging optical system based on the evaluation value,
The imaging device, wherein the control unit determines at least one of a driving speed of the imaging optical system and a focus position of the imaging optical system based on the intensity of the noise reduction process. The imaging device according to claim 1, wherein the acquisition unit acquires the intensity of the noise reduction process. The imaging device according to claim 1, wherein the number of images of the plurality of frames used for the noise reduction process is determined based on the intensity of the noise reduction process. The imaging device according to claim 3, wherein the control unit determines the driving speed of the imaging optical system or the focus position based on the number of images of the plurality of frames. When determining the driving speed of the imaging optical system, the control unit determines the driving speed of the imaging optical system such that the driving speed of the imaging optical system becomes slower as the intensity of the noise reduction processing increases. The imaging device according to claim 1, characterized in that: When determining the focus position of the imaging optical system, the control unit determines the focus position of the imaging optical system by correcting the current focus position based on the intensity of the noise reduction process. The imaging device according to claim 1 further comprising a storage unit that stores a plurality of focus positions controlled by the control unit,
The control unit may correct the current focus position based on the intensity of the noise reduction process so that the current focus position becomes one of the plurality of focus positions. The imaging device according to claim 6. an imaging step of capturing an image of the subject;
a signal processing step of performing image noise reduction processing based on the plurality of frames of images captured in the step;
an acquisition step of acquiring an evaluation value indicating the contrast of the image on which the noise reduction process has been performed;
a control step of controlling a focus position of the imaging optical system based on the evaluation value,
A method for controlling an imaging apparatus, wherein in the control step, a driving speed of the imaging optical system or a focus position of the imaging optical system is determined based on the intensity of the noise reduction process. 9. The method of controlling an imaging device according to claim 8, wherein in the acquisition step, the intensity of the noise reduction process is acquired. 9. The method of controlling an imaging device according to claim 8, wherein the number of images of the plurality of frames used for the noise reduction process is determined based on the intensity of the noise reduction process. 11. The method of controlling an imaging device according to claim 10, wherein in the control step, the driving speed of the imaging optical system or the focus position is determined based on the number of images of the plurality of frames. In the control step, when determining the driving speed of the imaging optical system, the driving speed of the imaging optical system is determined such that the driving speed of the imaging optical system becomes slower as the intensity of the noise reduction processing increases. 9. The method of controlling an imaging device according to claim 8. In the control step, when determining the focus position of the imaging optical system, the focus position of the imaging optical system is determined by correcting the current focus position based on the intensity of the noise reduction process. The method for controlling an imaging device according to claim 8. The control unit may perform correction based on the intensity of the noise reduction process so that the focus position is any one of a plurality of focus positions that were controlled before the current focus position. The method for controlling an imaging device according to claim 13. A program for executing the method for controlling an imaging device according to claim 8. A computer-readable storage medium storing the program according to claim 15.

Images