JP2015099337A - Focusing direction detector, imaging device, focusing direction detection processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily, highly accurately detect a direction of driving an optical system to a focused position of the optical system if an axial chromatic aberration is present.SOLUTION: A focusing direction detector includes: imaging means generating an image including a first image signal relating to a first color component and a second image signal relating to a second color component; correction means correcting at least one of the first image signal and the second image signal so that a first pixel value of the first image signal generally matches a second pixel value of the second image signal, and so that a first gradient of a signal waveform of the first image signal generally matches a second gradient of a signal waveform of the second image signal; and direction detection means detecting a direction of a focused position of an optical system on the basis of a magnitude relation between the first image signal and the second image signal after correction by the correction means.

Description

  The present invention relates to a focus direction detection device, an imaging device, and a focus direction detection processing program.

  At the edge part in the image, the focal position is detected by using the fact that the difference in blur width calculated between the color components due to the presence of axial chromatic aberration changes before and after focusing. A technique for determining a direction to move is disclosed (for example, see Patent Document 1).

JP2012-22286A

  According to the invention disclosed in Patent Document 1, if a plurality of edges are included in the edge region between two sub-flat regions, a detection error of the focus detection position may occur.

(1) The in-focus direction detecting device according to claim 1 is a first image signal related to the first color component constituting the color of the subject, the color of the subject, and the first color component. An imaging means for generating an image including a second image signal relating to a different second color component, and a first pixel position in the image among a plurality of first pixel values constituting the first image signal And the second pixel value at the second pixel position in the image corresponding to the first pixel position among the plurality of second pixel values constituting the second image signal And the first gradient at the first pixel position of the signal waveform of the first image signal substantially coincides with the second gradient at the second pixel position of the signal waveform of the second image signal. To correct at least one of the first image signal and the second image signal And a direction detecting means for detecting the direction of the in-focus position of the optical system based on the magnitude relationship between the first image signal and the second image signal corrected by the correcting means. To do.
(2) According to a thirteenth aspect of the present invention, there is provided a focusing direction detection processing program comprising: a first image signal related to a first color component constituting a color of a subject; a color of the subject; Is a first pixel in the image out of an imaging process for generating an image including a second image signal relating to a different second color component, and a first plurality of pixel values constituting the first image signal Of the first pixel value at the position and the second plurality of pixel values constituting the second image signal, the second pixel value at the second pixel position in the image corresponding to the first pixel position And the first gradient at the first pixel position of the signal waveform of the first image signal and the second gradient at the second pixel position of the signal waveform of the second image signal are substantially equal to each other. At least one of the first image signal and the second image signal to match. A correction process for correcting one and a direction detection process for detecting the direction of the in-focus position of the optical system based on the magnitude relationship between the first image signal and the second image signal corrected by the correction process. It is made to perform.

  In the case where there is axial chromatic aberration, the direction in which the optical system is driven to the in-focus position of the optical system can be detected easily and with high accuracy, so that high-speed focus adjustment is possible.

It is a block block diagram of the imaging device which has a focusing direction detection apparatus in one embodiment. It is a flowchart of a focusing direction detection process. It is a figure showing a part of pixel arrange | positioned according to a two-dimensional arrangement | sequence in the horizontal direction and the vertical direction which comprise an interpolation image. It is a figure showing the signal waveform of the image signal regarding the green component in a back pin state, and a red component. It is a figure showing the signal waveform of the image signal regarding the green component in a front pin state, and a red component. It is a figure showing a mode that the outline of the edge is formed in the pixel arrange | positioned according to the Bayer arrangement | sequence which comprises a RAW image. It is a figure which shows the provision form of the computer program product for the focusing direction detection apparatus by this invention for performing a focusing direction detection process.

  FIG. 1 is a block configuration diagram of an imaging apparatus 100 having an in-focus direction detection apparatus 105 according to an embodiment of the present invention. The imaging apparatus 100 includes a driving device 140 and an optical system 150 driven by the driving device 140 together with the focusing direction detection device 105. The in-focus direction detection device 105 includes a control device 110 and an image sensor 120. Subject light 85 coming from the subject 80 enters the image sensor 120 through the optical system 150. The image sensor 120 has a green image signal (G image signal) related to a green component (G component) constituting the color of the subject 80 and a red image signal (R image signal) related to a red component (R component) constituting the color of the subject 80. ) And a blue image signal (B image signal) relating to the blue component (B component) constituting the color of the subject 80 is generated and output.

  The control device 110 is a computer that executes a focusing direction detection processing program. The control device 110 includes an imaging control unit 111, an ISO sensitivity determination unit 112, a white balance correction unit 113, an interpolated image generation unit 114, an in-focus direction detection image signal determination unit 115, an image signal correction unit 116, a front pin / rear pin. The determination index value calculation unit 117 and the focusing direction detection unit 118 are functionally provided. Imaging control unit 111, ISO sensitivity determination unit 112, white balance correction unit 113, interpolation image generation unit 114, focus direction detection image signal determination unit 115, image signal correction unit 116, front pin / rear pin determination index value calculation unit 117 and the focus direction detection unit 118 perform each step of the focus direction detection process, which will be described later with reference to FIG.

  In imaging apparatus 100 having focusing direction detection apparatus 105 in the present embodiment, R component and G component of RGB color components have axial chromatic aberration. When the light corresponding to the R component of the subject light 85 forms an image behind the light corresponding to the G component of the subject light 85, that is, the subject image of the subject 80 at a position away from the optical system 150. In the case of forming an image, an example in which the direction determination of the in-focus position of the optical system 150 due to axial chromatic aberration is performed from an edge whose pixel value changes in the horizontal direction in an interpolated image generated based on a RAW image as will be described later. Show. Note that in this specification, “pixel” means not the photoelectric conversion element arranged on the imaging surface of the imaging apparatus 100 but the smallest unit constituting an image. In this case, if the focus state of the optical system 150 is a back-pin state with respect to the imaging surface, an R plane image formed by subject light corresponding to the R component is formed by subject light corresponding to the G component on the imaging surface. More blurry than the G-plane image. If the focus state of the optical system 150 is the front pin state with respect to the imaging surface, the G plane image is more blurred than the R plane image. A flowchart of the focus direction detection process performed by the control device 110 included in the focus direction detection device 105 is shown in FIG.

  In step S210, the imaging control unit 111 of the control device 110 acquires a RAW image from the imaging device 120, and the ISO sensitivity determination unit 112 of the control device 110 determines the ISO sensitivity. With the ISO sensitivity determined by the ISO sensitivity determination unit 112, the image sensor 120 generates a RAW image. The imaging control unit 111 acquires a RAW image from the imaging element 120. The white balance correction unit 113 of the control device 110 performs white balance correction on the image signal of the RAW image acquired by the imaging control unit 111. The interpolated image generation unit 114 of the control device 110 performs an interpolating process on the image signal of the RAW image corrected by the white balance correction unit 113, and the G pixel value, the R pixel value, and the B pixel at all two-dimensional pixel positions. An interpolated image in which values are associated is generated. A plurality of G pixel values, a plurality of R pixel values, and a plurality of B pixel values in the interpolated image constitute a G image signal, an R image signal, and a B image signal, respectively. The focus direction detection image signal determination unit 115 of the control device 110 includes three types of interpolated image signals, that is, two types of focus direction detection image signals of the R image signal, the G image signal, and the B image signal. To decide. In this example, since the R component and the G component have axial chromatic aberration, the focus direction detection image signal determination unit 115 determines the R image signal and the G image signal as two types of focus direction detection image signals. To do.

  In step S220, the front pin / rear pin determination index value calculation unit 117 of the control device 110 sets an initial coordinate value of the target pixel position in the interpolation image. As will be described later, in the subsequent steps, the front pin / rear pin determination index value calculation unit 117 of the control device 110 performs two types of focusing direction detection image signal determination unit 115 at the target pixel position in the interpolation image. The sum of the horizontal gradient, the secondary gradient, the right pixel value, and the left pixel value of the R image signal and G image signal determined as the focus direction detection image signal is calculated. In step S220, the front pin / rear pin determination index value calculation unit 117 sets an initial coordinate value of the target pixel position in the interpolated image while ensuring a margin necessary for the calculation in step S250 described later. For example, when calculating the gradient and secondary gradient at the target pixel position in the interpolated image from adjacent pixels in the horizontal direction and calculating the sum of the pixel values of the right N pixels and the left N pixels, A margin of N + 1 pixels in the direction and 0 pixel in the vertical direction are required, and the initial coordinate value of the target pixel position is (N + 1, 0). In step S220, the front pin / rear pin determination index value calculation unit 117 initializes a later-described front pin determination counter N_FRONT and a rear pin determination counter N_REAR so that N_REAR = 0 and N_FRONT = 0.

In step S230, the front pin / rear pin determination index value calculation unit 117 of the control device 110 performs the horizontal gradients dxR of the signal waveforms of the two colors of the R image signal and the G image signal at the current target pixel position in the interpolation image, and dxG and the horizontal quadratic gradients d ² xR and d ² xG are calculated. In the present embodiment, the gradients dxR and dxG are calculated by a difference filter that takes a pixel value difference between pixels adjacent in the horizontal direction, and the secondary gradient is calculated by a Laplacian filter using pixels adjacent in the horizontal direction. FIG. 3 is a diagram illustrating a part of pixels arranged according to a two-dimensional array in the horizontal direction (x direction) and the vertical direction (y direction) constituting the interpolation image. Using the pixel value R (x, y) and the pixel value G (x, y) at the current target pixel position (x, y) in the interpolated image, the gradient dxR at the current target pixel position (x, y) and dxG and quadratic gradients d ² xR and d ² xG are represented by the following formula (1). Pixel values R (x-1, y), R (x, y), and R (x + 1, y) are included in the R image signal. Pixel values G (x-1, y), G (x, y), and G (x + 1, y) are included in the G image signal.

  As i ≧ 2, pixel positions (x−i, y) and (x + i, y) sandwich the target pixel position (x, y) in the vicinity of the target pixel position (x, y). From the pixel position (x−i, y) side of the target pixel position (x, y), that is, the left side, the pixel value constituting the R image signal increases from the pixel position (x + i, y) side, that is, the right side. When indicating a tendency, the gradient dxR is a positive gradient, and when indicating a tendency that the pixel values constituting the R image signal become smaller, the gradient dxR is a negative gradient. As shown in Expression (1), the gradient dxR at the target pixel position (x, y) is, for example, a pixel position (x + 1, y) between the target pixel position (x, y) and the pixel position (x + i, y). ) And a pixel value R (x−y) at a pixel position (x−1, y) between the target pixel position (x, y) and the pixel position (xi, y). 1, y).

  Pixels constituting the G image signal from the pixel position (x−i, y) side of the target pixel position (x, y) in the interpolated image, that is, from the left side toward the pixel position (x + i, y) side, that is, the right side. The gradient dxG is a positive gradient when the value tends to increase, and the gradient dxG is a negative gradient when the pixel value constituting the G image signal tends to decrease. As shown in Expression (1), the gradient dxG at the target pixel position (x, y) is, for example, the pixel position (x + 1, y) between the target pixel position (x, y) and the pixel position (x + i, y). ) And the pixel value G (x−y) at the pixel position (x−1, y) between the target pixel position (x, y) and the pixel position (xi, y). 1, y).

As shown in the equation (1), the quadratic gradient d ² xR is calculated from the difference between the pixel value R (x, y) and the pixel value R (x−1, y) and the pixel value R (x + 1, y). This is a value obtained by subtracting the difference from the pixel value R (x, y). From the difference between the pixel value G (x, y) and the pixel value G (x−1, y), the quadratic gradient d ² xG is calculated using the pixel value G (x + 1, y) and the pixel value G (x, y). This is the value obtained by subtracting the difference.

In step S240, the front pin / rear pin determination index value calculator 117 calculates the pixel values R (x, y) and G (x, y) of two colors at the target pixel position (x, y) in the interpolated image, and the gradient. It is determined whether dxR and dxG and quadratic gradients d ² xR and d ² xG satisfy the following conditions (a), (b), and (c). The front / rear pin determination described later at the target pixel position (x, y) that does not satisfy any of the following three conditions is not used because it deteriorates the accuracy of detecting the direction of the focus position described later.

(A) Since the accuracy deteriorates due to the influence of noise, the two-color pixel values R (x, y) and G (x, y) are both greater than or equal to the predetermined pixel value TH_VALUE. If the pixel value is small, the influence of noise is large, and the accuracy of detecting the direction of the in-focus position is likely to deteriorate.

(B) The gradients dxR and dxG of the two colors at the target pixel position (x, y) in the interpolated image are greater than or equal to a predetermined gradient value TH_GRAD. For example, when the gradients dxR and dxG are small as in an image flat region, the change in the relationship between the gradient of the R image signal and the gradient of the G image signal due to axial chromatic aberration before and after the in-focus position (front pin and rear pin) Therefore, it is easy to deteriorate the direction detection accuracy of the in-focus position.

(C) The secondary gradients d ² xR and d ² xG of the two colors at the target pixel position (x, y) in the interpolation image are substantially zero. For example, the secondary gradients d ² xR and d ² xG may both be equal to or smaller than a predetermined secondary gradient value TH_GRAD2. When the target pixel position (x, y) in the interpolated image is located at the boundary between the image flat area and the edge area (edge of the edge area) or at the boundary between two different edge areas, the front pin / rear to be described later The pin determination index value is not calculated correctly.

The conditions (a), (b) and (c) described above are expressed by the following formula (2). Pixel values R (x, y) and G (x, y) at the target pixel position (x, y) in the interpolated image, gradients dxR and dxG, and secondary gradients d ² xR and d ² xG are determined according to the condition (a ), (B) and (c), that is, when it is determined that the expression (2) is satisfied, the focus direction detection process proceeds to step S250. Pixel values R (x, y) and G (x, y) at the target pixel position (x, y) in the interpolated image, gradients dxR and dxG, and secondary gradients d ² xR and d ² xG are determined according to the condition (a ), (B) and (c), that is, when it is determined that the expression (2) is not satisfied, the focus direction detection process proceeds to step S270.

In step S250, the image signal correction unit 116 of the control device 110 corrects the G image signal of the R image signal and the G image signal determined by the focus direction detection image signal determination unit 115 in step S210. Specifically, the pixel value G (x, y) at the target pixel position (x, y) in the interpolation image is substantially matched with the pixel value R (x, y), and the target pixel position (x in the interpolation image) , Y) approximately matches the gradient dxG with the gradient dxR. The corrected pixel value G ′ (x, y) and the corrected gradient dxG ′ at the target pixel position (x, y) are expressed by the following equation (3) using the coefficient a and the intercept b.

As described above, the corrected pixel value G ′ (x, y) at the target pixel position (x, y) substantially matches the pixel value R (x, y), and the target pixel position (x, y). The corrected gradient dxG ′ in FIG. 2 substantially matches the gradient dxR. When the pixel value G ′ (x, y) and the pixel value R (x, y), and the gradient dxG ′ and the gradient dxR at the target pixel position (x, y) completely coincide with each other, the following formula (4 ). When equation (4) is solved for coefficient a and intercept b, equation (5) is obtained. The pixel value G ′ (x, y) and the pixel value R (x, y) at the target pixel position (x, y), and the gradient dxG ′ and the gradient dxR do not completely coincide with each other, but only by a small predetermined amount. May be different. For example, in consideration of the case where noise, density unevenness, or the like occurs at the target pixel position (x, y) or the surrounding pixel position, a pixel value that is presumed to be statistically possible due to such noise, density unevenness, or the like. Is the predetermined amount.

  4 and 5 are diagrams showing signal waveforms of image signals relating to the green component and the red component in the rear pin state and the front pin state, respectively. In FIG. 4A and FIG. 5A, the signal waveform of the R image signal related to the R component is shown by a solid line, and the signal waveform of the G image signal related to the G component before correction is shown by a broken line. In FIG. 4B and FIG. 5B, the signal waveform of the R image signal related to the R component is shown by a solid line, and the signal waveform of the G image signal related to the corrected G component is shown by a broken line.

  As described above, the corrected pixel value G ′ (x, y) at the target pixel position (x, y) substantially matches the pixel value R (x, y), and the correction at the target pixel position (x, y). A state in which the subsequent gradient dxG ′ substantially coincides with the gradient dxR is shown in FIGS. 4B and 5B. According to FIGS. 4B and 5B, the magnitude relationship between the corrected G ′ image signal and the R image signal is determined depending on whether the state is the rear pin state or the front pin state. Yes. Specifically, in the rear pin state, as shown in FIG. 4B, both the pixel value G ′ and the pixel value R increase from the left to the right of the target pixel position (x, y). The pixel value G ′ (x−i, y) at the pixel position on the left side of the target pixel position (x, y) is larger than the pixel value R (x−i, y), and the target pixel position (x, y) The pixel value G ′ (x + i, y) at the pixel position on the right side of) is smaller than the pixel value R (x + i, y). Accordingly, the pixel value sum ΣG ′ (xi, y) of the left pixel values G ′ (xi, y) is the sum of the left pixel values R (xi, y). A sum of pixel values ΣG ′ (x + i, y) of a plurality of pixel values G ′ (x + i, y) on the right side that is larger than ΣR (x−i, y) is a plurality of pixel values R (x + i, y) on the right side. Is smaller than the sum of pixel values ΣR (x + i, y).

  In the rear pin state, contrary to FIG. 4B, when the pixel value G ′ and the pixel value R both decrease from the left side to the right side of the target pixel position (x, y), The pixel value G ′ (x−i, y) at the pixel position on the left side of the pixel position (x, y) is smaller than the pixel value R (x−i, y), and is on the right side of the target pixel position (x, y). The pixel value G ′ (x + i, y) at the pixel position is larger than the pixel value R (x + i, y) (not shown). Accordingly, the pixel value sum ΣG ′ (xi, y) of the left pixel values G ′ (xi, y) is the sum of the left pixel values R (xi, y). A sum of pixel values ΣG ′ (x + i, y) of a plurality of pixel values G ′ (x + i, y) on the right side that is smaller than ΣR (x−i, y) is a plurality of pixel values R (x + i, y) on the right side. Is greater than the sum of pixel values ΣR (x + i, y).

  In the front pin state, as shown in FIG. 5B, when the pixel value G ′ and the pixel value R both increase from the left to the right of the target pixel position (x, y), The pixel value G ′ (x−i, y) at the pixel position on the left side of the position (x, y) is smaller than the pixel value R (x−i, y), and the pixel on the right side of the target pixel position (x, y). The pixel value G ′ (x + i, y) at the position is larger than the pixel value R (x + i, y). Accordingly, the pixel value sum ΣG ′ (xi, y) of the left pixel values G ′ (xi, y) is the sum of the left pixel values R (xi, y). A sum of pixel values ΣG ′ (x + i, y) of a plurality of pixel values G ′ (x + i, y) on the right side that is smaller than ΣR (x−i, y) is a plurality of pixel values R (x + i, y) on the right side. Is greater than the sum of pixel values ΣR (x + i, y).

  In the front pin state, contrary to FIG. 5B, when both the pixel value G ′ and the pixel value R decrease downward from the left side to the right side of the target pixel position (x, y), The pixel value G ′ (x−i, y) at the pixel position on the left side of the pixel position (x, y) is larger than the pixel value R (x−i, y), and the pixel value on the right side of the target pixel position (x, y). The pixel value G ′ (x + i, y) at the pixel position is smaller than the pixel value R (x + i, y) (not shown). Accordingly, the pixel value sum ΣG ′ (xi, y) of the left pixel values G ′ (xi, y) is the sum of the left pixel values R (xi, y). A sum of pixel values ΣG ′ (x + i, y) of a plurality of pixel values G ′ (x + i, y) on the right side that is larger than ΣR (x−i, y) is a plurality of pixel values R (x + i, y) on the right side. Is smaller than the sum of pixel values ΣR (x + i, y).

In step S250, the front pin / rear pin determination index value calculation unit 117 calculates the front pin / rear pin determination index value I _RG represented by the equation (6). In Expression (6), pixel values R (x−N−1, y), R (x−N, y),..., R (x−) of the N pixels on the left side of the target pixel position (x, y). 2, y) pixel value sum lsumR, pixel values R (x + 2, y), R (x + 3, y),..., R (x + N + 1, y) on the right side of the target pixel position (x, y) Pixel value sum rsumR, corrected pixel values G ′ (x−N−1, y), G ′ (x−N, y), N ′ on the left side of the target pixel position (x, y),. , G ′ (x−2, y) pixel value sum lsumG ′, corrected pixel values G ′ (x + 2, y), G ′ (x + 3, y) of N pixels on the right side of the target pixel position (x, y) ),..., G ′ (x + N + 1, y) pixel value sums rsumG ′ are all pixel-of-interest positions (including pixel-of-interest positions (x, y)) included in the horizontal pixel row in the interpolated image. x, y) neighborhood It is calculated from the original. The value of the constant N is an integer value of about 1 to 4, for example. That is, the front pin / rear pin determination index value calculation unit 117 has a pixel value G ′ (xi) on the left side of the target pixel position (x, y) in the G ′ image signal and the R image signal in which the G image signal is corrected. , Y), the difference between the pixel value sum ΣG ′ (xi, y) and the pixel value sum ΣR (xi, y) of the pixel value R (xi, y), and the target pixel position (x, y ) Based on the difference between the pixel value sum ΣG ′ (x + i, y) of the right pixel value G ′ (x + i, y) and the pixel value sum ΣR (x + i, y) of the pixel value R (x + i, y), the front pin / The rear pin determination index value I _RG is calculated.

In step S260, the front pin / rear pin determination index value calculation unit 117 determines whether the front pin determination counter N_FRONT or the front pin / rear pin determination index value I _R−G is based on the product of the gradient dxG ′ = dxR and the front pin / rear pin determination index value IRG. The rear pin determination counter N_REAR is updated. When the product of the gradient dxG ′ = dxR and the front pin / rear pin determination index value IR _−G takes a positive value or is larger than, for example, the upper threshold value TH_I1, it is determined that the rear pin state described above is present, The rear pin determination counter N_REAR is incremented by one. When the product of the gradient dxG ′ = dxR and the front pin / rear pin determination index value IR _−G takes a negative value or is smaller than, for example, the lower threshold TH_I2, it is determined that the front pin state described above is present. The front pin determination counter N_FRONT is incremented by 1. Although upper threshold TH_I1 and lower threshold TH_I2 is a value close to 0 both the front pin / rear focus determination index value I _R-G is not only when the image pickup surface is positioned at the focus position of the optical system 150 When the imaging surface is too far from the in-focus position of the optical system 150, a value close to 0 is shown. On the imaging surface is measured in advance before the pin / post pin determination index value I _R-G in the case where too far away from the in-focus position of the optical system 150, the upper threshold TH_I1 and lower in consideration of the measured values By defining the threshold value TH_I2, erroneous detection of the direction of the in-focus position of the optical system 150 can be prevented.

  In step S270, the front pin / rear pin determination index value calculation unit 117 can shift the target pixel position to the pixel position on the right side of the current target pixel position in the interpolation image even in consideration of the margin described above. In this case, the target pixel position is shifted to the right pixel position. In this case, an affirmative determination is made in step S270, and the focus direction detection processing returns to step S230, and the processing from step S230 described above is repeated at the shifted pixel position. If it is impossible to shift the target pixel position to the pixel position to the right of the current target pixel position in the interpolated image, the above margin is taken into consideration below the current target pixel position in the interpolated image. The target pixel position is shifted to the leftmost shiftable pixel position. In this case, an affirmative determination is made in step S270, and the focus direction detection processing returns to step S230, and the processing from step S230 described above is repeated at the shifted pixel position. If it is impossible to shift the target pixel position to the lower side of the current target pixel position in the interpolated image, a negative determination is made in step S270, and the focus direction detection process proceeds to step S280.

  In step S280, the focus direction detection unit 118 of the control device 110 detects the direction of the focus position of the optical system 150 based on the counter values of the front pin determination counter N_FRONT and the rear pin determination counter N_REAR. Specifically, when the counter value of the rear pin determination counter N_REAR is larger than the counter value of the front pin determination counter N_FRONT, it is determined as the rear pin. The in-focus direction detection unit 118 detects that the in-focus position of the optical system 150 is away from the image plane of the image sensor 120. When the counter value of the rear pin determination counter N_REAR is smaller than the counter value of the front pin determination counter N_FRONT, it is determined as the front pin. The in-focus direction detection unit 118 detects that the in-focus position direction of the optical system 150 is a direction in which the optical system 150 approaches the image plane of the image sensor 120.

  In step S290, the focus direction detection unit 118 controls the drive device 140 so that the drive device 140 drives the optical system 150 in the direction of the focus position detected in step S280. Thereafter, the in-focus direction detection process ends.

  In the in-focus direction detection apparatus 105 according to the present embodiment, in the case of axial chromatic aberration, the direction in which the optical system 150 is driven to the in-focus position of the optical system 150 can be detected easily and with high accuracy. High-speed focus adjustment is possible. In addition, it is possible to implement the front pin / rear pin determination as a filtering process by hardware for each target pixel position.

---- Modified example ---
(1) In the above-described embodiment, the gradient calculation is calculated by the horizontal direction adjacent pixel difference filter, and the secondary gradient is calculated by the horizontal direction adjacent pixel Laplacian filter. However, pixels in the upper and lower lines may be used as in the Sobel filter. . For example, using the upper and lower lines one by one, the following equation (8) is obtained. When the Sobel filter is used, in step S220, initial coordinate values (N + 1, 1) of the target pixel position in the interpolation image are set. With this processing, a buffer area for three lines is required, but a strong determination against noise can be made.

(2) In the above-described embodiment, axial chromatic aberration is present between the R component and the G component in the subject light 85, and the light corresponding to the R component in the subject light 85 is included in the subject light 85. In the case where an image is formed behind the light corresponding to the G component, an example is shown in which determination is performed using the R image signal and the G image signal of two colors. The same may be applied when there is axial chromatic aberration between the G component and the B component or between the R component and the B component. There is an axial chromatic aberration between the G component and the B component, and the light corresponding to the B component of the subject light 85 forms an image behind the light corresponding to the G component of the subject light 85. Front pin / rear pin determination index value I _B−G , there is axial chromatic aberration between the R component and the B component, and the light corresponding to the B component of the subject light 85 is the R component of the subject light 85 The index value IB _-R when the image is formed behind the light corresponding to is represented by the following expressions (9) and (10), respectively. The same applies to other axial chromatic aberrations.

  For example, in a subject outline having no contrast difference in either the R component or the G component, the determination performance is improved by using a combination of other color components.

In general, in the case of a zoom lens as the optical system 150, the axial chromatic aberration balance varies depending on the zoom position. Therefore, based on the optical design value of the lens or the axial chromatic aberration measured in advance, any one of the front pin / rear pin determination index values I _RG , I _BG , I _B-R, etc. depending on the zoom position. Need to decide whether to use. For this purpose, in step S210 described above, the focus direction detection image signal determination unit 115 of the control device 110 performs the R image signal, the G image signal, and the B image signal, which are three types of image signals constituting the interpolation image. The zoom position may be taken into consideration when determining two types of focus direction detection image signals. If the two types of in-focus direction detection image signals thus determined are the R image signal and the G image signal, the front pin / rear pin determination index value calculation unit 117 of the control device 110 performs the front pin / rear pin. Determination index value I _RG is calculated. If the two types of determined focus direction detection image signals are the B image signal and the G image signal, the front pin / rear pin determination index value calculation unit 117 of the control device 110 performs the front pin / rear pin determination index value I. _B-G is calculated. If the two types of determined focus direction detection image signals are the B image signal and the R image signal, the front pin / rear pin determination index value calculation unit 117 of the control device 110 performs the front pin / rear pin determination index value I. _B-R is calculated.

Further, when there are a plurality of combinations of color components having axial chromatic aberration, a combination of color components having a higher S / N is selected in consideration of the subject color, ISO sensitivity at the time of imaging by the image sensor 120, or white balance. It may be selected to perform the in-focus direction detection process. At this time, for example, the front pin / rear pin determination index values I _RG , I _B-G , and I _B-R may all be used for the in-focus direction detection process after being weighted.

(3) In the above-described embodiment, the example in which the in-focus direction is detected from the horizontal edge by the longitudinal chromatic aberration is shown. Further, the in-focus direction may also be detected from the vertical edge by axial chromatic aberration. Front focus / rear focus determination index value J _R-G in this case, as shown below, and slope dyR vertical R component in the target pixel position in the interpolated image (x, y), in the vertical direction R The upper pixel value sum usumR and the lower pixel value sum dsumR of the components, the gradient dyG of the vertical G component at the target pixel position (x, y) in the interpolation image calculated in the same manner, and the vertical G component The upper pixel value sum usumG and the lower pixel value sum dsumG may be calculated by the following equation (11). A front pin / rear focus determination index value I _R-G, front focus / rear focus determination index value J previous _R-G and a were calculated using the respective pin determining counter one of the counter values of N_FRONT, and before Optically based on one counter value of the rear pin determination counter N_REAR calculated using the pin / rear pin determination index value I _RG and the front pin / rear pin determination index value J _RG respectively. The direction of the in-focus position of the system 150 may be detected. Alternatively, front focus / rear focus determination index value I and _R-G, front focus / rear focus determination index value J _R-G before the calculated with each pin determining counter counter sum of N_FRONT, and the front focus / Rear pin determination index value I _RG and the rear pin determination counter value N RG calculated using the front pin / rear pin determination index value J _RG respectively. The direction of the focal position may be detected.

(4) In the above-described embodiment, in step S250, the image signal correction unit 116 of the control device 110 determines the R image signal and the G image signal determined by the focus direction detection image signal determination unit 115 in step S210. The G image signal is corrected. However, the image signal correction unit 116 corrects the R image signal so that the pixel value R (x, y) at the target pixel position (x, y) in the interpolated image is substantially equal to the pixel value G (x, y). In addition, the gradient dxR at the target pixel position (x, y) in the interpolation image may be substantially matched with the gradient dxG. Alternatively, both the R image signal and the G image signal are corrected, and the pixel value G (x, y) and the pixel value R (x, y) at the target pixel position (x, y) in the interpolation image are set to predetermined pixel values. The gradient dxG and the gradient dxR at the target pixel position (x, y) in the interpolation image may be substantially matched with a predetermined gradient value.

(5) In the above-described embodiment, the configuration in which the determination result is output for one image has been described. However, the present invention is applied to a range corresponding to the focus detection area in one image, and The direction of the in-focus position of the optical system 150 may be detected based on the counter value of the rear pin determination counter N_REAR and the counter value of the front pin determination counter N_FRONT within the focus detection area.

(6) In the above-described embodiment, the configuration in which the determination result is output for one image has been described. However, a plurality of frame images generated and output from the image sensor 120 at predetermined intervals or their interpolated images. , Based on the counter integrated value of the rear pin determination counter N_REAR and the counter integrated value of the front pin determination counter N_FRONT, that is, the integrated value obtained by integrating each counter value for each image without initialization. The direction of the in-focus position of the optical system 150 may be detected based on the above.

(7) In the above-described embodiment, the interpolated RGB interpolation image has been described as an object. However, a raw image before interpolation may be an object. In this case, the processing load can be reduced by calculating the gradient and the sum of pixel values on both the left and right sides of the target pixel position in the RAW image directly from the RAW image before interpolation. For example, when there is axial chromatic aberration between the R component and the G component for a RAW image in the Bayer array, the front pin / rear position can be detected using Expression (12) in order to detect the in-focus direction from the horizontal edge. The pin determination index value I _RG may be calculated. When the pixel value P (x, y) at the first target pixel position (x, y) in the Bayer array RAW image has an R component, the pixel at the second target pixel position (x, y + 1) immediately below it The value P (x, y) has a G component. Therefore, the calculation according to Expression (12) is performed while shifting the first target pixel position (x, y) by selecting a pixel position having an R component.

  FIG. 6 is a diagram illustrating a state in which an edge outline is formed on pixels that are two-dimensionally arranged according to a Bayer array that configures a RAW image. As shown in FIG. 6, the present invention is applied if the edge contour position is included in the first target pixel position (x, y) and the second target pixel position (x, y + 1) in the RAW image. Accordingly, the rear pin determination counter N_REAR and the front pin determination counter N_FRONT are updated at the first target pixel position (x, y) and the second target pixel position (x, y + 1), and the optical system 150 is focused. The direction of the position can be detected.

  In Expression (12), the pixel value sum lsumR is, for example, the pixel value R (x−2N−2, y) of the N pixel on the left side of the first target pixel position (x, y) in the RAW image illustrated in FIG. 6. , R (x−2N, y),..., R (x−6, y), R (x−4, y). The pixel value sum rsumR is, for example, the pixel values R (x + 4, y), R (x + 6, y),..., R () of the N pixels on the right side of the first target pixel position (x, y) in the RAW image. x + 2N, y) and R (x + 2N + 2, y). The pixel value sum lsumG ′ is, for example, pixel values G (x−2N−2, y + 1) and G (x−2N, y + 1) of N pixels on the left side of the second target pixel position (x, y) in the RAW image. ,..., G (x−6, y + 1) and G (x−4, y + 1) are pixel value sums of pixel values corrected by the image signal correction unit 116 of the control device 110. The pixel value sum rsumG ′ is, for example, the pixel values G (x + 4, y + 1), G (x + 6, y + 1),..., G on the right side of the second target pixel position (x, y) in the RAW image. (X + 2N, y + 1) and G (x + 2N + 2, y + 1) are pixel value sums of pixel values corrected by the image signal correction unit 116 of the control device 110.

  In Expression (12), as shown in FIG. 6, the gradient dxR is a pixel position (x + 2, y) between the first target pixel position (x, y) and the pixel position (x + 4, y) in the RAW image. At the pixel position (x-2, y) between the pixel value R (x + 2, y) at the first pixel position (x, y) and the pixel position (x-4, y) in the RAW image. The difference from the value R (x−2, y). As shown in FIG. 6, the gradient dxG is a pixel value G (x + 2) at a pixel position (x + 2, y + 1) between the second target pixel position (x, y + 1) and the pixel position (x + 4, y + 1) in the RAW image. , Y + 1) and the pixel value G (x−2) at the pixel position (x−2, y + 1) between the second target pixel position (x, y + 1) and the pixel position (x−4, y + 1) in the RAW image. , Y + 1).

Even when there is axial chromatic aberration between the B component and the G component, or when there is axial chromatic aberration between the B component and the R component, as in the case where there is axial chromatic aberration between the R component and the G component, The pin / rear pin determination index value IB _-G or IB _-R may be calculated.

(8) As shown in FIG. 7, a computer program product for the control device 110 of the imaging device 100 to execute the focus direction detection processing in each of the above-described embodiments and modifications is a program storage medium 2000 (for example, The data processing device 1500 (for example, a PC) is provided through a data signal transmitted through a CD-ROM) or a communication network 4000 (for example, the Internet). The imaging apparatus 100 reads a focusing direction detection processing program from the data processing apparatus 1500 via communication.

  The data processing device 1500 is provided with a focusing direction detection processing program via the program storage medium 2000. Alternatively, the focus direction detection processing program may be provided in the following supply form. The data processing device 1500 has a connection function with the communication network 4000. The server 3000 is a computer that provides a focusing direction detection processing program, and stores the focusing direction detection processing program in a storage medium such as a hard disk. The server 3000 places the focusing direction detection processing program read from the hard disk on a carrier wave as a data signal, and transfers it to the data processing device 1500 via the communication network 4000. As described above, the focusing direction detection processing program is preferably supplied by various forms of computer program products including provision as data signals via the program storage medium 2000 and the communication network 4000.

  The above-described embodiments and modifications may be combined.

80 subject, 85 subject light,
100 imaging device, 105 focusing direction detection device, 110 control device,
111 imaging control unit, 112 ISO sensitivity determination unit, 113 white balance correction unit,
114 interpolation image generation unit, 115 in-focus direction detection image signal determination unit,
116 image signal correction unit, 117 front pin / rear pin determination index value calculation unit,
118 in-focus direction detection unit, 120 imaging device, 140 driving device, 150 optical system,
1500 data processing device, 2000 program storage medium,
3000 servers, 4000 communication networks

Claims (13)

A first image signal relating to a first color component constituting the color of the subject, and a second image signal relating to a second color component constituting the color of the subject and different from the first color component. Imaging means for generating an image including:
Of the first plurality of pixel values constituting the first image signal, the first pixel value at the first pixel position in the image and the second plurality constituting the second image signal. Among the pixel values of the first image signal, the second pixel value at the second pixel position in the image corresponding to the first pixel position is substantially matched, and the signal waveform of the first image signal is The first image signal and the second image signal so that the first gradient at one pixel position and the second gradient at the second pixel position of the signal waveform of the second image signal substantially coincide with each other. Correcting means for correcting at least one of the image signals of
Direction detecting means for detecting the direction of the in-focus position of the optical system based on the magnitude relationship between the first image signal and the second image signal corrected by the correcting means. Focal direction detection device. The in-focus direction detecting device according to claim 1,
The first plurality of pixel values are the first pixel value in the third pixel position among the third pixel position and the fourth pixel position sandwiching the first pixel position in the vicinity of the first pixel position. 3 pixel values and a fourth pixel value at the fourth pixel position,
The second plurality of pixel values are defined by a fifth pixel position corresponding to the third pixel position and a fourth pixel position sandwiching the second pixel position in the vicinity of the second pixel position. Of the corresponding sixth pixel positions, including a fifth pixel value at the fifth pixel position and a sixth pixel value at the sixth pixel position;
The light corresponding to the second color component of the subject light coming from the subject and passing through the optical system is more optical than the light corresponding to the first color component of the subject light. When forming the subject image of the subject at a position away from the system, the direction detection means
The pixel value increases from the third pixel position side of the first pixel position toward the fourth pixel position side, and from the fifth pixel position side of the second pixel position to the sixth pixel position side. The third pixel value is larger than the fifth pixel value, and the fourth pixel value is larger than the sixth pixel value. When small, it is detected that the direction of the in-focus position is a direction away from the image plane of the imaging unit of the optical system,
The pixel value decreases from the third pixel position side of the first pixel position toward the fourth pixel position side, and from the fifth pixel position side of the second pixel position to the sixth pixel position side. The third pixel value is smaller than the fifth pixel value, and the fourth pixel value is smaller than the sixth pixel value. When large, it detects that the direction of the in-focus position is a direction away from the image plane of the image pickup means, and
The pixel value increases from the third pixel position side of the first pixel position toward the fourth pixel position side, and from the fifth pixel position side of the second pixel position to the sixth pixel position side. The third pixel value is smaller than the fifth pixel value, and the fourth pixel value is larger than the sixth pixel value. When large, the direction of the in-focus position is detected to be a direction in which the optical system approaches the image plane of the imaging unit,
The pixel value decreases from the third pixel position side of the first pixel position toward the fourth pixel position side, and from the fifth pixel position side of the second pixel position to the sixth pixel position side. The third pixel value is greater than the fifth pixel value, and the fourth pixel value is greater than the sixth pixel value. When it is small, it is detected that the direction of the in-focus position is a direction away from the image plane of the image pickup means, and the in-focus position is a direction in which the optical system approaches the image plane of the image pickup means. An in-focus direction detecting device for detecting that it is. The in-focus direction detecting device according to claim 2,
The first gradient includes a seventh pixel value included in the first plurality of pixel values at a seventh pixel position between the first pixel position and the third pixel position; A value based on a difference from an eighth pixel value included in the first plurality of pixel values at an eighth pixel position between the first pixel position and the fourth pixel position;
The second gradient includes a ninth pixel value included in the second plurality of pixel values at a ninth pixel position between the second pixel position and the fifth pixel position; and A value based on a difference from a tenth pixel value included in the second plurality of pixel values at a tenth pixel position between the second pixel position and the sixth pixel position. In-focus direction detection device. The in-focus direction detecting device according to claim 2 or 3,
A value obtained by subtracting a difference between the eighth pixel value and the first pixel value from a difference between the first pixel value and the seventh pixel value is equal to or less than a predetermined secondary gradient value; A value obtained by subtracting the difference between the tenth pixel value and the second pixel value from the difference between the second pixel value and the ninth pixel value is equal to or less than the predetermined secondary gradient value. An in-focus direction detecting device. In the focusing direction detection apparatus according to any one of claims 1 to 4,
The in-focus direction detecting device, wherein the first gradient and the second gradient are values greater than or equal to a predetermined gradient value. In the in-focus direction detection device according to any one of claims 1 to 5,
The in-focus direction detecting device, wherein the first pixel value and the second pixel value are equal to or greater than a predetermined pixel value. The in-focus direction detection device according to any one of claims 4 to 6,
Each of the first plurality of pixel positions in the image corresponding to the first plurality of pixel values corresponds to the first pixel position;
Each of the second plurality of pixel positions in the image corresponding to the second plurality of pixel values corresponds to the second pixel position;
The correcting means corrects at least one of the first image signal and the second image signal for each of the first plurality of pixel signal positions and the second plurality of pixel positions;
The direction detecting means detects the direction of the in-focus position based on the magnitude relationship for each of the first plurality of pixel signal positions and the second plurality of pixel positions. Direction detection device. In the focusing direction detection apparatus according to any one of claims 1 to 7,
The in-focus direction detecting device, wherein the first pixel position and the second pixel position are the same. In the focusing direction detection apparatus according to any one of claims 1 to 7,
The in-focus direction detecting device, wherein the first pixel position and the second pixel position are adjacent to each other. The in-focus direction detection device according to any one of claims 1 to 9,
The image generated by the imaging means includes three types of image signals relating to three types of color components that constitute the color of the subject,
The in-focus direction detection device includes a zoom position of the optical system, a color of the subject, an ISO sensitivity when the three types of image signals are output, and a white balance for the three types of image signals. According to any one of the above, further comprising a determining means for determining two types of image signals from the three types of image signals,
The in-focus direction detecting device, wherein the two types of image signals determined by the determining means are the first image signal and the second image signal. The in-focus direction detection device according to any one of claims 1 to 10,
The imaging means outputs a frame image including the first image signal and the second image signal at predetermined intervals,
The in-focus direction detecting apparatus, wherein the direction detecting unit detects a direction of the in-focus position based on the magnitude relationship in the plurality of frame images output at the predetermined intervals. The in-focus direction detection device according to any one of claims 1 to 11,
The optical system;
An imaging apparatus comprising: drive means for driving the optical system in the direction of the in-focus position detected by the direction detection means. A first image signal relating to a first color component constituting the color of the subject, and a second image signal relating to a second color component constituting the color of the subject and different from the first color component. Imaging processing to generate an image including,
Of the first plurality of pixel values constituting the first image signal, the first pixel value at the first pixel position in the image and the second plurality constituting the second image signal. Among the pixel values of the first image signal, the second pixel value at the second pixel position in the image corresponding to the first pixel position is substantially matched, and the signal waveform of the first image signal is The first image signal and the second image signal so that the first gradient at one pixel position and the second gradient at the second pixel position of the signal waveform of the second image signal substantially coincide with each other. Correction processing for correcting at least one of the image signals of
A direction detection process for detecting a direction of a focus position of the optical system based on a magnitude relationship between the first image signal and the second image signal after correction by the correction process; A focusing direction detection processing program.

Images