JP2008304629A - Focus controller, imaging device, focus controlling method and integrated circuit - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To manage frequency components at or above the Nyquist frequency without increasing the frequency of a process clock and obtain an effective focus evaluation value even when the high frequency components at or above the Nyquist frequency of a picked up image are dominant. <P>SOLUTION: This focus controller is equipped with a pixel shift compensation part, a first signal processing part, a moire component extraction part, a focus evaluation value generation part, and a focus control part. The pixel shift compensation part compensates phase shifts among the first image signals and generates a second image signal, and the first signal processing part extracts a first predetermined frequency component from the second image signal and generates a third image signal. The moire component extraction part subtracts among the third image signal, restrains the signal components in the Nyquist frequency band, and extracts the moire components folded back in the Nyquist frequency band. The focus evaluation value generation part computes the focus evaluation value based on the moire components, and focus control is performed based on the focus evaluation value. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a focus control device, an imaging device, a focus control method, and an integrated circuit of an imaging device that performs focus control.

  Conventionally, various methods have been proposed and put into practical use for autofocusing of image pickup apparatuses. In particular, a contrast detection method is widely used in which various processes are performed on an image signal captured by an image sensor to detect the contrast of the image signal. The high frequency component of the image signal present in the contour portion or the like when the subject is imaged increases as the focus is adjusted. This means that the brightness level difference between the bright part and the dark part of the contour portion of the subject, that is, the contrast becomes large. The contrast detection method uses this property. That is, in order to focus on a desired subject, the focus lens may be controlled automatically or manually so that the high-frequency component of the image signal is increased. Here, as a method of detecting contrast, a method of extracting a high frequency component as a contrast value by applying a filter process using an HPF (High Pass Filter) having a predetermined frequency characteristic to an image signal is often used.

  Various frequency characteristics of the HPF for extracting the high-frequency component are conceivable, but the frequency component included in the captured image signal varies depending on the imaging conditions such as the type of subject to be imaged and the zoom position of the lens. It is desirable to select an HPF having an appropriate frequency characteristic. For example, when the frequency component contained in the subject is relatively low, an HPF having a low cutoff frequency is selected to extract the frequency component, and when the frequency component contained in the subject is relatively high, the frequency is selected. A better focus evaluation value can be obtained by selecting an HPF having a high cutoff frequency in order to extract components. Similarly, even when the same subject is imaged, if the zoom position of the lens is on the tele (tele) side (telephoto side), the frequency component included in the captured image signal is relatively low, so the cutoff frequency If the zoom position of the lens is on the wide side (wide angle side), the frequency component contained in the captured image signal is relatively high, so that the HPF with a high cutoff frequency is selected. Is preferable.

  As a conventional focus control device that switches the cut-off frequency of an HPF used for extraction of a high-frequency component in accordance with the zoom position of the lens as described above, for example, the one described in Patent Document 1 is known.

  FIG. 8 is a diagram showing a configuration of a conventional focus control apparatus.

  The focus control apparatus shown in FIG. 8 includes a lens unit unit 1, a zoom position detection unit 11, imaging units 12G, 12B, and 12R including imaging elements such as a CCD and a CMOS, a high-frequency component extraction unit 13, and an evaluation value generator. Unit 14 and focus control unit 10. The lens unit 1 can control zoom and focus on the subject. The zoom position detection unit 11 detects the zoom position of the lens unit unit 1. The imaging units 12G, 12B, and 12R image a subject through the lens unit unit 1. The high frequency component extraction unit 13 generates a luminance signal from the three-channel signals G0, B0, and R0 that are photoelectrically converted and output by the imaging units 12G, 12B, and 12R, and then the lens unit detected by the zoom position detection unit 11 A high frequency component is extracted by an HPF selected from N based on the zoom position of the unit 1. The focus evaluation value generation unit 14 generates a focus evaluation value by integrating the absolute value of the high frequency component extracted by the high frequency component extraction unit 13 over the region where the focus state is evaluated. The focus control unit 10 determines the focus state based on the focus evaluation value and performs focus control of the lens unit unit 1.

  FIG. 9 is a flowchart showing the HPF selection method of the high-frequency component extraction unit 13 shown in FIG.

  The focus operation of the conventional focus control apparatus will be described below with reference to FIGS. When the subject is imaged using the imaging device including the focus control device shown in FIG. 8, the light incident through the lens unit 1 is converted into 3-channel electrical signals by the imaging units 12G, 12B, and 12R. The high frequency component extraction unit 13 generates a luminance signal from the three-channel electrical signal, and extracts a high frequency component by the HPF selected according to the flowchart shown in FIG. This operation will be described below.

As shown in FIG. 9, the high frequency component extraction unit 13 confirms the zoom position of the lens unit 1 detected by the zoom position detection unit 11, and if the zoom position is at the tele end (telephoto end). The tele end HPF is selected, and if the zoom position is not at the tele end, it is confirmed whether the zoom position is at position 1 next. If the zoom position is at position 1, then the position 1 HPF is selected. If the zoom position is not at position 1, then it is confirmed whether it is at position 2. Similarly, the positions 3, 4,..., N-2 are confirmed, and if not, the zoom position is determined to be at the wide end (wide-angle end), and the wide end HPF is selected. In this manner, the high frequency component is extracted from the luminance signal by the HPF selected according to the zoom position from the tele end HPF having a low cut-off frequency to the wide end HPF having a high cut-off frequency. The focus evaluation value generation unit 14 integrates the extracted absolute value of the high-frequency component over a preset region to generate a focus evaluation value that serves as an index for evaluating the focus state. Since the focus evaluation value becomes larger as the focus position of the lens unit unit 1 approaches the focus position with respect to the subject, the focus control unit 10 controls the lens unit unit 1 so that the focus evaluation value takes a maximum value. By doing so, a focusing operation according to the zoom position of the lens is realized.
JP 2006-215105 A

  However, in the above conventional focus control device, a high frequency component equal to or higher than the Nyquist frequency is included as a frequency component included in the captured image, such as when a subject at a distant place is imaged at the wide end or the pattern of the subject itself is very fine. May become dominant. In such a case, it is difficult to control the focus because a sufficient focus evaluation value is not generated even in the in-focus state. Hereinafter, this will be described with reference to FIG.

  FIG. 10 is a diagram illustrating frequency characteristics of an image signal imaged by a general imaging apparatus. In FIG. 10, (a), (b), and (c) are frequency characteristics of the three-channel image signals imaged by the imaging units 12G, 12B, and 12R, and (d) is a predetermined image signal of the three channels. It is the frequency characteristic of the luminance signal generated by adding at the ratio. Generally, when an image signal obtained by imaging a subject includes a frequency component higher than the Nyquist frequency fs / 2 of the drive frequency fs of the imaging unit, imaging is performed when the image signal is analog / digital (A / D) converted. By sampling at the drive frequency fs of the unit, a signal having a frequency equal to or higher than the Nyquist frequency is turned back into the Nyquist frequency band and mixed into the signal component as a moire component.

  Once this moire component is mixed, it cannot be removed by subsequent signal processing. Therefore, the subject is usually imaged through an optical LPF (Low Pass Filter) for removing a frequency component higher than the Nyquist frequency in advance. As a result, the signal components of the obtained three-channel image signal are limited to a frequency band equal to or lower than the Nyquist frequency fs / 2, as shown in (a), (b) and (c) of FIG. Therefore, the frequency band of the luminance signal generated from the three-channel image signal is also limited to a frequency band equal to or lower than the Nyquist frequency fs / 2, as shown in FIG. Naturally, no matter what HPF is selected in the high frequency component extraction unit 13, the high frequency component extracted to generate the focus evaluation value from this luminance signal is a frequency component equal to or lower than the Nyquist frequency. Therefore, when a high frequency component equal to or higher than the Nyquist frequency is dominant as a frequency component included in the captured image as described above, a sufficient focus evaluation value is not generated even in the in-focus state. It becomes difficult to control.

  In addition to such an imaging apparatus, there is an imaging apparatus that employs a spatial pixel shift method as a technique for improving the resolution of an image signal. In this case, the Nyquist frequency fccd / 2 or more with respect to the drive frequency fccd of the imaging unit. An optical LPF that also passes the frequency component is used. Although details will be described later in the spatial pixel shift method, if the clock frequency after performing the process of correcting the spatial pixel position shift between the channels is represented by fs, the Nyquist frequency fs / 2 or more is used in this case as well. Therefore, it is necessary to limit the frequency band to the Nyquist frequency fs / 2 or less by a digital filter or the like. Accordingly, in a case where a high frequency component equal to or higher than the Nyquist frequency is dominant as a frequency component included in the captured image, as in the case of imaging a subject through an optical LPF for removing a frequency component higher than the Nyquist frequency in advance. It becomes difficult to control the focus.

  The present invention solves the above-described conventional problems, and even when a high frequency component equal to or higher than the Nyquist frequency is dominant as a frequency component included in a captured image, a moiré component folded in the Nyquist frequency band is used. Focus control device and imaging device capable of handling frequency components higher than the Nyquist frequency without increasing the frequency of the processing clock by using as an index of the degree of focus on the subject, and obtaining an effective focus evaluation value A focus control method and an integrated circuit are provided.

  A focus control device according to a first aspect of the present invention includes a lens unit section that collects light from a subject and is capable of focus control, and approximately half the pixel pitch in at least one of horizontal and vertical directions. A focus control device applied to an imaging device including a plurality of imaging units that are shifted from each other and each output a first image signal, a pixel shift correction unit, a first signal processing unit, and moire component extraction Unit, a focus evaluation value generation unit, and a focus control unit. The pixel shift correction unit corrects a phase shift between the first image signals and generates a second image signal. The first signal processing unit extracts a first predetermined frequency component from the second image signal and generates a third image signal. The moiré component extraction unit performs subtraction between the third image signals, thereby suppressing the signal component in the Nyquist frequency band and extracting the moiré component that is turned back into the Nyquist frequency band. The focus evaluation value generation unit calculates a focus evaluation value based on the moire component. The focus control unit performs focus control of the lens unit unit based on the focus evaluation value.

  The first image signal is corrected for phase shift by the pixel shift correction unit to generate a second image signal, and the second image signal is extracted by extracting a first predetermined frequency component by the first signal processing unit. Three image signals are generated, and the third image signal is subtracted by a predetermined ratio by the moire component extraction unit, so that the signal component in the Nyquist frequency band is suppressed and the moire component folded back in the Nyquist frequency band is extracted. . Then, the focus evaluation value generation unit generates a focus evaluation value from a frequency component higher than the Nyquist frequency that is folded back as a moire component in the Nyquist frequency band, and the focus control unit performs focus control based on the focus evaluation value. Is called.

  Thereby, even when a high frequency component equal to or higher than the Nyquist frequency is dominant as a frequency component included in the image, by using the moire component folded back in the Nyquist frequency band as an index of the degree of focus on the subject, A frequency component higher than the Nyquist frequency can be handled without increasing the frequency of the processing clock, and an effective focus evaluation value can be obtained.

  A focus control apparatus according to a second aspect of the present invention is the focus control apparatus according to the first aspect, wherein the first predetermined frequency component is a total frequency component or a partial frequency component of the second image signal. is there.

  A focus control apparatus according to a third aspect of the present invention is the focus control apparatus according to the first or second aspect, wherein the first signal processing unit extracts a low-frequency component from each second image signal, and A predetermined ratio is determined so that the amplitude levels of the low frequency components are substantially equal, and the moire component extraction unit performs subtraction between the third image signals at the predetermined ratio.

  Here, the predetermined ratio means, for example, a ratio of gain adjustment performed by the first signal processing unit so that the levels of the signal components of the respective image signals are equal. When the subject to be imaged is a chromatic color, each image signal output from the G channel, B channel, and R channel imaging units has an unbalanced signal component. Therefore, by adjusting the level of each signal component to be equal by the first signal processing unit, it is possible to suppress the signal component by subtraction in the moire component extraction unit even when the subject to be imaged is a chromatic color. Moreover, the suppression effect of the signal component can be further enhanced even with an achromatic color.

  A focus control apparatus according to a fourth aspect of the present invention is the focus control apparatus according to any one of the first to third aspects, wherein the second signal processing unit extracts a second predetermined frequency component from the moire component. The focus evaluation value generation unit integrates at least a part of the second predetermined frequency component to generate a focus evaluation value.

  A focus control device according to a fifth aspect of the present invention is the focus control device according to the fourth aspect, wherein the second predetermined frequency component extracted from the moire component is the total frequency component of the moire component or a part of the frequency component. It is a frequency component.

  According to a sixth aspect of the present invention, there is provided a focus control device that collects light from a subject and has a lens controllable configuration, and approximately half the pixel pitch in at least one of horizontal and vertical directions. A focus control device applied to an imaging device including a plurality of imaging units that are shifted from each other and each output a first image signal, a pixel shift correction unit, a first signal processing unit, and moire component extraction Unit, a signal component extraction unit, a focus evaluation value generation unit, and a focus control unit. The pixel shift correction unit corrects a phase shift between the first image signals and generates a second image signal. The first signal processing unit extracts a first predetermined frequency component from the second image signal and generates a third image signal. The moiré component extraction unit performs subtraction between the third image signals, thereby suppressing the signal component in the Nyquist frequency band and extracting the moiré component that is turned back into the Nyquist frequency band. The signal component extraction unit performs addition between the third image signals, thereby suppressing a moire component that is folded back in the Nyquist frequency band and extracting a predetermined signal component. The focus evaluation value generation unit calculates a focus evaluation value from at least one of the extracted moire component and the extracted predetermined signal component. The focus control unit performs focus control of the lens unit unit based on the focus evaluation value.

  This makes it possible to handle frequency components higher than the Nyquist frequency without increasing the frequency of the processing clock by using the moiré component that is folded back in the Nyquist frequency band, and to combine the signal component and moiré component in the Nyquist frequency band. Since either or both of the generated amounts can be used as an indicator of the degree of focus on the subject, either the frequency component below the Nyquist frequency or the high frequency component above the Nyquist frequency is dominant as the frequency component included in the captured image. Even in this case, an effective focus evaluation value can be obtained.

  A focus control apparatus according to a seventh aspect of the present invention is the focus control apparatus according to the sixth aspect, wherein the first predetermined frequency component is the total frequency component or a part of the frequency component of the second image signal. is there.

  A focus control apparatus according to an eighth aspect of the present invention is the focus control apparatus according to the sixth or seventh aspect, wherein the first signal processing unit extracts a low frequency component from each second image signal, A predetermined ratio is determined so that the amplitude levels of the frequency components are substantially equal, the moire component extraction unit performs subtraction between the third image signals by the predetermined ratio, and the signal component extraction unit performs the predetermined ratio. Addition is performed between the third image signals.

  A focus control apparatus according to a ninth aspect of the present invention is the focus control apparatus according to any of the sixth to eighth aspects, wherein the second predetermined frequency is obtained from one or both of the moire component and the predetermined signal component. A second signal processing unit that extracts components is further provided, and the focus evaluation value generation unit integrates at least a part of the second predetermined frequency component to generate a focus evaluation value.

  A focus control apparatus according to a tenth aspect of the present invention is the focus control apparatus according to the ninth aspect, wherein the second predetermined frequency component extracted from the moire component and the predetermined signal component is the moire component and the predetermined frequency component. It is the total frequency component or a part of the frequency component of the signal component.

  A focus control apparatus according to an eleventh aspect of the present invention is the focus control apparatus according to any one of the sixth to tenth aspects, wherein a moiré component is determined by at least one of the zoom position of the lens unit and the calculated focus evaluation value. And a selector that selects one or both of the predetermined signal components.

  A focus control apparatus according to a twelfth aspect of the present invention is the focus control apparatus according to the ninth or tenth aspect of the present invention, wherein the second signal processing unit includes a filter, and the filter includes a moire component and a predetermined signal component. Are common filters for extracting the second predetermined frequency component.

  An imaging device according to a thirteenth aspect of the present invention includes the focus control device according to the first or sixth aspect.

  Thereby, it is possible to realize an imaging device including a focus control device that achieves the same effects as the first invention or the sixth invention.

  According to a fourteenth aspect of the present invention, there is provided a lens control unit that collects light from an object and has a focus controllable structure, and approximately half the pixel pitch in at least one of the horizontal and vertical directions. A focus control method applied to an imaging apparatus including a plurality of imaging units that are shifted from each other and output a first image signal, respectively, a pixel shift correction step, a signal processing step, a moire component extraction step, And a focus evaluation value generation step and a focus control step. In the pixel shift correction step, a phase shift between the first image signals is corrected to generate a second image signal. In the signal processing step, a first predetermined frequency component is extracted from the second image signal to generate a third image signal. In the moire component extraction step, subtraction is performed between the third image signals to suppress the signal component in the Nyquist frequency band and extract the moiré component that is turned back in the Nyquist frequency band. In the focus evaluation value generation step, a focus evaluation value is calculated based on the moire component. In the focus control step, focus control of the lens unit unit is performed based on the focus evaluation value.

  As a result, a focus control method that achieves the same effects as the first invention can be realized.

  A focus control method according to a fifteenth aspect of the present invention includes a lens unit portion that collects light from a subject and has a focus controllable structure, and approximately half the pixel pitch in at least one of the horizontal and vertical directions. A focus control method applied to an imaging apparatus including a plurality of imaging units that are shifted from each other and output a first image signal, respectively, a pixel shift correction step, a signal processing step, a moire component extraction step, A signal component extraction step, a focus evaluation value generation step, and a focus control step. In the pixel shift correction step, a phase shift between the first image signals is corrected to generate a second image signal. In the signal processing step, a first predetermined frequency component is extracted from the second image signal to generate a third image signal. In the moire component extraction step, subtraction is performed between the third image signals to suppress the signal component in the Nyquist frequency band and extract the moiré component that is turned back into the Nyquist frequency band. In the signal component extraction step, by adding between the third image signals, a predetermined signal component is extracted while suppressing the moire component that is turned back in the Nyquist frequency band. In the focus evaluation value generation step, a focus evaluation value is calculated from at least one of the extracted moire component and the extracted predetermined signal component. In the focus control step, focus control of the lens unit unit is performed based on the focus evaluation value.

  As a result, a focus control method that achieves the same effects as the sixth invention can be realized.

  According to a sixteenth aspect of the present invention, there is provided an integrated circuit that collects light from a subject and has a lens controllable configuration, and at least one half of a pixel pitch in at least one of horizontal and vertical directions. An integrated circuit applied to an imaging apparatus including a plurality of imaging units that are shifted from each other and that respectively output a first image signal and a focus control unit that performs focus control of a lens unit unit based on a focus evaluation value A pixel shift correction unit, a signal processing unit, a moire component extraction unit, and a focus evaluation value generation unit. The pixel shift correction unit corrects a phase shift between the first image signals and generates a second image signal. The signal processing unit extracts a first predetermined frequency component from the second image signal to generate a third image signal. The moiré component extraction unit performs subtraction between the third image signals, thereby suppressing the signal component in the Nyquist frequency band and extracting the moiré component that is turned back into the Nyquist frequency band. The focus evaluation value generation unit calculates a focus evaluation value based on the moire component.

  Thus, an integrated circuit that exhibits the same effect as that of the first invention can be realized.

  According to a seventeenth aspect of the present invention, there is provided an integrated circuit that collects light from a subject and has a lens controllable configuration, and a lens unit portion that is substantially half the pixel pitch in at least one horizontal and vertical direction. An integrated circuit applied to an imaging apparatus including a plurality of imaging units that are shifted from each other and that respectively output a first image signal and a focus control unit that performs focus control of a lens unit unit based on a focus evaluation value A pixel shift correction unit, a signal processing unit, a moire component extraction unit, a signal component extraction unit, and a focus evaluation value generation unit. The pixel shift correction unit corrects a phase shift between the first image signals and generates a second image signal. The signal processing unit extracts a first predetermined frequency component from the second image signal to generate a third image signal. The moiré component extraction unit performs subtraction between the third image signals, thereby suppressing the signal component in the Nyquist frequency band and extracting the moiré component that is turned back into the Nyquist frequency band. The signal component extraction unit performs addition between the third image signals, thereby suppressing a moire component that is folded back in the Nyquist frequency band and extracting a predetermined signal component. The focus evaluation value generation unit calculates a focus evaluation value from at least one of the extracted moire component and the extracted predetermined signal component.

  As a result, an integrated circuit having the same effect as that of the sixth invention can be realized.

  In the present invention, even when a high frequency component equal to or higher than the Nyquist frequency is dominant as a frequency component included in a captured image, the moire component folded in the Nyquist frequency band is used as an index of the degree of focus on the subject. As a result, it is possible to handle frequency components higher than the Nyquist frequency without increasing the frequency of the processing clock, and an effective focus evaluation value can be obtained.

Hereinafter, embodiments of the present invention will be described with reference to FIGS. 1, 2, 3, 4, 5, and 6.
(Embodiment 1)
<1.1: Configuration of the focus control device>
FIG. 1 is a diagram showing a schematic configuration of a focus control apparatus 100 according to Embodiment 1 of the present invention.

  A focus control apparatus 100 shown in FIG. 1 includes a lens unit unit 1, imaging units 2G, 2B, and 2R, a pixel shift correction unit 3, a first filter processing unit 4, a gain adjustment unit 5, and a moire component extraction unit. 6, a second filter processing unit 8, a focus evaluation value generation unit 9, and a focus control unit 10. The focus control device 100 constitutes a part of the imaging device.

  The lens unit unit 1 is composed of a plurality of lenses, and adjusts the focal length of a light flux from the subject (hereinafter simply referred to as light) by condensing light from the subject and moving the lens. It has a configuration capable of controlling zoom and focus on a subject. The lens unit 1 outputs the condensed light from the subject to the imaging units 2G, 2B, and 2R. The lens unit 1 or the imaging units 2G, 2B, 2R includes an optical LPF (Low Pass Filter) that allows light including a frequency component equal to or higher than the Nyquist frequency of the driving frequency of the imaging units 2G, 2B, 2R to pass therethrough. .

  The imaging units 2G, 2B, and 2R include imaging elements such as CCDs and CMOSs, receive light output from the lens unit unit 1 as input, and convert the light output from the lens unit unit 1 into electrical signals by photoelectric conversion. The converted electrical signals G0, B0, R0 are output to the pixel shift correction unit 3. As shown in FIG. 2 and the like, the imaging units 2G, 2B, and 2R are arranged so that the pixel positions for imaging the subject through the lens unit unit 1 are shifted.

  The pixel shift correction unit 3 corrects the pixel position shift between the electrical signals G0, B0, and R0 input from the imaging units 2G, 2B, and 2R, that is, the phase shift by a method that will be described later. Image signals G1, B1, and R1 are output. In the present embodiment, the processing after the pixel shift correction unit 3 is digital processing.

  As a part of the first signal processing unit, the first filter processing unit 4 extracts a first predetermined frequency component from each of the image signals G1, B1, and R1 of each channel output by the pixel shift correction unit 3. Image signals G2, B2, and R2 are output. The first filter processing unit 4 also outputs signals GL2, BL2, and RL2 composed of low-frequency components for use in a gain adjustment unit 5 described later. The signals G2, B2, and R2 output from the first filter processing unit 4 are not limited to some frequency components, and may include all frequency components. Here, the total frequency component means all frequencies included in the image signals G1, B1, and R1.

  As a part of the first signal processing unit, the gain adjustment unit 5 adjusts the gain according to the ratio of the low frequency components of the image signal that is the output of the first filter processing unit 4, and the image signals G3, B3, R3 is generated. As will be described later with reference to FIG. 6, when the subject to be imaged is a chromatic color, the image signals output from the G channel, B channel, and R channel imaging units 2G, 2B, and 2R are out of balance. Become. Therefore, when the subject to be imaged is a chromatic color or when it is desired to further enhance the suppression effect of the signal component even if it is an achromatic color, the gain adjustment unit 5 performs gain adjustment so that the level of each signal component becomes equal. I do.

  The moiré component extraction unit 6 extracts the moiré component M4 from the image signals G3, B3, and R3 of each channel generated by the gain adjustment unit 5 (FIG. 6C) and outputs it.

  As part of the second signal processing unit, the second filter processing unit 8 extracts a second predetermined frequency component from the moire component M4 extracted and output by the moire component extraction unit 6, and outputs the moire component M5. . Extracting the second predetermined frequency component may be, for example, extraction of all frequency components, or extraction of a part of moire components as shown in FIG.

  For example, the focus evaluation value generation unit 9 integrates the absolute value of the second predetermined frequency component of the moire component M5 in a certain region determined for evaluating the focus state in the screen, and obtains the focus evaluation value. And output to the focus control unit 10. The focus evaluation value may be generated from all the moire components in the same region, or the focus evaluation value may be generated from a part (for example, extracted by thinning out) in order to save the processing amount. .

The focus control unit 10 determines a focus state based on the input focus evaluation value, and outputs a focus control signal to the lens unit unit 1. The lens unit 1 adjusts the focal length of the light from the subject by moving the lens or the like in accordance with the input focus control signal.
<1.2: Operation of the focus control device>
Hereinafter, the operation of the focus control apparatus according to the first embodiment of the present invention will be described.

  Light from the subject is collected by the lens unit 1 and input to the imaging units 2G, 2B, and 2R. Light input to the imaging units 2G, 2B, and 2R is converted into an electrical signal by photoelectric conversion, and is output to the pixel shift correction unit 3 as electrical signals G0, B0, and R0. The signals G0, B0, R0 input to the pixel shift correction unit 3 are corrected for pixel position shift, that is, phase shift, by a method as described later, and are subjected to first filter processing as channel image signals G1, B1, R1. To the unit 4. From the signals G1, B1, and R1 input to the first filter processing unit 4, first predetermined frequency components are extracted and output to the gain adjustment unit 5 as image signals G2, B2, and R2. Further, the signals G1, B1, and R1 input to the first filter processing unit 4 are extracted as low frequency components for use in the gain adjusting unit 5 and output as low frequency component signals GL2, BL2, and RL2. Is done. The signals G2, B2, and R2 input to the gain adjusting unit 5 are gain-adjusted based on the low frequency components GL2, BL2, and RL2, and are output to the moire extracting unit 6 as image signals G3, B3, and R3. . The image signals G3, B3, and R3 input to the moire extraction unit 6 are extracted with the moire component M4 and output to the second filter processing unit 8. From the moire component M4 input to the second filter processing unit 8, a second predetermined frequency component is extracted and output to the focus evaluation value generation unit 9 as the moire component M5.

  The focus evaluation value generation unit 9 generates a focus evaluation value by integrating the absolute values of some of the frequency components of the moire component extracted in this way over the region where the focus state is to be evaluated. In this case, the focus evaluation value may be generated by integrating only a part of the data by reducing the number of data instead of all the absolute values. The focus evaluation value generated in this way is output to the focus control unit 10. Then, the focus control unit 10 realizes a focusing operation by controlling the lens unit 1 so that the focus evaluation value becomes a maximum value.

  In the focus control apparatus 100 according to the first embodiment of the present invention, the spatial pixel shifting method is employed. As shown in FIGS. 2A and 3A, the B-channel and R-channel imaging units 2B and 2R are spatially spaced in the horizontal direction by ½ the pixel pitch with respect to the G-channel imaging unit 2G. It is arranged to shift to. Therefore, image signals whose phases are shifted by 1 / (2 fccd) are output from the image capturing units 2G, 2B, and 2R of each channel, where the drive frequency of the image capturing unit is fccd. The pixel shift correction unit 3 corrects the phase shift between the channels.

  As a method of correcting the pixel shift, various methods can be considered depending on the imaging device. For example, as shown in FIG. 2, in the case of an HD (High Definition) camera, the image signal of each channel output from the imaging unit is processed with the same frequency as the drive frequency fccd (for example, 74 MHz) of the imaging unit. Interpolation processing is performed by a digital filter or the like operating with the clock fs, and image signals of the B channel and the R channel having the same phase as the G channel are generated as shown in FIG. On the other hand, as shown in FIG. 3, in the case of an SD (Standard Definition) camera, pixel shift correction is performed with a processing clock fs having a frequency twice the drive frequency fccd (for example, 18 MHz) of the imaging unit. In this case, two methods are conceivable. First, after inserting 0 into the image signal of each channel output from the image pickup unit at a position shifted by 1 / (2 fccd) in the G channel, the B channel, and the R channel, the same processing as described above is performed. In this method, interpolation processing is performed using a digital filter or the like that operates at the clock fs. The second is a method of performing interpolation processing with the processing clock fs by zero-order hold. In either case, as shown in FIG. 3B, an image signal having the same phase for each channel is generated.

  An optical LPF (not shown) used in an imaging apparatus employing the spatial pixel shifting method also passes a frequency component having a Nyquist frequency of fccd / 2 or higher. Therefore, when the subject including such a frequency component is imaged, the image signals G0, B0, and R0 output by the G channel, B channel, and R channel imaging units 2G, 2B, and 2R are shown in FIG. As shown in (b), the frequency component having a Nyquist frequency of fccd / 2 or higher has a frequency characteristic that is turned back into a frequency band of Nyquist frequency fccd / 2 or lower as a moire component. Here, as described above, the B channel and R channel imaging units 2B and 2R are spatially shifted in the horizontal direction by 1/2 of the pixel pitch with respect to the G channel imaging unit 2G. Therefore, the moire component of the B channel and the R channel is a moire component having a phase inverted by 180 ° with respect to the moire component of the G channel, as shown in FIG. 4B. In order to improve the resolution in the spatial pixel shifting method, for example, G channel, B channel, and R channel image signals are separated into a high frequency component and a low frequency component, respectively, and the G channel high frequency component, the B channel, and the R channel are separated. By adding and averaging the sum of the high frequency components of the channels, it is possible to extract the high frequency components in which the moire components whose phases are reversed are suppressed. This is added to each low frequency component as a high frequency component common to each channel, and as shown in FIG. 4D, an image signal in which the moire component is suppressed is obtained. The extraction of the image signal with the moire component suppressed is performed by a predetermined signal component extraction unit (not shown) that is a part of the imaging apparatus.

  In the focus control apparatus 100 according to Embodiment 1 of the present invention, on the contrary, the moire component extraction unit 6 subtracts, for example, the sum of the B channel and R channel image signals from the G channel image signal. Thus, as shown in FIG. 4C, the in-phase signal components are suppressed and the anti-phase moire components are extracted.

  At this time, if the subject to be imaged is achromatic, the sum of the G channel image signal and the B channel and R channel image signals is not the same level of signal level, but is relatively high. It is close in size. Therefore, even if subtraction is performed as it is, a certain degree of signal component suppression effect can be obtained. However, when the subject to be imaged is a chromatic color, the image signals output from the G channel, B channel, and R channel imaging units 2G, 2B, and 2R are balanced as shown in FIGS. 6 (a) and 6 (b). It will collapse. Therefore, when the subject to be imaged is a chromatic color or when it is desired to further enhance the suppression effect of the signal component even if it is an achromatic color, the gain adjustment unit 5 performs gain adjustment so that the level of each signal component becomes equal. I do. In this gain adjustment, the relationship between the average value of the G channel image signal and the addition of the B channel and R channel image signals may be used, and either the B channel or the R channel may be used. The relationship with the image signal may be used.

As the method, there are the following methods on the premise that there is a correlation between the signal level of the low frequency component and the high frequency component of the image signal. For example, the first filter processing unit 4 extracts the low frequency component of each channel as indicated by the oblique lines in FIGS. 6A and 6B, and sets the signal level of the G channel low frequency component to g and the B channel low. Assuming that the signal level obtained by adding the frequency component and the low frequency component of the R channel is br, the gain adjusting unit 5 multiplies the sum of the B channel and R channel image signals by g / br and the moire component extracting unit 6 By subtracting from the image signal of the G channel, as shown in FIG. 6C, the in-phase signal components are greatly suppressed, and the anti-phase moire components are extracted.
<1.3: Others>
In the present embodiment, the second filter processing unit 8 narrows down the frequency component of the subject to be focused by using only a part of the frequency components of the moire component. You may make it use all the frequency components of a moire component, without narrowing down.

  Further, the restriction of the frequency band of the moire component performed by the second filter processing unit 8 may be performed by the first filter processing unit 4. That is, if the first filter processing unit 4 extracts the frequency component of the image signal used for the subtraction in advance in the shaded area in FIGS. 5A and 5B, the second filter processing unit 8 is not required. The moire component shown in FIG. 5C can be extracted.

  Also, in the present embodiment, a case where the B channel and R channel imaging units 2B and 2R are spatially shifted in the horizontal direction by 1/2 of the pixel pitch with respect to the G channel imaging unit 2G will be described. However, processing may be performed by spatially shifting in the vertical direction and replacing each horizontal clock frequency described above with a vertical clock frequency, that is, a line frequency. Alternatively, both the horizontal direction and the vertical direction may be spatially shifted from each other by 1/2 of the pixel pitch and processed in the same manner.

In the present embodiment, image signals output from image pickup units for different color signals of the G channel, the B channel, and the R channel are used. Instead, for example, the pixel pitch in at least one direction of the horizontal direction and the vertical direction is used. Are provided with two image pickup units for the same color signal that are spatially shifted by 1/2 (for example, two 2Gs are provided using a four-plate image pickup unit and are shifted and arranged), and each outputs. An image signal may be used. In this case, if the light amounts received by the two imaging units are made equal to each other and subtracted or subtracted at a rate corresponding to the received light amounts, the gain adjusting unit 5 is unnecessary, and by subtraction It becomes possible to remove the signal component almost completely. Note that a 6-plate imaging unit provided with two G channels, two B channels, and two R channels may be used. In this case, the second imaging unit is arranged spatially shifted by 1/2 of the pixel pitch in at least one of the horizontal direction and the vertical direction with respect to the first imaging unit of G, B, and R. With such a configuration, the focus evaluation value can be obtained more reliably regardless of the color of the subject.
<1.3: Features of the focus control device>
As described above, according to the first embodiment, by using the moiré component that is turned back in the Nyquist frequency band, it is possible to handle a frequency component that is higher than the Nyquist frequency without increasing the frequency of the processing clock. Is used as an index of the degree of focus on the subject, so that an effective focus evaluation value can be obtained even when a high frequency component equal to or higher than the Nyquist frequency is dominant as a frequency component included in the captured image. A possible focus control device or the like can be provided.
(Embodiment 2)
<2.1: Configuration of focus control device>
FIG. 7 is a diagram showing a schematic configuration of the focus control apparatus 200 according to the second embodiment of the present invention. The focus control device 200 is the same as the focus control device 100 of the first embodiment in FIG. 1 except that the focus control device 200 includes a signal component extraction unit 7 and a zoom position detection unit 11 for use in generating a focus evaluation value. It has a configuration.

  A focus control device 200 illustrated in FIG. 7 includes a lens unit unit 1, imaging units 2G, 2B, and 2R, a pixel shift correction unit 3, a first filter processing unit 4, a gain adjustment unit 5, and a moire component extraction unit. 6, a signal component extraction unit 7, a second filter processing unit 8, a focus evaluation value generation unit 9, a focus control unit 10, and a zoom position detection unit 11.

  The lens unit unit 1 is composed of a plurality of lenses, and adjusts the focal length of a light flux from the subject (hereinafter simply referred to as light) by condensing light from the subject and moving the lens. It has a configuration capable of controlling zoom and focus on a subject. The lens unit 1 outputs the condensed light from the subject to the imaging units 2G, 2B, and 2R. The lens unit 1 or the imaging units 2G, 2B, and 2R includes an optical LPF that allows light including a frequency component equal to or higher than the Nyquist frequency of the driving frequency of the imaging units 2G, 2B, and 2R to pass therethrough.

  The imaging units 2G, 2B, and 2R include imaging elements such as CCDs and CMOSs, receive light output from the lens unit unit 1 as input, and convert the light output from the lens unit unit 1 into electrical signals by photoelectric conversion. The converted electrical signals G0, B0, R0 are output to the pixel shift correction unit 3. As shown in FIG. 2 and the like, the imaging units 2G, 2B, and 2R are arranged so that the pixel positions for imaging the subject through the lens unit unit 1 are shifted.

  The pixel shift correction unit 3 corrects the pixel position shift between the electrical signals G0, B0, and R0 input from the imaging units 2G, 2B, and 2R, that is, the phase shift, in the same manner as in the first embodiment. Channel image signals G1, B1, and R1 are output. In the present embodiment, the processing after the pixel shift correction unit 3 is digital processing.

  As a part of the first signal processing unit, the first filter processing unit 4 extracts a first predetermined frequency component from each of the image signals G1, B1, and R1 of each channel output by the pixel shift correction unit 3. Image signals G2, B2, and R2 are output. Further, the first filter processing unit 4 also outputs image signals GL2, BL2, and RL2 composed of low-frequency components for use in a gain adjustment unit 5 described later. Note that the image signals G2, B2, and R2 output by the first filter processing unit 4 are not limited to a part of the frequency components, and include all the frequency components as in the first embodiment. There may be.

  As a part of the first signal processing unit, the gain adjustment unit 5 adjusts the gain according to the ratio of the low frequency components of the image signal that is the output of the first filter processing unit 4, and the image signals G3, B3, R3 is generated. As described in the first embodiment, when the subject to be imaged is a chromatic color, the image signals output from the G channel, B channel, and R channel imaging units 2G, 2B, and 2R are out of balance. Become. Therefore, when the subject to be imaged is a chromatic color or when it is desired to further enhance the suppression effect of the signal component even if it is an achromatic color, the gain adjustment unit 5 performs gain adjustment so that the level of each signal component becomes equal. I do.

  The moiré component extraction unit 6 extracts the moiré component M4 from the image signals G3, B3, and R3 of each channel generated by the gain adjustment unit 5 (FIG. 6C) and outputs it.

  The signal component extraction unit 7 adds, for example, a G-channel image signal and a sum of the B-channel and R-channel image signals, thereby generating moiré components having opposite phases as shown in FIG. The in-phase signal component S4 is extracted while being suppressed. Note that such signal component extraction is performed for signal processing for the purpose of improving resolution in the imaging apparatus, but in the second embodiment, this signal component is used as the focus evaluation value. It is characterized in that it is used for generation.

  As part of the second signal processing unit, the second filter processing unit 8 extracts a second predetermined frequency component from the moire component M4 extracted and output by the moire component extraction unit 6, and outputs the moire component M5. . Extracting the second predetermined frequency component may be, for example, extraction of all frequency components, or extraction of a part of moire components as shown in FIG. The second filter processing unit 8 also extracts a second predetermined frequency component from the signal component S4 extracted and output by the signal component extraction unit 7 as a part of the second signal processing unit, and outputs the signal component S5. To do. Extracting the second predetermined frequency component may be, for example, extraction of all frequency components, or may be extraction of some frequency components as shown in FIG. In addition, filters having the same characteristics may be used for extracting the moire component M5 and the signal component S5, or filters having different characteristics may be used.

  For example, the focus evaluation value generation unit 9 is an absolute value of the second predetermined frequency component of one or both of the moire component M5 and the signal component S5 within a certain area determined for evaluating the focus state in the screen. Are integrated to generate a focus evaluation value, which is output to the focus control unit 10. In addition, focus evaluation values may be generated for all frequency components of the moire component M5 and the signal component S5 in the same region, or a part thereof (for example, a sample extracted by thinning out) in order to save processing amount. Focus evaluation values may be generated only for).

  The focus control unit 10 determines a focus state based on the input focus evaluation value, and outputs a focus control signal to the lens unit unit 1. The lens unit 1 adjusts the focal length of the light from the subject by moving the lens or the like in accordance with the input focus control signal.

The zoom position detection unit 11 receives the zoom position signal (Z position signal) output from the lens unit unit 1 and detects the zoom position. The zoom position detection signal (Z position detection signal) is transmitted to the first filter processing unit 4 and the first filter processing unit 4. 2 output to at least one of the filter processing unit 8 and the focus evaluation value generation unit 9.
<2.2: Operation of the focus control device>
Hereinafter, the operation of the focus control apparatus 200 according to the second embodiment of the present invention will be described.

  In the focus control apparatus 200 according to the second embodiment of the present invention, the spatial pixel shifting method is also employed. From the imaging units 2G, 2B, and 2R of each channel, if the imaging unit drive frequency is fccd, the phase is ½ fccd. Are output, and the pixel shift correction unit 3 corrects the phase shift between the channels.

  Similar to the first embodiment, the optical LPF provided in the lens unit 1 or the imaging units 2G, 2B, and 2R also passes a frequency component having a Nyquist frequency of fccd / 2 or higher. When a subject including such a frequency component is imaged, the image signals output by the imaging units 2G, 2B, and 2R have frequency characteristics as shown in FIGS. 4 (a) and 4 (b). The pixel shift correction unit 3, the first filter processing unit 4, the gain adjustment unit 5, and the moire component extraction unit 6 operate in the same manner as in the first embodiment.

  In the signal component extraction unit 7, for example, by adding the G-channel image signal and the B-channel and R-channel image signals, as shown in FIG. In-phase signal components are extracted with suppression.

  When generating the focus evaluation value, either one of the moire component extracted by the moire component extraction unit 6 and the signal component extracted by the signal component extraction unit 7 may be used depending on the selection, or both may be used. good. Further, all the frequency components of the extracted moire component and signal component may be used. Furthermore, the frequency component of the subject to be focused may be narrowed down by extracting only a part of the frequency components by the second filter processing unit 8. For example, the second filter processing unit 8 takes out the hatched portions in FIGS. 5A and 5B and shows the moire component in FIG. 5C and the signal component in FIG. 5D. The part shown may be used.

  Note that the frequency component of the moire component performed by the second filter processing unit 8 can be limited by the first filter processing unit 4. That is, if the first filter processing unit 4 extracts the frequency component of the image signal used for the subtraction in advance by extracting the hatched portion in FIGS. 5A and 5B, the second filter processing unit 8 can be omitted. Also, the moire component shown in FIG. 5C and the signal component shown in FIG. 5D can be extracted. The focus evaluation value generation unit 9 generates the focus evaluation value by integrating the absolute values of all or some of the frequency components of the moire component and signal component extracted in this manner over the region where the focus state is to be evaluated. To do. In this case, the focus evaluation value may be generated by integrating only a part of the data by reducing the number of data instead of all the absolute values. Then, the focus control unit 10 realizes a focusing operation by controlling the lens unit 1 so that the focus evaluation value becomes a maximum value.

  Here, in the first filter processing unit 4, the second filter processing unit 8, or the focus evaluation value generation unit 9, a selection unit for determining / selecting which one of the moire component and the signal component is selected and which is used is selected. It may be provided. This selection is performed based on the zoom position of the lens unit section 1 detected by the zoom position detection section 11 as shown in FIG. 7, the result of analyzing the generated focus evaluation value, or the like. For example, when the lens unit 1 is on the tele (tele) side (telephoto side), the frequency of the obtained image signal is relatively low, and it is determined that the Nyquist frequency or less is dominant, and focus evaluation is performed from the signal component. Generate a value. On the other hand, when the lens unit 1 is on the wide side (wide angle side), the frequency of the obtained image signal is relatively high, and it is determined that the Nyquist frequency or higher is dominant, and focus evaluation is performed from the moire component. Generate a value. Alternatively, the focus evaluation values may always be generated and used from both or one of the signal component and the moire component. Also, when a sufficient level is not obtained as the focus evaluation value generated from the signal component, a focus evaluation value is generated from the moire component, and conversely, a sufficient level is not obtained as the focus evaluation value generated from the moire component. Alternatively, the focus evaluation value may be generated from the signal component.

  As described above, by selecting either the signal component or the moire component and performing the filter processing, the filter characteristics used in the first filter processing unit 4 and the second filter processing unit 8 are the same, that is, they are common. Even if a filter is used, two types of frequency components to be extracted are selected. Therefore, there is an advantage that an effect equivalent to changing the filter characteristics can be obtained. The filter characteristics are not limited to one type, and a plurality of types of filters may be used. For example, when two types of filters are used, since the frequency components of the signal component and the moire component are extracted by each filter, it is possible to extract four types of frequency components in total.

  Here, the case where the B channel and R channel imaging units 2B and 2R are spatially shifted in the horizontal direction by 1/2 of the pixel pitch with respect to the G channel imaging unit 2G has been described. The horizontal clock frequency described above may be replaced with the vertical clock frequency, that is, the line frequency, and processing may be performed, or the pixel pitch of 1 may be set in both the horizontal and vertical directions. The processing may be performed in the same manner with the arrangement being spatially shifted by / 2.

Here, the image signals output from the image pickup units for different color signals of the G channel, the B channel, and the R channel are used, but instead, for example, 1/1 of the pixel pitch in at least one of the horizontal direction and the vertical direction. Two image pickup units for the same color signal that are spatially shifted by 2 are provided (for example, two 2G are provided by using a four-plate image pickup unit and are arranged to be shifted), and the image signals output by each are May be used. In this case, the gain adjustment unit 5 is not required if the light amounts received by the two imaging units are made equal to each other and are subtracted and added, or subtracted and added at a rate corresponding to the received light amounts. Further, the signal component and the moire component can be almost completely removed by subtraction and addition. As in the first embodiment, a 6-plate imaging unit provided with two G channels, two B channels, and two R channels may be used.
<2.3: Features of the focus control device>
As described above, according to the present embodiment, by using the moiré component folded in the Nyquist frequency band, it is possible to handle a frequency component higher than the Nyquist frequency without increasing the frequency of the processing clock. In order to use either or both of the signal component in the band and the amount of generated moire component as an index of the degree of focus on the subject, the frequency component included in the captured image is a frequency component equal to or lower than the Nyquist frequency, and the Nyquist frequency It is possible to provide a focus control device and the like that can obtain an effective focus evaluation value regardless of which of the above high-frequency components becomes dominant.
(Other embodiments)
A part of the above-described focus control device may be integrated into one chip by a semiconductor device such as an LSI.

  Further, although it is referred to as LSI here, it may be referred to as IC, system LSI, super LSI, or ultra LSI depending on the degree of integration.

  Further, the method of circuit integration is not limited to LSI's, and implementation using dedicated circuitry or general purpose processors is also possible. An FPGA (Field Programmable Gate Array) that can be programmed after mounting on a printed circuit board or a reconfigurable processor that can reconfigure the connection and settings of internal circuit cells may be used.

  Furthermore, if integrated circuit technology comes out to replace LSI's as a result of the advancement of semiconductor technology or a derivative other technology, it is naturally also possible to carry out function block integration using this technology. Biotechnology can be applied as a possibility.

  Moreover, each process of the said embodiment may be implement | achieved by hardware, and may be implement | achieved by software. Further, it may be realized by mixed processing of software and hardware.

  The specific configuration of the present invention is not limited to the above-described embodiment, and various changes and modifications can be made without departing from the scope of the invention.

  A focus control device, an imaging device, a focus control method, and an integrated circuit according to the present invention include, among image signals obtained by imaging a subject, a high-frequency component having a Nyquist frequency or higher that is not originally handled as an output signal of the imaging device. In order to evaluate the in-focus state, the frequency component included in the captured image is higher than the Nyquist frequency, such as when the subject at a distant location is imaged at the wide end or the subject's own pattern is very fine. Focus control can be performed based on an effective focus evaluation value even when the high-frequency component is dominant, and it is useful as an autofocus function and a manual focus function of the imaging apparatus.

The figure explaining schematic structure of the focus control apparatus in Embodiment 1 of this invention. The figure explaining the pixel arrangement | positioning in HD camera which employ | adopted the spatial pixel shift method The figure explaining the pixel arrangement | positioning in SD camera which employ | adopted the spatial pixel shift method The figure which shows the frequency characteristic of the image signal in Embodiment 1 and 2 of this invention The figure which shows the frequency characteristic of the image signal in Embodiment 1 and 2 of this invention The figure which shows the frequency characteristic of the image signal in Embodiment 1 and 2 of this invention The figure explaining schematic structure of the focus control apparatus in Embodiment 2 of this invention. The figure explaining schematic structure of the conventional focus control apparatus The flowchart which shows the selection method of HPF of the conventional focus control apparatus. The figure which shows the frequency characteristic of the image signal in the conventional focus control apparatus

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Lens unit part 2G, 2B, 2R 12G, 12B, 12R Image pick-up part 3 Image shift correction part 4, 8 1st, 2nd filter process part (1st, 2nd signal process part)
5 Gain adjuster (first signal processor)
6 Moire component extraction unit 7 Signal component extraction unit 9, 14 Focus evaluation value generation unit 10 Focus control unit 11 Zoom position detection unit 13 High frequency component extraction unit

Claims (17)

A lens unit unit that collects light from the subject and has a configuration capable of focus control, and is shifted from each other by approximately half the pixel pitch in at least one of the horizontal and vertical directions, and outputs a first image signal. A focus control device applied to an imaging device including a plurality of imaging units,
A pixel shift correction unit that corrects a phase shift between the first image signals to generate a second image signal;
A first signal processing unit that extracts a first predetermined frequency component from the second image signal to generate a third image signal;
A moire component extraction unit that suppresses signal components in the Nyquist frequency band and extracts a moire component that is turned back into the Nyquist frequency band by performing subtraction between the third image signals;
A focus evaluation value generation unit that calculates a focus evaluation value based on the moire component;
A focus control unit that performs focus control of the lens unit unit based on the focus evaluation value;
A focus control device. The first predetermined frequency component is a total frequency component or a partial frequency component of the second image signal.
The focus control apparatus according to claim 1. The first signal processing unit extracts a low frequency component from each of the second image signals, determines a predetermined ratio so that the amplitude level of the low frequency component is substantially equal,
The moire component extraction unit performs the subtraction between the third image signals at the predetermined ratio.
The focus control apparatus according to claim 1 or 2. A second signal processing unit for extracting a second predetermined frequency component from the moire component;
The focus evaluation value generation unit integrates at least a part of the second predetermined frequency component to generate the focus evaluation value;
The focus control apparatus according to claim 1.   5. The focus control device according to claim 4, wherein the second predetermined frequency component extracted from the moire component is all or a part of the frequency components of the moire component. A lens unit unit that collects light from the subject and has a configuration capable of focus control, and is shifted from each other by approximately half the pixel pitch in at least one of the horizontal and vertical directions, and outputs a first image signal. A focus control device applied to an imaging device including a plurality of imaging units,
A pixel shift correction unit that corrects a phase shift between the first image signals to generate a second image signal;
A first signal processing unit that extracts a first predetermined frequency component from the second image signal to generate a third image signal;
A moire component extraction unit that suppresses signal components in the Nyquist frequency band and extracts a moire component that is turned back into the Nyquist frequency band by performing subtraction between the third image signals;
A signal component extraction unit that extracts the predetermined signal component by suppressing the moire component that is folded back in the Nyquist frequency band by performing addition between the third image signals;
A focus evaluation value generator that calculates a focus evaluation value from at least one of the extracted moire component and the predetermined signal component extracted;
A focus control unit that performs focus control of the lens unit unit based on the focus evaluation value;
A focus control device. The first predetermined frequency component is a total frequency component or a partial frequency component of the second image signal.
The focus control apparatus according to claim 6. The first signal processing unit extracts a low frequency component from each of the second image signals, determines a predetermined ratio so that the amplitude level of the low frequency component is substantially equal,
The moire component extraction unit performs the subtraction between the third image signals at the predetermined ratio,
The signal component extraction unit performs the addition between the third image signals at the predetermined ratio.
The focus control apparatus according to claim 6 or 7. A second signal processing unit for extracting a second predetermined frequency component from one or both of the moire component and the predetermined signal component;
The focus evaluation value generation unit integrates at least a part of the second predetermined frequency component to generate the focus evaluation value;
The focus control apparatus according to claim 6. The second predetermined frequency component extracted from the moire component and the predetermined signal component is all or a part of the frequency components of the moire component and the predetermined signal component.
The focus control apparatus according to claim 9. A selection unit that selects one or both of the moire component and the predetermined signal component according to at least one of the zoom position of the lens unit and the calculated focus evaluation value;
The focus control apparatus according to claim 6. The second signal processing unit includes a filter;
The filter is a common filter that extracts the second predetermined frequency component for both the moire component and the predetermined signal component.
The focus control apparatus according to claim 9 or 10. The focus control device according to claim 1 or 6 is provided.
Imaging device. A lens unit unit that collects light from the subject and has a configuration capable of focus control, and is shifted from each other by approximately half the pixel pitch in at least one of the horizontal and vertical directions, and outputs a first image signal. A focus control method applied to an imaging device including a plurality of imaging units,
A pixel shift correction step of generating a second image signal by correcting a phase shift between the first image signals;
A signal processing step of generating a third image signal by extracting a first predetermined frequency component from the second image signal;
A moire component extraction step of subtracting between the third image signals to suppress a signal component in the Nyquist frequency band and extract a moiré component folded back in the Nyquist frequency band;
A focus evaluation value generating step for calculating a focus evaluation value based on the moire component;
A focus control step for performing focus control of the lens unit based on the focus evaluation value;
A focus control method comprising: A lens unit unit that collects light from the subject and has a configuration capable of focus control, and is shifted from each other by approximately half the pixel pitch in at least one of the horizontal and vertical directions, and outputs a first image signal. A focus control method applied to an imaging device including a plurality of imaging units,
A pixel shift correction step of generating a second image signal by correcting a phase shift between the first image signals;
A signal processing step of generating a third image signal by extracting a first predetermined frequency component from the second image signal;
A moire component extraction step of subtracting between the third image signals to suppress a signal component in the Nyquist frequency band and extract a moiré component folded back in the Nyquist frequency band;
A signal component extraction step of extracting a predetermined signal component while suppressing the moire component that is folded back in the Nyquist frequency band by performing addition between the third image signals;
A focus evaluation value generation step of calculating a focus evaluation value from at least one of the extracted moire component and the extracted predetermined signal component;
A focus control step for performing focus control of the lens unit based on the focus evaluation value;
A focus control method comprising: A lens unit unit that collects light from the subject and has a configuration capable of focus control, and is shifted from each other by approximately half the pixel pitch in at least one of the horizontal and vertical directions, and outputs a first image signal. An integrated circuit applied to an imaging device including a plurality of imaging units and a focus control unit that performs focus control of the lens unit unit based on a focus evaluation value,
A pixel shift correction unit that corrects a phase shift between the first image signals to generate a second image signal;
A signal processing unit for extracting a first predetermined frequency component from the second image signal to generate a third image signal;
A moire component extraction unit that suppresses signal components in the Nyquist frequency band and extracts a moire component that is turned back into the Nyquist frequency band by performing subtraction between the third image signals;
A focus evaluation value generation unit that calculates the focus evaluation value based on the moire component;
An integrated circuit comprising: A lens unit unit that collects light from the subject and has a configuration capable of focus control, and is shifted from each other by approximately half the pixel pitch in at least one of the horizontal and vertical directions, and outputs a first image signal. An integrated circuit applied to an imaging device including a plurality of imaging units and a focus control unit that performs focus control of the lens unit unit based on a focus evaluation value,
A pixel shift correction unit that corrects a phase shift between the first image signals to generate a second image signal;
A signal processing unit for extracting a first predetermined frequency component from the second image signal to generate a third image signal;
A moire component extraction unit that suppresses signal components in the Nyquist frequency band and extracts a moire component that is turned back into the Nyquist frequency band by performing subtraction between the third image signals;
A signal component extraction unit that extracts the predetermined signal component by suppressing the moire component that is folded back in the Nyquist frequency band by performing addition between the third image signals;
A focus evaluation value generating unit that calculates the focus evaluation value from at least one of the extracted moire component and the extracted predetermined signal component;
An integrated circuit comprising:

Images