JP2023034485A - Imaging device and autofocus control method - Google Patents

Abstract

To provide an imaging device with which it is possible to realize autofocus that excels in responsiveness and focusing accuracy.SOLUTION: Distortion aberration vector correction units 311 and 312 correct first and second image data on the basis of distortion aberration vector information for each pixel that indicates the distortion aberration vectors of first and second lens groups. A parallax calculation unit 313 calculates the parallax between the first and second image data on the basis of the image data corrected by the distortion aberration vector correction units 311 and 312. A focus lens position conversion unit 314 converts the parallax calculated by the parallax calculation unit 313 into the position information on a focus lens for focusing on the distance to a subject that is indicated by the parallax. A drive control unit 33 controls a drive unit so as to move the position of the focus lens to the position indicated by the position information.SELECTED DRAWING: Figure 5

Description

The present invention relates to an imaging device and an autofocus control method.

As a representative example of an autofocus (AF) method in which an imaging device automatically focuses on a subject when photographing the subject, the distance to the subject is measured using two imaging systems arranged in the horizontal direction. There are phase-difference AF and hill-climbing AF using a hill-climbing method (see Patent Document 1). In phase difference AF, the approximate distance to the subject is measured based on the horizontal parallax (phase difference) of the image obtained by the two-lens imaging system, and the focus lens is adjusted according to the measured distance. be moved. In the hill-climbing AF, the focus lens is moved so that the contrast of the image of the subject is maximized.

JP-A-2006-93860

Phase-difference AF has good responsiveness, but its focusing accuracy is not so good. On the other hand, the hill-climbing AF has good focusing accuracy, but the responsiveness is poor because it takes a relatively long time to achieve focusing. There is a demand for AF control with good responsiveness and focusing accuracy.

SUMMARY OF THE INVENTION It is an object of the present invention to provide an imaging apparatus and an autofocus control method capable of achieving autofocus with good responsiveness and focusing accuracy.

The present invention has a first imaging system having a first lens group including a first focus lens and a first imaging device, a second lens group and a second imaging device, The lens group includes: a second imaging system arranged horizontally or vertically with a predetermined distance from the first lens group; and a phase difference autofocus processing unit for focusing on a distance based on the parallax of the second image data, and a first driving the first focus lens based on the processing by the phase difference autofocus processing unit and a first drive control section that controls the drive section.

In the above configuration, the phase difference autofocus processing unit corrects the first image data based on distortion vector information for each pixel indicating a distortion vector of the first lens group. a second distortion vector correction unit configured to correct the second image data based on distortion vector information for each pixel indicating a distortion vector of the second lens group; a parallax calculation unit for calculating parallax between the first and second image data whose distortion vectors are corrected by first and second distortion aberration vector correction units, respectively; and the parallax calculated by the parallax calculation unit. into position information indicating the position of the first focus lens for focusing on the distance to the subject indicated by the parallax.

In the above configuration, the first drive control section controls the first drive section to move the position of the first focus lens to the position indicated by the position information.

In the present invention, a subject is imaged by a first imaging system having a first lens group including a focus lens and a first imaging element, first image data is generated, and the second lens group and the second and the second lens group is arranged horizontally or vertically with a predetermined distance from the first lens group. , generating second image data, correcting the first image data based on distortion vector information for each pixel indicating a distortion vector of the first lens group, and converting the second image data to , correcting based on distortion vector information for each pixel indicating the distortion vector of the second lens group, and calculating the parallax between the first and second image data in which the distortion vector is corrected. and converting it into position information indicating the position of the focus lens for focusing on the distance to the subject indicated by the parallax, and moving the position of the focus lens to the position indicated by the position information. do.

According to the imaging device and the autofocus control method of the present invention, autofocus with good responsiveness and good focusing accuracy can be realized.

It is a block diagram showing an imaging device of each embodiment. 1 is a perspective view showing a schematic appearance of an imaging device according to each embodiment; FIG. FIG. 4 is a diagram showing barrel distortion due to lens distortion; FIG. 10 is a diagram showing pincushion distortion due to distortion of a lens; FIG. 4 is a diagram showing distortion aberration vectors due to barrel distortion; FIG. 4 is a diagram showing distortion aberration vectors due to pincushion distortion; 3 is a block diagram showing a specific configuration example of a control unit in the imaging device of the first embodiment; FIG. 4 is a flow chart showing an autofocus control method of the first embodiment; FIG. 11 is a block diagram showing a specific configuration example of a control unit in the imaging apparatus according to the second embodiment; FIG. FIG. 10 is a diagram showing an example of an image containing a repeating pattern; FIG. FIG. 8C is a conceptual diagram showing a state in which high-frequency components are attenuated by a low-pass filter and low-frequency components are extracted from the image shown in FIG. 8A; 9 is a flow chart showing an autofocus control method of the second embodiment; FIG. 12 is a block diagram showing a specific configuration example of a control unit in an imaging apparatus according to a third embodiment; FIG. 10 is a flow chart showing an autofocus control method of the third embodiment; FIG. 12 is a block diagram showing a specific configuration example of a control unit in an imaging apparatus according to a fourth embodiment; FIG. 14 is a flow chart showing an autofocus control method of the fourth embodiment; FIG. 12 is a block diagram showing a specific configuration example of a control unit in an imaging apparatus according to a fifth embodiment; FIG. FIG. 4 is a diagram showing an example of a region set within a frame; FIG. 10 is a diagram showing a state in which the user touches the display with a touch panel with a finger in order to change the area to be focused. FIG. 4 is a diagram showing an example of a histogram of parallaxes in each region within a frame; FIG. 11 is a flow chart showing an autofocus control method of the fifth embodiment; FIG.

An imaging device and an autofocus control method of each embodiment will be described below with reference to the accompanying drawings.

The configuration and schematic operation of an imaging apparatus 100 common to each embodiment will be described with reference to FIGS. 1 and 2. FIG. As shown in FIG. 1, an imaging apparatus 100 includes a first imaging system having a lens group 11 (first lens group) and an imaging element 12 (first imaging element), a lens group 21 (second lens group) and a second imaging system having an imaging element 22 (second imaging element). As shown in FIG. 2, the lens group 11 and the lens group 21 are arranged horizontally on the front surface 101F of the housing 101 with a predetermined distance therebetween. The lens group 11 and the lens group 21 may be arranged vertically on the front surface 101F with a predetermined distance therebetween.

The first imaging system is a main imaging system for generating image data for photographing and recording a subject. The second imaging system is a secondary imaging system for focusing on a subject by phase difference AF. Lens groups 11 and 21 include focus lenses. Assuming that the focus lens included in the lens group 11 is the first focus lens, the focus lens included in the lens group 21 is the second focus lens. The lens group 11 may include a zoom lens in addition to the focus lens.

The imaging device 12 generates an analog captured image signal of an imaged subject based on the light incident through the lens group 11 . The A/D converter 13 A/D-converts the captured image signal output from the image sensor 12 and supplies digital image data D<b>13 (first image data) to the control unit 30 . The imaging element 12 and the A/D converter 13 may be integrated. The image data D13 is moving image data.

The imaging element 22 generates an analog captured image signal of an imaged subject based on the light incident through the lens group 21 . The A/D converter 23 A/D-converts the captured image signal output from the image sensor 22 and supplies digital image data D23 (second image data) to the control unit 30 . The imaging element 22 and the A/D converter 23 may be integrated. The image data D23 is moving image data.

The imaging elements 12 and 22 are CCD (Charge Coupled Device) image sensors or CMOS (Complementary Metal Oxide Semiconductor) image sensors. Although not shown in FIG. 1, if the imaging elements 12 and 22 are Bayer array image sensors, various video signal processing such as demosaicing and conversion to RGB data are performed. Image data D13 and D23 may be RGB data. The imaging elements 12 and 22 may be composed of three-plate image sensors.

The control unit 30 controls the driving units 14 and 24 to automatically focus on the subject based on the image data D13 and D23. Drive section 14 is a first drive section and drive section 24 is a second drive section. The driving units 14 and 24 can be composed of motors for moving the focus lenses of the lens groups 11 and 21, respectively. The control unit 30 can be composed of a central processing unit (CPU) of a microcomputer.

A ROM (Read Only Memory) 41 and a RAM (Random Access Memory) 42 are connected to the control unit 30 . ROM41 is an example of a non-temporary storage medium. Data necessary for autofocus may be stored in the ROM 41 . The ROM 41 may store an autofocus control program. The RAM 42 is used as a working memory when the controller 30 executes various processes.

A display 50 with a touch panel (hereinafter abbreviated as the display 50), an encoder 61, and an operation section 70 are further connected to the control section 30. As shown in FIG. The display 50 displays an image based on the image data D13. The imaging device 100 may have a display without a touch panel instead of the display 50 . The encoder 61 compression-encodes the image data D13, and the storage unit 62 stores the compression-encoded image data. The operation unit 70 is an operation button provided on the housing 101 . Here, the touch panel of the display 50 is included in the operation unit 70 .

A plurality of embodiments for realizing autofocus with good responsiveness and good focusing accuracy in the imaging apparatus 100 configured as described above will be described in order. Each embodiment described later achieves autofocus with good responsiveness and good focusing accuracy by using both phase difference AF and hill-climbing AF. Furthermore, each embodiment improves focusing accuracy when phase difference AF is used by a configuration unique to each embodiment.

<First embodiment>
Each frame of image data D13 and D23 may exhibit barrel distortion as shown in FIG. 3A or pincushion distortion as shown in FIG. 3B due to lens distortion aberrations in lens groups 11 and 21 . The degree of barrel distortion or pincushion distortion may differ between each frame of the image data D13 and each frame of the image data D23. There is also a difference between distortion and pincushion distortion.

FIG. 4A shows how each pixel in the frame is displaced by barrel distortion, and FIG. 4B shows how each pixel in the frame is displaced by pincushion distortion. In FIGS. 4A and 4B, each bullet indicates a pixel at its original pixel position, unaffected by barrel or pincushion distortion. Each vector indicates that the pixel located at the starting point of the vector is displaced to the position of the ending point of the vector under the influence of barrel distortion or pincushion distortion. In FIGS. 4A and 4B, the black circles are some of the pixels unaffected by barrel or pincushion distortion, and the vector is one of the pixels affected by barrel or pincushion distortion. are the pixels of the part.

The position, direction and amount of displacement of pixels affected by barrel distortion or pincushion distortion are not the same between each frame of the image data D13 and each frame of the image data D23. Therefore, if the horizontal parallax between the image data D13 and the image data D23 is calculated using the image data D13 and D23 as they are, the original correct parallax will be affected by lens distortion due to barrel distortion or pincushion distortion. cannot be asked for.

In the first embodiment, after correcting a vector caused by lens distortion aberration in the lens groups 11 and 21 (hereinafter referred to as a distortion aberration vector), the horizontal parallax between the image data D13 and the image data D23 is obtained, Focusing accuracy is improved when phase difference AF is used.

FIG. 5 shows a specific configuration example of the controller 30 in the first embodiment. The ROM 41 preliminarily stores distortion aberration vector information for each pixel indicating the distortion aberration vectors of the lens groups 11 and 21, which are measured by photographing an object at infinity with the image pickup apparatus 100. FIG. When the imaging apparatus 100 photographs an object at infinity, the parallax is almost 0, so the distortion vectors of the lens groups 11 and 21 can be measured.

The right direction of the frame is positive in the horizontal direction, and the downward direction is positive in the vertical direction. The information may be expressed as +10 in the horizontal direction and -5 in the vertical direction.

As shown in FIG. 5, the control unit 30 includes a phase difference autofocus processing unit (hereinafter referred to as phase difference AF processing unit) 31A, a hill climbing autofocus processing unit (hereinafter referred to as hill climbing AF processing unit) 32, drive control units 33 and 34 Prepare. Assuming that the drive control section 33 is the first drive control section, the drive control section 34 is the second drive control section. The phase difference AF processing unit 31A has distortion aberration vector correction units 311 and 312, a parallax calculation unit 313, a focus lens position conversion unit 314, and a conversion table holding unit 315. Assuming that the distortion vector correction section 311 is a first distortion vector correction section, the distortion vector correction section 312 is a second distortion vector correction section.

Image data D<b>13 and distortion vector information of the lens group 11 are input to the distortion vector correction unit 311 . The distortion aberration vector correction unit 311 corrects each pixel in each frame of the image data D13 based on the distortion aberration vector information and supplies it to the parallax calculation unit 313 . If the distortion vector information of a certain pixel is +10 in the horizontal direction and -5 in the vertical direction, the distortion vector correction unit 311 displaces the pixel by -10 pixels in the horizontal direction and +5 pixels in the vertical direction. .

Image data D<b>23 and distortion vector information of the lens group 21 are input to the distortion vector correction unit 312 . Similar to the distortion vector correction unit 311 , the distortion vector correction unit 312 corrects each pixel in each frame of the image data D23 based on the distortion vector information and supplies the corrected pixels to the parallax calculation unit 313 .

The distortion vector information to be stored in the ROM 41 may be the values themselves for displacing the pixels by the distortion vector correction units 311 and 312 . That is, when the distortion vector is a vector of +10 pixels in the horizontal direction and -5 pixels in the vertical direction, the distortion vector information to be stored in the ROM 41 is obtained by inverting the sign of the distortion vector to obtain -10 pixels in the horizontal direction and +5 pixels in the vertical direction. may be In this case, for example, if the distortion vector information in a certain pixel is −10 in the horizontal direction and +5 in the vertical direction, the distortion vector correction unit 311 displaces the pixel by −10 pixels in the horizontal direction and +5 pixels in the vertical direction. It should be displaced.

The parallax calculator 313 calculates the parallax between the image data D13 and D23 whose distortion vectors are corrected by the distortion vector correctors 311 and 312, respectively. The parallax calculation unit 313 may calculate parallax in a partial region within the frame. Typically, the parallax calculator 313 calculates the parallax in a predetermined area located in the center of the frame in order to focus on the center of the frame. The parallax calculation unit 313 may calculate the parallax in a central area of a total of 9 areas obtained by equally dividing the frame into 3 parts in the horizontal direction and 3 parts in the vertical direction.

The parallax calculation unit 313 searches for a pixel with the smallest difference (the pixel with the highest correlation) in the image data D23 at the same vertical position as the pixel of the image data D13 within the region for calculating the parallax. A horizontal parallax with the image data D23 is calculated. The parallax calculator 313 may use, for example, the average value (or the maximum value) of the parallaxes obtained for each pixel in the region where the parallax is calculated as the final parallax between the image data D13 and the image data D23.

As the focus position moves away from the imaging device 100 and approaches infinity, the parallax between the image data D13 and the image data D23 becomes smaller. The parallax calculated by the parallax calculation unit 313 is generally a value corresponding to the focusing distance.

The conversion table holding unit 315 holds a conversion table that associates the parallax calculated by the parallax calculation unit 313 with position information indicating the position of the focus lens that can focus at a distance corresponding to the parallax. ing. A conversion table may be stored in the ROM 41 .

The focus lens position conversion unit 314 refers to the conversion table, converts the parallax supplied from the parallax calculation unit 313 into focus lens position information, and supplies the position information to the drive control units 33 and 34 . The drive control units 33 and 34 control the drive units 14 and 24 to move the focus lenses of the lens groups 11 and 21 to the positions obtained by the focus lens position conversion unit 314, respectively.

After controlling the drive units 14 and 24 to move the focus lenses of the lens groups 11 and 21 , the drive control units 33 and 34 control the drive units 14 and 24 according to instructions from the hill-climbing AF processing unit 32 . The hill-climbing AF processing section 32 obtains the contrast of the image data D13. The drive control units 33 and 34 control the drive units 14 and 24 to move the focus lenses of the lens groups 11 and 21 until the hill-climbing AF processing unit 32 obtains the maximum contrast.

The focus lens position conversion unit 314 refers to the conversion table, converts the parallax supplied from the parallax calculation unit 313 into focus lens position information, and supplies the information to the drive control unit 33 . The position information of the focus lens may not be supplied. The drive control unit 33 controls the drive unit 14 to move the focus lens of the lens group 11 to the position obtained by the focus lens position conversion unit 314 . On the other hand, the drive control section 34 does not have to control the drive section 24 to move the focus lens of the lens group 21 to the position obtained by the focus lens position conversion section 314 . The same applies to second to fifth embodiments, which will be described later.

The autofocus control method executed in the first embodiment will be described using the flowchart shown in FIG. In FIG. 6, when the imaging apparatus 100 starts shooting, in step S101, the control unit 30 adjusts the pixels in each frame of the image data D13 and D23 based on the distortion aberration vector information of the lens groups 11 and 21, respectively. to correct.

The control unit 30 calculates the parallax of the image data D13 and D23 in step S102. The control unit 30 refers to the conversion table to convert the parallax into the position of the focus lens in step S103, and controls the driving units 14 and 24 to move the focus lens in step S104.

Thereafter, in step S105, the control unit 30 executes hill-climbing AF. In step S106, the control unit 30 determines whether or not an operation to end shooting has been performed. If the shooting end operation is not performed (NO), the control unit 30 repeats the processing of steps S105 and S106. If the shooting end operation is performed (YES), the control unit 30 ends the processing.

According to the first embodiment, the phase-difference AF is executed first, the focus lens is moved to an approximate in-focus position, and then the hill-climbing AF is executed. can be done. In addition, according to the first embodiment, the parallax is obtained after correcting the distortion vectors of the lens groups 11 and 21, so that the focusing accuracy when using the phase difference AF can be improved. .

<Second embodiment>
In the second embodiment, the imaging apparatus 100 is in the initial state (state before focusing), and the phase difference AF is used by configuring so as not to calculate the inaccurate parallax when the focus is coincident. Improves focusing accuracy.

FIG. 7 shows a specific configuration example of the controller 30 in the second embodiment. In FIG. 7, the same parts as those in FIG. 5 are denoted by the same reference numerals, and the description thereof may be omitted. As shown in FIG. 7, the controller 30 includes a phase difference AF processor 31B, a hill-climbing AF processor 32, and drive controllers 33 and . The phase difference AF processing unit 31B has low-pass filters (hereinafter referred to as LPFs) 316 and 317, a parallax calculation unit 313, a focus lens position conversion unit 314, and a conversion table holding unit 315. If LPF 316 is the first low-pass filter, LPF 317 is the second low-pass filter.

The LPFs 316 and 317 respectively attenuate the high-frequency components of the image data D13 and D23 to extract low-frequency components, and supply the extracted low-frequency components of the image data D13 and D23 to the parallax calculator 313 . The parallax calculator 313 calculates parallax based on the low-frequency components of the image data D13 and D23 supplied from the LPFs 316 and 317 .

InPic(x,y) is the value of pixel (x, y) before LPF processing by LPF 316 and 317, OutPic(x, y) is the value of pixel (x, y) after LPF processing, and M is the number of additions Then, the LPFs 316 and 317 execute the processing shown in the following equation (1). Weighting processing may be performed according to the positions of the pixels to be added during the LPF processing.
OutPic(x,y)=ΣInPic(x,y)/M (1)

As shown in FIG. 8A, it is assumed that an image of a certain frame of the image data D13 includes a repetitive pattern 81 such as a checkered pattern in which high luminance and low luminance are repeated in the horizontal direction. If the frame image includes the repeating pattern 81, the parallax calculator 313 is likely to calculate an erroneous parallax.

As shown in FIG. 8B, the LPFs 316 and 317 apply the LPF processing to the image data D13 and D23, so that the repeating pattern 81 can be made into the constant value pattern 82. FIG. This makes it easier for the parallax calculator 313 to calculate a correct parallax. Note that FIG. 8B conceptually shows a state in which the image shown in FIG. 8A is subjected to LPF processing by showing the image with a dashed line.

When the parallax calculator 313 searches for a pixel with the smallest difference based on the image data D13 and D23 output from the LPFs 316 and 317, the minimum value may not exist. In such a case, the parallax calculator 313 may output 0 for that pixel. As a result, it is possible to avoid the fact that the minimum value is obtained by chance and the parallax is calculated as if it were correct.

The autofocus control method executed in the second embodiment will be described using the flowchart shown in FIG. In FIG. 9, when the imaging device 100 starts photographing, the control unit 30 performs LPF processing on the image data D13 and D23 in step S201. In step S202, the control unit 30 calculates parallax based on the LPF-processed image data D13 and D23.

The control unit 30 refers to the conversion table to convert the parallax into the position of the focus lens in step S203, and controls the driving units 14 and 24 to move the focus lens in step S204.

Thereafter, in step S205, the control unit 30 executes hill-climbing AF. In step S206, the control unit 30 determines whether or not an operation to end shooting has been performed. If the shooting end operation is not performed (NO), the control unit 30 repeats the processing of steps S205 and S206. If the shooting end operation is performed (YES), the control unit 30 ends the processing.

According to the second embodiment, the phase-difference AF is performed first, the focus lens is moved to an approximate in-focus position, and then the hill-climbing AF is performed. can be done. In addition, according to the second embodiment, the image data D13 and D23 are subjected to the LPF processing and then the parallax is obtained, so that the focusing accuracy when using the phase difference AF can be improved.

<Third Embodiment>
3rd Embodiment is a more preferable structure which improved 2nd Embodiment. FIG. 10 shows a specific configuration example of the controller 30 according to the third embodiment. In FIG. 10, the same parts as those in FIG. 7 are denoted by the same reference numerals, and the description thereof may be omitted. As shown in FIG. 10, the phase difference AF processing unit 31C includes a parallax calculation unit 318 (second parallax calculation unit), a difference calculation unit 319, and a parallax selection unit 3110 as components that the phase difference AF processing unit 31B does not have. have

The parallax calculator 313 (first parallax calculator) calculates parallax based on the low frequency components of the image data D13 and D23 supplied from the LPFs 316 and 317 . The parallax calculator 318 calculates parallax based on the input image data D13 and D23 that have not passed through the LPFs 316 and 317 . The difference calculation unit 319 calculates the difference between the parallax supplied from the parallax calculation unit 313 and the parallax supplied from the parallax calculation unit 318 , and supplies the difference to the parallax selection unit 3110 . The parallax calculated by the parallax calculator 313 and the parallax calculated by the parallax calculator 318 are input to the parallax selector 3110 .

The parallax selection unit 3110 determines whether or not the parallax difference supplied from the difference calculation unit 319 is equal to or greater than a predetermined threshold. If the parallax difference is greater than or equal to the threshold, the accuracy of the parallax calculated by the parallax calculation unit 318 is considered to be poor, and if the difference in parallax is less than the threshold, the accuracy of the parallax calculated by the parallax calculation unit 318 is considered to be good. be done. Therefore, the parallax selection unit 3110 selects the parallax calculated by the parallax calculation unit 313 if the parallax difference is equal to or greater than the threshold, and selects the parallax calculated by the parallax calculation unit 318 if the parallax difference is less than the threshold. and supplied to the focus lens position conversion unit 314 .

The parallax difference calculated by the difference calculator 319 may be supplied to the drive controllers 33 and 34, as indicated by the dashed arrow lines in FIG. The drive control units 33 and 34 may control the drive units 14 and 24 as follows when the input parallax difference is greater than or equal to the threshold. The drive control units 33 and 34 move the focus lenses of the lens groups 11 and 21 immediately according to the instruction from the hill-climbing AF processing unit 32 without moving the focus lenses of the lens groups 11 and 21 to the positions instructed by the phase difference AF processing unit 31C. It controls the drives 14 and 24 as follows.

The autofocus control method executed in the third embodiment will be described using the flowchart shown in FIG. In FIG. 11, when the imaging device 100 starts photographing, the control unit 30 performs LPF processing on the image data D13 and D23 in step S301. In step S302, the control unit 30 calculates parallax based on the image data D13 and D23 before LPF processing and after LPF processing.

In step S303, the control unit 30 calculates the difference in parallax between the two. In step S304, the control unit 30 determines whether or not the difference is greater than or equal to the threshold. If the difference is equal to or greater than the threshold (YES), in step S305, the control unit 30 selects the parallax calculated based on the image data D13 and D23 after LPF processing, and causes the process to proceed to step S307. If the difference is not equal to or greater than the threshold (NO), in step S306, the control unit 30 selects the parallax calculated based on the image data D13 and D23 before LPF processing, and moves the process to step S307.

The control unit 30 refers to the conversion table to convert the parallax into the position of the focus lens in step S307, and controls the driving units 14 and 24 to move the focus lens in step S308.

Thereafter, in step S309, the control unit 30 executes hill-climbing AF. In step S310, the control unit 30 determines whether or not an operation to end shooting has been performed. If the shooting end operation is not performed (NO), the control unit 30 repeats the processing of steps S309 and S310. If the shooting end operation is performed (YES), the control unit 30 ends the processing.

As indicated by the dashed arrow line in FIG. 11, when the difference is equal to or greater than the threshold in step S304, the process may immediately proceed to step S309.

According to the third embodiment, when the image is an image for which parallax is calculated with poor accuracy, the parallax can be obtained by the image data D13 and D23 subjected to the LPF processing, and the focusing accuracy when using phase difference AF can be improved. It is preferable not to apply the LPF process to an image for which parallax can be calculated with high accuracy. According to the third embodiment, the parallax can be obtained from the image data D13 and D23 that are not subjected to the LPF processing when the image is an image for which the parallax can be calculated with high accuracy.

<Fourth Embodiment>
4th Embodiment is a more preferable structure which improved 2nd Embodiment. FIG. 12 shows a specific configuration example of the controller 30 according to the fourth embodiment. In FIG. 12, the same parts as those in FIG. 7 are denoted by the same reference numerals, and the description thereof may be omitted. As shown in FIG. 12, the phase difference AF processing section 31D has a repeated pattern detection section 3111 as a configuration that the phase difference AF processing section 31B does not have.

The repetitive pattern detection unit 3111 detects a repetitive pattern such as the repetitive pattern 81 shown in FIG. 8A that may exist in the image by detecting edges of the image in each frame of the image data D13. When repeating pattern detection section 3111 detects a repeating pattern, it notifies LPFs 316 and 317 of the fact. For example, the repeat pattern detection unit 3111 may supply the value "0" to the LPFs 316 and 317 when no repeat pattern is detected, and the value "1" to the LPFs 316 and 317 when the repeat pattern is detected.

The LPFs 316 and 317 supply the image data D13 and D23 to the parallax calculator 313 as they are without LPF processing the image data D13 and D23 unless the repeat pattern detection unit 3111 detects a repeat pattern. When the repeat pattern detection unit 3111 detects a repeat pattern, the LPFs 316 and 317 perform LPF processing on the image data D13 and D23 and supply the processed image data to the parallax calculation unit 313 .

The autofocus control method executed in the fourth embodiment will be described using the flowchart shown in FIG. In FIG. 13, when the imaging device 100 starts photographing, the control unit 30 determines in step S401 whether or not a repeating pattern exists in the frame based on the image data D13. If there is a repeating pattern in the frame (YES), the control unit 30 performs LPF processing on the image data D13 and D23 in step S402, and proceeds to step S403. If no repeating pattern exists in the frame (NO), the control unit 30 causes the process to proceed to step S403.

In step S403, the control unit 30 calculates parallax based on the image data D13 and D23 that have undergone LPF processing or have not undergone LPF processing. The control unit 30 refers to the conversion table to convert the parallax into the position of the focus lens in step S404, and controls the driving units 14 and 24 to move the focus lens in step S405.

Thereafter, in step S406, the control unit 30 executes hill-climbing AF. In step S407, the control unit 30 determines whether or not an operation to end shooting has been performed. If the shooting end operation is not performed (NO), the control unit 30 repeats the processing of steps S406 and S407. If the shooting end operation is performed (YES), the control unit 30 ends the processing.

According to the fourth embodiment, when the images of the image data D13 and D23 are images including repetitive patterns that are likely to result in inaccurate parallax calculation, the LPF-processed image data D13 and D23 Parallax can be obtained, and focusing accuracy can be improved when phase difference AF is used. According to the fourth embodiment, when the images of the image data D13 and D23 do not include repetitive patterns, the parallax can be obtained from the image data D13 and D23 without LPF processing.

<Fifth Embodiment>
The fifth embodiment is configured to selectively use phase difference AF and hill-climbing AF according to the reliability of the calculated parallax in the focused area.

FIG. 14 shows a specific configuration example of the controller 30 according to the fifth embodiment. In FIG. 14, the same parts as in FIG. 5 are denoted by the same reference numerals, and the description thereof may be omitted. As shown in FIG. 14, the control section 30 includes a phase difference AF processing section 31E, a hill-climbing AF processing section 32, and drive control sections 33 and . The phase difference AF processing unit 31E has a parallax calculation unit 313, a focus lens position conversion unit 314, a conversion table storage unit 315, a histogram generation unit 3112, and a reliability determination unit 3113.

In the fifth embodiment, the parallax calculator 313 receives an area designation signal for designating one of a plurality of areas in each frame. As shown in FIG. 15, each frame D13F and D23F of the image data D13 and D23 is equally divided into three areas in the horizontal direction and equally divided into three areas in the vertical direction into a total of nine areas. The nine regions are called "upper left", "upper middle", "upper right", "left middle", "middle middle", "right middle", "lower left", "lower middle", and "lower right" regions. It is assumed that In the default state, the parallax calculation unit 313 receives an area designation signal that specifies a "medium-medium" area.

The parallax calculator 313 calculates the parallax in the area designated by the area designation signal among the entire areas of each frame of the image data D13 and D23. The parallax calculator 313 supplies the focus lens position converter 314 with, for example, the average value (or the maximum value) of the parallaxes calculated for the plurality of pixels in the specified area as the final parallax. In the default state, the parallax calculation unit 313 calculates the parallax in the “medium” area and supplies it to the focus lens position conversion unit 314 .

Assume that the user touches the display 50 with a finger to change the focused area, as shown in FIG. Control unit 30 generates an area designation signal that identifies the area of the position touched by the finger. In the example shown in FIG. 16 , the user touches the “middle right” area, and the parallax calculation unit 313 receives an area designation signal specifying the “middle right” area. In this case, the parallax calculator 313 calculates the parallax in the “middle right” region and supplies it to the focus lens position converter 314 .

The histogram generation unit 3112 generates a histogram of parallax within the area calculated by the parallax calculation unit 313 . The histogram generation unit 3112 may generate a histogram of parallax calculated with all pixels in the region, or generate a histogram of parallax calculated with some pixels thinned out in the horizontal and vertical directions. good too.

FIG. 17 shows an example of a histogram when the parallax calculation unit 313 calculates the parallax for the entire frame and the histogram generation unit 3112 generates a histogram for each region in a given frame. For simplification, the histogram shown in FIG. 17 shows the frequencies (the number of pixels) at which these parallaxes are obtained at parallaxes 0, 4, 8, . The histogram generation unit 3112 supplies the histogram of the area actually generated by the histogram generation unit 3112 to the reliability determination unit 3113 among the histograms of parallax of each area as shown in FIG.

Here, the parallax calculator 313 calculates the parallax in the area designated by the area designation signal, and the histogram generator 3112 generates the histogram of the parallax in the designated area. The parallax calculation unit 313 may calculate parallax in all areas of each frame, and the histogram generation unit 3112 may generate a histogram of parallax in all areas.

In this case, the parallax calculator 313 may calculate the final parallax in the area designated by the area designation signal and supply it to the focus lens position converter 314 . An area designation signal is also supplied to the histogram generation section 3112 , and the histogram generation section 3112 may supply the histogram of the area designated by the area designation signal to the reliability determination section 3113 .

With this configuration, the amount of calculation in the parallax calculation unit 313 and the histogram generation unit 3112 increases. However, when the user touches the display 50 to change the focused area, the reliability determination unit 3113 can immediately determine the reliability of the calculated parallax.

The reliability determination unit 3113 determines the reliability of the parallax calculated by statistically analyzing the histogram. The reliability determination unit 3113 obtains the standard deviation or variance of the histogram in order to determine the reliability of parallax. The reliability determination unit 3113 may obtain the maximum value or the average value in addition to the standard deviation or variance.

If the standard deviation or variance of the histogram is less than a predetermined size, it means that the disparity calculated within the region has a small variation and is highly reliable. If the standard deviation or variance is greater than or equal to a predetermined size, it means that the parallax calculated within the area varies and is unreliable. At least one of the maximum value and the average value may further be used for reliability determination.

When the standard deviation or the variance is less than a predetermined size, the reliability determination unit 3113 sets the value “0”, for example, indicating that the calculated parallax is highly reliable, as the reliability determination value to the drive control unit 33 and the 34. When the standard deviation or the variance is equal to or greater than a predetermined size, the reliability determination unit 3113 sets the value “1”, for example, indicating that the reliability of the calculated parallax is low, to the drive control unit 33 and the 34.

In the example shown in FIG. 17, for example, parallax 16 is the largest value in the “upper left” region, and parallaxes 12 to 20 are relatively concentrated. The distance to the subject is moderate, and the parallax is highly reliable. In the “upper right” region, parallax 0 is the maximum number of parallaxes, and parallaxes are relatively concentrated in 0-12. The distance to the subject is long, and the parallax is highly reliable. In the "middle and lower" region, parallax 28 is the largest number, and parallaxes 20 to 32 are relatively concentrated. The distance to the subject is close, and the parallax is highly reliable.

On the other hand, in the “middle right” area, the maximum number of parallaxes is small, and parallaxes occur in a wide range from 0 to 32 parallaxes. The standard deviation or variance is greater than or equal to a predetermined size, and the reliability of parallax is low. It is a matter of design to set a threshold value of standard deviation or variance for determining whether the reliability of parallax is high or low.

When the reliability determination value “0” is supplied from the reliability determination unit 3113 , the drive control units 33 and 34 move the focus lenses of the lens groups 11 and 21 to the positions determined by the focus lens position conversion unit 314 . Control the drives 14 and 24 to move. After controlling the drive units 14 and 24 to move the focus lenses of the lens groups 11 and 21 , the drive control units 33 and 34 control the drive units 14 and 24 according to instructions from the hill-climbing AF processing unit 32 .

When the reliability determination value "1" is supplied from the reliability determination unit 3113, the drive control units 33 and 34 move the focus lenses of the lens groups 11 and 21 to the positions obtained by the focus lens position conversion unit 314. The drive units 14 and 24 are controlled to move the focus lens according to the instruction from the hill-climbing AF processing unit 32 without delay.

In FIG. 17, in the default state, it is determined that the reliability of the parallax is high in the “medium” region, and the drive control units 33 and 34 are supplied with the reliability determination value “0”. Therefore, in the "medium-medium" area, the hill-climbing AF is performed following the phase-difference AF by the phase-difference AF processing unit 31E. When the user touches the "middle right" area to focus on the "middle right" area, it is determined that the reliability of the parallax in the "middle right" area is low, and the drive control unit 33 and 34 is supplied with the reliability determination value "1". Therefore, hill-climbing AF is immediately executed in the "middle right" area.

The autofocus control method executed in the fifth embodiment will be described using the flowchart shown in FIG. In FIG. 18, when the imaging device 100 starts photographing, the control unit 30 calculates the parallax for each pixel in the area designated by the area designation signal in step S501. In step S502, the control unit 30 generates a histogram of parallax within the area.

In step S503, the control unit 30 determines whether or not the parallax is highly reliable. If the parallax is highly reliable (YES), in step S504, the control unit 30 refers to the conversion table to convert the parallax into the position of the focus lens, and in step S505, instructs the drive unit to move the focus lens. 14 and 24 are controlled to shift the process to step S506. If the reliability of the parallax is not high (NO), the control unit 30 causes the process to proceed to step S506.

The control unit 30 executes hill-climbing AF in step S506. In step S507, the control unit 30 determines whether or not an operation to end shooting has been performed. If the operation to end shooting is not performed (NO), the control unit 30 determines in step S508 whether or not the focused area has been changed. If the focused area is not changed (NO), the control unit 30 repeats the processing of steps S506 to S508.

If the area to be focused is changed in step S508 (YES), the control unit 30 returns the process to step S501 and repeats the process of moving to step S501. If an operation to end shooting is performed in step S507 (YES), the control unit 30 ends the processing.

In the above description, the reliability of the parallax is set to two stages of "high" and "low", but the reliability may be determined in multiple stages. The drive control units 33 and 34 may change the speed of moving the focus lens or change the target position of the focus lens according to the value of the reliability level. The drive control units 33 and 34 may change the timing of transition from phase difference AF to hill-climbing AF according to the value of the reliability level.

According to the fifth embodiment, phase difference AF and hill-climbing AF are selectively used, or phase-difference AF is used for hill-climbing depending on whether or not the reliability of parallax calculated in an area to be focused is reliable or the degree of reliability. It is possible to shift to AF. As a result, autofocus with good responsiveness and good focusing accuracy can be realized.

In FIG. 14 , the reliability determination unit 3113 determines the reliability of parallax based on the histogram of parallax generated by the histogram generation unit 3112 . It is not always necessary to create a histogram that graphs the parallax and the frequency at which the parallax is obtained, and the phase difference AF processing unit 31E only generates tallied data that aggregates the parallax and the frequency at which the parallax is obtained. It's okay. Furthermore, the reliability determination unit 3113 may determine the reliability of parallax based on the dispersion of parallax in the area designated by the area designation signal. In this case, the phase-difference AF processing unit 31E may include a variance calculation unit instead of the histogram generation unit 3112. FIG. The method of determining the reliability of parallax by the reliability determining unit 3113 is not limited. Aggregated data or variances as described above, however, are suitable for determining parallax reliability.

By the way, the first embodiment can be combined with the second to fourth embodiments. A combination of the first embodiment and the fifth embodiment is possible. The second to fourth embodiments can be combined with the fifth embodiment. Combinations of the first embodiment, the second to fourth embodiments, and the fifth embodiment are possible.

The present invention is not limited to the first to fifth embodiments described above, and can be variously modified without departing from the gist of the present invention. As described above, each embodiment can be arbitrarily combined, and the present invention also includes a configuration in which each embodiment is arbitrarily combined.

The first to fifth embodiments may be configured with an autofocus control program. In this case, the autofocus control program causes the computer to execute the processes shown in FIGS. The proper use of hardware and software is optional.

Reference Signs List 11, 21 lens group 12, 22 image sensor 13, 23 A/D converter 14, 24 drive section 30 control section 31A, 31B, 31C, 31D, 31E phase difference autofocus processing section 32 hill climbing autofocus processing section 33, 34 Drive control unit 41 ROM
42 RAMs
50 display with touch panel 61 encoder 62 storage unit 70 operation unit 311, 312 distortion aberration vector correction unit 313, 318 parallax calculation unit 314 focus lens position conversion unit 315 conversion table holding unit 316, 317 low-pass filter 319 difference calculation unit 3110 parallax selection unit 3111 repetitive pattern detection unit 3112 histogram generation unit 3113 reliability determination unit

Claims (4)

a first imaging system having a first lens group including a first focus lens and a first imaging element;
A second image pickup having a second lens group and a second image sensor, wherein the second lens group is arranged horizontally or vertically with a predetermined distance from the first lens group system and
a phase difference autofocus processing unit for focusing on a distance based on the parallax of the first and second image data respectively generated by the first and second imaging systems;
a first drive control unit that controls a first drive unit that drives the first focus lens based on processing by the phase difference autofocus processing unit;
with
The phase difference autofocus processing unit is
a first distortion vector correction unit that corrects the first image data based on distortion vector information for each pixel indicating a distortion vector of the first lens group;
a second distortion vector correction unit that corrects the second image data based on distortion vector information for each pixel indicating a distortion vector of the second lens group;
a parallax calculation unit that calculates a parallax between the first and second image data whose distortion vectors are corrected by the first and second distortion vector correction units, respectively;
a focus lens position conversion unit that converts the parallax calculated by the parallax calculation unit into position information indicating the position of the first focus lens for focusing on the distance to the subject indicated by the parallax;
has
The imaging device, wherein the first drive control section controls the first drive section to move the position of the first focus lens to a position indicated by the position information. Further comprising a hill-climbing autofocus processing unit for focusing on a subject using a hill-climbing method based on the contrast of the first image data,
The first drive control unit, after controlling the first drive unit to move the position of the first focus lens to the position indicated by the position information, based on processing by the hill-climbing autofocus processing unit , controlling the first drive unit to move the position of the first focus lens. the second lens group includes a second focus lens;
further comprising a second drive control unit that controls a second drive unit that drives the second focus lens;
3. The imaging apparatus according to claim 1, wherein the second drive control section controls the second drive section to move the position of the second focus lens to the position indicated by the position information. imaging a subject with a first imaging system having a first lens group including a focus lens and a first imaging element to generate first image data;
A second image pickup having a second lens group and a second image sensor, wherein the second lens group is arranged horizontally or vertically with a predetermined distance from the first lens group imaging the subject with a system to generate second image data;
correcting the first image data based on distortion vector information for each pixel indicating a distortion vector of the first lens group;
correcting the second image data based on distortion vector information for each pixel indicating a distortion vector of the second lens group;
calculating a parallax between the first and second image data in which the distortion aberration vector is corrected;
converting into position information indicating the position of the focus lens for focusing on the distance to the subject indicated by the parallax;
An autofocus control method of moving a position of the focus lens to a position indicated by the position information.

Images