JP2014077830A - Imaging apparatus and focus control method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus capable of detecting a phase difference with high reliability and to provide a focus control method or the like.SOLUTION: The imaging apparatus includes an imaging part 10, a contrast value calculation part 20, a detection position determination part 30, a phase difference detection part 40, and an AF control part 50. The imaging part 10 obtains a first image and a second image having parallax to the first image as parallax images. The contrast value calculation part 20 calculates a contrast value from at least one of the parallax images. The detection position determination part 30 determines a detection position where the phase difference is detected, based on the contrast value. The phase difference detection part 40 detects the phase difference between the first image and the second image in the detection position. The AF control part 50 performs the focus control of the imaging part 10 on the basis of the phase difference.

Description

  The present invention relates to an imaging apparatus, a focus control method, and the like.

  As typical auto-focus (AF) methods, contrast AF (for example, focus adjustment by a hill-climbing method) and phase difference AF are known. Since the phase difference AF can be expected to be smoother and faster than the contrast AF, the phase difference AF is optimal when it is necessary to always keep focus. For example, stereoscopic (3D) video and stereo video have video information related to the depth of an object using parallax, and thus are suitable for remote operation such as moving an article while watching the video, and have a sense of distance. It can be operated while holding it. When performing such an operation, it is important that the AF is always in focus.

  As a method of phase difference AF, for example, in Patent Document 1, phase difference detection pixels are provided for every six rows in the image pickup pixels of the image pickup element, and the phase difference AF is based on a signal obtained by the phase difference detection pixels. A technique for detecting a phase difference is disclosed.

JP 2012-124766 A

  In the phase difference AF, there is a possibility that the subject cannot be focused if the phase difference cannot be detected properly, and thus there is a problem that it is necessary to perform highly reliable phase difference detection.

  For example, although phase difference detection is performed by correlation calculation, there may be a case where the phase difference cannot be calculated correctly by a flat portion of an image or the like. As a method for solving this, for example, a method of providing a plurality of phase difference detection points as in Patent Document 1 and selecting a phase difference with high reliability from the phase differences detected at each point can be considered. However, in this method, since the phase difference detection points are provided discretely, a portion suitable for phase difference detection such as an edge portion does not always form an image on the point, and the phase difference cannot be calculated correctly. there is a possibility.

  According to some aspects of the present invention, it is possible to provide an imaging apparatus, a focus control method, and the like that can detect a phase difference with high reliability.

  According to one aspect of the present invention, an imaging unit that acquires a first image and a second image having a parallax with respect to the first image as a parallax image, and a contrast value from at least one of the parallax images. A contrast value calculation unit to calculate; a detection position determination unit for determining a detection position for detecting a phase difference between the first image and the second image based on the contrast value; and the first image at the detection position. And an image pickup apparatus comprising: a phase difference detection unit that detects the phase difference between the first image and the second image; and a focus control unit that performs focus control of the image pickup unit based on the phase difference.

  According to another aspect of the present invention, a first image and a second image having a parallax with respect to the first image are acquired as a parallax image, and a contrast value is calculated from at least one of the parallax images. And determining a detection position for detecting a phase difference between the first image and the second image based on the contrast value, and determining a phase difference between the first image and the second image at the detection position. The present invention relates to a focus control method that detects and performs focus control of the imaging unit based on the phase difference.

1 is a configuration example of an imaging apparatus according to a first embodiment. 6 is a flowchart of AF control processing performed by the imaging apparatus according to the first embodiment. Explanatory drawing of operation | movement of the contrast map production | generation part in 2nd Embodiment. Explanatory drawing of operation | movement of the contrast map production | generation part in 2nd Embodiment. Explanatory drawing of operation | movement of the detection position determination part in 2nd Embodiment. Explanatory drawing of operation | movement of the detection position determination part in 2nd Embodiment. The detailed structural example of the phase difference detection part in 2nd Embodiment. Operation | movement explanatory drawing of the phase difference detection part in 2nd Embodiment. Operation | movement explanatory drawing of the phase difference detection part in 2nd Embodiment. 10 is a flowchart of AF control processing performed by the imaging apparatus according to the second embodiment. The structural example of the imaging device in 3rd Embodiment. Explanatory drawing of operation | movement of the contrast map production | generation part in 3rd Embodiment. Explanatory drawing of operation | movement of the display control part in 3rd Embodiment. 10 is a flowchart of AF control processing performed by the imaging apparatus according to the third embodiment. The 1st structural example of the imaging device in 4th Embodiment. 10 is a second configuration example of an imaging apparatus according to a fourth embodiment. The detailed structural example of the imaging part in 4th Embodiment.

  Hereinafter, preferred embodiments of the present invention will be described in detail. The present embodiment described below does not unduly limit the contents of the present invention described in the claims, and all the configurations described in the present embodiment are indispensable as means for solving the present invention. Not necessarily.

1. First Embodiment In phase difference AF (AF: Auto-Focus), for example, a phase difference is detected by correlation calculation such as SAD (Sum of Absolute Difference) calculation. In such a correlation calculation, a phase difference can be accurately calculated in a portion including an edge component, but in a gradation portion without an edge component or a portion where unevenness having a dense hole such as a sponge continues, The phase difference may not be calculated. If the phase difference cannot be calculated correctly, the subject cannot be focused. Therefore, the phase difference AF requires a highly reliable phase difference detection.

  As a technique for solving such a problem, the phase difference information calculated based on the shift value in the phase difference calculation and the shape (reliability) of the SAD curve represented by the SAD value in the shift value is used. There is a method to judge whether it is good. In addition, there is a method of providing a number of phase difference detection points in an image, calculating a phase difference for each point, and selecting phase difference information having high reliability from the many phase differences (for example, Patent Document 1). . With this method, it is possible to obtain more accurate phase difference information with one imaging.

  However, with this technique, even if a plurality of phase difference detection points are provided, the arrangement is discrete, and an image portion suitable for phase difference calculation is not always formed on the phase difference detection points. In order to form an image portion suitable for phase difference calculation on the phase difference detection point, a method of increasing the number of points can be considered, but increasing the number of points increases the cost and calculation time of the phase difference calculation circuit. There are challenges.

  Further, in the method of providing a phase difference detection pixel in an imaging pixel of an image sensor as in Patent Document 1, and using the phase difference detection pixel as a phase difference detection point, the phase difference detection pixel becomes a pixel defect. There is a problem. That is, since the image pickup pixels and the phase difference detection pixels are mixed on the image pickup element, it is necessary to replace the phase difference detection pixels with data interpolated from surrounding photographing pixels. Therefore, there is a problem that it is disadvantageous in terms of resolution as compared with a normal image sensor. In addition, a dedicated imaging element in which imaging pixels and phase difference detection pixels are mixed must be used.

  This embodiment that can solve the above problems will be described below. First, the first embodiment will be described.

  FIG. 1 shows a configuration example of an imaging apparatus according to the first embodiment. The imaging apparatus includes an imaging unit 10, a contrast value calculation unit 20, a detection position determination unit 30, a phase difference detection unit 40, an AF control unit 50, and a lens driving unit 60.

  Processing performed by the imaging apparatus will be described with reference to the flowchart of FIG. As shown in FIG. 2, first, the imaging unit 10 acquires a first image and a second image having parallax as parallax images (step S1). In the following description, an example in which there is parallax in the horizontal scanning direction of the imaging element included in the imaging unit 10 is described as a first image as a left image and a second image as a right image.

  Next, the contrast value calculation unit 20 calculates a contrast value from at least one of the left image and the right image (step S2). In the following description, the case where the contrast value is calculated from the right image will be described as an example. For example, the contrast value calculation unit 20 divides the right image into a plurality of sub-blocks, calculates a contrast value in each sub-block (for example, a contrast average value in the sub-block), and sets the contrast value as a contrast map. Store in the illustrated memory. In the present embodiment, the present invention is not limited to this. For example, a contrast value at each pixel position may be calculated. In addition, it is not always necessary to store the contrast value as a contrast map, and a statistical value (for example, the maximum value in the horizontal scanning direction) for determining the detection position of the phase difference is obtained, and once the statistical value is obtained, the contrast value is sequentially obtained. May be deleted from the memory.

  Next, the detection position determination unit 30 determines the detection position of the phase difference based on the contrast value (step S3). Specifically, it is considered that the higher the contrast value, the higher the accuracy of phase difference detection by correlation calculation. Therefore, the position where the contrast value is determined to be relatively high in the right image is set as the phase difference detection position. decide. For example, as will be described later, the position where the contrast value is maximum in the right image, the barycentric position of the contrast value, and the like are determined as the phase difference detection position.

  Next, the phase difference detection unit 40 detects the phase difference between the right image and the left image at the determined detection position (step S4). Specifically, correlation calculation is performed using the right image used for determination of the detection position as a reference image and the left image as a search image, and a phase difference at the detection position is detected. In the correlation calculation, for example, an image near the detection position is extracted from the right image, the image is shifted on the left image with reference to the detection position, and a correlation value (for example, SAD value) at each shift amount is obtained. A shift amount that becomes a value (for example, a minimum value in the case of an SAD value) is detected as a correlation value.

  Next, the AF control unit 50 controls the lens driving unit 60 based on the detected phase difference to adjust the focus of the imaging unit 10 (step S5). For example, the AF control unit 50 calculates a lens driving amount that focuses on the subject based on the phase difference, drives the focus lens of the imaging unit 10 with the lens driving amount, and focuses the subject. Alternatively, the AF control unit 50 moves the focus lens of the imaging unit 10 by a predetermined lens driving amount in a direction in which the subject is focused according to the shift direction of the phase difference, and the subject is subjected to the phase difference AF multiple times. The focus may be adjusted.

  According to the above embodiment, as shown in FIG. 1, the imaging apparatus includes the imaging unit 10, the contrast value calculation unit 20, the detection position determination unit 30, the phase difference detection unit 40, and the AF control unit 50 (focus control unit). Prepare. As described with reference to FIG. 2, the imaging unit 10 acquires a right image (first image) and a left image (second image) having a parallax with respect to the right image as parallax images. The contrast value calculation unit 20 calculates the contrast value CTV from at least one of the parallax images (the right image in the example of the present embodiment). The detection position determination unit 30 determines a detection position for detecting a phase difference between the right image and the left image based on the contrast value CTV. The phase difference detection unit 40 detects the phase difference between the right image and the left image at the detection position. The AF control unit 50 performs focus control of the imaging unit 10 based on the phase difference.

  Now, in the case where a plurality of discrete positions are used as phase difference detection positions as in the prior art, there is a possibility that a subject area suitable for phase difference detection may not form an image at the discrete positions, There is a problem that the processing load is large because the correlation calculation is performed for the plurality of positions. Further, in the method of providing a phase difference detection pixel in the image sensor as in Patent Document 1, there is a problem that the phase difference detection pixel becomes a pixel defect.

  In this regard, in the present embodiment, since the phase difference detection position is determined based on the contrast value CTV, an arbitrary position suitable for phase difference detection in the image can be determined as the phase difference detection position. As a result, the phase difference can be detected at an arbitrary pixel position instead of being discrete, so that the detection reliability of the phase difference is improved, and a stable and highly accurate phase difference AF can be realized. Further, since the correlation calculation is performed only at the determined detection position, the cost and calculation time of the phase difference detection circuit can be minimized. In addition, since it is not necessary to provide a phase difference detection pixel, it is not necessary to perform pixel interpolation, and a high-definition image can be captured. Further, it is not necessary to prepare a special image sensor provided with phase difference detection pixels, and for example, a general RGB image sensor can be used.

  In the present embodiment, the contrast value calculation unit 20 sets an AF target area in at least one of the parallax images (the right image in the example of the present embodiment), and calculates a contrast value CTV in the AF target area. . The detection position determination unit 30 detects a position having a relatively high contrast value in the AF target area, and determines the position having the relatively high contrast value as a detection position of the phase difference. For example, the position having a relatively high contrast value is a position corresponding to the maximum value of the contrast value in the AF target area or the barycentric position of the contrast value in the AF target area.

  In this way, the detection position of the phase difference can be set not in a low contrast region such as a gradation portion having no edge component but in a high contrast region considered suitable for phase difference detection. Thereby, highly reliable phase difference AF can be realized.

2. Second Embodiment Next, a second embodiment will be described. Note that the description of the same components, operations, and processes as in the first embodiment will be omitted as appropriate. For example, the imaging apparatus of the second embodiment can be configured in the same manner as the imaging apparatus of FIG. In the following, the case where the parallax direction is the horizontal scanning direction and the contrast value is calculated from the right image will be described as an example, but the parallax direction may be a direction other than the horizontal scanning direction, or the contrast value may be calculated from the left image. It may be calculated.

  As shown in FIG. 3, the contrast value calculation unit 20 sets an AF target area for the right image, and divides the AF target area into M × N sub-blocks. Each sub-block is composed of n × m pixels. FIG. 3 shows an example of division into M = 10 sub-blocks in the x direction which is the horizontal scanning direction and N = 6 sub-blocks in the y direction which is the vertical scanning direction. Here, the AF target area may be, for example, a predetermined area in the right image or the entire right image.

The contrast value calculation unit 20 calculates the contrast value CTV for each sub-block using the following equation (1). Here, i represents the pixel position in the x direction within the sub-block, and j represents the pixel position in the y direction within the sub-block. D _ij is a luminance value at the pixel at the position (i, j), and is obtained by the following equation (2). R _ij , G _ij , and B _ij are red, green, and blue pixel values, respectively.

  The method for calculating the contrast value is not limited to the above method, and various methods can be used. For example, the contrast value may be calculated by various filter processes (for example, one-dimensional or two-dimensional high-pass filter process or band-pass filter process) that extract high-frequency components of the image, or the contrast value may be calculated by obtaining a differential image. It may be calculated.

  As shown in FIG. 4, the contrast value calculation unit 20 generates a contrast map based on the contrast value CTV of each sub-block. The contrast map is obtained by storing data in which a position of a sub block (for example, xy coordinates at the center of the sub block) and a contrast value CTV are associated with each other in a memory (not shown). In FIG. 4, sub-blocks with higher contrast values CTV are represented by darker hatching.

  As illustrated in FIG. 5, the detection position determination unit 30 detects the maximum value Dxmax in the x direction from the contrast values arranged in the x direction in the contrast map. In the example of FIG. 5, ten contrast values are arranged in the x direction, and six Dxmax are obtained by obtaining the x direction maximum value Dxmax for each position in the y direction. Further, the detection position determination unit 30 detects the y-direction maximum value Dymax from the contrast values arranged in the y-direction in the contrast map. In the example of FIG. 5, six contrast values are arranged in the y direction, and ten Dymax are obtained by obtaining the y direction maximum value Dymax for each position in the x direction.

  The detection position determination unit 30 detects the maximum value from Dxmax, and obtains the position in the y direction corresponding to the maximum value (for example, the center position of the sub-block corresponding to the maximum value). Further, the maximum value is detected from Dymax, and the position in the x direction corresponding to the maximum value (for example, the center position of the sub-block corresponding to the maximum value) is obtained. Then, the detection position determination unit 30 outputs the obtained positions in the x direction and the y direction as detection positions of the phase difference.

  In the above description, the case where the position having the maximum contrast value is adopted as the detection position has been described. However, the present embodiment is not limited to this. For example, the barycentric position of the contrast value may be adopted as the detection position.

Specifically, as shown in FIG. 6, the detection position determination unit 30 obtains the center-of-gravity position GPx in the x direction by the following equation (3), obtains the center-of-gravity position GPy in the y-direction by the following equation (4), GPx, GPy) is output as a phase difference detection position. Here, k represents the numbering of M subblocks arranged in the x direction, and l represents the numbering of N subblocks arranged in the y direction. Dyadd _k is a value obtained by adding the contrast value of the k th of N aligned sub-blocks in the y-direction in the x-direction, Dxadd _l is the contrast value of the M aligned sub-blocks in the x direction in the l-th in the y-direction It is the added value. _xk is the position of the kth subblock in the x direction (for example, the center position of the subblock), and _yl is the position of the lth subblock in the y direction (for example, the center position of the subblock).

  Thus, by setting the barycentric position of the contrast value as the phase difference detection position, it becomes strong against image noise, and it is possible to realize a stable phase difference AF. That is, even if the contrast value of the image is locally high due to noise in a portion where the contrast of the subject is actually low, such as a flat portion, the high contrast of the subject can be obtained by obtaining the center of gravity position of the contrast value. The portion can be detected as a phase difference detection position.

  FIG. 7 shows a detailed configuration example of the phase difference detection unit 40. The phase difference detection unit 40 includes a readout position shift instruction unit 200, an image acquisition unit 210 (video signal acquisition unit), a SAD calculation unit 220, a minimum value detection unit 230, a shift amount holding unit 240, and a reliability detection unit 250. .

  As illustrated in FIG. 8, the image acquisition unit 210 acquires a search image used for the SAD calculation from the left image, and acquires a reference image used for the SAD calculation from the right image. The search image is 128 pixels continuous in the horizontal scanning direction (parallax direction) with the detection position (xd, yd) determined by the detection position determination unit 30 as the center. The reference image is 64 pixels continuous in the horizontal scanning direction (parallax direction) with the detection position (xd, yd) as the center.

The SAD calculation unit 220 obtains an SAD value from the reference image and the search image by the following equation (5). Here, s is a shift amount of the reference image with respect to the search image, and the case where the center of the search image matches the center of the reference image is set as a standard (s = 0). SAD (s) is the SAD value at the shift amount s. x ′ represents a pixel position in the reference image, and SP _{x ′} represents a pixel value at the pixel position x ′ of the reference image. x ′ + s represents a pixel position in the search image, and is a pixel position corresponding to the pixel position x ′ when the reference image and the search image are overlapped with a shift amount s. TP _{x ′ + s} represents the pixel value at the pixel position x ′ + s of the search image.

  As shown in FIG. 9, the read position shift instruction unit 200 outputs the shift amount s while changing the shift amount s by one, and the SAD operation unit 220 obtains the SAD value SAD (s) at the shift amount s. The minimum value detection unit 230 detects the minimum value of SAD (s), and the shift amount holding unit 240 holds the detected shift amount s ′ of the minimum value, and outputs the shift amount s ′ as a phase difference. .

  The reliability detection unit 250 digitizes the valley shape (acute angle) of the SAD value curve and outputs the numerical value as reliability information. For example, the reliability detection unit 250 detects the SAD value SAD (s) having a difference of 2 or more with respect to the minimum value SAD (s ′) of the SAD value, and shift amount s of the detected SAD (s). And the difference pxn (number of pixels) between the shift amount s ′ and the shift amount s ′ are output as reliability. In this case, the smaller the number of pixels pxn, the sharper the fall of the valley of the SAD value curve, and the higher the reliability of the phase difference s ′.

  The AF control unit 50 gradually moves the lens position of the imaging unit 10 in each frame in a direction in which the phase difference s ′ is zero based on the phase difference s ′ detected in each frame of the video. Specifically, the AF control unit 50 decreases the phase difference s ′ when the sign of the phase difference s ′ in the current frame is positive (for example, when the focused position is far from the subject). Move the lens position by a predetermined amount in the direction (the direction to bring the focused position closer). On the other hand, when the sign of the phase difference s ′ in the current frame is negative (for example, when the in-focus position is closer to the subject), the direction in which the phase difference s ′ is increased (the in-focus position is changed). The lens position is moved by a predetermined amount in the direction of moving away. Then, the lens position is controlled so that the phase difference s ′ becomes zero over a plurality of frames. In this operation, if the reliability represented by the reliability information is lower than the predetermined reliability (for example, the number of pixels pxn is larger than the predetermined value), the AF control unit 50 does not move the lens position in that frame.

  In this way, it is possible to suppress malfunction of phase difference AF and prevent an unnatural image that is suddenly out of focus from being displayed. In other words, when AF control is performed such that the phase difference s ′ is set to zero by one lens movement, if the phase difference s ′ is erroneously detected, the lens can be moved erroneously to a lens position where the subject is not in focus. There is sex. In this regard, according to the present embodiment, since the lens is moved over a plurality of frames by a predetermined value, even if there is an erroneous detection of the phase difference s ′, the lens moves only a little, and an error occurs. It is possible to minimize the influence of detection.

  FIG. 10 shows a flowchart of AF control processing performed by the imaging apparatus according to the second embodiment. As shown in FIG. 10, the imaging unit 10 first acquires RGB pixel value data of a parallax image (right image, left image) (step S20).

  Next, the contrast value calculation unit 20 converts the RGB pixel value data of the parallax image into luminance value data (step S21). Next, the contrast value calculation unit 20 calculates the contrast value of each sub-block from the luminance value data of at least one of the parallax images (for example, the right image) (step S22).

  Next, the detection position determination unit 30 calculates the maximum y-direction value Dymax of the contrast value, detects the maximum value of the Dymax as the maximum value in the x-direction, and corresponds to the maximum value in the x-direction. The x position of the sub block to be output is output as the x position of the phase difference detection position (step S23). Next, the detection position determination unit 30 calculates the maximum value Dxmax of the contrast value in the x direction, detects the maximum value of the Dxmax as the maximum value in the y direction, and corresponds to the maximum value in the y direction. The y position of the sub-block to be output is output as the y position of the phase difference detection position (step S24).

  Next, the phase difference detection unit 40 acquires a search image (128 pixels centered on the detection position) from the left image based on the detection position of the phase difference (step S25). Next, the phase difference detection unit 40 acquires a reference image (64 pixels centered on the detection position) from the right image based on the detection position of the phase difference (step S26). Next, the phase difference detection unit 40 calculates the SAD value SAD (s) while shifting the read start position (shift amount s) of the search image, and calculates the shift amount s that minimizes SAD (s) as the phase difference. It detects as s' (step S27).

  Next, when the sign of the phase difference s ′ is positive, the AF control unit 50 moves the lens position by a predetermined amount in the direction of decreasing the phase difference s ′, and the sign of the phase difference s ′ is negative. In order to increase the phase difference s ′, the lens position is moved by a predetermined amount (step S28).

  According to the above embodiment, the contrast value calculation unit 20 generates a contrast map of the AF target area (autofocus target area). The detection position determination unit 30 detects the maximum value of the contrast value CTV in the contrast map, and detects a position corresponding to the maximum value as a position having a relatively high contrast value. Alternatively, the detection position determination unit 30 obtains the centroid position of the contrast value CTV in the contrast map, and detects the centroid position as a position having a relatively high contrast value.

  In this way, a position having a relatively high contrast value can be detected in the AF target area, and the position having the relatively high contrast value can be determined as a detection position for detecting the phase difference of the parallax image. it can.

  In the present embodiment, the imaging unit 10 captures a parallax image as a moving image. The phase difference detection unit 40 detects the phase difference s ′ for each frame of the moving image. The AF control unit 50 (focus control unit) determines the perspective relationship between the in-focus position of the imaging unit 10 and the subject based on the phase difference s ′, and based on the determination result, the AF control unit 50 (focus control unit). Control is performed to move the focus lens of the imaging unit 10 by a predetermined amount in a direction in which the in-focus position approaches the subject. Then, the subject is focused by repeating the control of moving by the predetermined amount in each frame.

  In this way, as described above, it is possible to suppress the malfunction of the phase difference AF and prevent the display of an unnatural image that is suddenly out of focus.

3. Third Embodiment Next, a third embodiment will be described. Note that description of components, operations, and processes similar to those in the first and second embodiments is omitted as appropriate.

  FIG. 11 shows a configuration example of the imaging apparatus in the third embodiment. The imaging apparatus includes an imaging unit 10, a contrast value calculation unit 20, a detection position determination unit 30, a phase difference detection unit 40, an AF control unit 50, a lens driving unit 60, a phase difference selection unit 70, and a display control unit 80.

  As shown in FIG. 12, the contrast value calculation unit 20 divides the AF target area into a plurality (a) of blocks BL1 to BL6. FIG. 12 shows an example of division into a = 2 × 3 blocks. Each block is divided into M × N sub-blocks, and each sub-block is composed of n × m pixels. The contrast value calculation unit 20 generates a contrast map for each block and outputs the contrast map to the detection position determination unit 30.

The detection position determination unit 30 determines the detection position of the phase difference for each block based on the contrast map. The detection position determination unit 30 obtains the variance value YDS of the maximum value Dymax in the y direction for each block by the following equation (6), and sorts the detection positions in ascending order of the variance value YDS. Here, k represents the numbering of M sub-blocks arranged in the x direction. Dymax _k is the maximum value among the contrast values of the k sub blocks arranged in the x direction and N sub blocks arranged in the y direction.

  The detection position determination unit 30 selects a plurality (b, b <a) of detection positions in order from the smallest variance value YDS, and outputs it to the phase difference detection unit 40. FIG. 11 shows an example of outputting a detection position of b = 3.

  The smaller the variance value YDS, the smaller the distribution variation of the y-direction maximum value Dymax, so there is a high possibility that the block is suitable for phase difference detection. For example, in a region where similar shapes are repeatedly arranged or in a high noise region, it is considered that a plurality of contrast value peaks are arranged and the variance value YDS is large. Therefore, it is possible to select a detection position that is more suitable for phase difference detection by sorting in order of increasing dispersion value YDS.

  Although the case where sorting is performed in ascending order of the variance value YDS has been described above, the present embodiment is not limited to this. For example, the reciprocal of the variance value YDS and the maximum value of the contrast value may be added by multiplying by a weighting coefficient, sorted in descending order of the added value, and a plurality of detection positions may be selected in descending order of the added value.

  The phase difference detection unit 40 includes first to third detection units 41 to 43. Three detection positions selected by the detection position determination unit 30 are input to the detection units 41 to 43 one by one. The detectors 41 to 43 obtain a phase difference and reliability for the input detection position, and output the phase difference and reliability to the phase difference selector 70.

  The phase difference selection unit 70 selects the phase difference having the highest reliability from the three phase differences and outputs the phase difference to the AF control unit 50.

  In the above description, the case where the phase difference for focus control is automatically selected based on the reliability has been described. However, the present embodiment is not limited to this. For example, the phase difference for focus control may be selected based on a selection instruction from the user. Specifically, the phase difference selection unit 70 outputs information on detection positions corresponding to the three phase differences from the phase difference detection unit 40 to the display control unit 80. As illustrated in FIG. 13, the display control unit 80 performs control to superimpose display AP <b> 1 to AP <b> 3 (for example, a rectangular frame) representing the three detection positions on a display image and display the display image on a display unit (not illustrated). Then, the user selects a detection position via an operation unit (not shown), and the phase difference selection unit 70 outputs the phase difference of the selected detection position to the AF control unit 50.

  By determining the detection position based on the user instruction in this way, it is possible to prevent autofocusing to a position not desired by the user. Further, by displaying a plurality of candidate positions in ascending order of the variance value YDS, it is possible to select an appropriate location desired by the user from candidates that can detect the phase difference with high reliability.

  FIG. 14 shows a flowchart of AF control processing performed by the imaging apparatus of the third embodiment. As shown in FIG. 14, first, the imaging unit 10 acquires RGB pixel value data of a parallax image (right image, left image) (step S40).

  Next, the contrast value calculation unit 20 converts the RGB pixel value data of the parallax image into luminance value data (step S41). Next, the contrast value calculation unit 20 sets a plurality of blocks BL1 to BL6 in at least one of the parallax images (for example, the right image), and the contrast of each sub-block for the selected block among the plurality of blocks BL1 to BL6. A value is calculated from the luminance data (step S42).

  Next, the detection position determination unit 30 calculates the maximum y-direction value Dymax of the contrast value, detects the maximum value of the Dymax as the maximum value in the x-direction, and corresponds to the maximum value in the x-direction. The x position of the sub block to be calculated is obtained as the x position of the phase difference detection position. Further, the variance value YDS is calculated (step S43). Next, the detection position determination unit 30 calculates the maximum value Dxmax of the contrast value in the x direction, detects the maximum value of the Dxmax as the maximum value in the y direction, and corresponds to the maximum value in the y direction. The y position of the sub-block to be obtained is obtained as the y position of the phase difference detection position (step S44).

  Next, the detection position determination unit 30 determines whether or not the phase difference detection position and the dispersion value YDS have been obtained for all of the plurality of blocks BL1 to BL6 (step S45). When there is a block for which the phase difference detection position and the variance value YDS are not obtained, the contrast value calculation unit 20 newly selects a block and executes step S42 again. When the phase difference detection position and the dispersion value YDS are obtained for all the blocks, the detection position determining unit 30 sorts the dispersion values YDS of the plurality of blocks BL1 to BL6 in ascending order, and the dispersion value YDS is in ascending order. A plurality of detection positions are output (step S46).

  Next, the phase difference detection unit 40 selects one detection position from a plurality of detection positions, and acquires a search image (128 pixels centered on the detection position) from the left image for the selected detection position (step S47). ). Next, the phase difference detection unit 40 acquires a reference image (64 pixels centered on the detection position) from the right image for the selected detection position (step S48). Next, the phase difference detection unit 40 calculates the SAD value SAD (s) while shifting the read start position (shift amount s) of the search image, and calculates the shift amount s that minimizes SAD (s) as the phase difference. Detect as s'. Further, the phase difference detection unit 40 obtains the reliability from the curve shape of SAD (s) (step S49).

  Next, the phase difference detection unit 40 determines whether or not the phase difference and the reliability have been obtained for all of the plurality of detection positions output in step S46 (step S50). If there is a detection position for which the phase difference and the reliability are not obtained, the phase difference detection unit 40 executes step S47 again. When the phase difference and the reliability are obtained for all the detection positions, the phase difference selecting unit 70 selects one phase difference having the highest reliability from the plurality of phase differences, and sends it to the AF control unit 50. Output (step S51).

  Next, when the sign of the phase difference is positive, the AF control unit 50 moves the lens position by a predetermined amount in the direction of decreasing the phase difference. When the sign of the phase difference is negative, the AF control unit 50 sets the phase difference. The lens position is moved by a predetermined amount in the increasing direction (step S52).

  According to the above embodiment, the detection position determination unit 30 sets a plurality of regions BL1 to BL6 in at least one of the parallax images (the right image in the example of the present embodiment), and the plurality of regions BL1 to BL1. Based on the contrast value (in each region) for each BL6, the detection position for each of the plurality of regions BL1 to BL6 is determined. The phase difference detection unit 40 detects a plurality of phase differences s ′ for each of the plurality of detection positions. The phase difference selector 70 selects a phase difference used for focus control from among the plurality of phase differences s ′.

  In this way, it is possible to determine a plurality of positions as candidates for the detection position of the phase difference and select a detection position more suitable for the phase difference detection from the plurality of candidates. As a result, the possibility that the phase difference cannot be detected and the possibility of erroneous detection of the phase difference can be reduced, and a more stable phase difference AF can be realized. For example, when the parallax is large, there is a possibility that the same subject is not captured in the right image and the left image in some of the plurality of regions BL1 to BL6. Even in such a case, a detection position can be determined in each region, and an appropriate detection position can be selected from among the detection positions, so that a phase difference can be detected.

  Further, in the present embodiment, the detection position determination unit 30 sets a region (a = 6 in the example of the present embodiment) as the plurality of regions BL1 to BL6, and the contrast value in each region of the a regions. The variance value YDS is calculated, and b regions (b <a, b = 3 in the example of the present embodiment) are selected from the a regions having the smallest variance value YDS. The phase difference detection unit 40 calculates b phase differences corresponding to the detection positions in the b regions by correlation calculation.

  In this way, correlation calculation can be performed only for detection positions with high dispersion values that are considered more suitable for phase difference detection, and useless correlation calculation is omitted for detection positions with low dispersion values that are not suitable for phase difference detection. it can. As a result, it is possible to reduce the circuit scale of the phase difference detection unit and the calculation time.

  In the present embodiment, the imaging apparatus uses information AP1 to AP3 representing a plurality of detection positions as at least one of parallax images (for example, a 3D image as in the fourth embodiment, or a right image and a left image). And a display control unit 80 that performs control to display on the display unit. The phase difference selection unit 70 selects a phase difference corresponding to the detection position selected by the user from among a plurality of detection positions as a phase difference used for focus control.

  In this way, as described above, by determining the detection position based on the user instruction, it is possible to prevent the user from being autofocused at a position not desired by the user.

4). Fourth Embodiment Next, a fourth embodiment will be described. Note that description of the same components, operations, and processes as those in the first to third embodiments will be omitted as appropriate.

  FIG. 15 shows a first configuration example of the imaging apparatus according to the fourth embodiment. The imaging apparatus includes an imaging unit 10, a detection position determination unit 30, a phase difference detection unit 40, an AF control unit 50, a lens driving unit 60, a contrast map generation unit 100, a 3D conversion circuit 110, and a 3D display monitor 120.

  FIG. 16 shows a second configuration example of the imaging apparatus according to the fourth embodiment. The imaging apparatus includes an imaging unit 10, a detection position determination unit 30, a phase difference detection unit 40, an AF control unit 50, a lens driving unit 60, a phase difference selection unit 70, a contrast map generation unit 100, a 3D conversion circuit 110, and a 3D display. A monitor 120, a character superimposing circuit 130, and an operation unit 150 are included.

  The contrast map generation unit 100 generates a contrast map from the luminance value of the right image and outputs the contrast map to the detection position determination unit 30 in the same manner as described in the second embodiment.

  The 3D conversion circuit 110 adjusts the position in the horizontal operation direction with respect to the left image and the right image, and alternately switches the left image and the right image after the position adjustment to output them.

  The 3D display monitor 120 displays the left image and the right image that are alternately output from the 3D conversion circuit 110. The 3D display monitor 120 is a display device that can individually view a left image (left viewpoint image) and a right image (right viewpoint image) by wearing dedicated glasses such as polarized glasses and liquid crystal shutter glasses. The 3D display monitor 120 is a stereoscopic display device that can display a stereoscopic image (left viewpoint image and right viewpoint image) as a directional image having a predetermined directivity by a parallax barrier, or a stereoscopic display that uses a lenticular lens. It may be a display device.

  The character superimposing circuit 130 superimposes a display representing a plurality of detection positions output from the phase difference selecting unit 70 on the image output from the 3D conversion circuit 110 on the image output from the 3D conversion circuit 110. Then, the superimposed image is output to the 3D display monitor 120.

  The operation unit 150 is an operation unit for performing an operation of selecting a detection position to be focused from among a plurality of detection positions displayed on the 3D display monitor 120. For example, the operation unit 150 can be configured by a mouse, a button, a touch panel, or the like.

  FIG. 17 shows a detailed configuration example of the imaging unit 10. The imaging unit 10 includes a focus lens LN1, a pair of left and right zoom optical systems LN2 and LN3, a pair of left and right imaging lenses LN4 and LN5, a pair of left and right imaging elements IS1 and IS2, and a pair of left and right imaging control units CU1. CU2 (camera control unit). The imaging elements IS1 and IS2 are constituted by a CCD (Charge-Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor) image sensor. For example, a general imaging element such as a Bayer array RGB imaging element can be used.

  A subject image is formed on the imaging elements IS1 and IS2 by the focus lens LN1, the zoom optical systems LN2 and LN3, and the imaging lenses LN4 and LN5, and the imaged subject images are captured by the imaging elements IS1 and IS2. The imaging control units CU1 and CU2 perform offset processing, white balance correction, gain control processing including sensitivity correction, gamma correction processing, synchronization processing, YC processing, sharpness on the image signals from the imaging elements IS1 and IS2. Image processing such as correction processing is performed. The imaging control units CU1 and CU2 output the image-processed images as a right image and a left image.

  In such an imaging unit 10, the imaging position is displaced according to the focus state regardless of the zoom magnification. That is, when the focus is in front of the subject, the right image is shifted to the left with respect to the left image, and the shift amount increases as the focus moves to the front. On the other hand, when the focus is behind the subject, the right image is shifted to the right with respect to the left image, and the shift amount increases as the focus moves rearward from the subject. When the subject is in focus, the right image and the left image have the same imaging position.

  The imaging apparatus as described above is assumed to be applied to, for example, a binocular microscope for brain surgery. In a binocular microscope for brain surgery, it is only necessary to adjust the focus within a relatively narrow range of about several centimeters in depth, and therefore, the parallax between the right image and the left image is expected to be relatively small. For example, when there is a focus on the near side within a few centimeters of depth, suppose that it is desired to perform autofocus on the far side within a few centimeters of depth. Even in such a case, since the parallax of the subject on the back side does not increase so much, it is considered that almost the same subject area is shown in the right image and the left image.

  In phase difference AF, the same subject needs to appear in the right image and the left image at the phase difference detection position, but if the parallax is not large, even if an arbitrary position in the image is selected as the detection position, It is very likely that the same subject appears in the right image and the left image at that position. Therefore, even if a position with a high contrast value is selected from an arbitrary position in the image as in this embodiment, it is possible to detect a phase difference at the arbitrary position, and a phase difference at a position with a high contrast value. Since AF is performed, high-quality phase difference AF can be realized. Thus, this embodiment is expected to be very effective when applied to a binocular microscope for brain surgery.

  The present embodiment is not limited to a binocular microscope for brain surgery or the like, and can be applied to, for example, a general binocular camera or a binocular video camera.

  Although the present embodiment has been described in detail as described above, it will be readily understood by those skilled in the art that many modifications can be made without departing from the novel matters and effects of the present invention. . Accordingly, all such modifications are intended to be included in the scope of the present invention. For example, a term described with a different term having a broader meaning or the same meaning at least once in the specification or the drawings can be replaced with the different term in any part of the specification or the drawings. In addition, the configuration, operation, contrast value calculation method, detection position determination method, phase difference detection method, AF control method, and the like of the imaging unit, the phase difference detection unit, and the imaging device are not limited to those described in this embodiment. Various modifications are possible.

10 imaging unit, 20 contrast value calculating unit, 30 detection position determining unit,
40 phase difference detection unit, 41 to 43 detection unit,
50 AF control unit (focus control unit), 60 lens drive unit,
70 phase difference selector, 80 display controller,
100 contrast map generation unit, 110 3D conversion circuit,
120 3D display monitor, 130 character superimposing circuit, 150 operation unit,
200 read position shift instruction unit, 210 image acquisition unit,
220 SAD calculation unit, 230 minimum value detection unit, 240 shift amount holding unit,
250 reliability detector,
BL1 to BL6 Multiple blocks (multiple areas), CTV contrast value,
Dxmax maximum value in x direction, Dymax maximum value in y direction,
LN1 focus lens, pxn number of pixels (reliability information),
s ′ phase difference, x horizontal scanning direction, y vertical scanning direction, xd, yd detection position

Claims (11)

An imaging unit that acquires a first image and a second image having parallax with respect to the first image as a parallax image;
A contrast value calculator that calculates a contrast value from at least one of the parallax images;
A detection position determination unit that determines a detection position for detecting a phase difference between the first image and the second image based on the contrast value;
A phase difference detector that detects the phase difference between the first image and the second image at the detection position;
A focus control unit that performs focus control of the imaging unit based on the phase difference;
An imaging apparatus comprising: In claim 1,
A phase difference selector, and
The detection position determination unit
Setting a plurality of regions for at least one of the parallax images, determining the detection positions for each of the plurality of regions based on the contrast values for the plurality of regions,
The phase difference detector is
Detecting a plurality of phase differences for each of the plurality of detection positions;
The phase difference selector is
An imaging apparatus, wherein a phase difference used for the focus control is selected from a plurality of the phase differences. In claim 2,
The detection position determination unit
An imaging apparatus, wherein a position having a relatively high contrast value is detected for each of the plurality of regions, and a position having the high contrast value is determined as the detection position. In claim 2 or 3,
The detection position determination unit
A regions are set as the plurality of regions, the variance of the contrast value in each of the a regions is calculated, and b regions (b Select <a)
The phase difference detector is
B phase differences corresponding to the detected positions in the b regions are calculated by correlation calculation, and reliability information of the correlation calculation is obtained for each phase difference of the b phase differences;
The phase difference selector is
An imaging apparatus, wherein a phase difference having the highest reliability represented by the reliability information among the b phase differences is selected as the phase difference for focus control. In any of claims 2 to 4,
A display control unit that performs control to superimpose information representing a plurality of the detection positions on the at least one image of the parallax image and display the information on a display unit;
The phase difference selector is
An imaging apparatus, wherein the phase difference corresponding to a detection position selected by a user is selected as a phase difference for focus control among the plurality of detection positions. In claim 1,
The contrast value calculation unit
Setting an autofocus target area on at least one of the parallax images, calculating a contrast value in the autofocus target area;
The detection position determination unit
An image pickup apparatus that detects a position having a relatively high contrast value in the autofocus target region and determines the position having the high contrast value as the detection position. In claim 6,
The contrast value calculation unit
Generating a contrast map of the autofocus target area;
The detection position determination unit
An imaging apparatus, wherein a maximum value of the contrast value in the contrast map is detected, and a position corresponding to the maximum value is detected as a position having a relatively high contrast value. In claim 6,
The contrast value calculation unit
Generating a contrast map of the autofocus target area;
The detection position determination unit
An imaging apparatus, wherein a gravity center position of the contrast value in the contrast map is obtained, and the gravity center position is detected as a position having a relatively high contrast value. In any one of Claims 1 thru | or 8.
The imaging unit
Capturing the parallax image as a moving image;
The phase difference detector is
Detecting the phase difference for each frame of the moving image;
The focus control unit
A direction in which the imaging unit is in focus and a subject is determined based on the phase difference, and a direction in which the imaging unit is in focus is brought closer to the subject based on the determination result. The image pickup apparatus is characterized in that the focus lens of the image pickup unit is controlled to move by a predetermined amount, and the control to move the focus lens by the predetermined amount is repeated for each frame to focus on the subject. Obtaining a first image and a second image having a parallax with respect to the first image as a parallax image;
A contrast value is calculated from at least one of the parallax images;
Determining a detection position for detecting a phase difference between the first image and the second image based on the contrast value;
Detecting a phase difference between the first image and the second image at the detection position;
A focus control method, wherein focus control of the imaging unit is performed based on the phase difference. In claim 10,
Setting a plurality of regions for the at least one image of the parallax images, determining the detection positions for the plurality of regions based on the contrast values for the plurality of regions,
Detecting a plurality of phase differences corresponding to the plurality of determined detection positions;
Select a phase difference to be used for focus control from a plurality of the phase differences,
A focus control method comprising: performing focus control of the imaging unit based on a phase difference used for the focus control.